l2k18 ahoy [s_graf@telus.net: FW: SSL connection failure with ftp but not wget [...]

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

l2k18 ahoy [s_graf@telus.net: FW: SSL connection failure with ftp but not wget [...]

Stuart Henderson
Probably worth pinging this one since l2k18 is on.

s_graf is trying to fetch ports sources from https://pypi.org/ and is
getting a hang and eventually a timeout when attempting connection from
ftp(1) on armv7. (From recent ports@ posts it seems like this still occurs).

curl/wget were working ok when tested before.

Can anyone with armv7 confirm/deny that they can replicate this? (just try
"ftp https://pypi.io/packages/source/s/six/six-1.11.0.tar.gz").

Any ideas?




> ftp: SSL write error: handshake failed: Operation timed out

----- Forwarded message from [hidden email] -----

From: [hidden email]
Date: Wed, 15 Aug 2018 15:02:40 -0700
To: [hidden email]
Cc: 'Stuart Henderson' <[hidden email]>
X-Mailer: Microsoft Outlook 16.0
Subject: FW: SSL connection failure with ftp but not wget [was Re: python files moved]  -- compare with another
        system

Trying to get a file during a php build fails on arm but not on i386
systems.
Stuart Henderson suggested I forward this.


 From the arm system (op1bsdtest2)

op1bsdtest2# curl -v https://pypi.io/packages/source/s/six/six-1.11.0.tar.gz
*   Trying 151.101.0.223...
* TCP_NODELAY set
* Connected to pypi.io (151.101.0.223) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: businessCategory=Private Organization;
jurisdictionCountryName=US; jurisdictionStateOrProvinceName=Delaware;
serialNumber=3359300; C=US; ST=New Hampshire; L=Wolfeboro; O=Python Software
Foundation; CN=www.python.org
*  start date: Mar 28 00:00:00 2018 GMT
*  expire date: Sep 27 12:00:00 2018 GMT
*  subjectAltName: host "pypi.io" matched cert's "pypi.io"
*  issuer: C=US; O=DigiCert Inc; OU=www.digicert.com; CN=DigiCert SHA2
Extended Validation Server CA
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade:
len=0
* Using Stream ID: 1 (easy handle 0x8063b000)
> GET /packages/source/s/six/six-1.11.0.tar.gz HTTP/2
> Host: pypi.io
> User-Agent: curl/7.61.0
> Accept: */*
>
* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
< HTTP/2 301
< server: Varnish
< retry-after: 0
< location: https://pypi.org/packages/source/s/six/six-1.11.0.tar.gz
< content-type: text/html; charset=UTF-8
< accept-ranges: bytes
< date: Wed, 15 Aug 2018 21:55:38 GMT
< x-served-by: cache-sea1033-SEA
< x-cache: HIT
< x-cache-hits: 0
< x-timer: S1534370139.898309,VS0,VE0
< strict-transport-security: max-age=31536000; includeSubDomains; preload
< x-frame-options: deny
< x-xss-protection: 1; mode=block
< x-content-type-options: nosniff
< x-permitted-cross-domain-policies: none
< content-length: 122
<
* Connection #0 to host pypi.io left intact
<html><head><title>301 Moved Permanently</title></head><body><center><h1>301
Moved Permanently</h1></center></body></html>op1bsdtest2#



op1bsdtest2# dmesg
OpenBSD 6.3-current (GENERIC) #32: Fri Aug 10 10:32:37 MDT 2018
    [hidden email]:/usr/src/sys/arch/armv7/compile/GENERIC
real mem  = 536870912 (512MB)
avail mem = 516112384 (492MB)
mainbus0 at root: Xunlong Orange Pi One
cpu0 at mainbus0: ARM Cortex-A7 r0p5 (ARMv7)
cpu0: DC enabled IC enabled WB disabled EABT branch prediction enabled
cpu0: 32KB(32b/l,2way) I-cache, 32KB(64b/l,4way) wr-back D-cache
cortex0 at mainbus0
psci0 at mainbus0: PSCI 0.0
sxiccmu0 at mainbus0
simplebus0 at mainbus0: "soc"
syscon0 at simplebus0: "syscon"
sxiccmu1 at simplebus0
sxipio0 at simplebus0: 94 pins
ampintc0 at simplebus0 nirq 160, ncpu 4: "interrupt-controller"
sxiccmu2 at simplebus0
sxipio1 at simplebus0: 12 pins
sximmc0 at simplebus0
sdmmc0 at sximmc0: 4-bit, sd high-speed, mmc high-speed, dma
ehci0 at simplebus0
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 configuration 1 interface 0 "Generic EHCI root hub" rev
2.00/1.00 addr 1
ehci1 at simplebus0
usb1 at ehci1: USB revision 2.0
uhub1 at usb1 configuration 1 interface 0 "Generic EHCI root hub" rev
2.00/1.00 addr 1
sxitemp0 at simplebus0
dwxe0 at simplebus0: address 02:81:b1:07:76:5e
ukphy0 at dwxe0 phy 1: Generic IEEE 802.3u media interface, rev. 0: OUI
0x001105, model 0x0000
sxidog0 at simplebus0
com0 at simplebus0: ns16550, no working fifo
com0: console
sxitwi0 at simplebus0
iic0 at sxitwi0
"bosch,bme280" at iic0 addr 0x76 not configured
sxitwi1 at simplebus0
iic1 at sxitwi1
"bosch,bme280" at iic1 addr 0x77 not configured
sxirtc0 at simplebus0
gpio0 at sxipio0: 32 pins
gpio1 at sxipio0: 32 pins
gpio2 at sxipio0: 32 pins
gpio3 at sxipio0: 32 pins
gpio4 at sxipio0: 32 pins
gpio5 at sxipio0: 32 pins
gpio6 at sxipio0: 32 pins
gpio7 at sxipio1: 32 pins
agtimer0 at mainbus0: tick rate 24000 KHz
scsibus0 at sdmmc0: 2 targets, initiator 0
sd0 at scsibus0 targ 1 lun 0: <SD/MMC, SL16G, 0080> SCSI2 0/direct removable
sd0: 15193MB, 512 bytes/sector, 31116288 sectors
vscsi0 at root
scsibus1 at vscsi0: 256 targets
softraid0 at root
scsibus2 at softraid0: 256 targets
bootfile: sd0a:/bsd
boot device: sd0
root on sd0a (88106578f2222800.a) swap on sd0b dump on sd0b
op1bsdtest2#

-----Original Message-----
From: Stuart Henderson <[hidden email]>
Sent: August 15, 2018 1:47 PM
To: [hidden email]
Subject: RE: SSL connection failure with ftp but not wget [was Re: python
files moved] -- compare with another system

Very interesting! Could you forward to bugs@ so people who might have a
better idea what's wrong will see it please?

--
Sent from a phone, apologies for poor formatting.

On 15 August 2018 20:01:10 <[hidden email]> wrote:

> It looks like the problem is specific to the arm system. I ran the ftp
> -d on both systems one after the other.  Both are on the same network.
> The arm system is a recent snapshot  base install with src and ports
> loaded and really nothing else.
> I have not seen any other connection problems on the arm system and it
> is doing many as part of the php build.
>
> I will try some of the network reconfigs when the build of php finishes.
>
> From my 6.2 stable server:
>
> # ftp -d https://pypi.io/packages/source/s/six/six-1.11.0.tar.gz
> host pypi.io, port https, path
> packages/source/s/six/six-1.11.0.tar.gz,
> save as six-1.11.0.tar.gz, auth none.
> Trying 151.101.0.223...
> Requesting https://pypi.io/packages/source/s/six/six-1.11.0.tar.gz
> GET /packages/source/s/six/six-1.11.0.tar.gz HTTP/1.0
> Host: pypi.io
> User-Agent: OpenBSD ftp
>
> received 'HTTP/1.1 301 Redirect to Primary Domain'
> received 'Server: Varnish'
> received 'Retry-After: 0'
> received 'Location:
https://pypi.org/packages/source/s/six/six-1.11.0.tar.gz'

> Redirected to https://pypi.org/packages/source/s/six/six-1.11.0.tar.gz
> host pypi.org, port https, path
> packages/source/s/six/six-1.11.0.tar.gz,
> save as six-1.11.0.tar.gz, auth none.
> Trying 151.101.0.223...
> Requesting https://pypi.org/packages/source/s/six/six-1.11.0.tar.gz
> GET /packages/source/s/six/six-1.11.0.tar.gz HTTP/1.0
> Host: pypi.org
> User-Agent: OpenBSD ftp
>
> received 'HTTP/1.1 301 Moved Permanently'
> received 'Content-Security-Policy: base-uri 'self';
> block-all-mixed-content; connect-src 'self'
> https://api.github.com/repos/ *.fastly-insights.com sentry.io
> https://2p66nmmycsj3.statuspage.io;
> default-src 'none'; font-src 'self' fonts.gstatic.com; form-action
> 'self'; frame-ancestors 'none'; frame-src 'none'; img-src 'self'
> https://warehouse-camo.cmh1.psfhosted.org/ www.google-analytics.com
> *.fastly-insights.com; script-src 'self' www.googletagmanager.com
> www.google-analytics.com *.fastly-insights.com
> https://cdn.ravenjs.com; style-src 'self' fonts.googleapis.com; worker-src
*.fastly-insights.com'
> received 'Content-Type: text/plain; charset=UTF-8'
> received 'Location:
> https://files.pythonhosted.org/packages/source/s/six/six-1.11.0.tar.gz'
> Redirected to
> https://files.pythonhosted.org/packages/source/s/six/six-1.11.0.tar.gz
> host files.pythonhosted.org, port https, path
> packages/source/s/six/six-1.11.0.tar.gz, save as six-1.11.0.tar.gz, auth
none.

> Trying 151.101.41.63...
> Requesting
> https://files.pythonhosted.org/packages/source/s/six/six-1.11.0.tar.gz
> GET /packages/source/s/six/six-1.11.0.tar.gz HTTP/1.0
> Host: files.pythonhosted.org
> User-Agent: OpenBSD ftp
>
> received 'HTTP/1.1 302 Found'
> received 'Cache-Control: max-age=604800, public'
> received 'Content-Type: application/octet-stream'
> received 'Location:
>
https://files.pythonhosted.org/packages/16/d8/bc6316cf98419719bd59c91742194c
111b6f2e85abac88e496adefaf7afe/six-1.11.0.tar.gz'

> Redirected to
> https://files.pythonhosted.org/packages/16/d8/bc6316cf98419719bd59c917
> 42194c111b6f2e85abac88e496adefaf7afe/six-1.11.0.tar.gz
> host files.pythonhosted.org, port https, path
> packages/16/d8/bc6316cf98419719bd59c91742194c111b6f2e85abac88e496adefa
> f7afe/six-1.11.0.tar.gz,
> save as six-1.11.0.tar.gz, auth none.
> Trying 151.101.41.63...
> Requesting
> https://files.pythonhosted.org/packages/16/d8/bc6316cf98419719bd59c917
> 42194c111b6f2e85abac88e496adefaf7afe/six-1.11.0.tar.gz
> GET
> /packages/16/d8/bc6316cf98419719bd59c91742194c111b6f2e85abac88e496adef
> af7afe/six-1.11.0.tar.gz
> HTTP/1.0
> Host: files.pythonhosted.org
> User-Agent: OpenBSD ftp
>
> received 'HTTP/1.1 200 OK'
> received 'x-amz-id-2:
>
ZPG4LCvWjZhEUNqY9PvtfV2e2YaS3x2TDj/kcEDliRXzdWXLkp8nYE68NEGm0yD2GIomC5Ns1hw=
'

> received 'x-amz-request-id: 27A0CF68EA8E91AB'
> received 'Last-Modified: Sun, 17 Sep 2017 18:46:56 GMT'
> received 'ETag: "d12789f9baf7e9fb2524c0c64f1773f8"'
> received 'x-amz-version-id: RwRLQ60RynDAt7f8Xqbv.StV0y_SRxXJ'
> received 'Content-Type: binary/octet-stream'
> received 'Server: AmazonS3'
> received 'Cache-Control: max-age=365000000, immutable'
> received 'Content-Length: 29860'
> received 'Accept-Ranges: bytes'
> received 'Date: Wed, 15 Aug 2018 18:38:52 GMT'
> received 'Age: 4781773'
> received 'Connection: close'
> received 'X-Served-By: cache-sea1041-SEA, cache-sjc3122-SJC'
> received 'X-Cache: HIT, HIT'
> received 'X-Cache-Hits: 1, 4'
> received 'X-Timer: S1534358332.351879,VS0,VE0'
> received 'Strict-Transport-Security: max-age=31536000;
> includeSubDomains; preload'
> received 'X-Frame-Options: deny'
> received 'X-XSS-Protection: 1; mode=block'
> received 'X-Content-Type-Options: nosniff'
> received 'X-Permitted-Cross-Domain-Policies: none'
> received 'X-Robots-Header: noindex'
> 100%
> |*********************************************************************
> |*****|
> 29860       00:00
> 29860 bytes received in 0.04 seconds (784.32 KB/s)
>
> From arm system:
>
> op1bsdtest2# ftp -d
> https://pypi.io/packages/source/s/six/six-1.11.0.tar.gz
> host pypi.io, port https, path
> packages/source/s/six/six-1.11.0.tar.gz,
> save as six-1.11.0.tar.gz, auth none.
> Trying 151.101.0.223...
> Requesting https://pypi.io/packages/source/s/six/six-1.11.0.tar.gz
> ftp: SSL write error: handshake failed: Operation timed out
> op1bsdtest2#
>
>
>
> -----Original Message-----
> From: [hidden email] <[hidden email]> On Behalf Of
> Stuart Henderson
> Sent: August 15, 2018 1:37 AM
> To: [hidden email]
> Cc: [hidden email]
> Subject: SSL connection failure with ftp but not wget [was Re: python
> files moved]
>
> On 2018/08/14 17:41, [hidden email] wrote:
>> The current setup failed on the last three builds I have done.
>>
>> Wget seems to understand redirection. Note one line from wget output
>> seems to imply that the site has moved permanently.
>>
>> Connecting to pypi.org (pypi.org)|151.101.0.223|:443... connected.
>> HTTP request sent, awaiting response... 301 Moved Permanently
>> Location:
>> https://files.pythonhosted.org/packages/source/s/six/six-1.11.0.tar.g
>> z
>> [following]
>> --2018-08-14 15:57:26--
>> https://files.pythonhosted.org/packages/source/s/six/six-1.11.0.tar.g
>> z
>>
>> ftp -d failed after a long time.
>>
>> op1bsdtest2# ftp -d
>> https://pypi.io/packages/source/s/six/six-1.11.0.tar.gz
>> host pypi.io, port https, path
>> packages/source/s/six/six-1.11.0.tar.gz, save as six-1.11.0.tar.gz, auth
none.

>> Trying 151.101.0.223...
>> Requesting https://pypi.io/packages/source/s/six/six-1.11.0.tar.gz
>> ftp: SSL write error: handshake failed: Operation timed out
>
> The redirection thing is a red herring. ftp and wget both understand
> it and it should happen quickly. As you aren't able to successfully
> connect to https://pypi.io/ with ftp it doesn't even see the
> redirection, just eventually times out and falls back to ftp.openbsd.org.
>
> I'm not sure why wget can connect but ftp can't - I don't think either
> are doing anything particularly unusual with the TLS connection and
> both use libressl for this..
>
> Can you try curl -v -o /dev/null
> https://pypi.io/packages/source/s/six/six-1.11.0.tar.gz ?
> Does that succeed or fail, and can you paste the output? (it has
> better TLS debug than ftp or wget).
>
> Is this on armv7 again? If so are you able to test on a machine of
> another arch on the same network?
>
> Do you have problems connecting to anything else hosted on fastly?
>
> Does the problem go away if you reduce MTU on the network interface?
> ("ifconfig em0 mtu 1200" or something?)
>
>
>
>> -----Original Message-----
>> From: Stuart Henderson <[hidden email]>
>> Sent: August 14, 2018 4:23 PM
>> To: [hidden email]
>> Cc: [hidden email]
>> Subject: Re: python files moved
>>
>> On 2018/08/14 16:03, [hidden email] wrote:
>> > When building php I get the following error which causes a multi
>> > minute timeout.  From a wget request it looks like the web site has
>> > moved.  This happens on many files and causes quite a slowdown in
>> > building
>> a port.
>>
>> If we point too far into the redirection chain for pypi we're more
>> likely to have failures next time they change things, the pypi.io
>> ones seems a more stable endpoint.
>>
>> > ===>  Checking files for py-six-1.11.0
>> >
>> > >> Fetch https://pypi.io/packages/source/s/six/six-1.11.0.tar.gz
>> >
>> > ftp: SSL write error: handshake failed: Operation timed out
>> >
>> > >> Fetch
>> > >> https://ftp.openbsd.org/pub/OpenBSD/distfiles/six-1.11.0.tar.gz
>> >
>> > six-1.11.0.tar.gz 100%
>> > |********************************************************| 29860
>> 00:00
>> >
>> >
>> >
>> > With wget:
>> >
>> >
>> >
>> > op1bsdtest2# wget
>> > https://pypi.io/packages/source/s/six/six-1.11.0.tar.gz
>> >
>> > --2018-08-14 15:57:26--
>> > https://pypi.io/packages/source/s/six/six-1.11.0.tar.gz
>> >
>> > Resolving pypi.io (pypi.io)... 151.101.0.223, 151.101.64.223,
>> > 151.101.128.223, ...
>> >
>> > Connecting to pypi.io (pypi.io)|151.101.0.223|:443... connected.
>>
>> It's rather odd that ftp(1) times out and wget succeeds. Does ftp -d
>> throw any light on it?
>>
>>





----- End forwarded message -----

Reply | Threaded
Open this post in threaded view
|

Re: l2k18 ahoy [s_graf@telus.net: FW: SSL connection failure with ftp but not wget [...]

Mark Kettenis
> Date: Mon, 5 Nov 2018 20:56:33 +0000
> From: Stuart Henderson <[hidden email]>
>
> Probably worth pinging this one since l2k18 is on.
>
> s_graf is trying to fetch ports sources from https://pypi.org/ and is
> getting a hang and eventually a timeout when attempting connection from
> ftp(1) on armv7. (From recent ports@ posts it seems like this still occurs).
>
> curl/wget were working ok when tested before.
>
> Can anyone with armv7 confirm/deny that they can replicate this? (just try
> "ftp https://pypi.io/packages/source/s/six/six-1.11.0.tar.gz").
>
> Any ideas?

Works for me with an install built from source last update Oct 29th or so.

> > ftp: SSL write error: handshake failed: Operation timed out
>
> ----- Forwarded message from [hidden email] -----
>
> From: [hidden email]
> Date: Wed, 15 Aug 2018 15:02:40 -0700
> To: [hidden email]
> Cc: 'Stuart Henderson' <[hidden email]>
> X-Mailer: Microsoft Outlook 16.0
> Subject: FW: SSL connection failure with ftp but not wget [was Re: python files moved]  -- compare with another
> system
>
> Trying to get a file during a php build fails on arm but not on i386
> systems.
> Stuart Henderson suggested I forward this.
>
>
>  From the arm system (op1bsdtest2)
>
> op1bsdtest2# curl -v https://pypi.io/packages/source/s/six/six-1.11.0.tar.gz
> *   Trying 151.101.0.223...
> * TCP_NODELAY set
> * Connected to pypi.io (151.101.0.223) port 443 (#0)
> * ALPN, offering h2
> * ALPN, offering http/1.1
> * successfully set certificate verify locations:
> *   CAfile: /etc/ssl/cert.pem
>   CApath: none
> * TLSv1.2 (OUT), TLS handshake, Client hello (1):
> * TLSv1.2 (IN), TLS handshake, Server hello (2):
> * TLSv1.2 (IN), TLS handshake, Certificate (11):
> * TLSv1.2 (IN), TLS handshake, Server key exchange (12):
> * TLSv1.2 (IN), TLS handshake, Server finished (14):
> * TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
> * TLSv1.2 (OUT), TLS change cipher, Client hello (1):
> * TLSv1.2 (OUT), TLS handshake, Finished (20):
> * TLSv1.2 (IN), TLS change cipher, Client hello (1):
> * TLSv1.2 (IN), TLS handshake, Finished (20):
> * SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
> * ALPN, server accepted to use h2
> * Server certificate:
> *  subject: businessCategory=Private Organization;
> jurisdictionCountryName=US; jurisdictionStateOrProvinceName=Delaware;
> serialNumber=3359300; C=US; ST=New Hampshire; L=Wolfeboro; O=Python Software
> Foundation; CN=www.python.org
> *  start date: Mar 28 00:00:00 2018 GMT
> *  expire date: Sep 27 12:00:00 2018 GMT
> *  subjectAltName: host "pypi.io" matched cert's "pypi.io"
> *  issuer: C=US; O=DigiCert Inc; OU=www.digicert.com; CN=DigiCert SHA2
> Extended Validation Server CA
> *  SSL certificate verify ok.
> * Using HTTP2, server supports multi-use
> * Connection state changed (HTTP/2 confirmed)
> * Copying HTTP/2 data in stream buffer to connection buffer after upgrade:
> len=0
> * Using Stream ID: 1 (easy handle 0x8063b000)
> > GET /packages/source/s/six/six-1.11.0.tar.gz HTTP/2
> > Host: pypi.io
> > User-Agent: curl/7.61.0
> > Accept: */*
> >
> * Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
> < HTTP/2 301
> < server: Varnish
> < retry-after: 0
> < location: https://pypi.org/packages/source/s/six/six-1.11.0.tar.gz
> < content-type: text/html; charset=UTF-8
> < accept-ranges: bytes
> < date: Wed, 15 Aug 2018 21:55:38 GMT
> < x-served-by: cache-sea1033-SEA
> < x-cache: HIT
> < x-cache-hits: 0
> < x-timer: S1534370139.898309,VS0,VE0
> < strict-transport-security: max-age=31536000; includeSubDomains; preload
> < x-frame-options: deny
> < x-xss-protection: 1; mode=block
> < x-content-type-options: nosniff
> < x-permitted-cross-domain-policies: none
> < content-length: 122
> <
> * Connection #0 to host pypi.io left intact
> <html><head><title>301 Moved Permanently</title></head><body><center><h1>301
> Moved Permanently</h1></center></body></html>op1bsdtest2#
>
>
>
> op1bsdtest2# dmesg
> OpenBSD 6.3-current (GENERIC) #32: Fri Aug 10 10:32:37 MDT 2018
>     [hidden email]:/usr/src/sys/arch/armv7/compile/GENERIC
> real mem  = 536870912 (512MB)
> avail mem = 516112384 (492MB)
> mainbus0 at root: Xunlong Orange Pi One
> cpu0 at mainbus0: ARM Cortex-A7 r0p5 (ARMv7)
> cpu0: DC enabled IC enabled WB disabled EABT branch prediction enabled
> cpu0: 32KB(32b/l,2way) I-cache, 32KB(64b/l,4way) wr-back D-cache
> cortex0 at mainbus0
> psci0 at mainbus0: PSCI 0.0
> sxiccmu0 at mainbus0
> simplebus0 at mainbus0: "soc"
> syscon0 at simplebus0: "syscon"
> sxiccmu1 at simplebus0
> sxipio0 at simplebus0: 94 pins
> ampintc0 at simplebus0 nirq 160, ncpu 4: "interrupt-controller"
> sxiccmu2 at simplebus0
> sxipio1 at simplebus0: 12 pins
> sximmc0 at simplebus0
> sdmmc0 at sximmc0: 4-bit, sd high-speed, mmc high-speed, dma
> ehci0 at simplebus0
> usb0 at ehci0: USB revision 2.0
> uhub0 at usb0 configuration 1 interface 0 "Generic EHCI root hub" rev
> 2.00/1.00 addr 1
> ehci1 at simplebus0
> usb1 at ehci1: USB revision 2.0
> uhub1 at usb1 configuration 1 interface 0 "Generic EHCI root hub" rev
> 2.00/1.00 addr 1
> sxitemp0 at simplebus0
> dwxe0 at simplebus0: address 02:81:b1:07:76:5e
> ukphy0 at dwxe0 phy 1: Generic IEEE 802.3u media interface, rev. 0: OUI
> 0x001105, model 0x0000
> sxidog0 at simplebus0
> com0 at simplebus0: ns16550, no working fifo
> com0: console
> sxitwi0 at simplebus0
> iic0 at sxitwi0
> "bosch,bme280" at iic0 addr 0x76 not configured
> sxitwi1 at simplebus0
> iic1 at sxitwi1
> "bosch,bme280" at iic1 addr 0x77 not configured
> sxirtc0 at simplebus0
> gpio0 at sxipio0: 32 pins
> gpio1 at sxipio0: 32 pins
> gpio2 at sxipio0: 32 pins
> gpio3 at sxipio0: 32 pins
> gpio4 at sxipio0: 32 pins
> gpio5 at sxipio0: 32 pins
> gpio6 at sxipio0: 32 pins
> gpio7 at sxipio1: 32 pins
> agtimer0 at mainbus0: tick rate 24000 KHz
> scsibus0 at sdmmc0: 2 targets, initiator 0
> sd0 at scsibus0 targ 1 lun 0: <SD/MMC, SL16G, 0080> SCSI2 0/direct removable
> sd0: 15193MB, 512 bytes/sector, 31116288 sectors
> vscsi0 at root
> scsibus1 at vscsi0: 256 targets
> softraid0 at root
> scsibus2 at softraid0: 256 targets
> bootfile: sd0a:/bsd
> boot device: sd0
> root on sd0a (88106578f2222800.a) swap on sd0b dump on sd0b
> op1bsdtest2#
>
> -----Original Message-----
> From: Stuart Henderson <[hidden email]>
> Sent: August 15, 2018 1:47 PM
> To: [hidden email]
> Subject: RE: SSL connection failure with ftp but not wget [was Re: python
> files moved] -- compare with another system
>
> Very interesting! Could you forward to bugs@ so people who might have a
> better idea what's wrong will see it please?
>
> --
> Sent from a phone, apologies for poor formatting.
>
> On 15 August 2018 20:01:10 <[hidden email]> wrote:
>
> > It looks like the problem is specific to the arm system. I ran the ftp
> > -d on both systems one after the other.  Both are on the same network.
> > The arm system is a recent snapshot  base install with src and ports
> > loaded and really nothing else.
> > I have not seen any other connection problems on the arm system and it
> > is doing many as part of the php build.
> >
> > I will try some of the network reconfigs when the build of php finishes.
> >
> > From my 6.2 stable server:
> >
> > # ftp -d https://pypi.io/packages/source/s/six/six-1.11.0.tar.gz
> > host pypi.io, port https, path
> > packages/source/s/six/six-1.11.0.tar.gz,
> > save as six-1.11.0.tar.gz, auth none.
> > Trying 151.101.0.223...
> > Requesting https://pypi.io/packages/source/s/six/six-1.11.0.tar.gz
> > GET /packages/source/s/six/six-1.11.0.tar.gz HTTP/1.0
> > Host: pypi.io
> > User-Agent: OpenBSD ftp
> >
> > received 'HTTP/1.1 301 Redirect to Primary Domain'
> > received 'Server: Varnish'
> > received 'Retry-After: 0'
> > received 'Location:
> https://pypi.org/packages/source/s/six/six-1.11.0.tar.gz'
> > Redirected to https://pypi.org/packages/source/s/six/six-1.11.0.tar.gz
> > host pypi.org, port https, path
> > packages/source/s/six/six-1.11.0.tar.gz,
> > save as six-1.11.0.tar.gz, auth none.
> > Trying 151.101.0.223...
> > Requesting https://pypi.org/packages/source/s/six/six-1.11.0.tar.gz
> > GET /packages/source/s/six/six-1.11.0.tar.gz HTTP/1.0
> > Host: pypi.org
> > User-Agent: OpenBSD ftp
> >
> > received 'HTTP/1.1 301 Moved Permanently'
> > received 'Content-Security-Policy: base-uri 'self';
> > block-all-mixed-content; connect-src 'self'
> > https://api.github.com/repos/ *.fastly-insights.com sentry.io
> > https://2p66nmmycsj3.statuspage.io;
> > default-src 'none'; font-src 'self' fonts.gstatic.com; form-action
> > 'self'; frame-ancestors 'none'; frame-src 'none'; img-src 'self'
> > https://warehouse-camo.cmh1.psfhosted.org/ www.google-analytics.com
> > *.fastly-insights.com; script-src 'self' www.googletagmanager.com
> > www.google-analytics.com *.fastly-insights.com
> > https://cdn.ravenjs.com; style-src 'self' fonts.googleapis.com; worker-src
> *.fastly-insights.com'
> > received 'Content-Type: text/plain; charset=UTF-8'
> > received 'Location:
> > https://files.pythonhosted.org/packages/source/s/six/six-1.11.0.tar.gz'
> > Redirected to
> > https://files.pythonhosted.org/packages/source/s/six/six-1.11.0.tar.gz
> > host files.pythonhosted.org, port https, path
> > packages/source/s/six/six-1.11.0.tar.gz, save as six-1.11.0.tar.gz, auth
> none.
> > Trying 151.101.41.63...
> > Requesting
> > https://files.pythonhosted.org/packages/source/s/six/six-1.11.0.tar.gz
> > GET /packages/source/s/six/six-1.11.0.tar.gz HTTP/1.0
> > Host: files.pythonhosted.org
> > User-Agent: OpenBSD ftp
> >
> > received 'HTTP/1.1 302 Found'
> > received 'Cache-Control: max-age=604800, public'
> > received 'Content-Type: application/octet-stream'
> > received 'Location:
> >
> https://files.pythonhosted.org/packages/16/d8/bc6316cf98419719bd59c91742194c
> 111b6f2e85abac88e496adefaf7afe/six-1.11.0.tar.gz'
> > Redirected to
> > https://files.pythonhosted.org/packages/16/d8/bc6316cf98419719bd59c917
> > 42194c111b6f2e85abac88e496adefaf7afe/six-1.11.0.tar.gz
> > host files.pythonhosted.org, port https, path
> > packages/16/d8/bc6316cf98419719bd59c91742194c111b6f2e85abac88e496adefa
> > f7afe/six-1.11.0.tar.gz,
> > save as six-1.11.0.tar.gz, auth none.
> > Trying 151.101.41.63...
> > Requesting
> > https://files.pythonhosted.org/packages/16/d8/bc6316cf98419719bd59c917
> > 42194c111b6f2e85abac88e496adefaf7afe/six-1.11.0.tar.gz
> > GET
> > /packages/16/d8/bc6316cf98419719bd59c91742194c111b6f2e85abac88e496adef
> > af7afe/six-1.11.0.tar.gz
> > HTTP/1.0
> > Host: files.pythonhosted.org
> > User-Agent: OpenBSD ftp
> >
> > received 'HTTP/1.1 200 OK'
> > received 'x-amz-id-2:
> >
> ZPG4LCvWjZhEUNqY9PvtfV2e2YaS3x2TDj/kcEDliRXzdWXLkp8nYE68NEGm0yD2GIomC5Ns1hw=
> '
> > received 'x-amz-request-id: 27A0CF68EA8E91AB'
> > received 'Last-Modified: Sun, 17 Sep 2017 18:46:56 GMT'
> > received 'ETag: "d12789f9baf7e9fb2524c0c64f1773f8"'
> > received 'x-amz-version-id: RwRLQ60RynDAt7f8Xqbv.StV0y_SRxXJ'
> > received 'Content-Type: binary/octet-stream'
> > received 'Server: AmazonS3'
> > received 'Cache-Control: max-age=365000000, immutable'
> > received 'Content-Length: 29860'
> > received 'Accept-Ranges: bytes'
> > received 'Date: Wed, 15 Aug 2018 18:38:52 GMT'
> > received 'Age: 4781773'
> > received 'Connection: close'
> > received 'X-Served-By: cache-sea1041-SEA, cache-sjc3122-SJC'
> > received 'X-Cache: HIT, HIT'
> > received 'X-Cache-Hits: 1, 4'
> > received 'X-Timer: S1534358332.351879,VS0,VE0'
> > received 'Strict-Transport-Security: max-age=31536000;
> > includeSubDomains; preload'
> > received 'X-Frame-Options: deny'
> > received 'X-XSS-Protection: 1; mode=block'
> > received 'X-Content-Type-Options: nosniff'
> > received 'X-Permitted-Cross-Domain-Policies: none'
> > received 'X-Robots-Header: noindex'
> > 100%
> > |*********************************************************************
> > |*****|
> > 29860       00:00
> > 29860 bytes received in 0.04 seconds (784.32 KB/s)
> >
> > From arm system:
> >
> > op1bsdtest2# ftp -d
> > https://pypi.io/packages/source/s/six/six-1.11.0.tar.gz
> > host pypi.io, port https, path
> > packages/source/s/six/six-1.11.0.tar.gz,
> > save as six-1.11.0.tar.gz, auth none.
> > Trying 151.101.0.223...
> > Requesting https://pypi.io/packages/source/s/six/six-1.11.0.tar.gz
> > ftp: SSL write error: handshake failed: Operation timed out
> > op1bsdtest2#
> >
> >
> >
> > -----Original Message-----
> > From: [hidden email] <[hidden email]> On Behalf Of
> > Stuart Henderson
> > Sent: August 15, 2018 1:37 AM
> > To: [hidden email]
> > Cc: [hidden email]
> > Subject: SSL connection failure with ftp but not wget [was Re: python
> > files moved]
> >
> > On 2018/08/14 17:41, [hidden email] wrote:
> >> The current setup failed on the last three builds I have done.
> >>
> >> Wget seems to understand redirection. Note one line from wget output
> >> seems to imply that the site has moved permanently.
> >>
> >> Connecting to pypi.org (pypi.org)|151.101.0.223|:443... connected.
> >> HTTP request sent, awaiting response... 301 Moved Permanently
> >> Location:
> >> https://files.pythonhosted.org/packages/source/s/six/six-1.11.0.tar.g
> >> z
> >> [following]
> >> --2018-08-14 15:57:26--
> >> https://files.pythonhosted.org/packages/source/s/six/six-1.11.0.tar.g
> >> z
> >>
> >> ftp -d failed after a long time.
> >>
> >> op1bsdtest2# ftp -d
> >> https://pypi.io/packages/source/s/six/six-1.11.0.tar.gz
> >> host pypi.io, port https, path
> >> packages/source/s/six/six-1.11.0.tar.gz, save as six-1.11.0.tar.gz, auth
> none.
> >> Trying 151.101.0.223...
> >> Requesting https://pypi.io/packages/source/s/six/six-1.11.0.tar.gz
> >> ftp: SSL write error: handshake failed: Operation timed out
> >
> > The redirection thing is a red herring. ftp and wget both understand
> > it and it should happen quickly. As you aren't able to successfully
> > connect to https://pypi.io/ with ftp it doesn't even see the
> > redirection, just eventually times out and falls back to ftp.openbsd.org.
> >
> > I'm not sure why wget can connect but ftp can't - I don't think either
> > are doing anything particularly unusual with the TLS connection and
> > both use libressl for this..
> >
> > Can you try curl -v -o /dev/null
> > https://pypi.io/packages/source/s/six/six-1.11.0.tar.gz ?
> > Does that succeed or fail, and can you paste the output? (it has
> > better TLS debug than ftp or wget).
> >
> > Is this on armv7 again? If so are you able to test on a machine of
> > another arch on the same network?
> >
> > Do you have problems connecting to anything else hosted on fastly?
> >
> > Does the problem go away if you reduce MTU on the network interface?
> > ("ifconfig em0 mtu 1200" or something?)
> >
> >
> >
> >> -----Original Message-----
> >> From: Stuart Henderson <[hidden email]>
> >> Sent: August 14, 2018 4:23 PM
> >> To: [hidden email]
> >> Cc: [hidden email]
> >> Subject: Re: python files moved
> >>
> >> On 2018/08/14 16:03, [hidden email] wrote:
> >> > When building php I get the following error which causes a multi
> >> > minute timeout.  From a wget request it looks like the web site has
> >> > moved.  This happens on many files and causes quite a slowdown in
> >> > building
> >> a port.
> >>
> >> If we point too far into the redirection chain for pypi we're more
> >> likely to have failures next time they change things, the pypi.io
> >> ones seems a more stable endpoint.
> >>
> >> > ===>  Checking files for py-six-1.11.0
> >> >
> >> > >> Fetch https://pypi.io/packages/source/s/six/six-1.11.0.tar.gz
> >> >
> >> > ftp: SSL write error: handshake failed: Operation timed out
> >> >
> >> > >> Fetch
> >> > >> https://ftp.openbsd.org/pub/OpenBSD/distfiles/six-1.11.0.tar.gz
> >> >
> >> > six-1.11.0.tar.gz 100%
> >> > |********************************************************| 29860
> >> 00:00
> >> >
> >> >
> >> >
> >> > With wget:
> >> >
> >> >
> >> >
> >> > op1bsdtest2# wget
> >> > https://pypi.io/packages/source/s/six/six-1.11.0.tar.gz
> >> >
> >> > --2018-08-14 15:57:26--
> >> > https://pypi.io/packages/source/s/six/six-1.11.0.tar.gz
> >> >
> >> > Resolving pypi.io (pypi.io)... 151.101.0.223, 151.101.64.223,
> >> > 151.101.128.223, ...
> >> >
> >> > Connecting to pypi.io (pypi.io)|151.101.0.223|:443... connected.
> >>
> >> It's rather odd that ftp(1) times out and wget succeeds. Does ftp -d
> >> throw any light on it?
> >>
> >>
>
>
>
>
>
> ----- End forwarded message -----
>
>

Reply | Threaded
Open this post in threaded view
|

Re: l2k18 ahoy [s_graf@telus.net: FW: SSL connection failure with ftp but not wget [...]

Matthieu Herrb-7
On Mon, Nov 05, 2018 at 10:03:31PM +0100, Mark Kettenis wrote:

> > Date: Mon, 5 Nov 2018 20:56:33 +0000
> > From: Stuart Henderson <[hidden email]>
> >
> > Probably worth pinging this one since l2k18 is on.
> >
> > s_graf is trying to fetch ports sources from https://pypi.org/ and is
> > getting a hang and eventually a timeout when attempting connection from
> > ftp(1) on armv7. (From recent ports@ posts it seems like this still occurs).
> >
> > curl/wget were working ok when tested before.
> >
> > Can anyone with armv7 confirm/deny that they can replicate this? (just try
> > "ftp https://pypi.io/packages/source/s/six/six-1.11.0.tar.gz").
> >
> > Any ideas?
>
> Works for me with an install built from source last update Oct 29th
> or so.

Works for me too on a -current built here this weekend.
Note that there is IPv6 involved. So IPv6 routing issues (IPMPv6
filtering causing MTU negociation failures) can interfere...

sabre% ftp https://pypi.iœpackages/source/s/six/six-1.11.0.tar.gz
Trying 2a04:4e42:400::223...
Requesting https://pypi.iœpackages/source/s/six/six-1.11.0.tar.gz
Redirected to https://pypi.org/packages/source/s/six/six-1.11.0.tar.gz
Trying 2a04:4e42:400::223...
Requesting https://pypi.org/packages/source/s/six/six-1.11.0.tar.gz
Redirected to
https://files.pythonhosted.org/packages/source/s/six/six-1.11.0.tar.gz
Trying 2a04:4e42:1d::319...
Requesting
https://files.pythonhosted.org/packages/source/s/six/six-1.11.0.tar.gz
Redirected to
https://files.pythonhosted.org/packages/16/d8/bc6316cf98419719bd59c91742194c111b6f2e85abac88e496adefaf7afe/six-1.11.0.tar.gz
Trying 2a04:4e42:1d::319...
Requesting
https://files.pythonhosted.org/packages/16/d8/bc6316cf98419719bd59c91742194c111b6f2e85abac88e496adefaf7afe/six-1.11.0.tar.gz
100%
|*****************************************************************************************|
29860       00:00
29860 bytes received in 0.07 seconds (412.27 KB/s)

--
Matthieu Herrb