TLS suddenly not working over IKED site-to-site

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

TLS suddenly not working over IKED site-to-site

Rachel Roch
I hope someone here can shed light on an infuriating problem I’ve spent a week trying to resolve without luck.

The problem concerns an IKED site-to-site VPN on OpenBSD 6.3 (both endpoints fully syspatched).

The VPN worked absolutely perfectly until it suddenly started behaving strangely.  Seriously, I’m talking about “pass any traffic you can think of”, then I go on holiday for a week (nobody else has physical or remote access to the machines, and I did not connect on holiday), then this behaviour starts.

Basically the behaviour I am seeing is that anything that uses TLS is no longer able to connect (or at least gets no further than trying to do a TLS handshake, e.g. Firefox hangs showing "performing TLS handshake..." at the bottom of the screen), so that means:

- HTTPS websites
- VoIP
- IMAP over TLS
- RDP over TLS

Are all broken on the VPN, but all TLS-based services continue to work perfectly off-site (or when the site-to-site VPN is bypassed with a third-party VPN).  This impacts multiple servers and multiple clients, so its not just one server or one desktop PC, its anything that tries to talk TLS over that VPN !


However:
- Ping (including large packet size, e.g. “-s 1600”)
- SSH
- DNS
- Anything else you care to name that doesn’t use TLS

All continue to work perfectly over the VPN.

My PF rules (which cannot possibly be the problem, because they have not changed a single bit between “working” and “not working) don’t even differentiate between traffic types, so it can’t be some sudden PF oddity :

pass in on enc from <remote_vpnets> to <local_vpnets> keep state (if-bound) $midPriority
pass out on enc from <ocal_vpnets> to <remote_vpnets> keep state (if-bound) $midPriority

Similarly, my IKED config is also completely unchanged between "working" and "not working", and ipsecctl -sa continues to show everything correctly established

ikev2 "to remote" active esp from $a_net to $b_net\
        local $local_ext peer $remote_ext \
        ikesa auth hmac-sha2-512 enc aes-256 prf hmac-sha2-512 group curve25519 \
        childsa enc chacha20-poly1305 group curve25519 \
        srcid $local_ext dstid $remote_ext \
        ikelifetime 4h lifetime 3h bytes 512M \
        ecdsa384


This whole thing is just driving me crazy !

Reply | Threaded
Open this post in threaded view
|

Re: TLS suddenly not working over IKED site-to-site

Theodore Wynnychenko-2


> -----Original Message-----
> From: [hidden email] [mailto:[hidden email]] On Behalf
> Of Rachel Roch
> Sent: Monday, December 03, 2018 11:19 AM
> To: [hidden email]
> Subject: TLS suddenly not working over IKED site-to-site
>
> I hope someone here can shed light on an infuriating problem I’ve spent
> a week trying to resolve without luck.
>
> The problem concerns an IKED site-to-site VPN on OpenBSD 6.3 (both
> endpoints fully syspatched).
.
.
.
>
>
> This whole thing is just driving me crazy !


Hello,
This appears to be the same thing I have been having issues with and mentioned in a post to misc last week ("Untable ssl connections over ikev2 VPN") - (yes, typo intact - it should be "unstable").

I have tried adding a "max-mss 1300" directive into pf.conf (i.e.: "match in all scrub (no-df random-id max-mss 1300)").

At first, I _thought_ this made a difference, but I am not sure if that is really true.

I have also noticed that the TLS failures seem to vary based on OS.  At this point, I was able to get an https connection to work with firefox on MacOS, but the TLS handshake continues to hang (100% of the time) with firefox on a Windows 7 PC.  With an openBSD laptop, it seems like it sometimes works and sometimes doesn't (using "openssl s_client" to test).

I also made no changes in pf.conf or iked.conf from the working to non-working period.

I have no idea what to do; I am just posting my observations if that helps.
Thanks



Reply | Threaded
Open this post in threaded view
|

Re: TLS suddenly not working over IKED site-to-site

Paul Suh-2
In reply to this post by Rachel Roch


> On Dec 3, 2018, at 12:18 PM, Rachel Roch <[hidden email]> wrote:
>
> I hope someone here can shed light on an infuriating problem I’ve spent a week trying to resolve without luck.
>
> The problem concerns an IKED site-to-site VPN on OpenBSD 6.3 (both endpoints fully syspatched).
>
> The VPN worked absolutely perfectly until it suddenly started behaving strangely.  Seriously, I’m talking about “pass any traffic you can think of”, then I go on holiday for a week (nobody else has physical or remote access to the machines, and I did not connect on holiday), then this behaviour starts.
>
> Basically the behaviour I am seeing is that anything that uses TLS is no longer able to connect (or at least gets no further than trying to do a TLS handshake, e.g. Firefox hangs showing "performing TLS handshake..." at the bottom of the screen), so that means:
>
> - HTTPS websites
> - VoIP
> - IMAP over TLS
> - RDP over TLS
>
> Are all broken on the VPN, but all TLS-based services continue to work perfectly off-site (or when the site-to-site VPN is bypassed with a third-party VPN).  This impacts multiple servers and multiple clients, so its not just one server or one desktop PC, its anything that tries to talk TLS over that VPN !
>
>
> However:
> - Ping (including large packet size, e.g. “-s 1600”)
> - SSH
> - DNS
> - Anything else you care to name that doesn’t use TLS
>
> All continue to work perfectly over the VPN.
>
> My PF rules (which cannot possibly be the problem, because they have not changed a single bit between “working” and “not working) don’t even differentiate between traffic types, so it can’t be some sudden PF oddity :
>
> pass in on enc from <remote_vpnets> to <local_vpnets> keep state (if-bound) $midPriority
> pass out on enc from <ocal_vpnets> to <remote_vpnets> keep state (if-bound) $midPriority
>
> Similarly, my IKED config is also completely unchanged between "working" and "not working", and ipsecctl -sa continues to show everything correctly established
>
> ikev2 "to remote" active esp from $a_net to $b_net\
>         local $local_ext peer $remote_ext \
>         ikesa auth hmac-sha2-512 enc aes-256 prf hmac-sha2-512 group curve25519 \
>         childsa enc chacha20-poly1305 group curve25519 \
>         srcid $local_ext dstid $remote_ext \
>         ikelifetime 4h lifetime 3h bytes 512M \
>         ecdsa384
>
>
> This whole thing is just driving me crazy !
>
Rachel,

As a first step, try using s_client to connect to a TLS service and see what comes back:

$ openssl s_client -connect <hostname>:<port> -showcerts

There are more possible options on s_client to debug more deeply but this is a good start.


--Paul



smime.p7s (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: TLS suddenly not working over IKED site-to-site

Rachel Roch
In reply to this post by Theodore Wynnychenko-2


>
> Hello,
> This appears to be the same thing I have been having issues with and mentioned in a post to misc last week ("Untable ssl connections over ikev2 VPN") - (yes, typo intact - it should be "unstable").
>
> I have tried adding a "max-mss 1300" directive into pf.conf (i.e.: "match in all scrub (no-df random-id max-mss 1300)").
>
> At first, I _thought_ this made a difference, but I am not sure if that is really true.
>
> I have also noticed that the TLS failures seem to vary based on OS.  At this point, I was able to get an https connection to work with firefox on MacOS, but the TLS handshake continues to hang (100% of the time) with firefox on a Windows 7 PC.  With an openBSD laptop, it seems like it sometimes works and sometimes doesn't (using "openssl s_client" to test).
>
> I also made no changes in pf.conf or iked.conf from the working to non-working period.
>
> I have no idea what to do; I am just posting my observations if that helps.
> Thanks
>

Hi,

Glad its just not me !!! Even if you don't know the fix, at least I now know I haven't gone completely crazy !

For me it more consistent, on OSX its 100% hang, on Windows 10 its 100% hang.  Haven't tried OpenBSD client yet, I've been too busy putting emergency workarounds in place to bypass the site-to-site stuff. Will try OpenBSD client though when I get a chance.

Appreciate you taking the time to email ... keep in touch !



Reply | Threaded
Open this post in threaded view
|

Re: TLS suddenly not working over IKED site-to-site

Rachel Roch
In reply to this post by Paul Suh-2


> Rachel,
>
> As a first step, try using s_client to connect to a TLS service and see what comes back:
>
> $ openssl s_client -connect <hostname>:<port> -showcerts
>
> There are more possible options on s_client to debug more deeply but this is a good start.
>
>
> --Paul
>

In answer to the above. Testing against three "random" servers  (see bottom of the email for full exchange, top three are through VPN, rest are bypassing VPN):

Through the VPN:
- Server "A" (HTTPS with "real" cert)- Nothing more than "CONNECTED (00000005)"
- Server "B" (HTTPS with "self-signed" cert)- Certificates get displayed (this correlates with behaviour seen in browser where I get shown the "do you want to continue" prompt, I can see details of the certs presented, but when I click continue it hangs)
- Server "C" (IMAPS) - Nothing more than "CONNECTED (00000005)"

Bypassing the VPN:
- Server A shows certs in openssl(and browser works ok)- Server "C" shows certs in openssl (and email client works ok)

foobarOVERVPN $ openssl s_client -connect web1.example.com:443 -showcerts
CONNECTED(00000005)
^C
foobarOVERVPN $ openssl s_client -connect web2.example.com:8443 -showcerts
CONNECTED(00000005)
depth=0 C = US, ST = CA, L = San Jose, O = example.com, OU = MyCorp, CN = MyCorp
verify error:num=18:self signed certificate
verify return:1
depth=0 C = US, ST = CA, L = San Jose, O = example.com, OU = MyCorp, CN = MyCorp
verify return:1
---
Certificate chain
0 s:/C=ZZ/ST=AA/L=BB/O=example.com/OU=MyCorp/CN=MyCorp <http://example.com/OU=MyCorp/CN=MyCorp>
   i:/C=ZZ/ST=AA/L=BB/O=example.com/OU=MyCorp/CN=MyCorp <http://example.com/OU=MyCorp/CN=MyCorp>
-----BEGIN CERTIFICATE-----
<SNIP>
-----END CERTIFICATE-----
---
Server certificate
subject=/C=ZZ/ST=AA/L=BB/O=example.com/OU=MyCorp/CN=MyCorp <http://example.com/OU=MyCorp/CN=MyCorp>
issuer=/C=ZZ/ST=AA/L=BB/O=example.com/OU=MyCorp/CN=MyCorp <http://example.com/OU=MyCorp/CN=MyCorp>
---
No client certificate CA names sent
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 1316 bytes and written 326 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-GCM-SHA256
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES128-GCM-SHA256
    Session-ID: 5C0575730056006E28542F880C1AB6541729337C0DDBEC95347E2B5B4669EAD7
    Session-ID-ctx:
    Master-Key: 66B8EB1A3FB0857509627840D8DDB595659A5794D365D462DED737AAD4532F4AD542663B8BAE27A7665539D15C14ADEA
    Start Time: 1543861619
    Timeout   : 7200 (sec)
    Verify return code: 18 (self signed certificate)
---
^C

foobarOVERVPN $ openssl s_client -connect imaps.example.com:993 -showcerts
CONNECTED(00000005)
^C
foobarBYPASSVPN $ openssl s_client -connect web1.example.com:443 -showcerts
CONNECTED(00000005)
depth=3 C = SE, O = AddTrust AB, OU = AddTrust External TTP Network, CN = AddTrust External CA Root
verify return:1
depth=2 C = GB, ST = Greater Manchester, L = Salford, O = COMODO CA Limited, CN = COMODO RSA Certification Authority
verify return:1
depth=1 C = GB, ST = Greater Manchester, L = Salford, O = COMODO CA Limited, CN = COMODO RSA Domain Validation Secure Server CA
verify return:1
depth=0 OU = Domain Control Validated, OU = PositiveSSL, CN = web1.example.com
verify return:1
---
Certificate chain
0 s:/OU=Domain Control Validated/OU=PositiveSSL/CN=web1.example.com
   i:/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Domain Validation Secure Server CA
-----BEGIN CERTIFICATE-----
<SNIP>
-----END CERTIFICATE-----
1 s:/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Domain Validation Secure Server CA
   i:/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Certification Authority
-----BEGIN CERTIFICATE-----
<SNIP>
-----END CERTIFICATE-----
2 s:/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Certification Authority
   i:/C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root
-----BEGIN CERTIFICATE-----
<SNIP>
-----END CERTIFICATE-----
3 s:/C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root
   i:/C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root
-----BEGIN CERTIFICATE-----
<SNIP>
-----END CERTIFICATE-----
---
Server certificate
subject=/OU=Domain Control Validated/OU=PositiveSSL/CN=web1.example.com
issuer=/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Domain Validation Secure Server CA
---
No client certificate CA names sent
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 6299 bytes and written 326 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID: 6D5EC20FC1493D55F28309A02B1B589268F251D625A7EB3B5958426225C51795
    Session-ID-ctx:
    Master-Key: D8B2A8181AB5FB4BC6A55ED226CFA9D0F77CF539CE3E4A9FAE6524D631B42BB057375E96BD4EB6014C02996BD6A645C4
    Start Time: 1543861687
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
---
^C
foobarBYPASSVPN $ openssl s_client -connect imaps.example.com:993 -showcerts
CONNECTED(00000005)
depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com <http://www.digicert.com>, CN = DigiCert High Assurance EV Root CA
verify return:1
depth=1 C = US, O = DigiCert Inc, OU = www.digicert.com <http://www.digicert.com>, CN = DigiCert SHA2 High Assurance Server CA
verify return:1
depth=0 C = ZZ, L = MYTOWN, O = MYCORP, CN = imaps.example.com
verify return:1
---
Certificate chain
0 s:/C=ZZ/L=MYTOWN/O=MYCORP/CN=imaps.example.com
   i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert <http://www.digicert.com/CN=DigiCert> SHA2 High Assurance Server CA
-----BEGIN CERTIFICATE-----
<SNIP>
-----END CERTIFICATE-----
1 s:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert <http://www.digicert.com/CN=DigiCert> SHA2 High Assurance Server CA
   i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert <http://www.digicert.com/CN=DigiCert> High Assurance EV Root CA
-----BEGIN CERTIFICATE-----
<SNIP>
-----END CERTIFICATE-----
2 s:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert <http://www.digicert.com/CN=DigiCert> High Assurance EV Root CA
   i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert <http://www.digicert.com/CN=DigiCert> High Assurance EV Root CA
-----BEGIN CERTIFICATE-----
<SNIP>
-----END CERTIFICATE-----
---
Server certificate
subject=/C=ZZ/L=MYTOWN/O=MYCORP/CN=imaps.example.com
issuer=/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert <http://www.digicert.com/CN=DigiCert> SHA2 High Assurance Server CA
---
No client certificate CA names sent
Server Temp Key: ECDH, P-384, 384 bits
---
SSL handshake has read 4243 bytes and written 358 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-GCM-SHA256
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES128-GCM-SHA256
    Session-ID: F8BB41516529FA657513FB23B803D7CA0990B674446CB78A9D71184C93A810FE
    Session-ID-ctx:
    Master-Key: FCA69ED068B34A1A3B1256A0390A9508357762AFC9E9EEA605979B6A6CD3C2EEA5CEB29E9A67DF219213C924E29328A7
    TLS session ticket lifetime hint: 300 (seconds)
    TLS session ticket:
<SNIP>
    Start Time: 1543861700
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
---
* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN AUTH=LOGIN] Dovecot ready.
^C

Reply | Threaded
Open this post in threaded view
|

Re: TLS suddenly not working over IKED site-to-site

Theodore Wynnychenko-2
>
> > Rachel,
> >
> > As a first step, try using s_client to connect to a TLS service and
> see what comes back:
> >
> > $ openssl s_client -connect <hostname>:<port> -showcerts
> >
> > There are more possible options on s_client to debug more deeply but
> this is a good start.
> >
> >
> > --Paul
> >
>
> In answer to the above. Testing against three "random" servers  (see
> bottom of the email for full exchange, top three are through VPN, rest
> are bypassing VPN):
>


I wanted to follow up on this.

I updated the servers that create the iked VPN to the 12/5 snapshot the other day.

I then took one machine on the "remote" net and ran openssl s_server.
I had another machine on the "local" net try to connect with openssl s_client.

So, the s_client shows:

SSL_connect:before/connect initialization
SSL_connect:SSLv3 write client hello A
... and nothing more.

The s_server shows:

Using auto DH parameters
Using default temp ECDH parameters
ACCEPT
SSL_accept:before/accept initialization
... and nothing more.

I also had tcpdump running at several places along the route.

On the outgoing/sending interface of the "s_client" machine I see:

21:19:54.735257 172.30.1.254.44715 > 172.30.7.205.443: S 950714671:950714671(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 6,nop,nop,timestamp 523073824 0> (DF)

21:19:54.773320 172.30.7.205.443 > 172.30.1.254.44715: S 668125506:668125506(0) ack 950714672 win 16384 <mss 1300,nop,nop,sackOK,nop,wscale 6,nop,nop,timestamp 590367976 523073824>

21:19:54.773391 172.30.1.254.44715 > 172.30.7.205.443: . ack 1 win 256 <nop,nop,timestamp 523073824 590367976> (DF)
21:19:54.774143 172.30.1.254.44715 > 172.30.7.205.443: P 1:197(196) ack 1 win 256 <nop,nop,timestamp 523073824 590367976> (DF)

21:19:56.272544 172.30.1.254.44715 > 172.30.7.205.443: P 1:197(196) ack 1 win 256 <nop,nop,timestamp 523073827 590367976> (DF)

21:19:59.272615 172.30.1.254.44715 > 172.30.7.205.443: P 1:197(196) ack 1 win 256 <nop,nop,timestamp 523073833 590367976> (DF)

21:20:05.272786 172.30.1.254.44715 > 172.30.7.205.443: P 1:197(196) ack 1 win 256 <nop,nop,timestamp 523073845 590367976>

21:20:10.743468 172.30.1.254.44715 > 172.30.7.205.443: F 197:197(0) ack 1 win 256 <nop,nop,timestamp 523073856 590367976>

21:20:10.781912 172.30.7.205.443 > 172.30.1.254.44715: . ack 1 win 261 <nop,nop,timestamp 590368008 523073824,nop,nop,sack 1 {197:197} >

21:20:12.124726 172.30.7.205.443 > 172.30.1.254.44715: F 1:1(0) ack 1 win 261 <nop,nop,timestamp 590368011 523073824>
21:20:12.124786 172.30.1.254.44715 > 172.30.7.205.443: F 197:197(0) ack 2 win 256 <nop,nop,timestamp 523073858 590368011>

21:20:12.162326 172.30.7.205.443 > 172.30.1.254.44715: . ack 1 win 261 <nop,nop,timestamp 590368011 523073824>
21:20:17.273069 172.30.1.254.44715 > 172.30.7.205.443: FP 1:197(196) ack 2 win 256 <nop,nop,timestamp 523073869 590368011> (DF)


On the incoming/receiving interface of the "local" iked machine I see:

21:19:54.737490 172.30.1.254.44715 > 172.30.7.205.443: S 950714671:950714671(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 6,nop,nop,timestamp 523073824 0> (DF)

21:19:54.775299 172.30.7.205.443 > 172.30.1.254.44715: S 668125506:668125506(0) ack 950714672 win 16384 <mss 1300,nop,nop,sackOK,nop,wscale 6,nop,nop,timestamp 590367976 523073824>

21:19:54.775625 172.30.1.254.44715 > 172.30.7.205.443: . ack 1 win 256 <nop,nop,timestamp 523073824 590367976> (DF)
21:19:54.776378 172.30.1.254.44715 > 172.30.7.205.443: P 1:197(196) ack 1 win 256 <nop,nop,timestamp 523073824 590367976> (DF)

21:19:56.274790 172.30.1.254.44715 > 172.30.7.205.443: P 1:197(196) ack 1 win 256 <nop,nop,timestamp 523073827 590367976> (DF)

21:19:59.274859 172.30.1.254.44715 > 172.30.7.205.443: P 1:197(196) ack 1 win 256 <nop,nop,timestamp 523073833 590367976> (DF)

21:20:05.275017 172.30.1.254.44715 > 172.30.7.205.443: P 1:197(196) ack 1 win 256 <nop,nop,timestamp 523073845 590367976>

21:20:10.745731 172.30.1.254.44715 > 172.30.7.205.443: F 197:197(0) ack 1 win 256 <nop,nop,timestamp 523073856 590367976>

21:20:10.783860 172.30.7.205.443 > 172.30.1.254.44715: . ack 1 win 261 <nop,nop,timestamp 590368008 523073824,nop,nop,sack 1 {197:197} >

21:20:12.126709 172.30.7.205.443 > 172.30.1.254.44715: F 1:1(0) ack 1 win 261 <nop,nop,timestamp 590368011 523073824>
21:20:12.127041 172.30.1.254.44715 > 172.30.7.205.443: F 197:197(0) ack 2 win 256 <nop,nop,timestamp 523073858 590368011>

21:20:12.164312 172.30.7.205.443 > 172.30.1.254.44715: . ack 1 win 261 <nop,nop,timestamp 590368011 523073824>


But, on the outgoing/sending interface of the "remote" iked machine, all that I see is:

21:19:54.733973 172.30.1.254.44715 > 172.30.7.205.443: S 4173630539:4173630539(0) win 16384 <mss 1300,nop,nop,sackOK,nop,wscale 6,nop,nop,timestamp 523073824 0>

21:19:54.734355 172.30.7.205.443 > 172.30.1.254.44715: S 2645985599:2645985599(0) ack 4173630540 win 16384 <mss 1300,nop,nop,sackOK,nop,wscale 6,nop,nop,timestamp 590367976 523073824>

21:19:54.773048 172.30.1.254.44715 > 172.30.7.205.443: . ack 1 win 256 <nop,nop,timestamp 523073824 590367976>
21:20:10.742843 172.30.1.254.44715 > 172.30.7.205.443: F 197:197(0) ack 1 win 256 <nop,nop,timestamp 523073856 590367976>

21:20:10.743111 172.30.7.205.443 > 172.30.1.254.44715: . ack 1 win 261 <nop,nop,timestamp 590368008 523073824,nop,nop,sack 1 {197:197} >

21:20:12.085788 172.30.7.205.443 > 172.30.1.254.44715: F 1:1(0) ack 1 win 261 <nop,nop,timestamp 590368011 523073824>
21:20:12.123252 172.30.1.254.44715 > 172.30.7.205.443: F 197:197(0) ack 2 win 256 <nop,nop,timestamp 523073858 590368011>

21:20:12.123472 172.30.7.205.443 > 172.30.1.254.44715: . ack 1 win 261 <nop,nop,timestamp 590368011 523073824>


And that is all that gets delivered to the incoming/receiving interface of the "s_server" machine:

21:19:54.710031 172.30.1.254.44715 > 172.30.7.205.443: S 4173630539:4173630539(0) win 16384 <mss 1300,nop,nop,sackOK,nop,wscale 6,nop,nop,timestamp 523073824 0>

21:19:54.710134 172.30.7.205.443 > 172.30.1.254.44715: S 2645985599:2645985599(0) ack 4173630540 win 16384 <mss 1300,nop,nop,sackOK,nop,wscale 6,nop,nop,timestamp 590367976 523073824>

21:19:54.749110 172.30.1.254.44715 > 172.30.7.205.443: . ack 1 win 256 <nop,nop,timestamp 523073824 590367976>
21:20:10.718972 172.30.1.254.44715 > 172.30.7.205.443: F 197:197(0) ack 1 win 256 <nop,nop,timestamp 523073856 590367976>

21:20:10.719023 172.30.7.205.443 > 172.30.1.254.44715: . ack 1 win 261 <nop,nop,timestamp 590368008 523073824,nop,nop,sack 1 {197:197} >

21:20:12.061678 172.30.7.205.443 > 172.30.1.254.44715: F 1:1(0) ack 1 win 261 <nop,nop,timestamp 590368011 523073824>
21:20:12.099433 172.30.1.254.44715 > 172.30.7.205.443: F 197:197(0) ack 2 win 256 <nop,nop,timestamp 523073858 590368011>

21:20:12.099484 172.30.7.205.443 > 172.30.1.254.44715: . ack 1 win 261 <nop,nop,timestamp 590368011 523073824>


Now, if I try connecting using s_client ON the "remote" iked machine (so, a connection that does not include the iked tunnel), everything works, and tcpdump shows ("the expected?") data and traffic leaving:

21:36:56.027413 172.30.7.1.33610 > 172.30.7.205.443: S 2112686897:2112686897(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 6,nop,nop,timestamp 3723402680 0> (DF)

21:36:56.027768 172.30.7.205.443 > 172.30.7.1.33610: S 3448062619:3448062619(0) ack 2112686898 win 16384 <mss 1300,nop,nop,sackOK,nop,wscale 6,nop,nop,timestamp 3091368224 3723402680>

21:36:56.027817 172.30.7.1.33610 > 172.30.7.205.443: . ack 1 win 256 <nop,nop,timestamp 3723402680 3091368224> (DF)
21:36:56.028403 172.30.7.1.33610 > 172.30.7.205.443: P 1:197(196) ack 1 win 256 <nop,nop,timestamp 3723402680 3091368224> (DF)

21:36:56.046516 172.30.7.205.443 > 172.30.7.1.33610: . 1:1289(1288) ack 197 win 261 <nop,nop,timestamp 3091368224 3723402680>

21:36:56.046518 172.30.7.205.443 > 172.30.7.1.33610: . 1289:2577(1288) ack 197 win 261 <nop,nop,timestamp 3091368224 3723402680>

21:36:56.046519 172.30.7.205.443 > 172.30.7.1.33610: . 2577:3865(1288) ack 197 win 261 <nop,nop,timestamp 3091368224 3723402680>

21:36:56.046520 172.30.7.205.443 > 172.30.7.1.33610: P 3865:4097(232) ack 197 win 261 <nop,nop,timestamp 3091368224 3723402680>

21:36:56.046607 172.30.7.1.33610 > 172.30.7.205.443: . ack 2577 win 215 <nop,nop,timestamp 3723402680 3091368224> (DF)
21:36:56.046793 172.30.7.1.33610 > 172.30.7.205.443: . ack 4097 win 192 <nop,nop,timestamp 3723402680 3091368224> (DF)
21:36:56.047147 172.30.7.205.443 > 172.30.7.1.33610: P 4097:4473(376) ack 197 win 261 <nop,nop,timestamp 3091368224 3723402680>

21:36:56.047196 172.30.7.1.33610 > 172.30.7.205.443: . ack 4473 win 246 <nop,nop,timestamp 3723402680 3091368224> (DF)
21:36:56.053509 172.30.7.1.33610 > 172.30.7.205.443: P 197:315(118) ack 4473 win 256 <nop,nop,timestamp 3723402680 3091368224> (DF)

21:36:56.055675 172.30.7.205.443 > 172.30.7.1.33610: P 4473:4691(218) ack 315 win 261 <nop,nop,timestamp 3091368224 3723402680>

21:36:56.250855 172.30.7.1.33610 > 172.30.7.205.443: . ack 4691 win 256 <nop,nop,timestamp 3723402681 3091368224> (DF)
21:36:58.540398 172.30.7.1.33610 > 172.30.7.205.443: F 315:315(0) ack 4691 win 256 <nop,nop,timestamp 3723402685 3091368224> (DF)

21:36:58.540581 172.30.7.205.443 > 172.30.7.1.33610: . ack 316 win 261 <nop,nop,timestamp 3091368229 3723402685>
21:36:58.541186 172.30.7.205.443 > 172.30.7.1.33610: F 4691:4691(0) ack 316 win 261 <nop,nop,timestamp 3091368229 3723402685>

21:36:58.541219 172.30.7.1.33610 > 172.30.7.205.443: . ack 4692 win 256 <nop,nop,timestamp 3723402685 3091368229> (DF)


I am no expert, but I can see that this "local" connection sends a lot more data.

So, today I was going to try this again, now looking at physical interfaces and the enc0 interface on the iked endpoints.

But, for whatever reason, I did not have to, because this morning https was working without a problem over the iked VPN.


Unfortunately, I noticed that there was problem with icinga2 (which monitors the hosts on the "remote" net).  I noticed that even though the hosts were up, icinga2 was reporting them as down.

I found that on the (alternately) remote or local iked host, icinga connections (over port 5665) were being blocked even though there is a specific "pass rule" in pf.conf to permit them.

For example, in the log I see:
Dec  8 15:50:01 ... pf: Dec 08 15:48:49.346816 rule 4/(match) block out on em0: 172.30.7.205.22112 > 172.30.2.99.5665: R 3963276584:3963276584(0) ack 252894831 win 0

But, pfctl is running with following:

# pfctl -s rules
match in all scrub (no-df random-id max-mss 1300)
pass in quick on em1 all flags S/SA
pass out quick on em1 all flags S/SA
block drop in log on em0 all
block drop out log on em0 all
...
pass quick inet proto tcp from 172.30.7.205 to 172.30.2.99 port = 5665 flags S/SA
... and on.

There are no other "quick" rules between the default block and the quick pass rule that should allow the packet.

I don't understand why the packet is blocked when it should specifically (and "quickly") be passed.

I played with my pf.conf for a while, and, all of a sudden, icinga was able to connect.  I undid my changes (things like removing the default block rules) and it continued to work.

I decided to reboot several of the hosts to see if things would be stable.  But, they are not.

I now find I can no longer connect to with TLS/SSL over the iked tunnel (the original behavior that seemed to have corrected itself).  Also, icinga continues to be unable to verify the status of the remote hosts over port 5665.

I don't have time right now to try using s_client and s_server and watching enc0 to see what is happening, but I will when I can.

If anyone has an ideas on what may be happening, please let me know.

Thanks
Ted


Reply | Threaded
Open this post in threaded view
|

Re: TLS suddenly not working over IKED site-to-site

Theodore Wynnychenko-2
In reply to this post by Rachel Roch
I would like to re-title this as something like "pf and iked instability on recent snapshots," but don’t know if doing so would break the mailing list thread, exiso, I left the subject unchanged...

> -----Original Message-----
> From: Theodore Wynnychenko [mailto:[hidden email]]
> Sent: Saturday, December 08, 2018 4:03 PM
> To: [hidden email]
> Cc: 'Rachel Roch'
> Subject: RE: TLS suddenly not working over IKED site-to-site
>
> >
.
.
.

> I now find I can no longer connect to with TLS/SSL over the iked tunnel
> (the original behavior that seemed to have corrected itself).  Also,
> icinga continues to be unable to verify the status of the remote hosts
> over port 5665.
>
> I don't have time right now to try using s_client and s_server and
> watching enc0 to see what is happening, but I will when I can.
>
> If anyone has an ideas on what may be happening, please let me know.
>
> Thanks
> Ted


Hello again;

So, I am at a complete loss to understand what is going on.
Today, I tried using openssl s_client and s_server to make a connection through the iked vpn (as I described in my last post).  However, with NO changes to iked.conf or pf.conf, today I had several connection attempts that completed correctly.  I have not included any output from those sporadic, completely functional connections.

Rather, today, most of the connections by s_client are not even acknowledged by the s_server on the other side of the iked vpn.

For example:
On the s_client machine:

# openssl s_client -state -connect "remote.host":https
SSL_connect:before/connect initialization
SSL_connect:SSLv3 write client hello A
... and nothing more ...

But on the s_server machine today all I see is:
# openssl s_sever -state -accept https ...certificate options...
Using auto DH parameters
Using default temp ECDH parameters
ACCEPT
... and no connection attempt is ever acknowledged ...

(Yesterday, at least this first part of the connection was received by the s_server:
Using auto DH parameters
Using default temp ECDH parameters
ACCEPT
SSL_accept:before/accept initialization
... and nothing more yesterday ...)


So, today using tcpdump on the outgoing interface of the s_client machine and the incoming interface of the "local" iked vpn endpoint shows:

16:43:05.107524 172.30.1.254.7305 > 172.30.7.205.443: S 1751796302:1751796302(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 6,nop,nop,timestamp 2698316052 0>

16:43:05.149146 172.30.1.254.7305 > 172.30.7.205.443: . ack 2119500805 win 256 <nop,nop,timestamp 2698316052 3536824996>

16:43:05.149895 172.30.1.254.7305 > 172.30.7.205.443: P 0:196(196) ack 1 win 256 <nop,nop,timestamp 2698316052 3536824996>

16:43:06.648487 172.30.1.254.7305 > 172.30.7.205.443: P 0:196(196) ack 1 win 256 <nop,nop,timestamp 2698316055 3536824996>

16:43:09.648557 172.30.1.254.7305 > 172.30.7.205.443: P 0:196(196) ack 1 win 256 <nop,nop,timestamp 2698316061 3536824996>

16:43:09.948433 172.30.1.254.7305 > 172.30.7.205.443: F 196:196(0) ack 1 win 256 <nop,nop,timestamp 2698316061 3536824996>

16:43:15.648712 172.30.1.254.7305 > 172.30.7.205.443: FP 0:196(196) ack 1 win 256 <nop,nop,timestamp 2698316073 3536825005>

And this traffic (incomplete thought it may be for an ssl handshake) appears to be passed to enc0 intact:

16:43:05.105044 (authentic,confidential): SPI 0x151333df: 172.30.1.254.7305 > 172.30.7.205.443: S 3570513915:3570513915(0) win 16384 <mss 1300,nop,nop,sackOK,nop,wscale 6,nop,nop,timestamp 2698316052 0> (encap)

16:43:05.146122 (authentic,confidential): SPI 0xe1c30e4a: 172.30.7.205.443 > 172.30.1.254.7305: S 1312941075:1312941075(0) ack 3570513916 win 16384 <mss 1300,nop,nop,sackOK,nop,wscale 6,nop,nop,timestamp 3536824996 2698316052> (encap)

16:43:05.146654 (authentic,confidential): SPI 0x151333df: 172.30.1.254.7305 > 172.30.7.205.443: . ack 1 win 256 <nop,nop,timestamp 2698316052 3536824996> (encap)

16:43:05.147365 (unprotected): SPI 0x0000ef27: 172.30.1.254.7305 > 172.30.7.205.443: P 1:197(196) ack 1 win 256 <nop,nop,timestamp 2698316052 3536824996> (encap)

16:43:06.645932 (unprotected): SPI 0x0000ef27: 172.30.1.254.7305 > 172.30.7.205.443: P 1:197(196) ack 1 win 256 <nop,nop,timestamp 2698316055 3536824996> (encap)

16:43:09.646049 (unprotected): SPI 0x0000ef27: 172.30.1.254.7305 > 172.30.7.205.443: P 1:197(196) ack 1 win 256 <nop,nop,timestamp 2698316061 3536824996> (encap)

16:43:09.945908 (authentic,confidential): SPI 0x151333df: 172.30.1.254.7305 > 172.30.7.205.443: F 197:197(0) ack 1 win 256 <nop,nop,timestamp 2698316061 3536824996> (encap)

16:43:09.981966 (authentic,confidential): SPI 0xe1c30e4a: 172.30.7.205.443 > 172.30.1.254.7305: . ack 1 win 261 <nop,nop,timestamp 3536825005 2698316052,nop,nop,sack 1 {197:197} > (encap)

16:43:15.646158 (unprotected): SPI 0x0000ef27: 172.30.1.254.7305 > 172.30.7.205.443: FP 1:197(196) ack 1 win 256 <nop,nop,timestamp 2698316073 3536825005> (encap)


BUT, at the other end of the VPN, on enc0, all that is seen leaving the iked VPN tunnel is:

16:43:05.130558 (authentic,confidential): SPI 0x151333df: 172.30.1.254.7305 > 172.30.7.205.443: S 3570513915:3570513915(0) win 16384 <mss 1300,nop,nop,sackOK,nop,wscale 6,nop,nop,timestamp 2698316052 0> (encap)

16:43:05.131049 (authentic,confidential): SPI 0xe1c30e4a: 172.30.7.205.443 > 172.30.1.254.7305: S 1312941075:1312941075(0) ack 3570513916 win 16384 <mss 1300,nop,nop,sackOK,nop,wscale 6,nop,nop,timestamp 3536824996 2698316052> (encap)

16:43:05.174802 (authentic,confidential): SPI 0x151333df: 172.30.1.254.7305 > 172.30.7.205.443: . ack 1 win 256 <nop,nop,timestamp 2698316052 3536824996> (encap)

16:43:09.966420 (authentic,confidential): SPI 0x151333df: 172.30.1.254.7305 > 172.30.7.205.443: F 197:197(0) ack 1 win 256 <nop,nop,timestamp 2698316061 3536824996> (encap)

16:43:09.966853 (authentic,confidential): SPI 0xe1c30e4a: 172.30.7.205.443 > 172.30.1.254.7305: . ack 1 win 261 <nop,nop,timestamp 3536825005 2698316052,nop,nop,sack 1 {197:197} > (encap)


I have no idea what this all means, or what to do with it.
But, I am following up in case anybody has any idea of what may be happening.

Also, yesterday I described how the local iked machine appeared to be blocking packets that were explicitly allowed by pf.conf.  From my post yesterday:

(   For example, in the log I see:
Dec  8 15:50:01 ... pf: Dec 08 15:48:49.346816 rule 4/(match) block out on em0: 172.30.7.205.22112 > 172.30.2.99.5665: R 3963276584:3963276584(0) ack 252894831 win 0

But, pfctl is running with following:

# pfctl -s rules
match in all scrub (no-df random-id max-mss 1300)
pass in quick on em1 all flags S/SA
pass out quick on em1 all flags S/SA
block drop in log on em0 all
block drop out log on em0 all
...
pass quick inet proto tcp from 172.30.7.205 to 172.30.2.99 port = 5665 flags S/SA
... and on.    )

Well, whatever was happening appears to have been resolved, because at about midnight local time on Sunday morning, icinga2 declared that the host was back up.

To be clear, I have made no changes to either pf.conf or iked.conf on any of the machines involved in this testing from Saturday.

Also, this had all been stable for the last (about) 2 years, until about two-three weeks ago.  I did have another post, where I discussed the fact the iked VPN had failed to be reestablished after an update about 3-4 snapshots back.  I got it working again by changing the local endpoint on the "remote" iked machine from the internal ip associated with the internal interface to an internal "alias" ip address associated with the outgoing/external interface of that machine.

But, again, it had been working for 2 years until the recent update.

I don't have any idea of what may be helpful in figuring out what I am doing wrong, or what has changed, but I am happy to provide any information that may be of help.

I don't believe I have the knowledge to do more on my own at this point.

Thanks for any advice.
Ted




Reply | Threaded
Open this post in threaded view
|

Re: TLS suddenly not working over IKED site-to-site

雷致强
I’m having the same issue on OpenBSD 6.4. My iked.conf is similar to Rachel’s:

include "/etc/iked/macros.conf"

ikev2 quick active ipcomp esp proto gre\
        from 192.168.1.0/24 to $iked_server \
        local egress peer $iked_server \
        ikesa auth hmac-sha2-512 enc aes-256 prf hmac-sha2-512 group curve25519 \
        childsa enc chacha20-poly130 group curve25519 \
        dstid "asgard.local"

Sites are reachable by ping, but https doesn’t respond at all. I was wondering if it is because the GRE part, will do more experiments.

> On Dec 11, 2018, at 9:04 AM, Theodore Wynnychenko <[hidden email]> wrote:
>
> I would like to re-title this as something like "pf and iked instability on recent snapshots," but don’t know if doing so would break the mailing list thread, exiso, I left the subject unchanged...
>
>> -----Original Message-----
>> From: Theodore Wynnychenko [mailto:[hidden email]]
>> Sent: Saturday, December 08, 2018 4:03 PM
>> To: [hidden email]
>> Cc: 'Rachel Roch'
>> Subject: RE: TLS suddenly not working over IKED site-to-site
>>
>>>
> .
> .
> .
>> I now find I can no longer connect to with TLS/SSL over the iked tunnel
>> (the original behavior that seemed to have corrected itself).  Also,
>> icinga continues to be unable to verify the status of the remote hosts
>> over port 5665.
>>
>> I don't have time right now to try using s_client and s_server and
>> watching enc0 to see what is happening, but I will when I can.
>>
>> If anyone has an ideas on what may be happening, please let me know.
>>
>> Thanks
>> Ted
>
>
> Hello again;
>
> So, I am at a complete loss to understand what is going on.
> Today, I tried using openssl s_client and s_server to make a connection through the iked vpn (as I described in my last post).  However, with NO changes to iked.conf or pf.conf, today I had several connection attempts that completed correctly.  I have not included any output from those sporadic, completely functional connections.
>
> Rather, today, most of the connections by s_client are not even acknowledged by the s_server on the other side of the iked vpn.
>
> For example:
> On the s_client machine:
>
> # openssl s_client -state -connect "remote.host":https
> SSL_connect:before/connect initialization
> SSL_connect:SSLv3 write client hello A
> ... and nothing more ...
>
> But on the s_server machine today all I see is:
> # openssl s_sever -state -accept https ...certificate options...
> Using auto DH parameters
> Using default temp ECDH parameters
> ACCEPT
> ... and no connection attempt is ever acknowledged ...
>
> (Yesterday, at least this first part of the connection was received by the s_server:
> Using auto DH parameters
> Using default temp ECDH parameters
> ACCEPT
> SSL_accept:before/accept initialization
> ... and nothing more yesterday ...)
>
>
> So, today using tcpdump on the outgoing interface of the s_client machine and the incoming interface of the "local" iked vpn endpoint shows:
>
> 16:43:05.107524 172.30.1.254.7305 > 172.30.7.205.443: S 1751796302:1751796302(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 6,nop,nop,timestamp 2698316052 0>
>
> 16:43:05.149146 172.30.1.254.7305 > 172.30.7.205.443: . ack 2119500805 win 256 <nop,nop,timestamp 2698316052 3536824996>
>
> 16:43:05.149895 172.30.1.254.7305 > 172.30.7.205.443: P 0:196(196) ack 1 win 256 <nop,nop,timestamp 2698316052 3536824996>
>
> 16:43:06.648487 172.30.1.254.7305 > 172.30.7.205.443: P 0:196(196) ack 1 win 256 <nop,nop,timestamp 2698316055 3536824996>
>
> 16:43:09.648557 172.30.1.254.7305 > 172.30.7.205.443: P 0:196(196) ack 1 win 256 <nop,nop,timestamp 2698316061 3536824996>
>
> 16:43:09.948433 172.30.1.254.7305 > 172.30.7.205.443: F 196:196(0) ack 1 win 256 <nop,nop,timestamp 2698316061 3536824996>
>
> 16:43:15.648712 172.30.1.254.7305 > 172.30.7.205.443: FP 0:196(196) ack 1 win 256 <nop,nop,timestamp 2698316073 3536825005>
>
> And this traffic (incomplete thought it may be for an ssl handshake) appears to be passed to enc0 intact:
>
> 16:43:05.105044 (authentic,confidential): SPI 0x151333df: 172.30.1.254.7305 > 172.30.7.205.443: S 3570513915:3570513915(0) win 16384 <mss 1300,nop,nop,sackOK,nop,wscale 6,nop,nop,timestamp 2698316052 0> (encap)
>
> 16:43:05.146122 (authentic,confidential): SPI 0xe1c30e4a: 172.30.7.205.443 > 172.30.1.254.7305: S 1312941075:1312941075(0) ack 3570513916 win 16384 <mss 1300,nop,nop,sackOK,nop,wscale 6,nop,nop,timestamp 3536824996 2698316052> (encap)
>
> 16:43:05.146654 (authentic,confidential): SPI 0x151333df: 172.30.1.254.7305 > 172.30.7.205.443: . ack 1 win 256 <nop,nop,timestamp 2698316052 3536824996> (encap)
>
> 16:43:05.147365 (unprotected): SPI 0x0000ef27: 172.30.1.254.7305 > 172.30.7.205.443: P 1:197(196) ack 1 win 256 <nop,nop,timestamp 2698316052 3536824996> (encap)
>
> 16:43:06.645932 (unprotected): SPI 0x0000ef27: 172.30.1.254.7305 > 172.30.7.205.443: P 1:197(196) ack 1 win 256 <nop,nop,timestamp 2698316055 3536824996> (encap)
>
> 16:43:09.646049 (unprotected): SPI 0x0000ef27: 172.30.1.254.7305 > 172.30.7.205.443: P 1:197(196) ack 1 win 256 <nop,nop,timestamp 2698316061 3536824996> (encap)
>
> 16:43:09.945908 (authentic,confidential): SPI 0x151333df: 172.30.1.254.7305 > 172.30.7.205.443: F 197:197(0) ack 1 win 256 <nop,nop,timestamp 2698316061 3536824996> (encap)
>
> 16:43:09.981966 (authentic,confidential): SPI 0xe1c30e4a: 172.30.7.205.443 > 172.30.1.254.7305: . ack 1 win 261 <nop,nop,timestamp 3536825005 2698316052,nop,nop,sack 1 {197:197} > (encap)
>
> 16:43:15.646158 (unprotected): SPI 0x0000ef27: 172.30.1.254.7305 > 172.30.7.205.443: FP 1:197(196) ack 1 win 256 <nop,nop,timestamp 2698316073 3536825005> (encap)
>
>
> BUT, at the other end of the VPN, on enc0, all that is seen leaving the iked VPN tunnel is:
>
> 16:43:05.130558 (authentic,confidential): SPI 0x151333df: 172.30.1.254.7305 > 172.30.7.205.443: S 3570513915:3570513915(0) win 16384 <mss 1300,nop,nop,sackOK,nop,wscale 6,nop,nop,timestamp 2698316052 0> (encap)
>
> 16:43:05.131049 (authentic,confidential): SPI 0xe1c30e4a: 172.30.7.205.443 > 172.30.1.254.7305: S 1312941075:1312941075(0) ack 3570513916 win 16384 <mss 1300,nop,nop,sackOK,nop,wscale 6,nop,nop,timestamp 3536824996 2698316052> (encap)
>
> 16:43:05.174802 (authentic,confidential): SPI 0x151333df: 172.30.1.254.7305 > 172.30.7.205.443: . ack 1 win 256 <nop,nop,timestamp 2698316052 3536824996> (encap)
>
> 16:43:09.966420 (authentic,confidential): SPI 0x151333df: 172.30.1.254.7305 > 172.30.7.205.443: F 197:197(0) ack 1 win 256 <nop,nop,timestamp 2698316061 3536824996> (encap)
>
> 16:43:09.966853 (authentic,confidential): SPI 0xe1c30e4a: 172.30.7.205.443 > 172.30.1.254.7305: . ack 1 win 261 <nop,nop,timestamp 3536825005 2698316052,nop,nop,sack 1 {197:197} > (encap)
>
>
> I have no idea what this all means, or what to do with it.
> But, I am following up in case anybody has any idea of what may be happening.
>
> Also, yesterday I described how the local iked machine appeared to be blocking packets that were explicitly allowed by pf.conf.  From my post yesterday:
>
> (   For example, in the log I see:
> Dec  8 15:50:01 ... pf: Dec 08 15:48:49.346816 rule 4/(match) block out on em0: 172.30.7.205.22112 > 172.30.2.99.5665: R 3963276584:3963276584(0) ack 252894831 win 0
>
> But, pfctl is running with following:
>
> # pfctl -s rules
> match in all scrub (no-df random-id max-mss 1300)
> pass in quick on em1 all flags S/SA
> pass out quick on em1 all flags S/SA
> block drop in log on em0 all
> block drop out log on em0 all
> ...
> pass quick inet proto tcp from 172.30.7.205 to 172.30.2.99 port = 5665 flags S/SA
> ... and on.    )
>
> Well, whatever was happening appears to have been resolved, because at about midnight local time on Sunday morning, icinga2 declared that the host was back up.
>
> To be clear, I have made no changes to either pf.conf or iked.conf on any of the machines involved in this testing from Saturday.
>
> Also, this had all been stable for the last (about) 2 years, until about two-three weeks ago.  I did have another post, where I discussed the fact the iked VPN had failed to be reestablished after an update about 3-4 snapshots back.  I got it working again by changing the local endpoint on the "remote" iked machine from the internal ip associated with the internal interface to an internal "alias" ip address associated with the outgoing/external interface of that machine.
>
> But, again, it had been working for 2 years until the recent update.
>
> I don't have any idea of what may be helpful in figuring out what I am doing wrong, or what has changed, but I am happy to provide any information that may be of help.
>
> I don't believe I have the knowledge to do more on my own at this point.
>
> Thanks for any advice.
> Ted
>
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: TLS suddenly not working over IKED site-to-site

雷致强
Hi,

This issue varies to different destinations on my side. I cannot browse www.google.com <http://www.google.com/> and twitter.com <http://twitter.com/> and www.facebook.com <http://www.facebook.com/> (and possibly more), while I can browse duckduckgo.com <http://duckduckgo.com/>. Firefox returns Network Protocol Error on www.google.com <http://www.google.com/> and timeout on twitter.com <http://twitter.com/> and www.facebook.com <http://www.facebook.com/>. Additionally, IMAPS and SMTPS is working on Gmail, this email is sending via the VPN.

~> openssl s_client -connect www.google.com:443 -showcerts
CONNECTED(00000005)
depth=2 OU = GlobalSign Root CA - R2, O = GlobalSign, CN = GlobalSign
verify return:1
depth=1 C = US, O = Google Trust Services, CN = Google Internet Authority G3
verify return:1
depth=0 C = US, ST = California, L = Mountain View, O = Google LLC, CN = www.google.com
verify return:1
---
Certificate chain
 0 s:/C=US/ST=California/L=Mountain View/O=Google LLC/CN=www.google.com
   i:/C=US/O=Google Trust Services/CN=Google Internet Authority G3
-----BEGIN CERTIFICATE-----
MIIEgjCCA2qgAwIBAgIIaLoHUPAeg/4wDQYJKoZIhvcNAQELBQAwVDELMAkGA1UE
BhMCVVMxHjAcBgNVBAoTFUdvb2dsZSBUcnVzdCBTZXJ2aWNlczElMCMGA1UEAxMc
R29vZ2xlIEludGVybmV0IEF1dGhvcml0eSBHMzAeFw0xODExMjcxNDAyMDBaFw0x
OTAyMTkxNDAyMDBaMGgxCzAJBgNVBAYTAlVTMRMwEQYDVQQIDApDYWxpZm9ybmlh
MRYwFAYDVQQHDA1Nb3VudGFpbiBWaWV3MRMwEQYDVQQKDApHb29nbGUgTExDMRcw
FQYDVQQDDA53d3cuZ29vZ2xlLmNvbTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCC
AQoCggEBAKhtF0MjB2unptytQW44dm3c9eL3f3YrxJ684rqX8c901CD73NCO+vcF
iVd0c6PiYpYK/kw9vRDXNrognt3CKYWVq/FdTlrjuKWExt7qD/sxZCKetTl5qekm
V4PwcbEvS98NF9vXIIIzOTK3WdYjz573ydyCWpA32qOLpCt8a6XoedFzX7KiUaGS
RPPNQI0+xZgow7jgNpZklB+iEwymSgFrM5SHxp4fouJMVG3BnJqXyhTdz4U4Ck+S
3gVEBNKc/y+ZKGplwbZ5NngYaov7U9uItceawa5xplcN0A1RcDmVaaRtsF51ktVQ
pZaAauc7LibJc9j4+ov5wTYYDwrxDK8CAwEAAaOCAUIwggE+MBMGA1UdJQQMMAoG
CCsGAQUFBwMBMBkGA1UdEQQSMBCCDnd3dy5nb29nbGUuY29tMGgGCCsGAQUFBwEB
BFwwWjAtBggrBgEFBQcwAoYhaHR0cDovL3BraS5nb29nL2dzcjIvR1RTR0lBRzMu
Y3J0MCkGCCsGAQUFBzABhh1odHRwOi8vb2NzcC5wa2kuZ29vZy9HVFNHSUFHMzAd
BgNVHQ4EFgQUz7BpFM91Drzs5f0URBWlje274G0wDAYDVR0TAQH/BAIwADAfBgNV
HSMEGDAWgBR3wrhQmmd2drEtwobQg6B+pn66SzAhBgNVHSAEGjAYMAwGCisGAQQB
1nkCBQMwCAYGZ4EMAQICMDEGA1UdHwQqMCgwJqAkoCKGIGh0dHA6Ly9jcmwucGtp
Lmdvb2cvR1RTR0lBRzMuY3JsMA0GCSqGSIb3DQEBCwUAA4IBAQABZMq2qOr84g64
B0gEEkvQKIuNX+a2PB5R/IkhehqlH13120pTF1uoUBob9GCpmn2HT3KF88FiVzZ9
hfP2VturJv2WSzn1XSQGsba4oO6uiKWn1bAcXKpHWy+r+vnmuNBgLDNQ6uX3nC3A
5MrhGVv6D/7k5NPzOnI6mclBA/U/dE2Tp48Ej9RYg8rTjWajxUX4TTSskJZwmwXx
klEVY9Q4Gi3s5TrmDknusXhsIN9INzWDypB7h8Q5/IwjosMe2/rVK28BeSVtXRRy
xDI/NgavhvJJdAYOPiO0JW38QEHxKO3Fp7eO2zmZlVTbrq/VARguWl3qag526weW
4CNWWXYg
-----END CERTIFICATE-----
 1 s:/C=US/O=Google Trust Services/CN=Google Internet Authority G3
   i:/OU=GlobalSign Root CA - R2/O=GlobalSign/CN=GlobalSign
-----BEGIN CERTIFICATE-----
MIIEXDCCA0SgAwIBAgINAeOpMBz8cgY4P5pTHTANBgkqhkiG9w0BAQsFADBMMSAw
HgYDVQQLExdHbG9iYWxTaWduIFJvb3QgQ0EgLSBSMjETMBEGA1UEChMKR2xvYmFs
U2lnbjETMBEGA1UEAxMKR2xvYmFsU2lnbjAeFw0xNzA2MTUwMDAwNDJaFw0yMTEy
MTUwMDAwNDJaMFQxCzAJBgNVBAYTAlVTMR4wHAYDVQQKExVHb29nbGUgVHJ1c3Qg
U2VydmljZXMxJTAjBgNVBAMTHEdvb2dsZSBJbnRlcm5ldCBBdXRob3JpdHkgRzMw
ggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDKUkvqHv/OJGuo2nIYaNVW
XQ5IWi01CXZaz6TIHLGp/lOJ+600/4hbn7vn6AAB3DVzdQOts7G5pH0rJnnOFUAK
71G4nzKMfHCGUksW/mona+Y2emJQ2N+aicwJKetPKRSIgAuPOB6Aahh8Hb2XO3h9
RUk2T0HNouB2VzxoMXlkyW7XUR5mw6JkLHnA52XDVoRTWkNty5oCINLvGmnRsJ1z
ouAqYGVQMc/7sy+/EYhALrVJEA8KbtyX+r8snwU5C1hUrwaW6MWOARa8qBpNQcWT
kaIeoYvy/sGIJEmjR0vFEwHdp1cSaWIr6/4g72n7OqXwfinu7ZYW97EfoOSQJeAz
AgMBAAGjggEzMIIBLzAOBgNVHQ8BAf8EBAMCAYYwHQYDVR0lBBYwFAYIKwYBBQUH
AwEGCCsGAQUFBwMCMBIGA1UdEwEB/wQIMAYBAf8CAQAwHQYDVR0OBBYEFHfCuFCa
Z3Z2sS3ChtCDoH6mfrpLMB8GA1UdIwQYMBaAFJviB1dnHB7AagbeWbSaLd/cGYYu
MDUGCCsGAQUFBwEBBCkwJzAlBggrBgEFBQcwAYYZaHR0cDovL29jc3AucGtpLmdv
b2cvZ3NyMjAyBgNVHR8EKzApMCegJaAjhiFodHRwOi8vY3JsLnBraS5nb29nL2dz
cjIvZ3NyMi5jcmwwPwYDVR0gBDgwNjA0BgZngQwBAgIwKjAoBggrBgEFBQcCARYc
aHR0cHM6Ly9wa2kuZ29vZy9yZXBvc2l0b3J5LzANBgkqhkiG9w0BAQsFAAOCAQEA
HLeJluRT7bvs26gyAZ8so81trUISd7O45skDUmAge1cnxhG1P2cNmSxbWsoiCt2e
ux9LSD+PAj2LIYRFHW31/6xoic1k4tbWXkDCjir37xTTNqRAMPUyFRWSdvt+nlPq
wnb8Oa2I/maSJukcxDjNSfpDh/Bd1lZNgdd/8cLdsE3+wypufJ9uXO1iQpnh9zbu
FIwsIONGl1p3A8CgxkqI/UAih3JaGOqcpcdaCIzkBaR9uYQ1X4k2Vg5APRLouzVy
7a8IVk6wuy6pm+T7HT4LY8ibS5FEZlfAFLSW8NwsVz9SBK2Vqn1N0PIMn5xA6NZV
c7o835DLAFshEWfC7TIe3g==
-----END CERTIFICATE-----
---
Server certificate
subject=/C=US/ST=California/L=Mountain View/O=Google LLC/CN=www.google.com
issuer=/C=US/O=Google Trust Services/CN=Google Internet Authority G3
---
No client certificate CA names sent
Server Temp Key: ECDH, X25519, 253 bits
---
SSL handshake has read 2945 bytes and written 285 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-CHACHA20-POLY1305
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-CHACHA20-POLY1305
    Session-ID: 2319C8A3053A44BBFC253AE53CF6A35D9A37D19CD0CBC46ABA47D7398F657925
    Session-ID-ctx:
    Master-Key: A0B7364DBE160E54FF0BA997618F6317A0E376BFF8A4C5B4373974735B370AE8DB8C397DCC2C608D9037CFBEB2901B13
    TLS session ticket lifetime hint: 100799 (seconds)
    TLS session ticket:
    0000 - 00 30 ac f3 8e b3 c5 a7-90 c9 65 8c 5e 88 17 e4   .0........e.^...
    0010 - d2 09 7e 4b 97 1b 97 b3-55 4c 08 01 cb 52 6f 3c   ..~K....UL...Ro<
    0020 - 84 87 85 47 d4 83 f4 78-df fc a2 68 88 a3 4c 00   ...G...x...h..L.
    0030 - d1 6e 38 ae b5 d6 27 19-e4 3f 48 b8 f4 cf e2 b0   .n8...'..?H.....
    0040 - b6 89 69 e5 ab 24 b3 27-64 b3 45 58 cb 21 69 81   ..i..$.'d.EX.!i.
    0050 - 35 61 a4 f3 59 e0 68 e4-0e c9 0b a2 75 85 f3 65   5a..Y.h.....u..e
    0060 - a0 0e 89 72 df 41 09 b1-31 18 05 7e 75 6c 45 73   ...r.A..1..~ulEs
    0070 - fa df a6 e6 e3 38 8e 04-e1 21 39 97 e7 91 59 c1   .....8...!9...Y.
    0080 - 98 3a 00 46 58 b8 73 a4-0f 0f 36 ba 8f d9 e4 3c   .:.FX.s...6....<
    0090 - 7b f2 f9 2c f1 70 06 86-3f 76 e4 e3 7a 9b 40 9f   {..,.p..?v..z.@.
    00a0 - f9 ab 16 3d e8 e6 54 81-67 a0 99 c1 c3 98 cd 27   ...=..T.g......'
    00b0 - c5 31 2a 96 00 e0 82 67-74 6d 84 1d 88 0b 26 ae   .1*....gtm....&.
    00c0 - 45 45 ee 04 97 a1 a1 d1-86 09 75 75 1d a9 e3 01   EE........uu....
    00d0 - 1b 0f 1c 44 ca                                    ...D.

    Start Time: 1544866145
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
---
read:errno=0



~> openssl s_client -connect twitter.com:443 -showcerts
CONNECTED(00000006)
depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert High Assurance EV Root CA
verify return:1
depth=1 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert SHA2 High Assurance Server CA
verify return:1
depth=0 C = US, ST = California, L = San Francisco, O = "Twitter, Inc.", OU = tsa_m Point of Presence, CN = twitter.com
verify return:1
---
Certificate chain
 0 s:/C=US/ST=California/L=San Francisco/O=Twitter, Inc./OU=tsa_m Point of Presence/CN=twitter.com
   i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 High Assurance Server CA
-----BEGIN CERTIFICATE-----
MIIGhDCCBWygAwIBAgIQCWOaUnUDLiExEJVsj5mYWTANBgkqhkiG9w0BAQsFADBw
MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3
d3cuZGlnaWNlcnQuY29tMS8wLQYDVQQDEyZEaWdpQ2VydCBTSEEyIEhpZ2ggQXNz
dXJhbmNlIFNlcnZlciBDQTAeFw0xODA3MTcwMDAwMDBaFw0xOTA3MjIxMjAwMDBa
MIGKMQswCQYDVQQGEwJVUzETMBEGA1UECBMKQ2FsaWZvcm5pYTEWMBQGA1UEBxMN
U2FuIEZyYW5jaXNjbzEWMBQGA1UEChMNVHdpdHRlciwgSW5jLjEgMB4GA1UECwwX
dHNhX20gUG9pbnQgb2YgUHJlc2VuY2UxFDASBgNVBAMTC3R3aXR0ZXIuY29tMIIB
IjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA2dPpr8iFvJu73+YyFXH+9NSR
VyjeeVBhUhilZPL+Ajnl2qct/D0Xq8/zX2YNM7UjRO3HxiTYaP4sZ2Lxw0PomLUt
gnzDSK8E/w+/GFCMJ/6GVuJJnpI2G9NTDbylQD+UYqzuivmvTlwmmzIAd2+JH3yj
46np7VkZaVqrliRmLbBNNW7rj6CdC2O78xR2L9NJufzIL2Y0bkGGo6yJj0mt7bLx
791TG5g60QdUGombIvAvzrtFKb3RGc5u2DE6d2mhE5L/EtMgVBZ/jyQGlZTmJjuW
k56uviiYO7SDEHZRGqm2DxiNTvQ/IgVcMwnBYxFuQPSnaEO4Y56d7fD4F7/v1wID
AQABo4IC/TCCAvkwHwYDVR0jBBgwFoAUUWj/kK8CB3U8zNllZGKiErhZcjswHQYD
VR0OBBYEFHPl7L1zGyD02cZRjdu6yNL3hIMkMCcGA1UdEQQgMB6CC3R3aXR0ZXIu
Y29tgg93d3cudHdpdHRlci5jb20wDgYDVR0PAQH/BAQDAgWgMB0GA1UdJQQWMBQG
CCsGAQUFBwMBBggrBgEFBQcDAjB1BgNVHR8EbjBsMDSgMqAwhi5odHRwOi8vY3Js
My5kaWdpY2VydC5jb20vc2hhMi1oYS1zZXJ2ZXItZzYuY3JsMDSgMqAwhi5odHRw
Oi8vY3JsNC5kaWdpY2VydC5jb20vc2hhMi1oYS1zZXJ2ZXItZzYuY3JsMEwGA1Ud
IARFMEMwNwYJYIZIAYb9bAEBMCowKAYIKwYBBQUHAgEWHGh0dHBzOi8vd3d3LmRp
Z2ljZXJ0LmNvbS9DUFMwCAYGZ4EMAQICMIGDBggrBgEFBQcBAQR3MHUwJAYIKwYB
BQUHMAGGGGh0dHA6Ly9vY3NwLmRpZ2ljZXJ0LmNvbTBNBggrBgEFBQcwAoZBaHR0
cDovL2NhY2VydHMuZGlnaWNlcnQuY29tL0RpZ2lDZXJ0U0hBMkhpZ2hBc3N1cmFu
Y2VTZXJ2ZXJDQS5jcnQwDAYDVR0TAQH/BAIwADCCAQQGCisGAQQB1nkCBAIEgfUE
gfIA8AB2AKS5CZC0GFgUh7sTosxncAo8NZgE+RvfuON3zQ7IDdwQAAABZKnLxH8A
AAQDAEcwRQIhAIv5Jd1U1I8M9mEenBA9ErrFBxzldJ2useWhCpXrEOQZAiAje6IA
AmiKgAAgl7YRvRYLtuLv7Owe9wvj45+zi1HIMAB2AId1v+dZfPiMQ5lfvfNu/1aN
R1Y2/0q1YMG06v9eoIMPAAABZKnLxRkAAAQDAEcwRQIgbgidp2Ag5bmFIv1Ovpwk
228o1rv8UnWcmLEykZ6LNiYCIQD/blrpcjTwA0sVv5YcwO2bLzMYC+jZvIbFCkyJ
Zy2/ijANBgkqhkiG9w0BAQsFAAOCAQEAHOaryx+H1h/L7Wjwjyccyd8f54k18gcr
W1jRxVjnXFyT32MIV8cg/l8Oh1J/oTKe4jkVbGhzX3w+L44v9rgcUDqtQHOxizpd
ygr2s88JsjickVCS5YRaSquLJsYX8LR2g1/FoFHYUSKQq2Vwd60czltlOvJoN09k
nqOM0wvjD30AO6hmuAz0icAxKKjAZMjkQ+GkLs3sOZw670QaFOfNj/CFT8IZ7JT0
iWNlHDC6eMd0u5Kk6lY/Q84EWVD26ZZxQyO9n1jjiErtSGfbHbpgaZASsPaIFlUE
XiQmuj3SqjfQb6Iue77d6zPDbJiweOVu9IVbv73sz7PkD5gvUM73KA==
-----END CERTIFICATE-----
 1 s:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 High Assurance Server CA
   i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert High Assurance EV Root CA
-----BEGIN CERTIFICATE-----
MIIEsTCCA5mgAwIBAgIQBOHnpNxc8vNtwCtCuF0VnzANBgkqhkiG9w0BAQsFADBs
MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3
d3cuZGlnaWNlcnQuY29tMSswKQYDVQQDEyJEaWdpQ2VydCBIaWdoIEFzc3VyYW5j
ZSBFViBSb290IENBMB4XDTEzMTAyMjEyMDAwMFoXDTI4MTAyMjEyMDAwMFowcDEL
MAkGA1UEBhMCVVMxFTATBgNVBAoTDERpZ2lDZXJ0IEluYzEZMBcGA1UECxMQd3d3
LmRpZ2ljZXJ0LmNvbTEvMC0GA1UEAxMmRGlnaUNlcnQgU0hBMiBIaWdoIEFzc3Vy
YW5jZSBTZXJ2ZXIgQ0EwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC2
4C/CJAbIbQRf1+8KZAayfSImZRauQkCbztyfn3YHPsMwVYcZuU+UDlqUH1VWtMIC
Kq/QmO4LQNfE0DtyyBSe75CxEamu0si4QzrZCwvV1ZX1QK/IHe1NnF9Xt4ZQaJn1
itrSxwUfqJfJ3KSxgoQtxq2lnMcZgqaFD15EWCo3j/018QsIJzJa9buLnqS9UdAn
4t07QjOjBSjEuyjMmqwrIw14xnvmXnG3Sj4I+4G3FhahnSMSTeXXkgisdaScus0X
sh5ENWV/UyU50RwKmmMbGZJ0aAo3wsJSSMs5WqK24V3B3aAguCGikyZvFEohQcft
bZvySC/zA/WiaJJTL17jAgMBAAGjggFJMIIBRTASBgNVHRMBAf8ECDAGAQH/AgEA
MA4GA1UdDwEB/wQEAwIBhjAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIw
NAYIKwYBBQUHAQEEKDAmMCQGCCsGAQUFBzABhhhodHRwOi8vb2NzcC5kaWdpY2Vy
dC5jb20wSwYDVR0fBEQwQjBAoD6gPIY6aHR0cDovL2NybDQuZGlnaWNlcnQuY29t
L0RpZ2lDZXJ0SGlnaEFzc3VyYW5jZUVWUm9vdENBLmNybDA9BgNVHSAENjA0MDIG
BFUdIAAwKjAoBggrBgEFBQcCARYcaHR0cHM6Ly93d3cuZGlnaWNlcnQuY29tL0NQ
UzAdBgNVHQ4EFgQUUWj/kK8CB3U8zNllZGKiErhZcjswHwYDVR0jBBgwFoAUsT7D
aQP4v0cB1JgmGggC72NkK8MwDQYJKoZIhvcNAQELBQADggEBABiKlYkD5m3fXPwd
aOpKj4PWUS+Na0QWnqxj9dJubISZi6qBcYRb7TROsLd5kinMLYBq8I4g4Xmk/gNH
E+r1hspZcX30BJZr01lYPf7TMSVcGDiEo+afgv2MW5gxTs14nhr9hctJqvIni5ly
/D6q1UEL2tU2ob8cbkdJf17ZSHwD2f2LSaCYJkJA69aSEaRkCldUxPUd1gJea6zu
xICaEnL6VpPX/78whQYwvwt/Tv9XBZ0k7YXDK/umdaisLRbvfXknsuvCnQsH6qqF
0wGjIChBWUMo0oHjqvbsezt3tkBigAVBRQHvFwY+3sAzm2fTYS5yh+Rp/BIAV0Ae
cPUeybQ=
-----END CERTIFICATE-----
---
Server certificate
subject=/C=US/ST=California/L=San Francisco/O=Twitter, Inc./OU=tsa_m Point of Presence/CN=twitter.com
issuer=/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 High Assurance Server CA
---
No client certificate CA names sent
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 3534 bytes and written 326 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-GCM-SHA256
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES128-GCM-SHA256
    Session-ID: 58375F7A637D8738C8705D81D8598949753B923C66CBCD9AADC7AB02F92641D0
    Session-ID-ctx:
    Master-Key: 00EC0FFE25E5605EC545802324E90A493DB94F829814CDA8828F1309141BE13C7B8BF348FFB674F2DC60B12BBFB9A31E
    TLS session ticket lifetime hint: 129600 (seconds)
    TLS session ticket:
    0000 - d1 a8 5a f8 f7 17 60 12-a1 14 ef 55 8e fc f5 c0   ..Z...`....U....
    0010 - 67 e7 0e cb 45 f1 bb d1-b5 1d 48 83 97 28 a7 0a   g...E.....H..(..
    0020 - c6 8d 41 ab f1 45 cc c1-53 21 47 52 51 b2 65 47   ..A..E..S!GRQ.eG
    0030 - d3 9d 95 b5 80 dc 6d c7-a2 2e 6e 23 63 04 96 fc   ......m...n#c...
    0040 - 45 57 ab ce 92 ff df 62-a8 c1 7e 2e 8d 95 33 85   EW.....b..~...3.
    0050 - da a8 9b 28 be 7e f6 31-a7 8f 6d 58 64 b7 9f aa   ...(.~.1..mXd...
    0060 - 9e ca b0 e0 c0 73 3c 4f-42 21 fb ed 4a 04 ff 31   .....s<OB!..J..1
    0070 - bb 08 2d 94 f9 36 72 33-cd 5e 23 3f 1d b9 bb ab   ..-..6r3.^#?....
    0080 - 3b 7b b0 c5 f1 49 e9 f2-29 f5 91 4a 2d b0 db c9   ;{...I..)..J-...
    0090 - c9 5a b8 11 c3 17 be a3-52 70 ed a3 45 22 3c de   .Z......Rp..E"<.

    Start Time: 1544866734
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
---
closed



~> openssl s_client -connect www.facebook.com:443 -showcerts
CONNECTED(00000005)
depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert High Assurance EV Root CA
verify return:1
depth=1 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert SHA2 High Assurance Server CA
verify return:1
depth=0 C = US, ST = California, L = Menlo Park, O = "Facebook, Inc.", CN = *.facebook.com
verify return:1
---
Certificate chain
 0 s:/C=US/ST=California/L=Menlo Park/O=Facebook, Inc./CN=*.facebook.com
   i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 High Assurance Server CA
-----BEGIN CERTIFICATE-----
MIIGsjCCBZqgAwIBAgIQCzw7YBoY9Z7itrsFYF7ywDANBgkqhkiG9w0BAQsFADBw
MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3
d3cuZGlnaWNlcnQuY29tMS8wLQYDVQQDEyZEaWdpQ2VydCBTSEEyIEhpZ2ggQXNz
dXJhbmNlIFNlcnZlciBDQTAeFw0xNzEyMTUwMDAwMDBaFw0xOTAzMjIxMjAwMDBa
MGkxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpDYWxpZm9ybmlhMRMwEQYDVQQHEwpN
ZW5sbyBQYXJrMRcwFQYDVQQKEw5GYWNlYm9vaywgSW5jLjEXMBUGA1UEAwwOKi5m
YWNlYm9vay5jb20wWTATBgcqhkjOPQIBBggqhkjOPQMBBwNCAASIA87IjqqM6JBX
puN20BXCVsDjoP9wnF2rSV60qC130oLTrgfOQ3Uk1dv1R6LFCx4gs2pJUu6iDKBS
/b+BXOUbo4IEGDCCBBQwHwYDVR0jBBgwFoAUUWj/kK8CB3U8zNllZGKiErhZcjsw
HQYDVR0OBBYEFMD9dPV9y8Yn8QPTYqJF14QcFSEIMIHHBgNVHREEgb8wgbyCDiou
ZmFjZWJvb2suY29tgg4qLnh4LmZiY2RuLm5ldIILKi5mYnNieC5jb22CDioueHou
ZmJjZG4ubmV0gg4qLmZhY2Vib29rLm5ldIIOKi54eS5mYmNkbi5uZXSCDyoubWVz
c2VuZ2VyLmNvbYIGZmIuY29tggsqLmZiY2RuLm5ldIIIKi5mYi5jb22CECoubS5m
YWNlYm9vay5jb22CDW1lc3Nlbmdlci5jb22CDGZhY2Vib29rLmNvbTAOBgNVHQ8B
Af8EBAMCB4AwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMHUGA1UdHwRu
MGwwNKAyoDCGLmh0dHA6Ly9jcmwzLmRpZ2ljZXJ0LmNvbS9zaGEyLWhhLXNlcnZl
ci1nNi5jcmwwNKAyoDCGLmh0dHA6Ly9jcmw0LmRpZ2ljZXJ0LmNvbS9zaGEyLWhh
LXNlcnZlci1nNi5jcmwwTAYDVR0gBEUwQzA3BglghkgBhv1sAQEwKjAoBggrBgEF
BQcCARYcaHR0cHM6Ly93d3cuZGlnaWNlcnQuY29tL0NQUzAIBgZngQwBAgIwgYMG
CCsGAQUFBwEBBHcwdTAkBggrBgEFBQcwAYYYaHR0cDovL29jc3AuZGlnaWNlcnQu
Y29tME0GCCsGAQUFBzAChkFodHRwOi8vY2FjZXJ0cy5kaWdpY2VydC5jb20vRGln
aUNlcnRTSEEySGlnaEFzc3VyYW5jZVNlcnZlckNBLmNydDAMBgNVHRMBAf8EAjAA
MIIBfgYKKwYBBAHWeQIEAgSCAW4EggFqAWgAdgCkuQmQtBhYFIe7E6LMZ3AKPDWY
BPkb37jjd80OyA3cEAAAAWBXnEHoAAAEAwBHMEUCIBC3Rn4i2bhLyR344u3vl7be
vxoi+WPJBhGT+j1gJmg5AiEAwQ3rzH1mmMSYNYKtVNDZMo+l6e8Z35t+X9NDR7Du
gWAAdwCHdb/nWXz4jEOZX73zbv9WjUdWNv9KtWDBtOr/XqCDDwAAAWBXnEL7AAAE
AwBIMEYCIQCRjvvPARW3J1ENmo2Nz1cxisa1BcbDuqvSrfuXkz8btAIhAPmllqgF
8JjlVHUChiFzghsKVBeTxRagi55tgsAciaoZAHUAu9nfvB+KcbWTlCOXqpJ7RzhX
lQqrUugakJZkNo4e0YUAAAFgV5xCUgAABAMARjBEAiBY6qdNgMoQAqVTl3zRrTmy
+X/1f/esBUczsb3MWdZ1ZgIgXdxZNTrDBgyTzxgbVRObkqU3tZZdaiwsw4WI0xI0
BtQwDQYJKoZIhvcNAQELBQADggEBAGu0uxZD+IRXXlFWLPvknRkXA7J08NyVKG70
M2vDi2xF2YB8qlZgoxW8YiiV86IpwtOhYLZinSO0iCBDQmTf627LTPfuDcF6qOuO
WFTvj1IbplPvGWIu5tNBiFWNQxFAIL2Rf+5vmIe+YezUHTLGGqwRtFa2ImS17IMk
YjZ90LYXXO5qb1RKkFJtAvEBTbJsv8kr+J6Rx+YNJy17LnBX+MbWiyBbvUQoM3sY
MmcWmcaQmECz9ZHWYjZeufSHbHKG6KDYLU8x6DyhgtxK2rsoIMlNnJkNHaLjw+b8
7VCYa+EMWppvVuNyXOk9Jkbx7Q3SEoodT77kkHUX0bF2OkZy6cc=
-----END CERTIFICATE-----
 1 s:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 High Assurance Server CA
   i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert High Assurance EV Root CA
-----BEGIN CERTIFICATE-----
MIIEsTCCA5mgAwIBAgIQBOHnpNxc8vNtwCtCuF0VnzANBgkqhkiG9w0BAQsFADBs
MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3
d3cuZGlnaWNlcnQuY29tMSswKQYDVQQDEyJEaWdpQ2VydCBIaWdoIEFzc3VyYW5j
ZSBFViBSb290IENBMB4XDTEzMTAyMjEyMDAwMFoXDTI4MTAyMjEyMDAwMFowcDEL
MAkGA1UEBhMCVVMxFTATBgNVBAoTDERpZ2lDZXJ0IEluYzEZMBcGA1UECxMQd3d3
LmRpZ2ljZXJ0LmNvbTEvMC0GA1UEAxMmRGlnaUNlcnQgU0hBMiBIaWdoIEFzc3Vy
YW5jZSBTZXJ2ZXIgQ0EwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC2
4C/CJAbIbQRf1+8KZAayfSImZRauQkCbztyfn3YHPsMwVYcZuU+UDlqUH1VWtMIC
Kq/QmO4LQNfE0DtyyBSe75CxEamu0si4QzrZCwvV1ZX1QK/IHe1NnF9Xt4ZQaJn1
itrSxwUfqJfJ3KSxgoQtxq2lnMcZgqaFD15EWCo3j/018QsIJzJa9buLnqS9UdAn
4t07QjOjBSjEuyjMmqwrIw14xnvmXnG3Sj4I+4G3FhahnSMSTeXXkgisdaScus0X
sh5ENWV/UyU50RwKmmMbGZJ0aAo3wsJSSMs5WqK24V3B3aAguCGikyZvFEohQcft
bZvySC/zA/WiaJJTL17jAgMBAAGjggFJMIIBRTASBgNVHRMBAf8ECDAGAQH/AgEA
MA4GA1UdDwEB/wQEAwIBhjAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIw
NAYIKwYBBQUHAQEEKDAmMCQGCCsGAQUFBzABhhhodHRwOi8vb2NzcC5kaWdpY2Vy
dC5jb20wSwYDVR0fBEQwQjBAoD6gPIY6aHR0cDovL2NybDQuZGlnaWNlcnQuY29t
L0RpZ2lDZXJ0SGlnaEFzc3VyYW5jZUVWUm9vdENBLmNybDA9BgNVHSAENjA0MDIG
BFUdIAAwKjAoBggrBgEFBQcCARYcaHR0cHM6Ly93d3cuZGlnaWNlcnQuY29tL0NQ
UzAdBgNVHQ4EFgQUUWj/kK8CB3U8zNllZGKiErhZcjswHwYDVR0jBBgwFoAUsT7D
aQP4v0cB1JgmGggC72NkK8MwDQYJKoZIhvcNAQELBQADggEBABiKlYkD5m3fXPwd
aOpKj4PWUS+Na0QWnqxj9dJubISZi6qBcYRb7TROsLd5kinMLYBq8I4g4Xmk/gNH
E+r1hspZcX30BJZr01lYPf7TMSVcGDiEo+afgv2MW5gxTs14nhr9hctJqvIni5ly
/D6q1UEL2tU2ob8cbkdJf17ZSHwD2f2LSaCYJkJA69aSEaRkCldUxPUd1gJea6zu
xICaEnL6VpPX/78whQYwvwt/Tv9XBZ0k7YXDK/umdaisLRbvfXknsuvCnQsH6qqF
0wGjIChBWUMo0oHjqvbsezt3tkBigAVBRQHvFwY+3sAzm2fTYS5yh+Rp/BIAV0Ae
cPUeybQ=
-----END CERTIFICATE-----
---
Server certificate
subject=/C=US/ST=California/L=Menlo Park/O=Facebook, Inc./CN=*.facebook.com
issuer=/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 High Assurance Server CA
---
No client certificate CA names sent
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 3412 bytes and written 326 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-ECDSA-AES128-GCM-SHA256
Server public key is 256 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-ECDSA-AES128-GCM-SHA256
    Session-ID: FCB725472E0112F95A596012CDD55AB38A73CE46BABF6226978651E9F259C609
    Session-ID-ctx:
    Master-Key: A35C3023DE520684DA23B557656DDC3478FCD3C66B5BA677F62B605BCC4CD395C085D3A980587DE72D9C6FFCA988F5F4
    TLS session ticket lifetime hint: 172800 (seconds)
    TLS session ticket:
    0000 - e1 ec 9e 74 56 16 6a 55-7a 80 74 3d 2e 44 3e 0d   ...tV.jUz.t=.D>.
    0010 - 5f c0 6f 4a 8c d8 ec 2a-2b 27 91 29 de 04 3d 49   _.oJ...*+'.)..=I
    0020 - f5 ae 40 d6 86 3e 3c 29-66 7b 3f ed c5 ad 80 b1   ..@..><)f{?.....
    0030 - bb 3e 0b b1 67 6b 07 4b-ac 9f 30 f3 9f 8b ba 3b   .>..gk.K..0....;
    0040 - bf 51 24 e7 25 19 74 d2-c2 b1 c6 12 cb 50 dd 10   .Q$.%.t......P..
    0050 - 4f 08 7f 12 36 de 4e 8d-50 05 32 99 71 e4 a0 aa   O...6.N.P.2.q...
    0060 - a6 35 30 5e ed a9 f1 f5-b4 59 46 62 60 d6 5b 9a   .50^.....YFb`.[.
    0070 - 66 2d 6b 1f 69 ad b4 d3-52 b2 3e 83 16 92 93 38   f-k.i...R.>....8
    0080 - 59 70 9c 4c e7 f1 a4 e0-89 d4 6e 9b 47 f6 b5 be   Yp.L......n.G...
    0090 - bd 60 9c a1 3e ae 1d f1-0e d6 43 cb 0a 61 37 5d   .`..>.....C..a7]
    00a0 - 9f f5 6d 46 e8 8a 75 3e-ea b6 8f c6 02 5e 67 1a   ..mF..u>.....^g.

    Start Time: 1544866854
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
---
closed


Thanks and best regards,
Siegfried


> On Dec 14, 2018, at 1:59 PM, Zhi-Qiang Lei <[hidden email]> wrote:
>
> I’m having the same issue on OpenBSD 6.4. My iked.conf is similar to Rachel’s:
>
> include "/etc/iked/macros.conf"
>
> ikev2 quick active ipcomp esp proto gre\
>        from 192.168.1.0/24 to $iked_server \
>        local egress peer $iked_server \
>        ikesa auth hmac-sha2-512 enc aes-256 prf hmac-sha2-512 group curve25519 \
>        childsa enc chacha20-poly130 group curve25519 \
>        dstid "asgard.local"
>
> Sites are reachable by ping, but https doesn’t respond at all. I was wondering if it is because the GRE part, will do more experiments.
>
>> On Dec 11, 2018, at 9:04 AM, Theodore Wynnychenko <[hidden email]> wrote:
>>
>> I would like to re-title this as something like "pf and iked instability on recent snapshots," but don’t know if doing so would break the mailing list thread, exiso, I left the subject unchanged...
>>
>>> -----Original Message-----
>>> From: Theodore Wynnychenko [mailto:[hidden email]]
>>> Sent: Saturday, December 08, 2018 4:03 PM
>>> To: [hidden email]
>>> Cc: 'Rachel Roch'
>>> Subject: RE: TLS suddenly not working over IKED site-to-site
>>>
>>>>
>> .
>> .
>> .
>>> I now find I can no longer connect to with TLS/SSL over the iked tunnel
>>> (the original behavior that seemed to have corrected itself).  Also,
>>> icinga continues to be unable to verify the status of the remote hosts
>>> over port 5665.
>>>
>>> I don't have time right now to try using s_client and s_server and
>>> watching enc0 to see what is happening, but I will when I can.
>>>
>>> If anyone has an ideas on what may be happening, please let me know.
>>>
>>> Thanks
>>> Ted
>>
>>
>> Hello again;
>>
>> So, I am at a complete loss to understand what is going on.
>> Today, I tried using openssl s_client and s_server to make a connection through the iked vpn (as I described in my last post).  However, with NO changes to iked.conf or pf.conf, today I had several connection attempts that completed correctly.  I have not included any output from those sporadic, completely functional connections.
>>
>> Rather, today, most of the connections by s_client are not even acknowledged by the s_server on the other side of the iked vpn.
>>
>> For example:
>> On the s_client machine:
>>
>> # openssl s_client -state -connect "remote.host":https
>> SSL_connect:before/connect initialization
>> SSL_connect:SSLv3 write client hello A
>> ... and nothing more ...
>>
>> But on the s_server machine today all I see is:
>> # openssl s_sever -state -accept https ...certificate options...
>> Using auto DH parameters
>> Using default temp ECDH parameters
>> ACCEPT
>> ... and no connection attempt is ever acknowledged ...
>>
>> (Yesterday, at least this first part of the connection was received by the s_server:
>> Using auto DH parameters
>> Using default temp ECDH parameters
>> ACCEPT
>> SSL_accept:before/accept initialization
>> ... and nothing more yesterday ...)
>>
>>
>> So, today using tcpdump on the outgoing interface of the s_client machine and the incoming interface of the "local" iked vpn endpoint shows:
>>
>> 16:43:05.107524 172.30.1.254.7305 > 172.30.7.205.443: S 1751796302:1751796302(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 6,nop,nop,timestamp 2698316052 0>
>>
>> 16:43:05.149146 172.30.1.254.7305 > 172.30.7.205.443: . ack 2119500805 win 256 <nop,nop,timestamp 2698316052 3536824996>
>>
>> 16:43:05.149895 172.30.1.254.7305 > 172.30.7.205.443: P 0:196(196) ack 1 win 256 <nop,nop,timestamp 2698316052 3536824996>
>>
>> 16:43:06.648487 172.30.1.254.7305 > 172.30.7.205.443: P 0:196(196) ack 1 win 256 <nop,nop,timestamp 2698316055 3536824996>
>>
>> 16:43:09.648557 172.30.1.254.7305 > 172.30.7.205.443: P 0:196(196) ack 1 win 256 <nop,nop,timestamp 2698316061 3536824996>
>>
>> 16:43:09.948433 172.30.1.254.7305 > 172.30.7.205.443: F 196:196(0) ack 1 win 256 <nop,nop,timestamp 2698316061 3536824996>
>>
>> 16:43:15.648712 172.30.1.254.7305 > 172.30.7.205.443: FP 0:196(196) ack 1 win 256 <nop,nop,timestamp 2698316073 3536825005>
>>
>> And this traffic (incomplete thought it may be for an ssl handshake) appears to be passed to enc0 intact:
>>
>> 16:43:05.105044 (authentic,confidential): SPI 0x151333df: 172.30.1.254.7305 > 172.30.7.205.443: S 3570513915:3570513915(0) win 16384 <mss 1300,nop,nop,sackOK,nop,wscale 6,nop,nop,timestamp 2698316052 0> (encap)
>>
>> 16:43:05.146122 (authentic,confidential): SPI 0xe1c30e4a: 172.30.7.205.443 > 172.30.1.254.7305: S 1312941075:1312941075(0) ack 3570513916 win 16384 <mss 1300,nop,nop,sackOK,nop,wscale 6,nop,nop,timestamp 3536824996 2698316052> (encap)
>>
>> 16:43:05.146654 (authentic,confidential): SPI 0x151333df: 172.30.1.254.7305 > 172.30.7.205.443: . ack 1 win 256 <nop,nop,timestamp 2698316052 3536824996> (encap)
>>
>> 16:43:05.147365 (unprotected): SPI 0x0000ef27: 172.30.1.254.7305 > 172.30.7.205.443: P 1:197(196) ack 1 win 256 <nop,nop,timestamp 2698316052 3536824996> (encap)
>>
>> 16:43:06.645932 (unprotected): SPI 0x0000ef27: 172.30.1.254.7305 > 172.30.7.205.443: P 1:197(196) ack 1 win 256 <nop,nop,timestamp 2698316055 3536824996> (encap)
>>
>> 16:43:09.646049 (unprotected): SPI 0x0000ef27: 172.30.1.254.7305 > 172.30.7.205.443: P 1:197(196) ack 1 win 256 <nop,nop,timestamp 2698316061 3536824996> (encap)
>>
>> 16:43:09.945908 (authentic,confidential): SPI 0x151333df: 172.30.1.254.7305 > 172.30.7.205.443: F 197:197(0) ack 1 win 256 <nop,nop,timestamp 2698316061 3536824996> (encap)
>>
>> 16:43:09.981966 (authentic,confidential): SPI 0xe1c30e4a: 172.30.7.205.443 > 172.30.1.254.7305: . ack 1 win 261 <nop,nop,timestamp 3536825005 2698316052,nop,nop,sack 1 {197:197} > (encap)
>>
>> 16:43:15.646158 (unprotected): SPI 0x0000ef27: 172.30.1.254.7305 > 172.30.7.205.443: FP 1:197(196) ack 1 win 256 <nop,nop,timestamp 2698316073 3536825005> (encap)
>>
>>
>> BUT, at the other end of the VPN, on enc0, all that is seen leaving the iked VPN tunnel is:
>>
>> 16:43:05.130558 (authentic,confidential): SPI 0x151333df: 172.30.1.254.7305 > 172.30.7.205.443: S 3570513915:3570513915(0) win 16384 <mss 1300,nop,nop,sackOK,nop,wscale 6,nop,nop,timestamp 2698316052 0> (encap)
>>
>> 16:43:05.131049 (authentic,confidential): SPI 0xe1c30e4a: 172.30.7.205.443 > 172.30.1.254.7305: S 1312941075:1312941075(0) ack 3570513916 win 16384 <mss 1300,nop,nop,sackOK,nop,wscale 6,nop,nop,timestamp 3536824996 2698316052> (encap)
>>
>> 16:43:05.174802 (authentic,confidential): SPI 0x151333df: 172.30.1.254.7305 > 172.30.7.205.443: . ack 1 win 256 <nop,nop,timestamp 2698316052 3536824996> (encap)
>>
>> 16:43:09.966420 (authentic,confidential): SPI 0x151333df: 172.30.1.254.7305 > 172.30.7.205.443: F 197:197(0) ack 1 win 256 <nop,nop,timestamp 2698316061 3536824996> (encap)
>>
>> 16:43:09.966853 (authentic,confidential): SPI 0xe1c30e4a: 172.30.7.205.443 > 172.30.1.254.7305: . ack 1 win 261 <nop,nop,timestamp 3536825005 2698316052,nop,nop,sack 1 {197:197} > (encap)
>>
>>
>> I have no idea what this all means, or what to do with it.
>> But, I am following up in case anybody has any idea of what may be happening.
>>
>> Also, yesterday I described how the local iked machine appeared to be blocking packets that were explicitly allowed by pf.conf.  From my post yesterday:
>>
>> (   For example, in the log I see:
>> Dec  8 15:50:01 ... pf: Dec 08 15:48:49.346816 rule 4/(match) block out on em0: 172.30.7.205.22112 > 172.30.2.99.5665: R 3963276584:3963276584(0) ack 252894831 win 0
>>
>> But, pfctl is running with following:
>>
>> # pfctl -s rules
>> match in all scrub (no-df random-id max-mss 1300)
>> pass in quick on em1 all flags S/SA
>> pass out quick on em1 all flags S/SA
>> block drop in log on em0 all
>> block drop out log on em0 all
>> ...
>> pass quick inet proto tcp from 172.30.7.205 to 172.30.2.99 port = 5665 flags S/SA
>> ... and on.    )
>>
>> Well, whatever was happening appears to have been resolved, because at about midnight local time on Sunday morning, icinga2 declared that the host was back up.
>>
>> To be clear, I have made no changes to either pf.conf or iked.conf on any of the machines involved in this testing from Saturday.
>>
>> Also, this had all been stable for the last (about) 2 years, until about two-three weeks ago.  I did have another post, where I discussed the fact the iked VPN had failed to be reestablished after an update about 3-4 snapshots back.  I got it working again by changing the local endpoint on the "remote" iked machine from the internal ip associated with the internal interface to an internal "alias" ip address associated with the outgoing/external interface of that machine.
>>
>> But, again, it had been working for 2 years until the recent update.
>>
>> I don't have any idea of what may be helpful in figuring out what I am doing wrong, or what has changed, but I am happy to provide any information that may be of help.
>>
>> I don't believe I have the knowledge to do more on my own at this point.
>>
>> Thanks for any advice.
>> Ted
>>
>>
>>
>>
>

Reply | Threaded
Open this post in threaded view
|

Re: TLS suddenly not working over IKED site-to-site

Theodore Wynnychenko-2
In reply to this post by Rachel Roch
Hello again:

I updated my iked endpoints to the most recent (12/14/18) amd64 snapshot today, and am still having problems with secure connections.

So, today, I am just looking at the gateway machines.

The iked vpn tunnel gets established without an issue.

# ipsecctl -s all
FLOWS:
flow esp in from xx.xx.xx.xx to 172.30.1.20 peer xx.xx.xx.xx srcid FQDN/x.x.x dstid FQDN/x.x.x type use
flow ipcomp in from 172.30.6.0/23 to 172.30.0.0/22 peer xx.xx.xx.xx srcid FQDN/x.x.x dstid FQDN/x.x.x type use
flow ipcomp out from 172.30.0.0/22 to 172.30.6.0/23 peer xx.xx.xx.xx srcid FQDN/x.x.x dstid FQDN/x.x.x type require
flow esp out from 172.30.1.20 to xx.xx.xx.xx peer xx.xx.xx.xx srcid FQDN/x.x.x com dstid FQDN/x.x.x type require

SAD:
ipcomp tunnel from 172.30.1.20 to xx.xx.xx.xx spi 0x000054ab comp deflate
ipcomp tunnel from xx.xx.xx.xx to 172.30.1.20 spi 0x00008bf1 comp deflate
esp tunnel from 172.30.1.20 to xx.xx.xx.xx spi 0x0934ef93 auth hmac-sha2-256 enc aes-256
esp transport from xx.xx.xx.xx to 172.30.1.20 spi 0x29a95d18 auth hmac-sha2-256 enc aes-256
esp transport from 172.30.1.20 to xx.xx.xx.xx spi 0x7b90f84c auth hmac-sha2-256 enc aes-256
esp tunnel from xx.xx.xx.xx to 172.30.1.20 spi 0xa9238cfb auth hmac-sha2-256 enc aes-256


I have also added "set skip on { lo enc0 }" to pf.conf on both gateways (trying to take that out of the equation).


If, on the remote gateway I run:
# nc -l https

Then, when, on the local gateway I run:
# nc remote.gateway https
(( and enter "some text" and <CR> ))

On the remote gateway I see:
"some text"

OK, traffic is flowing from one to the other and allowed on https.  Right?

If I run s_server on the remote gateway:
# openssl s_server -state -accept https -CAfile /etc/ssl/cert.pem -cert /etc/ssl/server.crt -key /etc/ssl/private/server.key

The s_server starts:
Using auto DH parameters
Using default temp ECDH parameters
ACCEPT

When I try connecting with s_client on the local gateway:
# openssl s_client -state -connect remote.gateway:https

All I see is:
CONNECTED(00000003)
SSL_connect:before/connect initialization
SSL_connect:SSLv3 write client hello A


And there is no output on the server side (the remote gateway).

At the same time, tcpdump of enc0 shows:

On the local gateway:

17:37:00.199269 (authentic,confidential): SPI 0x7b90f84c: 172.30.1.20.20692 > 172.30.6.201.443: S 3823001077:3823001077(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 6,nop,nop,timestamp 48604571 0> (encap)

17:37:00.235313 (authentic,confidential): SPI 0x29a95d18: 172.30.6.201.443 > 172.30.1.20.20692: S 833135398:833135398(0) ack 3823001078 win 16384 <mss 65496,nop,nop,sackOK,nop,wscale 6,nop,nop,timestamp 730029501 48604571> (DF) (encap)

17:37:00.235335 (authentic,confidential): SPI 0x7b90f84c: 172.30.1.20.20692 > 172.30.6.201.443: . ack 1 win 256 <nop,nop,timestamp 48604571 730029501> (encap)

17:37:00.236086 (unprotected): SPI 0x000054ab: 172.30.1.20.20692 > 172.30.6.201.443: P 1:197(196) ack 1 win 256 <nop,nop,timestamp 48604571 730029501> (encap)

17:37:01.726509 (unprotected): SPI 0x000054ab: 172.30.1.20.20692 > 172.30.6.201.443: P 1:197(196) ack 1 win 256 <nop,nop,timestamp 48604574 730029501> (encap)

17:37:04.726571 (unprotected): SPI 0x000054ab: 172.30.1.20.20692 > 172.30.6.201.443: P 1:197(196) ack 1 win 256 <nop,nop,timestamp 48604580 730029501> (encap)

17:37:07.777123 (authentic,confidential): SPI 0x7b90f84c: 172.30.1.20.20692 > 172.30.6.201.443: F 197:197(0) ack 1 win 256 <nop,nop,timestamp 48604586 730029501> (encap)

17:37:07.820525 (authentic,confidential): SPI 0x29a95d18: 172.30.6.201.443 > 172.30.1.20.20692: . ack 1 win 271 <nop,nop,timestamp 730029516 48604571,nop,nop,sack 1 {197:197} > (DF) (encap)

17:37:10.726697 (unprotected): SPI 0x000054ab: 172.30.1.20.20692 > 172.30.6.201.443: FP 1:197(196) ack 1 win 256 <nop,nop,timestamp 48604592 730029516> (encap)

17:37:11.724643 (authentic,confidential): SPI 0x29a95d18: 172.30.6.201.443 > 172.30.1.20.20692: F 1:1(0) ack 1 win 271 <nop,nop,timestamp 730029524 48604571> (DF) (encap)

17:37:11.724680 (authentic,confidential): SPI 0x7b90f84c: 172.30.1.20.20692 > 172.30.6.201.443: F 197:197(0) ack 2 win 256 <nop,nop,timestamp 48604594 730029524> (encap)

17:37:11.759035 (authentic,confidential): SPI 0x29a95d18: 172.30.6.201.443 > 172.30.1.20.20692: . ack 1 win 271 <nop,nop,timestamp 730029524 48604571> (DF) (encap)


But, tcpdump of enc0 on the remote gateway only shows:

17:37:00.207517 (authentic,confidential): SPI 0x7b90f84c: 172.30.1.20.20692 > 172.30.6.201.443: S 3823001077:3823001077(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 6,nop,nop,timestamp 48604571 0> (encap)

17:37:00.207551 (authentic,confidential): SPI 0x29a95d18: 172.30.6.201.443 > 172.30.1.20.20692: S 833135398:833135398(0) ack 3823001078 win 16384 <mss 65496,nop,nop,sackOK,nop,wscale 6,nop,nop,timestamp 730029501 48604571> (DF) (encap)

17:37:00.244461 (authentic,confidential): SPI 0x7b90f84c: 172.30.1.20.20692 > 172.30.6.201.443: . ack 1 win 256 <nop,nop,timestamp 48604571 730029501> (encap)

17:37:07.792702 (authentic,confidential): SPI 0x7b90f84c: 172.30.1.20.20692 > 172.30.6.201.443: F 197:197(0) ack 1 win 256 <nop,nop,timestamp 48604586 730029501> (encap)

17:37:07.792724 (authentic,confidential): SPI 0x29a95d18: 172.30.6.201.443 > 172.30.1.20.20692: . ack 1 win 271 <nop,nop,timestamp 730029516 48604571,nop,nop,sack 1 {197:197} > (DF) (encap)


Needless to say, if I try these connections WITHOUT traversing the iked vpn, the openssl s_client to s_server connection works as expected.

Not being in any way an expert on these things, I cannot understand why "regular" tcp packets are able to traverse the VPN and emerge on the other side, while tcp pakets to the same port attempting to establish a secure connection are lost.  Why are the tcp PUSH data packets that leave the local gateway on enc0 never seen on the remote gateway's ecn0?


I am really hoping that somebody has some sort idea of what I can do to fix whatever it is that I have broken.  I will be happy to send more specific information, I just don't know what that might be.

Thanks again
Ted