ikedv2 + rdomains + nat = tcp works, udp doesn't

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

ikedv2 + rdomains + nat = tcp works, udp doesn't

Hi everyone! I like to play with all the cool toys the devs give us,
because, you know, they are there, and it helps me learn. One of my
favorite walls to bang my head against is automatically connecting my
(OpenBSD-stable) laptop to the internet and automatically keeping it
connected as I open and close my laptop, change locations, etc. To make
this more of a challenge, I have wlan, wwan, and ethernet connections to
choose from (ethernet > wlan > wwan), and I like to connect via some
sort of VPN so I can be at the airport or a cafe and not worry too much
about the local folks listening in on my traffic.

I have all this working fine using ifstated, some scripts, and SSH based
VPN, though the CPU usage is a bit high and its not without hiccups.
Anyhow, I thought I'd try out a different way of accomplishing a similar
effect and see what happens. This time around I thought I'd try putting
my connections in separate routing domains so I can test their
connectivity separately and switch between them quickly. Then I layer
some PF config to NAT traffic from rdomain 0 to the rdomain I want to
use for internet access, i.e. rdomain 1 is wwan, rdomain 2 is wlan. That
all works great. As Darth Vader would say, all too easy.

So then I tried connecting to my vpn server (running OpenBSD) using
ikedv2 on rdomain 0 and that works great too. I have another NAT rule in
pf.conf to send traffic over the ipsec flow that ikedv2 negotiated. Ping
works, TCP requests to websites work, but UDP based DNS lookups do not.
I'm using a local unbound instance for DNS lookups, so I can work around
my lack of UDP-ness by configuring it to do TCP based lookups, but I'm
wondering if anyone might have some idea why TCP works with this setup
but UDP does not.

Some IPs and MAC addresses replaced with consistent, unique, obvious
fakes to protect the innocent without, hopefully, interfering with the
usefulness of the logs:

# cat /etc/pf.conf
match in all scrub (no-df random-id max-mss 1440)
pass out on egress to !egress:network nat-to (athn0:0) rtable 2
pass out on enc0 from vether0 nat-to vether0:0

Note: that IS the entire pf.conf, I'm passing everything while I test
this configuration out.

# cat /etc/iked.conf
ikev2 "vpn" active ipcomp \
    from egress to \
    peer 104.xxx.xxx.xxx \
    srcid client.ggr.com \
    tag IKED

# cat /etc/iked.conf (on vpn server)
ikev2 "vpn" ipcomp \
    from to \
    from to \
    from to \
    peer any \
    srcid server.ggr.com \
    tag IKED

$ ifconfig
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 32768
        index 4 priority 0 llprio 3
        groups: lo
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4
        inet netmask 0xff000000
em0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:zz:zz:zz:zz:zz
        index 1 priority 0 llprio 3
        media: Ethernet autoselect (none)
        status: no carrier
athn0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> rdomain 2 mtu 1500
        index 2 priority 4 llprio 3
        lladdr 00:yy:yy:yy:yy:yy
        index 2 priority 4 llprio 3
        groups: wlan
        media: IEEE802.11 autoselect (OFDM36 mode 11a)
        status: active
        ieee80211: nwid MyFakeNetwork chan 153 bssid f0:mm:mm:mm:mm:mm 53dBm wpakey <not displayed> wpaprotos wpa1,wpa2 wpaakms psk wpaciphers tkip,ccmp wpagroupcipher tkip
        inet netmask 0xffffff00 broadcast
enc0: flags=0<>
        index 3 priority 0 llprio 3
        groups: enc
        status: active
vether0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        lladdr fe:ii:ii:ii:ii:ii
        index 7 priority 0 llprio 3
        groups: vether egress
        media: Ethernet autoselect
        status: active
        inet netmask 0xffffff00 broadcast
pflog0: flags=141<UP,RUNNING,PROMISC> mtu 33144
        index 8 priority 0 llprio 3
        groups: pflog
ppp0: flags=8010<POINTOPOINT,MULTICAST> rdomain 1 mtu 1500
        index 27 priority 0 llprio 3
        groups: ppp

$ route -T 0 -n show -inet
Routing tables

Destination        Gateway            Flags   Refs      Use   Mtu  Prio Iface
default        UGS    146528   860228     -     8 vether0          UHl       42   115625 32768     1 lo0
192.168.211/24      UC         0        0     -     4 vether0      fe:ii:ii:ii:ii:ii  UHLl       1   367197     -     1 vether0      UHb        0        0     -     1 vether0

$ route -T 2 -n show -inet
Routing tables

Destination        Gateway            Flags   Refs      Use   Mtu  Prio Iface
default             UGS      238    54110     -    12 athn0
10.0.0/24         UC         1      795     -     4 athn0           64:kk:kk:kk:kk:kk  UHLc       1      275     -     4 athn0         00:yy:yy:yy:yy:yy  UHLl       0        6     -     1 athn0         UHb        0       18     -     1 athn0

$ route -T 2 exec host google.com
Using domain server:

google.com has address
google.com has IPv6 address 2607:f8b0:4006:801::200e
google.com mail is handled by 50 alt4.aspmx.l.google.com.
google.com mail is handled by 20 alt1.aspmx.l.google.com.
google.com mail is handled by 10 aspmx.l.google.com.
google.com mail is handled by 30 alt2.aspmx.l.google.com.
google.com mail is handled by 40 alt3.aspmx.l.google.com.

# tcpdump -vvopn -i athn0
07:53:24.518765 > [udp sum ok] 58253+ A? google.com. (28) (ttl 64, id 8337, len 56)
07:53:24.545896 > [udp sum ok] 58253 q: A? google.com.1/0/0 google.com. A (44) (ttl 45, id 16165, len 72)
07:53:24.546606 > [udp sum ok] 63849+ AAAA? google.com. (28) (ttl 64, id 1415, len 56)
07:53:24.578293 > [udp sum ok] 63849 q: AAAA? google.com. 1/0/0 google.com. AAAA 2607:f8b0:4006:801::200e (56) (ttl 45, id 4970, len 84)
07:53:24.578567 > [udp sum ok] 59450+ MX? google.com. (28) (ttl 64, id 20803, len 56)
07:53:24.604558 > 59450 q: MX? google.com. 5/0/0 google.com. MX alt4.aspmx.l.google.com. 50, google.com. MX[|domain] (ttl 45, id 10095, len 164)

$ route -T 0 exec host google.com
;; connection timed out; no servers could be reached

# tcpdump -vvopn -i enc0
07:50:36.745230 (authentic,confidential): SPI 0x5b1985a0: > 104.xxx.xxx.xxx: > [udp sum ok] 6983+ A? google.com. (28) (ttl 64, id 42910, len 56) (ttl 64, id 21277, len 76, bad ip cksum 0! -> 7436)
07:50:36.770427 (authentic,confidential): SPI 0xa4554845: 104.xxx.xxx.xxx > > [udp sum ok] 6983 q: A? google.com. 1/0/0 google.com. A (44) (ttl 45, id 15649, len 72) (ttl 55, id 21022, len 92)

To my untrained eye, it looks like the DNS lookup is going across the
ipsec flow ok, both request and reply, but for some reason I'm not
processing the reply correctly.