Broken handling of outgoing ESP packets with divert(4)

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Broken handling of outgoing ESP packets with divert(4)

Thomas Klute
>Synopsis: Broken handling of outgoing ESP packets with divert(4)
>Category: kernel
>Environment:
        System      : OpenBSD 6.0
        Details     : OpenBSD 6.0-current (LOCAL) #0: Mon Jan  2 16:25:32 CET 2017
                         [hidden email]:/usr/src/sys/arch/amd64/compile/LOCAL

        Architecture: OpenBSD.amd64
        Machine     : amd64
>Description:
Outgoing ESP packets that are captured and reinjected via a divert(4)
socket end up with a sequence number that is one higher than they
should have, effectively skipping every second possible sequence
number. Similarly, if the program attached to the divert socket is
manipulating the SPI field (I'm doing this in a test environment to see
how the peer reacts to unknown SPIs), the SPI is reset to the correct
value after reinjection. Apparently the reinjected packets undergo ESP
processing again, which should not happen.

It looks like a similar bug affecting incoming ESP packets was fixed in
2013: https://marc.info/?l=openbsd-tech&m=136467523323717&w=2

I have tested only ESP SAs set up with iked(8) (see How-To-Repeat), but
since the bug appears to be in the ESP stack I assume the mechanism used
to set up the ESP SA does not matter.

>How-To-Repeat:
Prerequisite: A working iked(8) configuration using ESP (or equivalent,
modify step 5 as needed).

1. Compile the example code from the divert(4) man page.

2. Load PF rules like these to catch IPsec and IKE traffic:
pass out quick inet proto udp from port {isakmp, ipsec-nat-t}
divert-packet port 700 no state
pass in quick inet proto udp to port {isakmp, ipsec-nat-t} divert-packet
port 700 no state
pass out quick inet proto esp divert-packet port 700 no state
pass in quick inet proto esp divert-packet port 700 no state

3. Start the example divert application.

4. Start tcpdump on the outgoing interface of your IPsec configuration.

5. Start iked(8) and establish the ESP SA. I tested with iked(8) as the
responder, but the direction should not matter.

6. Send traffic over the ESP SA. Example tcpdump output from my test
environment ("openbsd" is the hostname of the machine this report was
generated on) when sending pings to the peer:

10:18:46.066107 openbsd.ipsec-nat-t > 10.50.0.7.ipsec-nat-t:udpencap:
esp openbsd > 10.50.0.7 spi 0xcfb330b6 seq 3 len 180
10:18:46.067336 10.50.0.7.ipsec-nat-t > openbsd.ipsec-nat-t:udpencap:
esp 10.50.0.7 > openbsd spi 0xe8208d98 seq 1 len 132
10:18:47.067999 openbsd.ipsec-nat-t > 10.50.0.7.ipsec-nat-t:udpencap:
esp openbsd > 10.50.0.7 spi 0xcfb330b6 seq 5 len 180
10:18:47.068899 10.50.0.7.ipsec-nat-t > openbsd.ipsec-nat-t:udpencap:
esp 10.50.0.7 > openbsd spi 0xe8208d98 seq 2 len 132
10:18:48.057983 openbsd.ipsec-nat-t > 10.50.0.7.ipsec-nat-t:udpencap:
esp openbsd > 10.50.0.7 spi 0xcfb330b6 seq 7 len 180
10:18:48.058814 10.50.0.7.ipsec-nat-t > openbsd.ipsec-nat-t:udpencap:
esp 10.50.0.7 > openbsd spi 0xe8208d98 seq 3 len 132
10:18:49.057999 openbsd.ipsec-nat-t > 10.50.0.7.ipsec-nat-t:udpencap:
esp openbsd > 10.50.0.7 spi 0xcfb330b6 seq 9 len 180
10:18:49.059071 10.50.0.7.ipsec-nat-t > openbsd.ipsec-nat-t:udpencap:
esp 10.50.0.7 > openbsd spi 0xe8208d98 seq 4 len 132

Note how the sequence number of the outgoing ESP SA increases by 2 with
every packet.

>Fix:
Workaround: Use rdomain(4) to separate IPsec and divert processing. If
the rdomain hosting the divert socket is not aware of the ESP SA, the
ESP packets are not reprocessed. E.g. if I run iked in rdomain 1 and
apply the divert rules in the default rdomain capture, manipulation,
and reinjection work as expected.

dmesg:
OpenBSD 6.0-current (LOCAL) #0: Mon Jan  2 16:25:32 CET 2017
    [hidden email]:/usr/src/sys/arch/amd64/compile/LOCAL
real mem = 1056833536 (1007MB)
avail mem = 1020198912 (972MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.8 @ 0xf0c00 (12 entries)
bios0: vendor SeaBIOS version "1.7.5-20140531_083030-gandalf" date
04/01/2014
bios0: QEMU Standard PC (i440FX + PIIX, 1996)
acpi0 at bios0: rev 0
acpi0: sleep states S3 S4 S5
acpi0: tables DSDT FACP SSDT APIC HPET
acpi0: wakeup devices
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: QEMU Virtual CPU version 2.1.2, 2600.39 MHz
cpu0:
FPU,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,SSE3,CX16,x2APIC,POPCNT,HV,NXE,LONG,LAHF
cpu0: 64KB 64b/line 2-way I-cache, 64KB 64b/line 2-way D-cache, 512KB
64b/line 16-way L2 cache
cpu0: ITLB 255 4KB entries direct-mapped, 255 4MB entries direct-mapped
cpu0: DTLB 255 4KB entries direct-mapped, 255 4MB entries direct-mapped
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
cpu0: apic clock running at 999MHz
cpu1 at mainbus0: apid 1 (application processor)
cpu1: QEMU Virtual CPU version 2.1.2, 2600.02 MHz
cpu1:
FPU,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,SSE3,CX16,x2APIC,POPCNT,HV,NXE,LONG,LAHF
cpu1: 64KB 64b/line 2-way I-cache, 64KB 64b/line 2-way D-cache, 512KB
64b/line 16-way L2 cache
cpu1: ITLB 255 4KB entries direct-mapped, 255 4MB entries direct-mapped
cpu1: DTLB 255 4KB entries direct-mapped, 255 4MB entries direct-mapped
cpu1: smt 0, core 0, package 1
cpu2 at mainbus0: apid 2 (application processor)
cpu2: QEMU Virtual CPU version 2.1.2, 2600.03 MHz
cpu2:
FPU,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,SSE3,CX16,x2APIC,POPCNT,HV,NXE,LONG,LAHF
cpu2: 64KB 64b/line 2-way I-cache, 64KB 64b/line 2-way D-cache, 512KB
64b/line 16-way L2 cache
cpu2: ITLB 255 4KB entries direct-mapped, 255 4MB entries direct-mapped
cpu2: DTLB 255 4KB entries direct-mapped, 255 4MB entries direct-mapped
cpu2: smt 0, core 0, package 2
cpu3 at mainbus0: apid 3 (application processor)
cpu3: QEMU Virtual CPU version 2.1.2, 2600.04 MHz
cpu3:
FPU,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,SSE3,CX16,x2APIC,POPCNT,HV,NXE,LONG,LAHF
cpu3: 64KB 64b/line 2-way I-cache, 64KB 64b/line 2-way D-cache, 512KB
64b/line 16-way L2 cache
cpu3: ITLB 255 4KB entries direct-mapped, 255 4MB entries direct-mapped
cpu3: DTLB 255 4KB entries direct-mapped, 255 4MB entries direct-mapped
cpu3: smt 0, core 0, package 3
ioapic0 at mainbus0: apid 0 pa 0xfec00000, version 11, 24 pins
acpihpet0 at acpi0: 100000000 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpicpu0 at acpi0: C1(@1 halt!)
acpicpu1 at acpi0: C1(@1 halt!)
acpicpu2 at acpi0: C1(@1 halt!)
acpicpu3 at acpi0: C1(@1 halt!)
"ACPI0006" at acpi0 not configured
"PNP0303" at acpi0 not configured
"PNP0F13" at acpi0 not configured
"PNP0700" at acpi0 not configured
"PNP0501" at acpi0 not configured
"PNP0A06" at acpi0 not configured
"ACPI0007" at acpi0 not configured
"ACPI0007" at acpi0 not configured
"ACPI0007" at acpi0 not configured
"ACPI0007" at acpi0 not configured
pvbus0 at mainbus0: KVM
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel 82441FX" rev 0x02
pcib0 at pci0 dev 1 function 0 "Intel 82371SB ISA" rev 0x00
pciide0 at pci0 dev 1 function 1 "Intel 82371SB IDE" rev 0x00: DMA,
channel 0 wired to compatibility, channel 1 wired to compatibility
wd0 at pciide0 channel 0 drive 0: <QEMU HARDDISK>
wd0: 16-sector PIO, LBA48, 20480MB, 41943040 sectors
atapiscsi0 at pciide0 channel 0 drive 1
scsibus1 at atapiscsi0: 2 targets
cd0 at scsibus1 targ 0 lun 0: <QEMU, QEMU DVD-ROM, 2.1.> ATAPI 5/cdrom
removable
wd0(pciide0:0:0): using PIO mode 4, DMA mode 2
cd0(pciide0:0:1): using PIO mode 4, DMA mode 2
pciide0: channel 1 disabled (no drives)
uhci0 at pci0 dev 1 function 2 "Intel 82371SB USB" rev 0x01: apic 0 int 11
piixpm0 at pci0 dev 1 function 3 "Intel 82371AB Power" rev 0x03: apic 0
int 9
iic0 at piixpm0
vga1 at pci0 dev 2 function 0 "Cirrus Logic CL-GD5446" rev 0x00
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
virtio0 at pci0 dev 3 function 0 "Qumranet Virtio Network" rev 0x00
vio0 at virtio0: address 52:54:00:10:85:34
virtio0: msix shared
virtio1 at pci0 dev 4 function 0 "Qumranet Virtio SCSI" rev 0x00
vioscsi0 at virtio1: qsize 128
scsibus2 at vioscsi0: 255 targets
virtio1: msix shared
virtio2 at pci0 dev 5 function 0 "Qumranet Virtio Memory" rev 0x00
viomb0 at virtio2
virtio2: apic 0 int 10
virtio3 at pci0 dev 6 function 0 "Qumranet Virtio Console" rev 0x00
virtio3: no matching child driver; not configured
virtio4 at pci0 dev 7 function 0 "Qumranet Virtio Network" rev 0x00
vio1 at virtio4: address 52:54:00:cf:01:8c
virtio4: msix shared
isa0 at pcib0
isadma0 at isa0
fdc0 at isa0 port 0x3f0/6 irq 6 drq 2
fd0 at fdc0 drive 1: density unknown
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com0: console
pckbc0 at isa0 port 0x60/5 irq 1 irq 12
pckbd0 at pckbc0 (kbd slot)
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pms0 at pckbc0 (aux slot)
wsmouse0 at pms0 mux 0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
usb0 at uhci0: USB revision 1.0
uhub0 at usb0 configuration 1 interface 0 "Intel UHCI root hub" rev
1.00/1.00 addr 1
vmm at mainbus0 not configured
uhidev0 at uhub0 port 1 configuration 1 interface 0 "QEMU QEMU USB
Tablet" rev 2.00/0.00 addr 2
uhidev0: iclass 3/0
ums0 at uhidev0: 3 buttons, Z dir
wsmouse1 at ums0 mux 0
vscsi0 at root
scsibus3 at vscsi0: 256 targets
softraid0 at root
scsibus4 at softraid0: 256 targets
root on wd0a (d9f1fa61e43a2a3f.a) swap on wd0b dump on wd0b

usbdevs:
Controller /dev/usb0:
addr 1: full speed, self powered, config 1, UHCI root hub(0x0000),
Intel(0x8086), rev 1.00
 port 1 addr 2: full speed, power 100 mA, config 1, QEMU USB
Tablet(0x0001), QEMU(0x0627), rev 0.00, iSerialNumber 42
 port 2 powered


--
Dipl.-Ing. Thomas Klute

achelos GmbH
Vattmannstraße 1
33100 Paderborn / Germany

Geschäftsführung: Kathrin Asmuth, Frank Stehling
Registergericht: Paderborn, HRB 8817, USt-IdNr.: DE260414872