Kernel panics - /usr/src/sys/netinet/raw ip.c - with ps

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Kernel panics - /usr/src/sys/netinet/raw ip.c - with ps

Scott Vanderbilt-3
I am getting kernel panics every 30 to 120 minutes with latest amd64
snapshot. The same panic occurs on four different machines (all amd64)
running this snapshot:

--------------------------------------------------------------------------------------------------------
dmesg
--------------------------------------------------------------------------------------------------------
OpenBSD 6.2-current (GENERIC.MP) #268: Sun Dec 10 11:18:16 MST 2017
[hidden email]:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 1038864384 (990MB)
avail mem = 1000521728 (954MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.5 @ 0xe4410 (25 entries)
bios0: vendor Intel Corp. version "MOPNV10J.86A.0154.2009.1117.1624"
date 11/17/2009
bios0: Intel Corporation D510MO
acpi0 at bios0: rev 2
acpi0: sleep states S0 S1 S3 S4 S5
acpi0: tables DSDT FACP APIC MCFG HPET SSDT
acpi0: wakeup devices SLPB(S4) PS2M(S4) PS2K(S4) UAR1(S4) UAR2(S4)
P32_(S4) ILAN(S4) PEX0(S4) PEX1(S4) PEX2(S4) PEX3(S4) UHC1(S3) UHC2(S3)
UHC3(S3) UHC4(S3) EHCI(S3) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Atom(TM) CPU D510 @ 1.66GHz, 1666.96 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,TM2,SSSE3,CX16,xTPR,PDCM,MOVBE,NXE,LONG,LAHF,PERF,SENSOR
cpu0: 512KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 7 var ranges, 88 fixed ranges
cpu0: apic clock running at 166MHz
cpu0: mwait min=64, max=64, C-substates=0.1, IBE
cpu1 at mainbus0: apid 1 (application processor)
cpu1: Intel(R) Atom(TM) CPU D510 @ 1.66GHz, 1666.69 MHz
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,TM2,SSSE3,CX16,xTPR,PDCM,MOVBE,NXE,LONG,LAHF,PERF,SENSOR
cpu1: 512KB 64b/line 8-way L2 cache
cpu1: smt 1, core 0, package 0
cpu2 at mainbus0: apid 2 (application processor)
cpu2: Intel(R) Atom(TM) CPU D510 @ 1.66GHz, 1666.69 MHz
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,TM2,SSSE3,CX16,xTPR,PDCM,MOVBE,NXE,LONG,LAHF,PERF,SENSOR
cpu2: 512KB 64b/line 8-way L2 cache
cpu2: smt 0, core 1, package 0
cpu3 at mainbus0: apid 3 (application processor)
cpu3: Intel(R) Atom(TM) CPU D510 @ 1.66GHz, 1666.69 MHz
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,TM2,SSSE3,CX16,xTPR,PDCM,MOVBE,NXE,LONG,LAHF,PERF,SENSOR
cpu3: 512KB 64b/line 8-way L2 cache
cpu3: smt 1, core 1, package 0
ioapic0 at mainbus0: apid 8 pa 0xfec00000, version 20, 24 pins
, remapped to apid 8
acpimcfg0 at acpi0 addr 0xf8000000, bus 0-63
acpihpet0 at acpi0: 14318179 Hz
acpiprt0 at acpi0: bus 5 (P32_)
acpiprt1 at acpi0: bus 0 (PCI0)
acpiprt2 at acpi0: bus 1 (PEX0)
acpiprt3 at acpi0: bus 2 (PEX1)
acpiprt4 at acpi0: bus 3 (PEX2)
acpiprt5 at acpi0: bus 4 (PEX3)
acpicpu0 at acpi0: C1(1000@3 mwait.1), PSS
acpicpu1 at acpi0: C1(1000@3 mwait.1), PSS
acpicpu2 at acpi0: C1(1000@3 mwait.1), PSS
acpicpu3 at acpi0: C1(1000@3 mwait.1), PSS
acpibtn0 at acpi0: SLPB
"PNP0003" at acpi0 not configured
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel Pineview DMI" rev 0x02
inteldrm0 at pci0 dev 2 function 0 "Intel Pineview Video" rev 0x02
drm0 at inteldrm0
intagp0 at inteldrm0
agp0 at intagp0: aperture at 0xd0000000, size 0x10000000
inteldrm0: msi
inteldrm0: 1280x800, 32bpp
wsdisplay0 at inteldrm0 mux 1: console (std, vt100 emulation)
wsdisplay0: screen 1-5 added (std, vt100 emulation)
azalia0 at pci0 dev 27 function 0 "Intel 82801GB HD Audio" rev 0x01: msi
azalia0: codecs: Realtek ALC662
audio0 at azalia0
ppb0 at pci0 dev 28 function 0 "Intel 82801GB PCIE" rev 0x01: msi
pci1 at ppb0 bus 1
1:0:0: mem address conflict 0xfffe0000/0x20000
re0 at pci1 dev 0 function 0 "Realtek 8168" rev 0x03: RTL8168D/8111D
(0x2800), msi, address 00:27:0e:08:a9:f5
rgephy0 at re0 phy 7: RTL8169S/8110S/8211 PHY, rev. 2
ppb1 at pci0 dev 28 function 1 "Intel 82801GB PCIE" rev 0x01: msi
pci2 at ppb1 bus 2
ppb2 at pci0 dev 28 function 2 "Intel 82801GB PCIE" rev 0x01: msi
pci3 at ppb2 bus 3
ppb3 at pci0 dev 28 function 3 "Intel 82801GB PCIE" rev 0x01: msi
pci4 at ppb3 bus 4
uhci0 at pci0 dev 29 function 0 "Intel 82801GB USB" rev 0x01: apic 8 int 23
uhci1 at pci0 dev 29 function 1 "Intel 82801GB USB" rev 0x01: apic 8 int 19
uhci2 at pci0 dev 29 function 2 "Intel 82801GB USB" rev 0x01: apic 8 int 18
uhci3 at pci0 dev 29 function 3 "Intel 82801GB USB" rev 0x01: apic 8 int 16
ehci0 at pci0 dev 29 function 7 "Intel 82801GB USB" rev 0x01: apic 8 int 23
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 configuration 1 interface 0 "Intel EHCI root hub" rev
2.00/1.00 addr 1
ppb4 at pci0 dev 30 function 0 "Intel 82801BAM Hub-to-PCI" rev 0xe1
pci5 at ppb4 bus 5
pcib0 at pci0 dev 31 function 0 "Intel NM10 LPC" rev 0x01
pciide0 at pci0 dev 31 function 2 "Intel 82801GB SATA" rev 0x01: DMA,
channel 0 configured to native-PCI, channel 1 configured to native-PCI
pciide0: using apic 8 int 19 for native-PCI interrupt
wd0 at pciide0 channel 1 drive 0: <ST3250310AS>
wd0: 16-sector PIO, LBA48, 238474MB, 488395055 sectors
wd0(pciide0:1:0): using PIO mode 4, Ultra-DMA mode 6
ichiic0 at pci0 dev 31 function 3 "Intel 82801GB SMBus" rev 0x01: apic 8
int 19
iic0 at ichiic0
spdmem0 at iic0 addr 0x50: 1GB DDR2 SDRAM non-parity PC2-5300CL5
usb1 at uhci0: USB revision 1.0
uhub1 at usb1 configuration 1 interface 0 "Intel UHCI root hub" rev
1.00/1.00 addr 1
usb2 at uhci1: USB revision 1.0
uhub2 at usb2 configuration 1 interface 0 "Intel UHCI root hub" rev
1.00/1.00 addr 1
usb3 at uhci2: USB revision 1.0
uhub3 at usb3 configuration 1 interface 0 "Intel UHCI root hub" rev
1.00/1.00 addr 1
usb4 at uhci3: USB revision 1.0
uhub4 at usb4 configuration 1 interface 0 "Intel UHCI root hub" rev
1.00/1.00 addr 1
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5 irq 1 irq 12
pckbd0 at pckbc0 (kbd slot)
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pms0 at pckbc0 (aux slot)
wsmouse0 at pms0 mux 0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
lpt0 at isa0 port 0x378/4 irq 7
wbsio0 at isa0 port 0x4e/2: W83627THF rev 0x84
lm1 at wbsio0 port 0x290/8: W83627THF
vscsi0 at root
scsibus1 at vscsi0: 256 targets
softraid0 at root
scsibus2 at softraid0: 256 targets
root on wd0a (c4f8032150c83b5d.a) swap on wd0b dump on wd0b
WARNING: / was not properly unmounted

----------------------------------------------------------
ddb output
----------------------------------------------------------
panic kernel diagnostic assertion "divert != NULL" failed: file
"/usr/src/sys/netinet/raw ip.c", line 138
Stopped at db_enter+0x5: popq %rbp
     TID      PID UID PRFLAGS    PFLAGS    CPU COMMAND
*  9249    96383    0    0x14000    0x200    2K softnet
db_enter() at db_enter+0x5
panic() at panic+0x129
__assert (ffffffff815d6114, ffff8000250aa890, ffffff001e92254e,
ffffff002c328b00) at __assert+0x24
rip_input(ffffff001e92254e,ffffff002c328b00,ffffff001e922562,45) at
rip_input+0x350
icmp_input_if(1,2, ffff8000250aaa3c, ffff8000250aaa40, ffff80000011f090)
at icnp.input_if+0x71d
icmp_input(2,ffffffff81939040,2,1) at icmp_input+0x42
ip_deliver(ffff000250aaa3c,ffff8000250aaa40,ffff800000019080,ffff8000250aaa90)
at ip_deliver+0x1fe
ipintr() at ipintr+0x5a
if_netisr(ffffffff811fa370) at if_netisr+0x54
taskq_thread(0) at taskq_thread+0x67
end trace frame: 0x0, count: 5
https://www.openbsd.org/ddb.htnl describes the minimum info required in bug
reports. Insufficient info makes it difficult to find and fix bugs.

ddb{2}> show panic
kernel diagnostic assertion "divert != NULL” failed: file
"/usr/src/sys/netinet/raw.ip.c", line 138

ddb{2}> trace
db_enter() at db_enter+0x5
panic() at panic+0x129
__assert(ffffffff815d6114, ffff8000250aa890, ffffff001e92254e,
ffffff002c328b00) at _assert+0x24
rip_input(ffffff001e92254e,ffffff002c328b00,ffffffeei«922S62,45> at
rip_input+0x350
icmp_input_if(1,2,ffff8000250aaa3c ffff8000250aaa40 ffff80000011f090) at
icmp input_if+0x71d    Λ
icmp_input(2,ffffffff81939040,2,l) at icnp_input+0x42
ip_deliver(fffff8000250aaa3c, ffff8000250aaa40, f fff800000019080,
ffff8000250aaa90> at ip_de liver+0x1fe
ipintr() at ipintr+0x5a
if_netisr(ffffffff811fa370) at if_netisr+0x54
taskq_thread(0) at taskq_thread+0x67
end trace frame: 0x0, count: -10
ddb{2}> I

ddb{2}> machine ddbcpu 1
Stopped at     x86_ipi_db+0x5: popq    %rbp
x86_ipi_db(ffffffff81327125) at x86_ipi_db+0x5
x86_ipi_handler() at x86_ipi_handler+0x6a
Xresume_lapic_ipi() at Xresume_lapic_ipi+0x1fe
--- interrupt ---
end of kernel
end trace frame: 0x964b2405c75250cc, count: 12
0x41cb8c419c524153:

ddb{1}> machine ddbcpu 2
Stopped at db_enter+0x5:    popq   %rbp
db_enter() at db_enter+0x5
panic() at panic+0x129
__assert(ffffffff815d6114,ffff8000250aa890,ffffff001e92254e,ffffff002c328b00)
at __assert+0x24
rip_input(ffffff001e92254e,ffffff002c328b00,ffffff001e922562,45) at
rip_input+0x350
icmp_input_if(1,2,ffff8000250aaa3c,ffff8000250aaa40,ffff80000011f090) at
icmp_input_if+0x71d
icmp_input(2,ffffffff81939040,2,l) at icmp_input+0x42
ip_deliver <ffff8000250aaa3c, ffff8O0O250aaa4O, fff f800000019080,
ffff8000250aaa90) at ip_deliver+0x1fe
ipintr() at ipintr+0x5a
if_netisr(ffffffff811fa370) at if_netisr+0x54
taskq_thread(0) at taskq_thread+0x67
end trace frame: 0x0, count: 5
ddb{2}>

--------------------------------------------------------------------------------------------------------
ps output (attached as screenshots)
--------------------------------------------------------------------------------------------------------

PS_LIST_1a.jpg (87K) Download Attachment
PS_LIST_2a.jpg (123K) Download Attachment
PS_LIST_3a.jpg (122K) Download Attachment
PS_LIST_4a.jpg (116K) Download Attachment
PS_LIST_5b.jpg (125K) Download Attachment
PS_LIST_6b.jpg (129K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Kernel panics - /usr/src/sys/netinet/raw ip.c - with ps

Alexander Bluhm
On Sun, Dec 10, 2017 at 10:51:49PM -0800, Scott Vanderbilt wrote:
> I am getting kernel panics every 30 to 120 minutes with latest amd64
> snapshot. The same panic occurs on four different machines (all amd64)
> running this snapshot:

In icmp_input_if() all mbuf tags are deleted without clearing the
divert flag.  This creates an inconsistency later in rip_input()
that will trigger the assertion.

The m_tag_delete_chain() before icmp_reflect() is not necessary
anymore as I have added a m_resethdr() in the latter recently.

All code paths that lead to a m_tag_delete_chain() are covered by
the divert switch.  So the divert flag should be cleared there.  If
we process the ICMP packet in our stack, it should not be diverted
to raw sockets.

Although there is no m_tag_delete_chain() in the ICMP6 code, also
clear the flag there.  Locally processed packets should not be
diverted to raw sockets.

Does this diff fix the panic?

ok?

bluhm

Index: netinet/ip_icmp.c
===================================================================
RCS file: /data/mirror/openbsd/cvs/src/sys/netinet/ip_icmp.c,v
retrieving revision 1.173
diff -u -p -r1.173 ip_icmp.c
--- netinet/ip_icmp.c 18 Oct 2017 17:01:14 -0000 1.173
+++ netinet/ip_icmp.c 12 Dec 2017 01:09:39 -0000
@@ -386,12 +386,14 @@ icmp_input_if(struct ifnet *ifp, struct
  case ICMP_TIMXCEED:
  case ICMP_PARAMPROB:
  case ICMP_SOURCEQUENCH:
+ m->m_pkthdr.pf.flags &=~ PF_TAG_DIVERTED;
  break;
  /*
   * Although pf_icmp_mapping() considers redirects belonging
   * to a diverted connection, we must process it here anyway.
   */
  case ICMP_REDIRECT:
+ m->m_pkthdr.pf.flags &=~ PF_TAG_DIVERTED;
  break;
  default:
  goto raw;
@@ -585,10 +587,6 @@ reflect:
     &ip->ip_dst.s_addr, 1))
  goto freeit;
 #endif
- /* Free packet atttributes */
- if (m->m_flags & M_PKTHDR)
- m_tag_delete_chain(m);
-
  icmpstat_inc(icps_reflect);
  icmpstat_inc(icps_outhist + icp->icmp_type);
  if (!icmp_reflect(m, &opts, NULL)) {
Index: netinet6/icmp6.c
===================================================================
RCS file: /data/mirror/openbsd/cvs/src/sys/netinet6/icmp6.c,v
retrieving revision 1.220
diff -u -p -r1.220 icmp6.c
--- netinet6/icmp6.c 3 Nov 2017 14:28:57 -0000 1.220
+++ netinet6/icmp6.c 12 Dec 2017 01:12:36 -0000
@@ -431,6 +431,7 @@ icmp6_input(struct mbuf **mp, int *offp,
  case ICMP6_PACKET_TOO_BIG:
  case ICMP6_TIME_EXCEEDED:
  case ICMP6_PARAM_PROB:
+ m->m_pkthdr.pf.flags &=~ PF_TAG_DIVERTED;
  break;
  default:
  goto raw;