panic: rw_enter: netlock locking against myself (NFS related?)

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

panic: rw_enter: netlock locking against myself (NFS related?)

Darren Tucker
I see this reproduceably when, eg doing cvs ops with Feb 5 snap.  I
found a thread from a couple of weeks ago but AFAICT the diff in that
thread is already in.

panic: rw_enter: netlock locking against myself
Stopped at      Debugger+0x9:   leave
    TID    PID    UID     PRFLAGS     PFLAGS  CPU  COMMAND
*331497  65559    500    0x100003          0    0  ssh
Debugger() at Debugger+0x9
panic() at panic+0xfe
rw_enter() at rw_enter+0x1c1
sosend() at sosend+0x114
nfs_send() at nfs_send+0x60
nfs_request() at nfs_request+0x408
nfs_removerpc() at nfs_removerpc+0x12e
nfs_inactive() at nfs_inactive+0x88
VOP_INACTIVE() at VOP_INACTIVE+0x35
vrele() at vrele+0x5c
unp_detach() at unp_detach+0x59
uipc_usrreq() at uipc_usrreq+0x2cd
soclose() at soclose+0x1a3
soo_close() at soo_close+0x1c
end trace frame: 0xffff800021397dd0, count: 0


Copyright (c) 1982, 1986, 1989, 1991, 1993
        The Regents of the University of California.  All rights reserved.
Copyright (c) 1995-2017 OpenBSD. All rights reserved.  https://www.OpenBSD.org

OpenBSD 6.0-current (GENERIC) #162: Sun Feb  5 13:49:23 MST 2017
    [hidden email]:/usr/src/sys/arch/amd64/compile/GENERIC
real mem = 2130575360 (2031MB)
avail mem = 2061467648 (1965MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.8 @ 0xf6480 (9 entries)
bios0: vendor SeaBIOS version "Ubuntu-1.8.2-1ubuntu1" date 04/01/2014
bios0: QEMU Standard PC (i440FX + PIIX, 1996)
acpi0 at bios0: rev 0
acpi0: sleep states S3 S4 S5
acpi0: tables DSDT FACP SSDT APIC HPET
acpi0: wakeup devices
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: QEMU Virtual CPU version 2.4.0, 2400.54 MHz
cpu0: FPU,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,SSE3,VMX,CX16,x2APIC,POPCNT,HV,NXE,LONG,LAHF
cpu0: 64KB 64b/line 2-way I-cache, 64KB 64b/line 2-way D-cache, 512KB 64b/line 16-way L2 cache
cpu0: ITLB 255 4KB entries direct-mapped, 255 4MB entries direct-mapped
cpu0: DTLB 255 4KB entries direct-mapped, 255 4MB entries direct-mapped
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
cpu0: apic clock running at 1000MHz
ioapic0 at mainbus0: apid 0 pa 0xfec00000, version 11, 24 pins
acpihpet0 at acpi0: 100000000 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpicpu0 at acpi0: C1(@1 halt!)
"ACPI0006" at acpi0 not configured
"PNP0303" at acpi0 not configured
"PNP0F13" at acpi0 not configured
"PNP0700" at acpi0 not configured
"PNP0501" at acpi0 not configured
"PNP0A06" at acpi0 not configured
"PNP0A06" at acpi0 not configured
"PNP0A06" at acpi0 not configured
pvbus0 at mainbus0: KVM
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel 82441FX" rev 0x02
pcib0 at pci0 dev 1 function 0 "Intel 82371SB ISA" rev 0x00
pciide0 at pci0 dev 1 function 1 "Intel 82371SB IDE" rev 0x00: DMA, channel 0 wired to compatibility, channel 1 wired to compatibility
pciide0: channel 0 disabled (no drives)
atapiscsi0 at pciide0 channel 1 drive 0
scsibus1 at atapiscsi0: 2 targets
cd0 at scsibus1 targ 0 lun 0: <QEMU, QEMU DVD-ROM, 2.4.> ATAPI 5/cdrom removable
cd0(pciide0:1:0): using PIO mode 4, DMA mode 2
piixpm0 at pci0 dev 1 function 3 "Intel 82371AB Power" rev 0x03: apic 0 int 9
iic0 at piixpm0
vga1 at pci0 dev 2 function 0 "Cirrus Logic CL-GD5446" rev 0x00
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
virtio0 at pci0 dev 3 function 0 "Qumranet Virtio RNG" rev 0x00
viornd0 at virtio0
virtio0: apic 0 int 11
virtio1 at pci0 dev 4 function 0 "Qumranet Virtio Network" rev 0x00
vio0 at virtio1: address 52:54:00:f6:02:ea
virtio1: msix shared
virtio2 at pci0 dev 5 function 0 "Qumranet Virtio Storage" rev 0x00
vioblk0 at virtio2
scsibus2 at vioblk0: 2 targets
sd0 at scsibus2 targ 0 lun 0: <VirtIO, Block Device, > SCSI3 0/direct fixed
sd0: 16384MB, 512 bytes/sector, 33554432 sectors
virtio2: msix shared
virtio3 at pci0 dev 6 function 0 "Qumranet Virtio Memory" rev 0x00
viomb0 at virtio3
virtio3: apic 0 int 10
virtio4 at pci0 dev 7 function 0 "Qumranet Virtio Storage" rev 0x00
vioblk1 at virtio4
scsibus3 at vioblk1: 2 targets
sd1 at scsibus3 targ 0 lun 0: <VirtIO, Block Device, > SCSI3 0/direct fixed
sd1: 16384MB, 512 bytes/sector, 33554432 sectors
virtio4: msix shared
virtio5 at pci0 dev 8 function 0 "Qumranet Virtio SCSI" rev 0x00
vioscsi0 at virtio5: qsize 128
scsibus4 at vioscsi0: 255 targets
virtio5: msix shared
isa0 at pcib0
isadma0 at isa0
fdc0 at isa0 port 0x3f0/6 irq 6 drq 2
fd0 at fdc0 drive 1: density unknown
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com0: console
pckbc0 at isa0 port 0x60/5 irq 1 irq 12
pckbd0 at pckbc0 (kbd slot)
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pms0 at pckbc0 (aux slot)
wsmouse0 at pms0 mux 0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
vmm0 at mainbus0: VMX/EPT
vscsi0 at root
scsibus5 at vscsi0: 256 targets
softraid0 at root
scsibus6 at softraid0: 256 targets
root on sd0a (1afc9f32ece695a9.a) swap on sd0b dump on sd0b
Automatic boot in progress: starting file system checks.
/dev/rsd0a: file system is clean; not checking
/dev/rsd1a: file system is clean; not checking
setting tty flags
pf enabled
ddb.console: 0 -> 1
kern.splassert: 1 -> 2
kern.bufcachepercent: 20 -> 50
starting network
add net default: gateway 2001:44b8:3110:fb01::254
reordering libraries: done.
starting early daemons: syslogd pflogd ntpd.
starting RPC daemons: portmap mountd(failed) nfsd(failed).
savecore: /dev/sd0b: Device not configured
Feb  9 15:31:26 quoll savecore: /dev/sd0b: Device not configured
checking quotas: done.
clearing /tmp
starting pre-securelevel daemons:.
kern.securelevel: 0 -> 1
creating runtime link editor directory cache.
preserving editor files.
starting network daemons: sshd snmpd smtpd.
starting local daemons:.
starting local daemons: cron.
Thu Feb  9 15:31:27 AEDT 2017

OpenBSD/amd64 (quoll.dtucker.net) (tty00)

login:
panic: rw_enter: netlock locking against myself
Stopped at      Debugger+0x9:   leave
    TID    PID    UID     PRFLAGS     PFLAGS  CPU  COMMAND
*331497  65559    500    0x100003          0    0  ssh
Debugger() at Debugger+0x9
panic() at panic+0xfe
rw_enter() at rw_enter+0x1c1
sosend() at sosend+0x114
nfs_send() at nfs_send+0x60
nfs_request() at nfs_request+0x408
nfs_removerpc() at nfs_removerpc+0x12e
nfs_inactive() at nfs_inactive+0x88
VOP_INACTIVE() at VOP_INACTIVE+0x35
vrele() at vrele+0x5c
unp_detach() at unp_detach+0x59
uipc_usrreq() at uipc_usrreq+0x2cd
soclose() at soclose+0x1a3
soo_close() at soo_close+0x1c
end trace frame: 0xffff800021397dd0, count: 0
https://www.openbsd.org/ddb.html describes the minimum info required in bug
reports.  Insufficient info makes it difficult to find and fix bugs.
ddb> trace
Debugger() at Debugger+0x9
panic() at panic+0xfe
rw_enter() at rw_enter+0x1c1
sosend() at sosend+0x114
nfs_send() at nfs_send+0x60
nfs_request() at nfs_request+0x408
nfs_removerpc() at nfs_removerpc+0x12e
nfs_inactive() at nfs_inactive+0x88
VOP_INACTIVE() at VOP_INACTIVE+0x35
vrele() at vrele+0x5c
unp_detach() at unp_detach+0x59
uipc_usrreq() at uipc_usrreq+0x2cd
soclose() at soclose+0x1a3
soo_close() at soo_close+0x1c
fdrop() at fdrop+0x2c
closef() at closef+0xcb
sys_close() at sys_close+0x60
syscall() at syscall+0x197
--- syscall (number 6) ---
end of kernel
end trace frame: 0x7e97ab4ec24, count: -18
0x7e93b6da20a:
ddb> ps
   PID     TID   PPID    UID  S       FLAGS  WAIT          COMMAND
*65559  331497  40164    500  7    0x100003                ssh
 40164  104311  70153    500  3    0x100083  piperd        cvs
 70153  249167  95185    500  3        0x83  wait          bash
 95185  373830  76180    500  3        0x90  select        sshd
 76180   62162  89399      0  3        0x92  poll          sshd
 54245  465947      1      0  3    0x100083  ttyin         getty
  5221  143887      1      0  3    0x100083  ttyin         getty
 27348  322417      1      0  3    0x100083  ttyin         getty
 77509  254505      1      0  3    0x100083  ttyin         getty
 57466   58328      1      0  3    0x100083  ttyin         getty
  5005  309102      1      0  3    0x100083  ttyin         getty
 20667  269314      1      0  3    0x100098  poll          cron
 94443  503227  46597     95  3    0x100092  kqread        smtpd
 52565  489502  46597    103  3    0x100092  kqread        smtpd
 60176  197267  46597     95  3    0x100092  kqread        smtpd
 45501  295029  46597     95  3    0x100092  kqread        smtpd
 73632  219358  46597     95  3    0x100092  kqread        smtpd
 82552  191311  46597     95  3    0x100092  kqread        smtpd
 46597  342688      1      0  3    0x100080  kqread        smtpd
 89881  446559      1      0  3        0x80  kqread        snmpd
 31311  249207      1     91  3        0x92  kqread        snmpd
 36021  138167      1     91  3        0x92  kqread        snmpd
 89399  304637      1      0  3        0x80  select        sshd
 30260  373752      0      0  3     0x14280  nfsidl        nfsio
 87509  501265      0      0  3     0x14280  nfsidl        nfsio
  5763  287896      0      0  3     0x14280  nfsidl        nfsio
 63259   42336      0      0  3     0x14280  nfsidl        nfsio
 27894  376601      1     28  3    0x100090  poll          portmap
 56478   99692      1      0  3    0x100080  poll          ntpd
 30613   88365  82272     83  3    0x100092  poll          ntpd
 82272   39491      1     83  3    0x100092  poll          ntpd
 51297   38005  93138     74  3    0x100090  bpf           pflogd
 93138   15364      1      0  3        0x80  netio         pflogd
 64022    8096  39384     73  3    0x100090  kqread        syslogd
 39384  484755      1      0  3    0x100082  netio         syslogd
 39972   94155      1      0  3        0x80  mfsidl        mount_mfs
 95554  163307      0      0  3     0x14200  pgzero        zerothread
 32466  276221      0      0  3     0x14200  aiodoned      aiodoned
  2040  346712      0      0  3     0x14200  syncer        update
 68632  214983      0      0  3     0x14200  cleaner       cleaner
 32975  192724      0      0  3     0x14200  reaper        reaper
 80353  431894      0      0  3     0x14200  pgdaemon      pagedaemon
 19644  238730      0      0  3     0x14200  bored         bfd
  8400  177792      0      0  3     0x14200  bored         crynlk
 88703  341462      0      0  3     0x14200  bored         crypto
 82731  207716      0      0  3     0x14200  pftm          pfpurge
 54415  346439      0      0  3     0x14200  bored         viomb
 39436  300007      0      0  3  0x40014200  acpi0         acpi0
 69614   24706      0      0  3     0x14200  bored         softnet
 65266  358625      0      0  3     0x14200  bored         systqmp
 78420   65487      0      0  3     0x14200  bored         systq
 25519  499550      0      0  3  0x40014200  bored         softclock
 67706  213188      0      0  3  0x40014200                idle0
     1  179173      0      0  3        0x82  wait          init
     0       0     -1      0  3     0x10200  scheduler     swapper
ddb>
--
Darren Tucker (dtucker at zip.com.au)
GPG key 11EAA6FA / A86E 3E07 5B19 5880 E860  37F4 9357 ECEF 11EA A6FA (new)
    Good judgement comes with experience. Unfortunately, the experience
usually comes from bad judgement.