Recurring panic in _rb_remove

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Recurring panic in _rb_remove

Paul de Weerd
Hi all,

Seeing a recurring panic on my local router.  It's also acting as a
backup and fileserver (serving OpenBSD snapshots to internal hosts).
The panic is:

uvm_fault(0xfffffd86fc8abee8, 0x8, 0, 1) -> e
kernel: page fault trap, code=0
Stopped at      _rb_remove+0x3c8:       movq    0x8(%rdi),%rax
ddb{0}> trace
_rb_remove(ffffffff81c7cc18,ffffffff81dde478,fffffd8124b0f600) at _rb_remove+0x3c8
uvm_pagefree(fffffd8124b0f600) at uvm_pagefree+0x37
buf_free_pages(fffffd85e204eb00) at buf_free_pages+0xbd
buf_dealloc_mem(fffffd85e204eb00) at buf_dealloc_mem+0x7b
buf_put(fffffd85e204eb00) at buf_put+0xcf
bufbackoff(ffffffff81dc3308,8) at bufbackoff+0x108
buf_realloc_pages(fffffd83ada48000,ffffffff81dc3308,2) at buf_realloc_pages+0xc6
buf_flip_high(fffffd83ada48000) at buf_flip_high+0x6f
bufcache_recover_dmapages(0,8) at bufcache_recover_dmapages+0x10c
buf_get(fffffd85d5cfd338,af,8000) at buf_get+0xd7
getblk(fffffd85d5cfd338,af,8000,0,ffffffffffffffff) at getblk+0xaf
ffs1_balloc(fffffd883c278c40,578000,9f,fffffd876b933428,1,ffff8000331d1138) at ffs1_balloc+0xd43
ffs_write(ffff8000331d11c0) at ffs_write+0x262
VOP_WRITE(fffffd85d5cfd338,ffff8000331d1328,1,fffffd876b933428) at VOP_WRITE+0x4f
vn_write(fffffd807ac47720,ffff8000331d1328,0) at vn_write+0x11e
dofilewritev(ffff800033063a78,6,ffff8000331d1328,0,ffff8000331d1400) at dofilewritev+0xf9
sys_write(ffff800033063a78,ffff8000331d13a0,ffff8000331d1400) at sys_write+0x51
syscall(ffff8000331d1470) at syscall+0x389
Xsyscall(0,4,c70,4,33fbee48ca0,33f512efd40) at Xsyscall+0x128
end of kernel
end trace frame: 0x7f7ffffe6cd0, count: -19

I've upgraded to a newer snapshot but still see this behaviour; it
usually happens while making backups using rsync, but not every day.
ps shows rsync as the active process:

ddb{0}> ps
   PID     TID   PPID    UID  S       FLAGS  WAIT          COMMAND
*60680  474219   2041    864  7           0                rsync

Included below is the cleaned-up console log with ps, trace and dmesg
included (I removed the ^M's and the ascii spinner at boot).  Note
that lines >80 chars wrap for the ddb output, I've corrected that in
the above trace output.

I'm leaving on vacation in a few hours, wanted to send this out before
I leave.  I believe the next step is to build a bsd.gdb and run that,
but I won't be able to do that until after I come back.

Cheers,

Paul

ddb{0}> ps
   PID     TID   PPID    UID  S       FLAGS  WAIT          COMMAND
*60680  474219   2041    864  7           0                rsync
  2041   80282  24289    864  3         0x2  biowait       rsync
  6746  452021  64179    864  3    0x100083  poll          top
 64179  271077  31496    864  3    0x10008b  pause         ksh
 24289  402112   8227    864  3    0x10008a  pause         sh
  8227   93111   6131    864  3    0x10008a  pause         sh
  6131  443394    236      0  3    0x100090  piperd        cron
 62053  223702  60120      0  3        0x83  poll          systat
 18568  429066      0      0  3     0x14200  bored         srdis
  8829  168032    301      0  3    0x100083  ttyin         ksh
   301  102538  31496    864  3    0x10008b  pause         ksh
 74109     121      1    577  3        0x90  poll          openvpn
 52918  480270      1      0  3    0x100080  kqread        httpd
  6829  314993      1     67  3    0x100092  kqread        httpd
 61846  188976      1     67  3    0x100092  kqread        httpd
 33530   17158      1     67  3    0x100092  kqread        httpd
 41296   39009      1     67  3    0x100092  kqread        httpd
 75685  405806      1     67  3    0x100092  kqread        httpd
 57000   70236      1     67  3    0x100092  kqread        httpd
 35760  472569      1     67  3    0x100092  kqread        httpd
 14021  116305      1     67  3    0x100092  kqread        httpd
  2803   15453      1     67  3    0x100092  kqread        httpd
 75532  223061      1     67  3    0x100092  kqread        httpd
 37086   58085      1     67  3    0x100092  kqread        httpd
 40156  430012   2085    864  3    0x100083  kqread        tmux
  2085  226180  88532    864  3    0x10008b  pause         ksh
 88532  166130  69188    864  3        0x90  select        sshd
 69188   74364  33879      0  3        0x92  poll          sshd
 88162  523355  31496    864  3    0x100083  ttyin         ksh
 30561   72987      0      0  3     0x14200  bored         srdis
 60120  114946   8531      0  3    0x10008b  pause         ksh
  5308  312248  19787    864  3    0x100083  piperd        tee
 74364  137675  19787    864  3        0x83  nanosleep     perl
 19787  339670  31496    864  3    0x10008b  pause         ksh
 78410   76409  31496    864  3    0x100083  ttyin         ksh
  8531   66284  31496    864  3    0x10008b  pause         ksh
 31496  516703      1    864  3    0x100080  kqread        tmux
 41528  277034      1    864  3    0x1000b0  poll          ssh-agent
 81571  489757      1      0  3    0x100083  ttyin         getty
   236  421789      1      0  3    0x100098  poll          cron
  2643   15298      1      0  3        0x80  kqread        apmd
 61310  374975  63511    606  3        0x90  kqread        ladvd
 63511  444373      1      0  3        0x80  kqread        ladvd
 66403   45971      1     79  3    0x100090  kqread        tftpd
 69243  138389      1     67  3    0x100090  kqread        slowcgi
 28533   82842  89153     95  3    0x100092  kqread        smtpd
 45227  188297  89153    103  3    0x100092  kqread        smtpd
 84638   58186  89153     95  3    0x100092  kqread        smtpd
 44075  269454  89153     95  3    0x100092  kqread        smtpd
 41264  280548  89153     95  3    0x100092  kqread        smtpd
 78215  461288  89153     95  3    0x100092  kqread        smtpd
 89153  403538      1      0  3    0x100080  kqread        smtpd
 88627  429981  83417     94  3    0x100092  kqread        rad
 74649  143691  83417     94  3    0x100092  kqread        rad
 83417  481752      1      0  3    0x100080  kqread        rad
 72824  505279      1     77  3    0x100090  poll          dhcpd
 33879   15750      1      0  3        0x80  select        sshd
 62997  431705  88090      0  3        0x80  nfsd          nfsd
 75743   18740  88090      0  3        0x80  nfsd          nfsd
 78492  366018  88090      0  3        0x80  nfsd          nfsd
 60307  311789  88090      0  3        0x80  nfsd          nfsd
 88090  408746      1      0  3        0x80  netcon        nfsd
 20431  348522  43306      0  3        0x80  poll          mountd
 43306  508203      1      0  3    0x100080  poll          mountd
   409  159789      1     28  3    0x100090  poll          portmap
 22301  404781      1      0  3    0x100080  poll          ntpd
 36989  223581  64833     83  3    0x100092  poll          ntpd
 64833  116505      1     83  3    0x100092  poll          ntpd
 59285  326976      1     53  3        0x90  kqread        unbound
 30573  181967  46920     97  3    0x100090  kqread        nsd
 46920  256017  19340     97  3    0x100090  poll          nsd
 19340   96004      1     97  3    0x100090  kqread        nsd
 99631  372320  66831     74  3    0x100092  bpf           pflogd
 66831  120785      1      0  3        0x80  netio         pflogd
 83582   22272   8539     73  3    0x100090  kqread        syslogd
  8539  217233      1      0  3    0x100082  netio         syslogd
 62341  438623      1     77  3    0x100090  poll          dhclient
 34468   93431      1      0  3        0x80  poll          dhclient
 27477  229658      1     77  3    0x100090  poll          dhclient
 84276  423340      1      0  3        0x80  poll          dhclient
 67890  219083  49402    115  3    0x100092  kqread        slaacd
 46124  389340  49402    115  3    0x100092  kqread        slaacd
 49402  343308      1      0  3    0x100080  kqread        slaacd
 21349  309777      0      0  3     0x14280  schto         i915/signal:6
 93120   79546      0      0  3     0x14280  schto         i915/signal:2
 26979  307356      0      0  3     0x14280  schto         i915/signal:1
 60274  328100      0      0  3     0x14280  schto         i915/signal:0
 14654  392598      0      0  3     0x14200  bored         i915-userptr-acq
 59417  164718      0      0  3     0x14200  bored         i915_modeset
 21074   35944      0      0  3     0x14200  bored         i915-dp
   213  471497      0      0  3     0x14200  bored         i915
 61715  273039      0      0  3     0x14200  pgzero        zerothread
 72035  254000      0      0  3     0x14200  aiodoned      aiodoned
 98656  458610      0      0  3     0x14200  syncer        update
 54150  296058      0      0  3     0x14200  cleaner       cleaner
 24032   63346      0      0  3     0x14200  reaper        reaper
 79920  275510      0      0  3     0x14200  pgdaemon      pagedaemon
 94527  299597      0      0  3     0x14200  bored         crynlk
 94838  399994      0      0  3     0x14200  bored         crypto
 31370  382682      0      0  3     0x14200  usbtsk        usbtask
 10571  220841      0      0  3     0x14200  usbatsk       usbatsk
 71415  231437      0      0  3     0x14200  bored         drmtskl
 26261   94667      0      0  3     0x14200  bored         drmlwq
 37392   31366      0      0  3     0x14200  bored         drmlwq
 19125  441887      0      0  3     0x14200  bored         drmlwq
 22022  365455      0      0  3     0x14200  bored         drmlwq
 46349    3735      0      0  3     0x14200  bored         drmubwq
 31592  185355      0      0  3     0x14200  bored         drmubwq
 15533  481451      0      0  3     0x14200  bored         drmubwq
 30646  262337      0      0  3     0x14200  bored         drmubwq
 53694    3565      0      0  3     0x14200  bored         drmwq
  6451  315183      0      0  3     0x14200  bored         drmwq
 88684    5459      0      0  3     0x14200  bored         drmwq
 33807  215626      0      0  3     0x14200  bored         drmwq
 99999  312497      0      0  3  0x40014200  acpi0         acpi0
 56475  507932      0      0  7  0x40014200                idle1
 71903   91415      0      0  3     0x14200  bored         sensors
  4936  359762      0      0  3     0x14200  bored         softnet
 69286  183115      0      0  3     0x14200  bored         systqmp
 27045  447467      0      0  3     0x14200  bored         systq
 15589  135997      0      0  3  0x40014200  bored         softclock
  3257  290261      0      0  3  0x40014200                idle0
 19343  371152      0      0  3     0x14200  bored         smr
     1  349148      0      0  3        0x82  wait          init
     0       0     -1      0  3     0x10200  scheduler     swapper
ddb{0}> trace
_rb_remove(ffffffff81c7cc18,ffffffff81dde478,fffffd8124b0f600) at _rb_remove+0x
3c8
uvm_pagefree(fffffd8124b0f600) at uvm_pagefree+0x37
buf_free_pages(fffffd85e204eb00) at buf_free_pages+0xbd
buf_dealloc_mem(fffffd85e204eb00) at buf_dealloc_mem+0x7b
buf_put(fffffd85e204eb00) at buf_put+0xcf
bufbackoff(ffffffff81dc3308,8) at bufbackoff+0x108
buf_realloc_pages(fffffd83ada48000,ffffffff81dc3308,2) at buf_realloc_pages+0xc
6
buf_flip_high(fffffd83ada48000) at buf_flip_high+0x6f
bufcache_recover_dmapages(0,8) at bufcache_recover_dmapages+0x10c
buf_get(fffffd85d5cfd338,af,8000) at buf_get+0xd7
getblk(fffffd85d5cfd338,af,8000,0,ffffffffffffffff) at getblk+0xaf
ffs1_balloc(fffffd883c278c40,578000,9f,fffffd876b933428,1,ffff8000331d1138) at f
fs1_balloc+0xd43
ffs_write(ffff8000331d11c0) at ffs_write+0x262
VOP_WRITE(fffffd85d5cfd338,ffff8000331d1328,1,fffffd876b933428) at VOP_WRITE+0x
4f
vn_write(fffffd807ac47720,ffff8000331d1328,0) at vn_write+0x11e
dofilewritev(ffff800033063a78,6,ffff8000331d1328,0,ffff8000331d1400) at dofilew
ritev+0xf9
sys_write(ffff800033063a78,ffff8000331d13a0,ffff8000331d1400) at sys_write+0x51

syscall(ffff8000331d1470) at syscall+0x389
Xsyscall(0,4,c70,4,33fbee48ca0,33f512efd40) at Xsyscall+0x128
end of kernel
end trace frame: 0x7f7ffffe6cd0, count: -19
ddb{0}> bo re
rebooting...
<--- snip ascii spinner --->
[ using 2252752 bytes of bsd ELF symbol table ]
Copyright (c) 1982, 1986, 1989, 1991, 1993
        The Regents of the University of California.  All rights reserved.
Copyright (c) 1995-2019 OpenBSD. All rights reserved.  https://www.OpenBSD.org

OpenBSD 6.5-current (GENERIC.MP) #146: Fri Jul 26 17:22:26 MDT 2019
    [hidden email]:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 34263703552 (32676MB)
avail mem = 33215143936 (31676MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 3.0 @ 0x8d717000 (86 entries)
bios0: vendor American Megatrends Inc. version "5.12" date 05/28/2018
acpi0 at bios0: ACPI 6.1
acpi0: sleep states S0 S5
acpi0: tables DSDT FACP APIC FPDT MCFG SSDT FIDT SSDT HPET SSDT SSDT UEFI SSDT LPIT WSMT SSDT SSDT SSDT SSDT DBGP DBG2 SPCR DMAR ASF!
acpi0: wakeup devices PS2K(S0) PS2M(S0) PXSX(S0) RP09(S0) PXSX(S0) RP10(S0) PXSX(S0) RP11(S0) PXSX(S0) RP12(S0) PXSX(S0) RP13(S0) PXSX(S0) RP01(S0) PXSX(S0) RP02(S0) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz, 2395.20 MHz, 06-8e-09
cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,SGX,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PT,MD_CLEAR,TSXFA,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES,MELTDOWN
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 24MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4.1.1.1, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz, 2394.44 MHz, 06-8e-09
cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,SGX,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PT,MD_CLEAR,TSXFA,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES,MELTDOWN
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 20, 120 pins
acpimcfg0 at acpi0
acpimcfg0: addr 0xe0000000, bus 0-255
acpihpet0 at acpi0: 23999999 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus -1 (PEG0)
acpiprt2 at acpi0: bus -1 (PEG1)
acpiprt3 at acpi0: bus -1 (PEG2)
acpiprt4 at acpi0: bus -1 (RP09)
acpiprt5 at acpi0: bus -1 (RP10)
acpiprt6 at acpi0: bus -1 (RP11)
acpiprt7 at acpi0: bus -1 (RP12)
acpiprt8 at acpi0: bus -1 (RP13)
acpiprt9 at acpi0: bus 1 (RP01)
acpiprt10 at acpi0: bus 2 (RP02)
acpiprt11 at acpi0: bus 3 (RP03)
acpiprt12 at acpi0: bus 4 (RP04)
acpiprt13 at acpi0: bus 5 (RP05)
acpiprt14 at acpi0: bus 6 (RP06)
acpiprt15 at acpi0: bus -1 (RP07)
acpiprt16 at acpi0: bus -1 (RP08)
acpiprt17 at acpi0: bus -1 (RP17)
acpiprt18 at acpi0: bus -1 (RP18)
acpiprt19 at acpi0: bus -1 (RP19)
acpiprt20 at acpi0: bus -1 (RP20)
acpiprt21 at acpi0: bus -1 (RP21)
acpiprt22 at acpi0: bus -1 (RP22)
acpiprt23 at acpi0: bus -1 (RP23)
acpiprt24 at acpi0: bus -1 (RP24)
acpiprt25 at acpi0: bus -1 (RP14)
acpiprt26 at acpi0: bus -1 (RP15)
acpiprt27 at acpi0: bus -1 (RP16)
acpiec0 at acpi0: not present
acpicpu0 at acpi0: C3(200@1034 mwait.1@0x60), C2(200@151 mwait.1@0x33), C1(1000@1 mwait.1), PSS
acpicpu1 at acpi0: C3(200@1034 mwait.1@0x60), C2(200@151 mwait.1@0x33), C1(1000@1 mwait.1), PSS
acpipwrres0 at acpi0: WRST
acpipwrres1 at acpi0: WRST
acpipwrres2 at acpi0: WRST
acpipwrres3 at acpi0: WRST
acpipwrres4 at acpi0: WRST
acpipwrres5 at acpi0: WRST
acpipwrres6 at acpi0: WRST
acpipwrres7 at acpi0: WRST
acpipwrres8 at acpi0: WRST
acpipwrres9 at acpi0: WRST
acpipwrres10 at acpi0: WRST
acpipwrres11 at acpi0: WRST
acpipwrres12 at acpi0: WRST
acpipwrres13 at acpi0: WRST
acpipwrres14 at acpi0: WRST
acpipwrres15 at acpi0: WRST
acpipwrres16 at acpi0: WRST
acpipwrres17 at acpi0: WRST
acpipwrres18 at acpi0: WRST
acpipwrres19 at acpi0: WRST
acpipwrres20 at acpi0: FN00, resource for FAN0
acpipwrres21 at acpi0: FN01, resource for FAN1
acpipwrres22 at acpi0: FN02, resource for FAN2
acpipwrres23 at acpi0: FN03, resource for FAN3
acpipwrres24 at acpi0: FN04, resource for FAN4
acpitz0 at acpi0: critical temperature is 119 degC
acpitz1 at acpi0: critical temperature is 119 degC
acpipci0 at acpi0 PCI0: 0x00000000 0x00000011 0x00000001
acpicmos0 at acpi0
acpibtn0 at acpi0: SLPB
"INT33A1" at acpi0 not configured
acpibtn1 at acpi0: PWRB
"PNP0C0B" at acpi0 not configured
"PNP0C0B" at acpi0 not configured
"PNP0C0B" at acpi0 not configured
"PNP0C0B" at acpi0 not configured
"PNP0C0B" at acpi0 not configured
acpivideo0 at acpi0: GFX0
acpivout0 at acpivideo0: DD1F
cpu0: using VERW MDS workaround (except on vmm entry)
cpu0: Enhanced SpeedStep 2395 MHz: speeds: 2601, 2600, 2500, 2400, 2200, 2000, 1900, 1700, 1500, 1400, 1200, 1100, 800, 700, 600, 400 MHz
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel Core 7G Host" rev 0x02
inteldrm0 at pci0 dev 2 function 0 "Intel HD Graphics 620" rev 0x02
drm0 at inteldrm0
inteldrm0: msi
xhci0 at pci0 dev 20 function 0 "Intel 100 Series xHCI" rev 0x21: msi, xHCI 1.0
usb0 at xhci0: USB revision 3.0
uhub0 at usb0 configuration 1 interface 0 "Intel xHCI root hub" rev 3.00/1.00 addr 1
"Intel 100 Series MEI" rev 0x21 at pci0 dev 22 function 0 not configured
ahci0 at pci0 dev 23 function 0 "Intel 100 Series AHCI" rev 0x21: msi, AHCI 1.3.1
ahci0: port 0: 6.0Gb/s
ahci0: port 1: 6.0Gb/s
ahci0: PHY offline on port 2
scsibus1 at ahci0: 32 targets
sd0 at scsibus1 targ 0 lun 0: <ATA, Samsung SSD 860, RVT4> SCSI3 0/direct fixed naa.5002538e405cebf1
sd0: 476940MB, 512 bytes/sector, 976773168 sectors, thin
sd1 at scsibus1 targ 1 lun 0: <ATA, Samsung SSD 860, RVQ0> SCSI3 0/direct fixed naa.5002538e40edbf97
sd1: 3815447MB, 512 bytes/sector, 7814037168 sectors, thin
ppb0 at pci0 dev 28 function 0 "Intel 100 Series PCIE" rev 0xf1: msi
pci1 at ppb0 bus 1
em0 at pci1 dev 0 function 0 "Intel 82583V" rev 0x00: msi, address 00:e0:67:17:d6:96
ppb1 at pci0 dev 28 function 1 "Intel 100 Series PCIE" rev 0xf1: msi
pci2 at ppb1 bus 2
em1 at pci2 dev 0 function 0 "Intel 82583V" rev 0x00: msi, address 00:e0:67:17:d6:97
ppb2 at pci0 dev 28 function 2 "Intel 100 Series PCIE" rev 0xf1: msi
pci3 at ppb2 bus 3
em2 at pci3 dev 0 function 0 "Intel 82583V" rev 0x00: msi, address 00:e0:67:17:d6:98
ppb3 at pci0 dev 28 function 3 "Intel 100 Series PCIE" rev 0xf1: msi
pci4 at ppb3 bus 4
em3 at pci4 dev 0 function 0 "Intel 82583V" rev 0x00: msi, address 00:e0:67:17:d6:99
ppb4 at pci0 dev 28 function 4 "Intel 100 Series PCIE" rev 0xf1: msi
pci5 at ppb4 bus 5
em4 at pci5 dev 0 function 0 "Intel 82583V" rev 0x00: msi, address 00:e0:67:17:d6:9a
ppb5 at pci0 dev 28 function 5 "Intel 100 Series PCIE" rev 0xf1: msi
pci6 at ppb5 bus 6
em5 at pci6 dev 0 function 0 "Intel 82583V" rev 0x00: msi, address 00:e0:67:17:d6:9b
pcib0 at pci0 dev 31 function 0 "Intel 200 Series LPC" rev 0x21
"Intel 100 Series PMC" rev 0x21 at pci0 dev 31 function 2 not configured
ichiic0 at pci0 dev 31 function 4 "Intel 100 Series SMBus" rev 0x21: apic 2 int 16
iic0 at ichiic0
admtemp0 at iic0 addr 0x18: Xeon
admtemp1 at iic0 addr 0x1a: Xeon
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com0: console
pckbc0 at isa0 port 0x60/5 irq 1 irq 12
pckbd0 at pckbc0 (kbd slot)
wskbd0 at pckbd0: console keyboard
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
it0 at isa0 port 0x2e/2: IT8772F rev 1, EC port 0xa40
vmm0 at mainbus0: VMX/EPT
uhidev0 at uhub0 port 2 configuration 1 interface 0 "RDing TEMPERHUM1V1.2" rev 2.00/0.01 addr 2
uhidev0: iclass 3/1, 1 report id
ukbd0 at uhidev0 reportid 1: 8 variable keys, 5 key codes
wskbd1 at ukbd0 mux 1
uhidev1 at uhub0 port 2 configuration 1 interface 1 "RDing TEMPERHUM1V1.2" rev 2.00/0.01 addr 2
uhidev1: iclass 3/1
ugold0 at uhidev1
uftdi0 at uhub0 port 3 configuration 1 interface 0 "FTDI FT230X Basic UART" rev 2.00/10.00 addr 3
ucom0 at uftdi0 portno 1
uftdi1 at uhub0 port 4 configuration 1 interface 0 "FTDI SYNLEAF - FT232R USB UART" rev 2.00/6.00 addr 4
ucom1 at uftdi1 portno 1
umass0 at uhub0 port 13 configuration 1 interface 0 "Western Digital My Passport 259D" rev 3.00/10.12 addr 5
umass0: using SCSI over Bulk-Only
scsibus2 at umass0: 2 targets, initiator 0
sd2 at scsibus2 targ 1 lun 0: <WD, My Passport 259D, 1012> SCSI4 0/direct fixed
sd2: 3815415MB, 512 bytes/sector, 7813969920 sectors
ses0 at scsibus2 targ 1 lun 1: <WD, SES Device, 1012> SCSI4 13/enclosure services fixed
ses0: unable to read enclosure configuration
vscsi0 at root
scsibus3 at vscsi0: 256 targets
softraid0 at root
scsibus4 at softraid0: 256 targets
root on sd0a (aec06f0350f24bab.a) swap on sd0b dump on sd0b
WARNING: / was not properly unmounted

--
>++++++++[<++++++++++>-]<+++++++.>+++[<------>-]<.>+++[<+
+++++++++++>-]<.>++[<------------>-]<+.--------------.[-]
                 http://www.weirdnet.nl/                 

Reply | Threaded
Open this post in threaded view
|

Re: Recurring panic in _rb_remove

Theo de Raadt-2
vn_write.

You are using vnd.  There are other reports this got broken recently.

Paul de Weerd <[hidden email]> wrote:

> Hi all,
>
> Seeing a recurring panic on my local router.  It's also acting as a
> backup and fileserver (serving OpenBSD snapshots to internal hosts).
> The panic is:
>
> uvm_fault(0xfffffd86fc8abee8, 0x8, 0, 1) -> e
> kernel: page fault trap, code=0
> Stopped at      _rb_remove+0x3c8:       movq    0x8(%rdi),%rax
> ddb{0}> trace
> _rb_remove(ffffffff81c7cc18,ffffffff81dde478,fffffd8124b0f600) at _rb_remove+0x3c8
> uvm_pagefree(fffffd8124b0f600) at uvm_pagefree+0x37
> buf_free_pages(fffffd85e204eb00) at buf_free_pages+0xbd
> buf_dealloc_mem(fffffd85e204eb00) at buf_dealloc_mem+0x7b
> buf_put(fffffd85e204eb00) at buf_put+0xcf
> bufbackoff(ffffffff81dc3308,8) at bufbackoff+0x108
> buf_realloc_pages(fffffd83ada48000,ffffffff81dc3308,2) at buf_realloc_pages+0xc6
> buf_flip_high(fffffd83ada48000) at buf_flip_high+0x6f
> bufcache_recover_dmapages(0,8) at bufcache_recover_dmapages+0x10c
> buf_get(fffffd85d5cfd338,af,8000) at buf_get+0xd7
> getblk(fffffd85d5cfd338,af,8000,0,ffffffffffffffff) at getblk+0xaf
> ffs1_balloc(fffffd883c278c40,578000,9f,fffffd876b933428,1,ffff8000331d1138) at ffs1_balloc+0xd43
> ffs_write(ffff8000331d11c0) at ffs_write+0x262
> VOP_WRITE(fffffd85d5cfd338,ffff8000331d1328,1,fffffd876b933428) at VOP_WRITE+0x4f
> vn_write(fffffd807ac47720,ffff8000331d1328,0) at vn_write+0x11e
> dofilewritev(ffff800033063a78,6,ffff8000331d1328,0,ffff8000331d1400) at dofilewritev+0xf9
> sys_write(ffff800033063a78,ffff8000331d13a0,ffff8000331d1400) at sys_write+0x51
> syscall(ffff8000331d1470) at syscall+0x389
> Xsyscall(0,4,c70,4,33fbee48ca0,33f512efd40) at Xsyscall+0x128
> end of kernel
> end trace frame: 0x7f7ffffe6cd0, count: -19
>
> I've upgraded to a newer snapshot but still see this behaviour; it
> usually happens while making backups using rsync, but not every day.
> ps shows rsync as the active process:
>
> ddb{0}> ps
>    PID     TID   PPID    UID  S       FLAGS  WAIT          COMMAND
> *60680  474219   2041    864  7           0                rsync
>
> Included below is the cleaned-up console log with ps, trace and dmesg
> included (I removed the ^M's and the ascii spinner at boot).  Note
> that lines >80 chars wrap for the ddb output, I've corrected that in
> the above trace output.
>
> I'm leaving on vacation in a few hours, wanted to send this out before
> I leave.  I believe the next step is to build a bsd.gdb and run that,
> but I won't be able to do that until after I come back.
>
> Cheers,
>
> Paul
>
> ddb{0}> ps
>    PID     TID   PPID    UID  S       FLAGS  WAIT          COMMAND
> *60680  474219   2041    864  7           0                rsync
>   2041   80282  24289    864  3         0x2  biowait       rsync
>   6746  452021  64179    864  3    0x100083  poll          top
>  64179  271077  31496    864  3    0x10008b  pause         ksh
>  24289  402112   8227    864  3    0x10008a  pause         sh
>   8227   93111   6131    864  3    0x10008a  pause         sh
>   6131  443394    236      0  3    0x100090  piperd        cron
>  62053  223702  60120      0  3        0x83  poll          systat
>  18568  429066      0      0  3     0x14200  bored         srdis
>   8829  168032    301      0  3    0x100083  ttyin         ksh
>    301  102538  31496    864  3    0x10008b  pause         ksh
>  74109     121      1    577  3        0x90  poll          openvpn
>  52918  480270      1      0  3    0x100080  kqread        httpd
>   6829  314993      1     67  3    0x100092  kqread        httpd
>  61846  188976      1     67  3    0x100092  kqread        httpd
>  33530   17158      1     67  3    0x100092  kqread        httpd
>  41296   39009      1     67  3    0x100092  kqread        httpd
>  75685  405806      1     67  3    0x100092  kqread        httpd
>  57000   70236      1     67  3    0x100092  kqread        httpd
>  35760  472569      1     67  3    0x100092  kqread        httpd
>  14021  116305      1     67  3    0x100092  kqread        httpd
>   2803   15453      1     67  3    0x100092  kqread        httpd
>  75532  223061      1     67  3    0x100092  kqread        httpd
>  37086   58085      1     67  3    0x100092  kqread        httpd
>  40156  430012   2085    864  3    0x100083  kqread        tmux
>   2085  226180  88532    864  3    0x10008b  pause         ksh
>  88532  166130  69188    864  3        0x90  select        sshd
>  69188   74364  33879      0  3        0x92  poll          sshd
>  88162  523355  31496    864  3    0x100083  ttyin         ksh
>  30561   72987      0      0  3     0x14200  bored         srdis
>  60120  114946   8531      0  3    0x10008b  pause         ksh
>   5308  312248  19787    864  3    0x100083  piperd        tee
>  74364  137675  19787    864  3        0x83  nanosleep     perl
>  19787  339670  31496    864  3    0x10008b  pause         ksh
>  78410   76409  31496    864  3    0x100083  ttyin         ksh
>   8531   66284  31496    864  3    0x10008b  pause         ksh
>  31496  516703      1    864  3    0x100080  kqread        tmux
>  41528  277034      1    864  3    0x1000b0  poll          ssh-agent
>  81571  489757      1      0  3    0x100083  ttyin         getty
>    236  421789      1      0  3    0x100098  poll          cron
>   2643   15298      1      0  3        0x80  kqread        apmd
>  61310  374975  63511    606  3        0x90  kqread        ladvd
>  63511  444373      1      0  3        0x80  kqread        ladvd
>  66403   45971      1     79  3    0x100090  kqread        tftpd
>  69243  138389      1     67  3    0x100090  kqread        slowcgi
>  28533   82842  89153     95  3    0x100092  kqread        smtpd
>  45227  188297  89153    103  3    0x100092  kqread        smtpd
>  84638   58186  89153     95  3    0x100092  kqread        smtpd
>  44075  269454  89153     95  3    0x100092  kqread        smtpd
>  41264  280548  89153     95  3    0x100092  kqread        smtpd
>  78215  461288  89153     95  3    0x100092  kqread        smtpd
>  89153  403538      1      0  3    0x100080  kqread        smtpd
>  88627  429981  83417     94  3    0x100092  kqread        rad
>  74649  143691  83417     94  3    0x100092  kqread        rad
>  83417  481752      1      0  3    0x100080  kqread        rad
>  72824  505279      1     77  3    0x100090  poll          dhcpd
>  33879   15750      1      0  3        0x80  select        sshd
>  62997  431705  88090      0  3        0x80  nfsd          nfsd
>  75743   18740  88090      0  3        0x80  nfsd          nfsd
>  78492  366018  88090      0  3        0x80  nfsd          nfsd
>  60307  311789  88090      0  3        0x80  nfsd          nfsd
>  88090  408746      1      0  3        0x80  netcon        nfsd
>  20431  348522  43306      0  3        0x80  poll          mountd
>  43306  508203      1      0  3    0x100080  poll          mountd
>    409  159789      1     28  3    0x100090  poll          portmap
>  22301  404781      1      0  3    0x100080  poll          ntpd
>  36989  223581  64833     83  3    0x100092  poll          ntpd
>  64833  116505      1     83  3    0x100092  poll          ntpd
>  59285  326976      1     53  3        0x90  kqread        unbound
>  30573  181967  46920     97  3    0x100090  kqread        nsd
>  46920  256017  19340     97  3    0x100090  poll          nsd
>  19340   96004      1     97  3    0x100090  kqread        nsd
>  99631  372320  66831     74  3    0x100092  bpf           pflogd
>  66831  120785      1      0  3        0x80  netio         pflogd
>  83582   22272   8539     73  3    0x100090  kqread        syslogd
>   8539  217233      1      0  3    0x100082  netio         syslogd
>  62341  438623      1     77  3    0x100090  poll          dhclient
>  34468   93431      1      0  3        0x80  poll          dhclient
>  27477  229658      1     77  3    0x100090  poll          dhclient
>  84276  423340      1      0  3        0x80  poll          dhclient
>  67890  219083  49402    115  3    0x100092  kqread        slaacd
>  46124  389340  49402    115  3    0x100092  kqread        slaacd
>  49402  343308      1      0  3    0x100080  kqread        slaacd
>  21349  309777      0      0  3     0x14280  schto         i915/signal:6
>  93120   79546      0      0  3     0x14280  schto         i915/signal:2
>  26979  307356      0      0  3     0x14280  schto         i915/signal:1
>  60274  328100      0      0  3     0x14280  schto         i915/signal:0
>  14654  392598      0      0  3     0x14200  bored         i915-userptr-acq
>  59417  164718      0      0  3     0x14200  bored         i915_modeset
>  21074   35944      0      0  3     0x14200  bored         i915-dp
>    213  471497      0      0  3     0x14200  bored         i915
>  61715  273039      0      0  3     0x14200  pgzero        zerothread
>  72035  254000      0      0  3     0x14200  aiodoned      aiodoned
>  98656  458610      0      0  3     0x14200  syncer        update
>  54150  296058      0      0  3     0x14200  cleaner       cleaner
>  24032   63346      0      0  3     0x14200  reaper        reaper
>  79920  275510      0      0  3     0x14200  pgdaemon      pagedaemon
>  94527  299597      0      0  3     0x14200  bored         crynlk
>  94838  399994      0      0  3     0x14200  bored         crypto
>  31370  382682      0      0  3     0x14200  usbtsk        usbtask
>  10571  220841      0      0  3     0x14200  usbatsk       usbatsk
>  71415  231437      0      0  3     0x14200  bored         drmtskl
>  26261   94667      0      0  3     0x14200  bored         drmlwq
>  37392   31366      0      0  3     0x14200  bored         drmlwq
>  19125  441887      0      0  3     0x14200  bored         drmlwq
>  22022  365455      0      0  3     0x14200  bored         drmlwq
>  46349    3735      0      0  3     0x14200  bored         drmubwq
>  31592  185355      0      0  3     0x14200  bored         drmubwq
>  15533  481451      0      0  3     0x14200  bored         drmubwq
>  30646  262337      0      0  3     0x14200  bored         drmubwq
>  53694    3565      0      0  3     0x14200  bored         drmwq
>   6451  315183      0      0  3     0x14200  bored         drmwq
>  88684    5459      0      0  3     0x14200  bored         drmwq
>  33807  215626      0      0  3     0x14200  bored         drmwq
>  99999  312497      0      0  3  0x40014200  acpi0         acpi0
>  56475  507932      0      0  7  0x40014200                idle1
>  71903   91415      0      0  3     0x14200  bored         sensors
>   4936  359762      0      0  3     0x14200  bored         softnet
>  69286  183115      0      0  3     0x14200  bored         systqmp
>  27045  447467      0      0  3     0x14200  bored         systq
>  15589  135997      0      0  3  0x40014200  bored         softclock
>   3257  290261      0      0  3  0x40014200                idle0
>  19343  371152      0      0  3     0x14200  bored         smr
>      1  349148      0      0  3        0x82  wait          init
>      0       0     -1      0  3     0x10200  scheduler     swapper
> ddb{0}> trace
> _rb_remove(ffffffff81c7cc18,ffffffff81dde478,fffffd8124b0f600) at _rb_remove+0x
> 3c8
> uvm_pagefree(fffffd8124b0f600) at uvm_pagefree+0x37
> buf_free_pages(fffffd85e204eb00) at buf_free_pages+0xbd
> buf_dealloc_mem(fffffd85e204eb00) at buf_dealloc_mem+0x7b
> buf_put(fffffd85e204eb00) at buf_put+0xcf
> bufbackoff(ffffffff81dc3308,8) at bufbackoff+0x108
> buf_realloc_pages(fffffd83ada48000,ffffffff81dc3308,2) at buf_realloc_pages+0xc
> 6
> buf_flip_high(fffffd83ada48000) at buf_flip_high+0x6f
> bufcache_recover_dmapages(0,8) at bufcache_recover_dmapages+0x10c
> buf_get(fffffd85d5cfd338,af,8000) at buf_get+0xd7
> getblk(fffffd85d5cfd338,af,8000,0,ffffffffffffffff) at getblk+0xaf
> ffs1_balloc(fffffd883c278c40,578000,9f,fffffd876b933428,1,ffff8000331d1138) at f
> fs1_balloc+0xd43
> ffs_write(ffff8000331d11c0) at ffs_write+0x262
> VOP_WRITE(fffffd85d5cfd338,ffff8000331d1328,1,fffffd876b933428) at VOP_WRITE+0x
> 4f
> vn_write(fffffd807ac47720,ffff8000331d1328,0) at vn_write+0x11e
> dofilewritev(ffff800033063a78,6,ffff8000331d1328,0,ffff8000331d1400) at dofilew
> ritev+0xf9
> sys_write(ffff800033063a78,ffff8000331d13a0,ffff8000331d1400) at sys_write+0x51
>
> syscall(ffff8000331d1470) at syscall+0x389
> Xsyscall(0,4,c70,4,33fbee48ca0,33f512efd40) at Xsyscall+0x128
> end of kernel
> end trace frame: 0x7f7ffffe6cd0, count: -19
> ddb{0}> bo re
> rebooting...
> <--- snip ascii spinner --->
> [ using 2252752 bytes of bsd ELF symbol table ]
> Copyright (c) 1982, 1986, 1989, 1991, 1993
> The Regents of the University of California.  All rights reserved.
> Copyright (c) 1995-2019 OpenBSD. All rights reserved.  https://www.OpenBSD.org
>
> OpenBSD 6.5-current (GENERIC.MP) #146: Fri Jul 26 17:22:26 MDT 2019
>     [hidden email]:/usr/src/sys/arch/amd64/compile/GENERIC.MP
> real mem = 34263703552 (32676MB)
> avail mem = 33215143936 (31676MB)
> mpath0 at root
> scsibus0 at mpath0: 256 targets
> mainbus0 at root
> bios0 at mainbus0: SMBIOS rev. 3.0 @ 0x8d717000 (86 entries)
> bios0: vendor American Megatrends Inc. version "5.12" date 05/28/2018
> acpi0 at bios0: ACPI 6.1
> acpi0: sleep states S0 S5
> acpi0: tables DSDT FACP APIC FPDT MCFG SSDT FIDT SSDT HPET SSDT SSDT UEFI SSDT LPIT WSMT SSDT SSDT SSDT SSDT DBGP DBG2 SPCR DMAR ASF!
> acpi0: wakeup devices PS2K(S0) PS2M(S0) PXSX(S0) RP09(S0) PXSX(S0) RP10(S0) PXSX(S0) RP11(S0) PXSX(S0) RP12(S0) PXSX(S0) RP13(S0) PXSX(S0) RP01(S0) PXSX(S0) RP02(S0) [...]
> acpitimer0 at acpi0: 3579545 Hz, 24 bits
> acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
> cpu0 at mainbus0: apid 0 (boot processor)
> cpu0: Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz, 2395.20 MHz, 06-8e-09
> cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,SGX,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PT,MD_CLEAR,TSXFA,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES,MELTDOWN
> cpu0: 256KB 64b/line 8-way L2 cache
> cpu0: smt 0, core 0, package 0
> mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
> cpu0: apic clock running at 24MHz
> cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4.1.1.1, IBE
> cpu1 at mainbus0: apid 2 (application processor)
> cpu1: Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz, 2394.44 MHz, 06-8e-09
> cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,SGX,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PT,MD_CLEAR,TSXFA,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES,MELTDOWN
> cpu1: 256KB 64b/line 8-way L2 cache
> cpu1: smt 0, core 1, package 0
> ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 20, 120 pins
> acpimcfg0 at acpi0
> acpimcfg0: addr 0xe0000000, bus 0-255
> acpihpet0 at acpi0: 23999999 Hz
> acpiprt0 at acpi0: bus 0 (PCI0)
> acpiprt1 at acpi0: bus -1 (PEG0)
> acpiprt2 at acpi0: bus -1 (PEG1)
> acpiprt3 at acpi0: bus -1 (PEG2)
> acpiprt4 at acpi0: bus -1 (RP09)
> acpiprt5 at acpi0: bus -1 (RP10)
> acpiprt6 at acpi0: bus -1 (RP11)
> acpiprt7 at acpi0: bus -1 (RP12)
> acpiprt8 at acpi0: bus -1 (RP13)
> acpiprt9 at acpi0: bus 1 (RP01)
> acpiprt10 at acpi0: bus 2 (RP02)
> acpiprt11 at acpi0: bus 3 (RP03)
> acpiprt12 at acpi0: bus 4 (RP04)
> acpiprt13 at acpi0: bus 5 (RP05)
> acpiprt14 at acpi0: bus 6 (RP06)
> acpiprt15 at acpi0: bus -1 (RP07)
> acpiprt16 at acpi0: bus -1 (RP08)
> acpiprt17 at acpi0: bus -1 (RP17)
> acpiprt18 at acpi0: bus -1 (RP18)
> acpiprt19 at acpi0: bus -1 (RP19)
> acpiprt20 at acpi0: bus -1 (RP20)
> acpiprt21 at acpi0: bus -1 (RP21)
> acpiprt22 at acpi0: bus -1 (RP22)
> acpiprt23 at acpi0: bus -1 (RP23)
> acpiprt24 at acpi0: bus -1 (RP24)
> acpiprt25 at acpi0: bus -1 (RP14)
> acpiprt26 at acpi0: bus -1 (RP15)
> acpiprt27 at acpi0: bus -1 (RP16)
> acpiec0 at acpi0: not present
> acpicpu0 at acpi0: C3(200@1034 mwait.1@0x60), C2(200@151 mwait.1@0x33), C1(1000@1 mwait.1), PSS
> acpicpu1 at acpi0: C3(200@1034 mwait.1@0x60), C2(200@151 mwait.1@0x33), C1(1000@1 mwait.1), PSS
> acpipwrres0 at acpi0: WRST
> acpipwrres1 at acpi0: WRST
> acpipwrres2 at acpi0: WRST
> acpipwrres3 at acpi0: WRST
> acpipwrres4 at acpi0: WRST
> acpipwrres5 at acpi0: WRST
> acpipwrres6 at acpi0: WRST
> acpipwrres7 at acpi0: WRST
> acpipwrres8 at acpi0: WRST
> acpipwrres9 at acpi0: WRST
> acpipwrres10 at acpi0: WRST
> acpipwrres11 at acpi0: WRST
> acpipwrres12 at acpi0: WRST
> acpipwrres13 at acpi0: WRST
> acpipwrres14 at acpi0: WRST
> acpipwrres15 at acpi0: WRST
> acpipwrres16 at acpi0: WRST
> acpipwrres17 at acpi0: WRST
> acpipwrres18 at acpi0: WRST
> acpipwrres19 at acpi0: WRST
> acpipwrres20 at acpi0: FN00, resource for FAN0
> acpipwrres21 at acpi0: FN01, resource for FAN1
> acpipwrres22 at acpi0: FN02, resource for FAN2
> acpipwrres23 at acpi0: FN03, resource for FAN3
> acpipwrres24 at acpi0: FN04, resource for FAN4
> acpitz0 at acpi0: critical temperature is 119 degC
> acpitz1 at acpi0: critical temperature is 119 degC
> acpipci0 at acpi0 PCI0: 0x00000000 0x00000011 0x00000001
> acpicmos0 at acpi0
> acpibtn0 at acpi0: SLPB
> "INT33A1" at acpi0 not configured
> acpibtn1 at acpi0: PWRB
> "PNP0C0B" at acpi0 not configured
> "PNP0C0B" at acpi0 not configured
> "PNP0C0B" at acpi0 not configured
> "PNP0C0B" at acpi0 not configured
> "PNP0C0B" at acpi0 not configured
> acpivideo0 at acpi0: GFX0
> acpivout0 at acpivideo0: DD1F
> cpu0: using VERW MDS workaround (except on vmm entry)
> cpu0: Enhanced SpeedStep 2395 MHz: speeds: 2601, 2600, 2500, 2400, 2200, 2000, 1900, 1700, 1500, 1400, 1200, 1100, 800, 700, 600, 400 MHz
> pci0 at mainbus0 bus 0
> pchb0 at pci0 dev 0 function 0 "Intel Core 7G Host" rev 0x02
> inteldrm0 at pci0 dev 2 function 0 "Intel HD Graphics 620" rev 0x02
> drm0 at inteldrm0
> inteldrm0: msi
> xhci0 at pci0 dev 20 function 0 "Intel 100 Series xHCI" rev 0x21: msi, xHCI 1.0
> usb0 at xhci0: USB revision 3.0
> uhub0 at usb0 configuration 1 interface 0 "Intel xHCI root hub" rev 3.00/1.00 addr 1
> "Intel 100 Series MEI" rev 0x21 at pci0 dev 22 function 0 not configured
> ahci0 at pci0 dev 23 function 0 "Intel 100 Series AHCI" rev 0x21: msi, AHCI 1.3.1
> ahci0: port 0: 6.0Gb/s
> ahci0: port 1: 6.0Gb/s
> ahci0: PHY offline on port 2
> scsibus1 at ahci0: 32 targets
> sd0 at scsibus1 targ 0 lun 0: <ATA, Samsung SSD 860, RVT4> SCSI3 0/direct fixed naa.5002538e405cebf1
> sd0: 476940MB, 512 bytes/sector, 976773168 sectors, thin
> sd1 at scsibus1 targ 1 lun 0: <ATA, Samsung SSD 860, RVQ0> SCSI3 0/direct fixed naa.5002538e40edbf97
> sd1: 3815447MB, 512 bytes/sector, 7814037168 sectors, thin
> ppb0 at pci0 dev 28 function 0 "Intel 100 Series PCIE" rev 0xf1: msi
> pci1 at ppb0 bus 1
> em0 at pci1 dev 0 function 0 "Intel 82583V" rev 0x00: msi, address 00:e0:67:17:d6:96
> ppb1 at pci0 dev 28 function 1 "Intel 100 Series PCIE" rev 0xf1: msi
> pci2 at ppb1 bus 2
> em1 at pci2 dev 0 function 0 "Intel 82583V" rev 0x00: msi, address 00:e0:67:17:d6:97
> ppb2 at pci0 dev 28 function 2 "Intel 100 Series PCIE" rev 0xf1: msi
> pci3 at ppb2 bus 3
> em2 at pci3 dev 0 function 0 "Intel 82583V" rev 0x00: msi, address 00:e0:67:17:d6:98
> ppb3 at pci0 dev 28 function 3 "Intel 100 Series PCIE" rev 0xf1: msi
> pci4 at ppb3 bus 4
> em3 at pci4 dev 0 function 0 "Intel 82583V" rev 0x00: msi, address 00:e0:67:17:d6:99
> ppb4 at pci0 dev 28 function 4 "Intel 100 Series PCIE" rev 0xf1: msi
> pci5 at ppb4 bus 5
> em4 at pci5 dev 0 function 0 "Intel 82583V" rev 0x00: msi, address 00:e0:67:17:d6:9a
> ppb5 at pci0 dev 28 function 5 "Intel 100 Series PCIE" rev 0xf1: msi
> pci6 at ppb5 bus 6
> em5 at pci6 dev 0 function 0 "Intel 82583V" rev 0x00: msi, address 00:e0:67:17:d6:9b
> pcib0 at pci0 dev 31 function 0 "Intel 200 Series LPC" rev 0x21
> "Intel 100 Series PMC" rev 0x21 at pci0 dev 31 function 2 not configured
> ichiic0 at pci0 dev 31 function 4 "Intel 100 Series SMBus" rev 0x21: apic 2 int 16
> iic0 at ichiic0
> admtemp0 at iic0 addr 0x18: Xeon
> admtemp1 at iic0 addr 0x1a: Xeon
> isa0 at pcib0
> isadma0 at isa0
> com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
> com0: console
> pckbc0 at isa0 port 0x60/5 irq 1 irq 12
> pckbd0 at pckbc0 (kbd slot)
> wskbd0 at pckbd0: console keyboard
> pcppi0 at isa0 port 0x61
> spkr0 at pcppi0
> it0 at isa0 port 0x2e/2: IT8772F rev 1, EC port 0xa40
> vmm0 at mainbus0: VMX/EPT
> uhidev0 at uhub0 port 2 configuration 1 interface 0 "RDing TEMPERHUM1V1.2" rev 2.00/0.01 addr 2
> uhidev0: iclass 3/1, 1 report id
> ukbd0 at uhidev0 reportid 1: 8 variable keys, 5 key codes
> wskbd1 at ukbd0 mux 1
> uhidev1 at uhub0 port 2 configuration 1 interface 1 "RDing TEMPERHUM1V1.2" rev 2.00/0.01 addr 2
> uhidev1: iclass 3/1
> ugold0 at uhidev1
> uftdi0 at uhub0 port 3 configuration 1 interface 0 "FTDI FT230X Basic UART" rev 2.00/10.00 addr 3
> ucom0 at uftdi0 portno 1
> uftdi1 at uhub0 port 4 configuration 1 interface 0 "FTDI SYNLEAF - FT232R USB UART" rev 2.00/6.00 addr 4
> ucom1 at uftdi1 portno 1
> umass0 at uhub0 port 13 configuration 1 interface 0 "Western Digital My Passport 259D" rev 3.00/10.12 addr 5
> umass0: using SCSI over Bulk-Only
> scsibus2 at umass0: 2 targets, initiator 0
> sd2 at scsibus2 targ 1 lun 0: <WD, My Passport 259D, 1012> SCSI4 0/direct fixed
> sd2: 3815415MB, 512 bytes/sector, 7813969920 sectors
> ses0 at scsibus2 targ 1 lun 1: <WD, SES Device, 1012> SCSI4 13/enclosure services fixed
> ses0: unable to read enclosure configuration
> vscsi0 at root
> scsibus3 at vscsi0: 256 targets
> softraid0 at root
> scsibus4 at softraid0: 256 targets
> root on sd0a (aec06f0350f24bab.a) swap on sd0b dump on sd0b
> WARNING: / was not properly unmounted
>
> --
> >++++++++[<++++++++++>-]<+++++++.>+++[<------>-]<.>+++[<+
> +++++++++++>-]<.>++[<------------>-]<+.--------------.[-]
>                  http://www.weirdnet.nl/                 
>

Reply | Threaded
Open this post in threaded view
|

Re: Recurring panic in _rb_remove

Paul de Weerd
On Tue, Jul 30, 2019 at 10:18:54PM -0600, Theo de Raadt wrote:
| vn_write.
|
| You are using vnd.  There are other reports this got broken recently.

Not that I'm aware of.  All filesystems are either directly on
physical devices or through softraid crypto on physical devives.

Also, no cronjobs that use vnconfig.

Paul

| Paul de Weerd <[hidden email]> wrote:
|
| > Hi all,
| >
| > Seeing a recurring panic on my local router.  It's also acting as a
| > backup and fileserver (serving OpenBSD snapshots to internal hosts).
| > The panic is:
| >
| > uvm_fault(0xfffffd86fc8abee8, 0x8, 0, 1) -> e
| > kernel: page fault trap, code=0
| > Stopped at      _rb_remove+0x3c8:       movq    0x8(%rdi),%rax
| > ddb{0}> trace
| > _rb_remove(ffffffff81c7cc18,ffffffff81dde478,fffffd8124b0f600) at _rb_remove+0x3c8
| > uvm_pagefree(fffffd8124b0f600) at uvm_pagefree+0x37
| > buf_free_pages(fffffd85e204eb00) at buf_free_pages+0xbd
| > buf_dealloc_mem(fffffd85e204eb00) at buf_dealloc_mem+0x7b
| > buf_put(fffffd85e204eb00) at buf_put+0xcf
| > bufbackoff(ffffffff81dc3308,8) at bufbackoff+0x108
| > buf_realloc_pages(fffffd83ada48000,ffffffff81dc3308,2) at buf_realloc_pages+0xc6
| > buf_flip_high(fffffd83ada48000) at buf_flip_high+0x6f
| > bufcache_recover_dmapages(0,8) at bufcache_recover_dmapages+0x10c
| > buf_get(fffffd85d5cfd338,af,8000) at buf_get+0xd7
| > getblk(fffffd85d5cfd338,af,8000,0,ffffffffffffffff) at getblk+0xaf
| > ffs1_balloc(fffffd883c278c40,578000,9f,fffffd876b933428,1,ffff8000331d1138) at ffs1_balloc+0xd43
| > ffs_write(ffff8000331d11c0) at ffs_write+0x262
| > VOP_WRITE(fffffd85d5cfd338,ffff8000331d1328,1,fffffd876b933428) at VOP_WRITE+0x4f
| > vn_write(fffffd807ac47720,ffff8000331d1328,0) at vn_write+0x11e
| > dofilewritev(ffff800033063a78,6,ffff8000331d1328,0,ffff8000331d1400) at dofilewritev+0xf9
| > sys_write(ffff800033063a78,ffff8000331d13a0,ffff8000331d1400) at sys_write+0x51
| > syscall(ffff8000331d1470) at syscall+0x389
| > Xsyscall(0,4,c70,4,33fbee48ca0,33f512efd40) at Xsyscall+0x128
| > end of kernel
| > end trace frame: 0x7f7ffffe6cd0, count: -19
| >
| > I've upgraded to a newer snapshot but still see this behaviour; it
| > usually happens while making backups using rsync, but not every day.
| > ps shows rsync as the active process:
| >
| > ddb{0}> ps
| >    PID     TID   PPID    UID  S       FLAGS  WAIT          COMMAND
| > *60680  474219   2041    864  7           0                rsync
| >
| > Included below is the cleaned-up console log with ps, trace and dmesg
| > included (I removed the ^M's and the ascii spinner at boot).  Note
| > that lines >80 chars wrap for the ddb output, I've corrected that in
| > the above trace output.
| >
| > I'm leaving on vacation in a few hours, wanted to send this out before
| > I leave.  I believe the next step is to build a bsd.gdb and run that,
| > but I won't be able to do that until after I come back.
| >
| > Cheers,
| >
| > Paul
| >
| > ddb{0}> ps
| >    PID     TID   PPID    UID  S       FLAGS  WAIT          COMMAND
| > *60680  474219   2041    864  7           0                rsync
| >   2041   80282  24289    864  3         0x2  biowait       rsync
| >   6746  452021  64179    864  3    0x100083  poll          top
| >  64179  271077  31496    864  3    0x10008b  pause         ksh
| >  24289  402112   8227    864  3    0x10008a  pause         sh
| >   8227   93111   6131    864  3    0x10008a  pause         sh
| >   6131  443394    236      0  3    0x100090  piperd        cron
| >  62053  223702  60120      0  3        0x83  poll          systat
| >  18568  429066      0      0  3     0x14200  bored         srdis
| >   8829  168032    301      0  3    0x100083  ttyin         ksh
| >    301  102538  31496    864  3    0x10008b  pause         ksh
| >  74109     121      1    577  3        0x90  poll          openvpn
| >  52918  480270      1      0  3    0x100080  kqread        httpd
| >   6829  314993      1     67  3    0x100092  kqread        httpd
| >  61846  188976      1     67  3    0x100092  kqread        httpd
| >  33530   17158      1     67  3    0x100092  kqread        httpd
| >  41296   39009      1     67  3    0x100092  kqread        httpd
| >  75685  405806      1     67  3    0x100092  kqread        httpd
| >  57000   70236      1     67  3    0x100092  kqread        httpd
| >  35760  472569      1     67  3    0x100092  kqread        httpd
| >  14021  116305      1     67  3    0x100092  kqread        httpd
| >   2803   15453      1     67  3    0x100092  kqread        httpd
| >  75532  223061      1     67  3    0x100092  kqread        httpd
| >  37086   58085      1     67  3    0x100092  kqread        httpd
| >  40156  430012   2085    864  3    0x100083  kqread        tmux
| >   2085  226180  88532    864  3    0x10008b  pause         ksh
| >  88532  166130  69188    864  3        0x90  select        sshd
| >  69188   74364  33879      0  3        0x92  poll          sshd
| >  88162  523355  31496    864  3    0x100083  ttyin         ksh
| >  30561   72987      0      0  3     0x14200  bored         srdis
| >  60120  114946   8531      0  3    0x10008b  pause         ksh
| >   5308  312248  19787    864  3    0x100083  piperd        tee
| >  74364  137675  19787    864  3        0x83  nanosleep     perl
| >  19787  339670  31496    864  3    0x10008b  pause         ksh
| >  78410   76409  31496    864  3    0x100083  ttyin         ksh
| >   8531   66284  31496    864  3    0x10008b  pause         ksh
| >  31496  516703      1    864  3    0x100080  kqread        tmux
| >  41528  277034      1    864  3    0x1000b0  poll          ssh-agent
| >  81571  489757      1      0  3    0x100083  ttyin         getty
| >    236  421789      1      0  3    0x100098  poll          cron
| >   2643   15298      1      0  3        0x80  kqread        apmd
| >  61310  374975  63511    606  3        0x90  kqread        ladvd
| >  63511  444373      1      0  3        0x80  kqread        ladvd
| >  66403   45971      1     79  3    0x100090  kqread        tftpd
| >  69243  138389      1     67  3    0x100090  kqread        slowcgi
| >  28533   82842  89153     95  3    0x100092  kqread        smtpd
| >  45227  188297  89153    103  3    0x100092  kqread        smtpd
| >  84638   58186  89153     95  3    0x100092  kqread        smtpd
| >  44075  269454  89153     95  3    0x100092  kqread        smtpd
| >  41264  280548  89153     95  3    0x100092  kqread        smtpd
| >  78215  461288  89153     95  3    0x100092  kqread        smtpd
| >  89153  403538      1      0  3    0x100080  kqread        smtpd
| >  88627  429981  83417     94  3    0x100092  kqread        rad
| >  74649  143691  83417     94  3    0x100092  kqread        rad
| >  83417  481752      1      0  3    0x100080  kqread        rad
| >  72824  505279      1     77  3    0x100090  poll          dhcpd
| >  33879   15750      1      0  3        0x80  select        sshd
| >  62997  431705  88090      0  3        0x80  nfsd          nfsd
| >  75743   18740  88090      0  3        0x80  nfsd          nfsd
| >  78492  366018  88090      0  3        0x80  nfsd          nfsd
| >  60307  311789  88090      0  3        0x80  nfsd          nfsd
| >  88090  408746      1      0  3        0x80  netcon        nfsd
| >  20431  348522  43306      0  3        0x80  poll          mountd
| >  43306  508203      1      0  3    0x100080  poll          mountd
| >    409  159789      1     28  3    0x100090  poll          portmap
| >  22301  404781      1      0  3    0x100080  poll          ntpd
| >  36989  223581  64833     83  3    0x100092  poll          ntpd
| >  64833  116505      1     83  3    0x100092  poll          ntpd
| >  59285  326976      1     53  3        0x90  kqread        unbound
| >  30573  181967  46920     97  3    0x100090  kqread        nsd
| >  46920  256017  19340     97  3    0x100090  poll          nsd
| >  19340   96004      1     97  3    0x100090  kqread        nsd
| >  99631  372320  66831     74  3    0x100092  bpf           pflogd
| >  66831  120785      1      0  3        0x80  netio         pflogd
| >  83582   22272   8539     73  3    0x100090  kqread        syslogd
| >   8539  217233      1      0  3    0x100082  netio         syslogd
| >  62341  438623      1     77  3    0x100090  poll          dhclient
| >  34468   93431      1      0  3        0x80  poll          dhclient
| >  27477  229658      1     77  3    0x100090  poll          dhclient
| >  84276  423340      1      0  3        0x80  poll          dhclient
| >  67890  219083  49402    115  3    0x100092  kqread        slaacd
| >  46124  389340  49402    115  3    0x100092  kqread        slaacd
| >  49402  343308      1      0  3    0x100080  kqread        slaacd
| >  21349  309777      0      0  3     0x14280  schto         i915/signal:6
| >  93120   79546      0      0  3     0x14280  schto         i915/signal:2
| >  26979  307356      0      0  3     0x14280  schto         i915/signal:1
| >  60274  328100      0      0  3     0x14280  schto         i915/signal:0
| >  14654  392598      0      0  3     0x14200  bored         i915-userptr-acq
| >  59417  164718      0      0  3     0x14200  bored         i915_modeset
| >  21074   35944      0      0  3     0x14200  bored         i915-dp
| >    213  471497      0      0  3     0x14200  bored         i915
| >  61715  273039      0      0  3     0x14200  pgzero        zerothread
| >  72035  254000      0      0  3     0x14200  aiodoned      aiodoned
| >  98656  458610      0      0  3     0x14200  syncer        update
| >  54150  296058      0      0  3     0x14200  cleaner       cleaner
| >  24032   63346      0      0  3     0x14200  reaper        reaper
| >  79920  275510      0      0  3     0x14200  pgdaemon      pagedaemon
| >  94527  299597      0      0  3     0x14200  bored         crynlk
| >  94838  399994      0      0  3     0x14200  bored         crypto
| >  31370  382682      0      0  3     0x14200  usbtsk        usbtask
| >  10571  220841      0      0  3     0x14200  usbatsk       usbatsk
| >  71415  231437      0      0  3     0x14200  bored         drmtskl
| >  26261   94667      0      0  3     0x14200  bored         drmlwq
| >  37392   31366      0      0  3     0x14200  bored         drmlwq
| >  19125  441887      0      0  3     0x14200  bored         drmlwq
| >  22022  365455      0      0  3     0x14200  bored         drmlwq
| >  46349    3735      0      0  3     0x14200  bored         drmubwq
| >  31592  185355      0      0  3     0x14200  bored         drmubwq
| >  15533  481451      0      0  3     0x14200  bored         drmubwq
| >  30646  262337      0      0  3     0x14200  bored         drmubwq
| >  53694    3565      0      0  3     0x14200  bored         drmwq
| >   6451  315183      0      0  3     0x14200  bored         drmwq
| >  88684    5459      0      0  3     0x14200  bored         drmwq
| >  33807  215626      0      0  3     0x14200  bored         drmwq
| >  99999  312497      0      0  3  0x40014200  acpi0         acpi0
| >  56475  507932      0      0  7  0x40014200                idle1
| >  71903   91415      0      0  3     0x14200  bored         sensors
| >   4936  359762      0      0  3     0x14200  bored         softnet
| >  69286  183115      0      0  3     0x14200  bored         systqmp
| >  27045  447467      0      0  3     0x14200  bored         systq
| >  15589  135997      0      0  3  0x40014200  bored         softclock
| >   3257  290261      0      0  3  0x40014200                idle0
| >  19343  371152      0      0  3     0x14200  bored         smr
| >      1  349148      0      0  3        0x82  wait          init
| >      0       0     -1      0  3     0x10200  scheduler     swapper
| > ddb{0}> trace
| > _rb_remove(ffffffff81c7cc18,ffffffff81dde478,fffffd8124b0f600) at _rb_remove+0x
| > 3c8
| > uvm_pagefree(fffffd8124b0f600) at uvm_pagefree+0x37
| > buf_free_pages(fffffd85e204eb00) at buf_free_pages+0xbd
| > buf_dealloc_mem(fffffd85e204eb00) at buf_dealloc_mem+0x7b
| > buf_put(fffffd85e204eb00) at buf_put+0xcf
| > bufbackoff(ffffffff81dc3308,8) at bufbackoff+0x108
| > buf_realloc_pages(fffffd83ada48000,ffffffff81dc3308,2) at buf_realloc_pages+0xc
| > 6
| > buf_flip_high(fffffd83ada48000) at buf_flip_high+0x6f
| > bufcache_recover_dmapages(0,8) at bufcache_recover_dmapages+0x10c
| > buf_get(fffffd85d5cfd338,af,8000) at buf_get+0xd7
| > getblk(fffffd85d5cfd338,af,8000,0,ffffffffffffffff) at getblk+0xaf
| > ffs1_balloc(fffffd883c278c40,578000,9f,fffffd876b933428,1,ffff8000331d1138) at f
| > fs1_balloc+0xd43
| > ffs_write(ffff8000331d11c0) at ffs_write+0x262
| > VOP_WRITE(fffffd85d5cfd338,ffff8000331d1328,1,fffffd876b933428) at VOP_WRITE+0x
| > 4f
| > vn_write(fffffd807ac47720,ffff8000331d1328,0) at vn_write+0x11e
| > dofilewritev(ffff800033063a78,6,ffff8000331d1328,0,ffff8000331d1400) at dofilew
| > ritev+0xf9
| > sys_write(ffff800033063a78,ffff8000331d13a0,ffff8000331d1400) at sys_write+0x51
| >
| > syscall(ffff8000331d1470) at syscall+0x389
| > Xsyscall(0,4,c70,4,33fbee48ca0,33f512efd40) at Xsyscall+0x128
| > end of kernel
| > end trace frame: 0x7f7ffffe6cd0, count: -19
| > ddb{0}> bo re
| > rebooting...
| > <--- snip ascii spinner --->
| > [ using 2252752 bytes of bsd ELF symbol table ]
| > Copyright (c) 1982, 1986, 1989, 1991, 1993
| > The Regents of the University of California.  All rights reserved.
| > Copyright (c) 1995-2019 OpenBSD. All rights reserved.  https://www.OpenBSD.org
| >
| > OpenBSD 6.5-current (GENERIC.MP) #146: Fri Jul 26 17:22:26 MDT 2019
| >     [hidden email]:/usr/src/sys/arch/amd64/compile/GENERIC.MP
| > real mem = 34263703552 (32676MB)
| > avail mem = 33215143936 (31676MB)
| > mpath0 at root
| > scsibus0 at mpath0: 256 targets
| > mainbus0 at root
| > bios0 at mainbus0: SMBIOS rev. 3.0 @ 0x8d717000 (86 entries)
| > bios0: vendor American Megatrends Inc. version "5.12" date 05/28/2018
| > acpi0 at bios0: ACPI 6.1
| > acpi0: sleep states S0 S5
| > acpi0: tables DSDT FACP APIC FPDT MCFG SSDT FIDT SSDT HPET SSDT SSDT UEFI SSDT LPIT WSMT SSDT SSDT SSDT SSDT DBGP DBG2 SPCR DMAR ASF!
| > acpi0: wakeup devices PS2K(S0) PS2M(S0) PXSX(S0) RP09(S0) PXSX(S0) RP10(S0) PXSX(S0) RP11(S0) PXSX(S0) RP12(S0) PXSX(S0) RP13(S0) PXSX(S0) RP01(S0) PXSX(S0) RP02(S0) [...]
| > acpitimer0 at acpi0: 3579545 Hz, 24 bits
| > acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
| > cpu0 at mainbus0: apid 0 (boot processor)
| > cpu0: Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz, 2395.20 MHz, 06-8e-09
| > cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,SGX,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PT,MD_CLEAR,TSXFA,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES,MELTDOWN
| > cpu0: 256KB 64b/line 8-way L2 cache
| > cpu0: smt 0, core 0, package 0
| > mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
| > cpu0: apic clock running at 24MHz
| > cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4.1.1.1, IBE
| > cpu1 at mainbus0: apid 2 (application processor)
| > cpu1: Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz, 2394.44 MHz, 06-8e-09
| > cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,SGX,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PT,MD_CLEAR,TSXFA,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES,MELTDOWN
| > cpu1: 256KB 64b/line 8-way L2 cache
| > cpu1: smt 0, core 1, package 0
| > ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 20, 120 pins
| > acpimcfg0 at acpi0
| > acpimcfg0: addr 0xe0000000, bus 0-255
| > acpihpet0 at acpi0: 23999999 Hz
| > acpiprt0 at acpi0: bus 0 (PCI0)
| > acpiprt1 at acpi0: bus -1 (PEG0)
| > acpiprt2 at acpi0: bus -1 (PEG1)
| > acpiprt3 at acpi0: bus -1 (PEG2)
| > acpiprt4 at acpi0: bus -1 (RP09)
| > acpiprt5 at acpi0: bus -1 (RP10)
| > acpiprt6 at acpi0: bus -1 (RP11)
| > acpiprt7 at acpi0: bus -1 (RP12)
| > acpiprt8 at acpi0: bus -1 (RP13)
| > acpiprt9 at acpi0: bus 1 (RP01)
| > acpiprt10 at acpi0: bus 2 (RP02)
| > acpiprt11 at acpi0: bus 3 (RP03)
| > acpiprt12 at acpi0: bus 4 (RP04)
| > acpiprt13 at acpi0: bus 5 (RP05)
| > acpiprt14 at acpi0: bus 6 (RP06)
| > acpiprt15 at acpi0: bus -1 (RP07)
| > acpiprt16 at acpi0: bus -1 (RP08)
| > acpiprt17 at acpi0: bus -1 (RP17)
| > acpiprt18 at acpi0: bus -1 (RP18)
| > acpiprt19 at acpi0: bus -1 (RP19)
| > acpiprt20 at acpi0: bus -1 (RP20)
| > acpiprt21 at acpi0: bus -1 (RP21)
| > acpiprt22 at acpi0: bus -1 (RP22)
| > acpiprt23 at acpi0: bus -1 (RP23)
| > acpiprt24 at acpi0: bus -1 (RP24)
| > acpiprt25 at acpi0: bus -1 (RP14)
| > acpiprt26 at acpi0: bus -1 (RP15)
| > acpiprt27 at acpi0: bus -1 (RP16)
| > acpiec0 at acpi0: not present
| > acpicpu0 at acpi0: C3(200@1034 mwait.1@0x60), C2(200@151 mwait.1@0x33), C1(1000@1 mwait.1), PSS
| > acpicpu1 at acpi0: C3(200@1034 mwait.1@0x60), C2(200@151 mwait.1@0x33), C1(1000@1 mwait.1), PSS
| > acpipwrres0 at acpi0: WRST
| > acpipwrres1 at acpi0: WRST
| > acpipwrres2 at acpi0: WRST
| > acpipwrres3 at acpi0: WRST
| > acpipwrres4 at acpi0: WRST
| > acpipwrres5 at acpi0: WRST
| > acpipwrres6 at acpi0: WRST
| > acpipwrres7 at acpi0: WRST
| > acpipwrres8 at acpi0: WRST
| > acpipwrres9 at acpi0: WRST
| > acpipwrres10 at acpi0: WRST
| > acpipwrres11 at acpi0: WRST
| > acpipwrres12 at acpi0: WRST
| > acpipwrres13 at acpi0: WRST
| > acpipwrres14 at acpi0: WRST
| > acpipwrres15 at acpi0: WRST
| > acpipwrres16 at acpi0: WRST
| > acpipwrres17 at acpi0: WRST
| > acpipwrres18 at acpi0: WRST
| > acpipwrres19 at acpi0: WRST
| > acpipwrres20 at acpi0: FN00, resource for FAN0
| > acpipwrres21 at acpi0: FN01, resource for FAN1
| > acpipwrres22 at acpi0: FN02, resource for FAN2
| > acpipwrres23 at acpi0: FN03, resource for FAN3
| > acpipwrres24 at acpi0: FN04, resource for FAN4
| > acpitz0 at acpi0: critical temperature is 119 degC
| > acpitz1 at acpi0: critical temperature is 119 degC
| > acpipci0 at acpi0 PCI0: 0x00000000 0x00000011 0x00000001
| > acpicmos0 at acpi0
| > acpibtn0 at acpi0: SLPB
| > "INT33A1" at acpi0 not configured
| > acpibtn1 at acpi0: PWRB
| > "PNP0C0B" at acpi0 not configured
| > "PNP0C0B" at acpi0 not configured
| > "PNP0C0B" at acpi0 not configured
| > "PNP0C0B" at acpi0 not configured
| > "PNP0C0B" at acpi0 not configured
| > acpivideo0 at acpi0: GFX0
| > acpivout0 at acpivideo0: DD1F
| > cpu0: using VERW MDS workaround (except on vmm entry)
| > cpu0: Enhanced SpeedStep 2395 MHz: speeds: 2601, 2600, 2500, 2400, 2200, 2000, 1900, 1700, 1500, 1400, 1200, 1100, 800, 700, 600, 400 MHz
| > pci0 at mainbus0 bus 0
| > pchb0 at pci0 dev 0 function 0 "Intel Core 7G Host" rev 0x02
| > inteldrm0 at pci0 dev 2 function 0 "Intel HD Graphics 620" rev 0x02
| > drm0 at inteldrm0
| > inteldrm0: msi
| > xhci0 at pci0 dev 20 function 0 "Intel 100 Series xHCI" rev 0x21: msi, xHCI 1.0
| > usb0 at xhci0: USB revision 3.0
| > uhub0 at usb0 configuration 1 interface 0 "Intel xHCI root hub" rev 3.00/1.00 addr 1
| > "Intel 100 Series MEI" rev 0x21 at pci0 dev 22 function 0 not configured
| > ahci0 at pci0 dev 23 function 0 "Intel 100 Series AHCI" rev 0x21: msi, AHCI 1.3.1
| > ahci0: port 0: 6.0Gb/s
| > ahci0: port 1: 6.0Gb/s
| > ahci0: PHY offline on port 2
| > scsibus1 at ahci0: 32 targets
| > sd0 at scsibus1 targ 0 lun 0: <ATA, Samsung SSD 860, RVT4> SCSI3 0/direct fixed naa.5002538e405cebf1
| > sd0: 476940MB, 512 bytes/sector, 976773168 sectors, thin
| > sd1 at scsibus1 targ 1 lun 0: <ATA, Samsung SSD 860, RVQ0> SCSI3 0/direct fixed naa.5002538e40edbf97
| > sd1: 3815447MB, 512 bytes/sector, 7814037168 sectors, thin
| > ppb0 at pci0 dev 28 function 0 "Intel 100 Series PCIE" rev 0xf1: msi
| > pci1 at ppb0 bus 1
| > em0 at pci1 dev 0 function 0 "Intel 82583V" rev 0x00: msi, address 00:e0:67:17:d6:96
| > ppb1 at pci0 dev 28 function 1 "Intel 100 Series PCIE" rev 0xf1: msi
| > pci2 at ppb1 bus 2
| > em1 at pci2 dev 0 function 0 "Intel 82583V" rev 0x00: msi, address 00:e0:67:17:d6:97
| > ppb2 at pci0 dev 28 function 2 "Intel 100 Series PCIE" rev 0xf1: msi
| > pci3 at ppb2 bus 3
| > em2 at pci3 dev 0 function 0 "Intel 82583V" rev 0x00: msi, address 00:e0:67:17:d6:98
| > ppb3 at pci0 dev 28 function 3 "Intel 100 Series PCIE" rev 0xf1: msi
| > pci4 at ppb3 bus 4
| > em3 at pci4 dev 0 function 0 "Intel 82583V" rev 0x00: msi, address 00:e0:67:17:d6:99
| > ppb4 at pci0 dev 28 function 4 "Intel 100 Series PCIE" rev 0xf1: msi
| > pci5 at ppb4 bus 5
| > em4 at pci5 dev 0 function 0 "Intel 82583V" rev 0x00: msi, address 00:e0:67:17:d6:9a
| > ppb5 at pci0 dev 28 function 5 "Intel 100 Series PCIE" rev 0xf1: msi
| > pci6 at ppb5 bus 6
| > em5 at pci6 dev 0 function 0 "Intel 82583V" rev 0x00: msi, address 00:e0:67:17:d6:9b
| > pcib0 at pci0 dev 31 function 0 "Intel 200 Series LPC" rev 0x21
| > "Intel 100 Series PMC" rev 0x21 at pci0 dev 31 function 2 not configured
| > ichiic0 at pci0 dev 31 function 4 "Intel 100 Series SMBus" rev 0x21: apic 2 int 16
| > iic0 at ichiic0
| > admtemp0 at iic0 addr 0x18: Xeon
| > admtemp1 at iic0 addr 0x1a: Xeon
| > isa0 at pcib0
| > isadma0 at isa0
| > com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
| > com0: console
| > pckbc0 at isa0 port 0x60/5 irq 1 irq 12
| > pckbd0 at pckbc0 (kbd slot)
| > wskbd0 at pckbd0: console keyboard
| > pcppi0 at isa0 port 0x61
| > spkr0 at pcppi0
| > it0 at isa0 port 0x2e/2: IT8772F rev 1, EC port 0xa40
| > vmm0 at mainbus0: VMX/EPT
| > uhidev0 at uhub0 port 2 configuration 1 interface 0 "RDing TEMPERHUM1V1.2" rev 2.00/0.01 addr 2
| > uhidev0: iclass 3/1, 1 report id
| > ukbd0 at uhidev0 reportid 1: 8 variable keys, 5 key codes
| > wskbd1 at ukbd0 mux 1
| > uhidev1 at uhub0 port 2 configuration 1 interface 1 "RDing TEMPERHUM1V1.2" rev 2.00/0.01 addr 2
| > uhidev1: iclass 3/1
| > ugold0 at uhidev1
| > uftdi0 at uhub0 port 3 configuration 1 interface 0 "FTDI FT230X Basic UART" rev 2.00/10.00 addr 3
| > ucom0 at uftdi0 portno 1
| > uftdi1 at uhub0 port 4 configuration 1 interface 0 "FTDI SYNLEAF - FT232R USB UART" rev 2.00/6.00 addr 4
| > ucom1 at uftdi1 portno 1
| > umass0 at uhub0 port 13 configuration 1 interface 0 "Western Digital My Passport 259D" rev 3.00/10.12 addr 5
| > umass0: using SCSI over Bulk-Only
| > scsibus2 at umass0: 2 targets, initiator 0
| > sd2 at scsibus2 targ 1 lun 0: <WD, My Passport 259D, 1012> SCSI4 0/direct fixed
| > sd2: 3815415MB, 512 bytes/sector, 7813969920 sectors
| > ses0 at scsibus2 targ 1 lun 1: <WD, SES Device, 1012> SCSI4 13/enclosure services fixed
| > ses0: unable to read enclosure configuration
| > vscsi0 at root
| > scsibus3 at vscsi0: 256 targets
| > softraid0 at root
| > scsibus4 at softraid0: 256 targets
| > root on sd0a (aec06f0350f24bab.a) swap on sd0b dump on sd0b
| > WARNING: / was not properly unmounted
| >
| > --
| > >++++++++[<++++++++++>-]<+++++++.>+++[<------>-]<.>+++[<+
| > +++++++++++>-]<.>++[<------------>-]<+.--------------.[-]
| >                  http://www.weirdnet.nl/                 
| >
|

--
>++++++++[<++++++++++>-]<+++++++.>+++[<------>-]<.>+++[<+
+++++++++++>-]<.>++[<------------>-]<+.--------------.[-]
                 http://www.weirdnet.nl/