X hangs

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

X hangs

Stuart Henderson
I have some hangs with my main workstation running -current if I'm
running X. I've seen it a couple of times in a day - unsure of anything
in particular that triggers it (I thought maybe browser-related but
I've seen it while not actively using a browser too). When it happens
I can't access DDB via BREAK, and there's nothing on the console
from an MP_LOCKDEBUG kernel. Nothing stands out with WITNESS either
(vmmaplk/inode lock order reversal during boot after attaching root,
but I don't think that's unusual).

I haven't triggered it if X is not running. I've now installed 6.7-release
files over the top and haven't triggered it yet with that either (12h
uptime on that now - I was lucky if it got to 3 or 4 hours before).
(The last couple of times I did trigger it, fsck had to work quite hard
afterwards).

Any ideas where to poke or of any suspicious commits? I had a look
through the post-6.7 history but nothing stood out. The window of when
it started in snapshots seems to be between release and something like
GENERIC.MP #201: Tue May 19 18:18:29 MDT.


OpenBSD 6.7 (GENERIC.MP) #182: Thu May  7 11:11:58 MDT 2020
    [hidden email]:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 34247069696 (32660MB)
avail mem = 33196531712 (31658MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xec400 (92 entries)
bios0: vendor Dell Inc. version "A12" date 05/11/2017
bios0: Dell Inc. PowerEdge T20
acpi0 at bios0: ACPI 5.0
acpi0: sleep states S0 S4 S5
acpi0: tables DSDT FACP APIC FPDT SLIC LPIT SSDT SSDT SSDT HPET SSDT MCFG SSDT ASF! DMAR
acpi0: wakeup devices UAR1(S4) RP01(S4) PXSX(S4) RP02(S4) PXSX(S4) PXSX(S4) RP05(S4) PXSX(S4) PXSX(S4) PXSX(S4) PXSX(S4) GLAN(S4) EHC1(S3) EHC2(S3) XHC_(S4) HDEF(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(R) CPU E3-1225 v3 @ 3.20GHz, 3392.66 MHz, 06-3c-03
cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Xeon(R) CPU E3-1225 v3 @ 3.20GHz, 3392.16 MHz, 06-3c-03
cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Xeon(R) CPU E3-1225 v3 @ 3.20GHz, 3392.16 MHz, 06-3c-03
cpu2: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 6 (application processor)
cpu3: Intel(R) Xeon(R) CPU E3-1225 v3 @ 3.20GHz, 3392.16 MHz, 06-3c-03
cpu3: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu3: 256KB 64b/line 8-way L2 cache
cpu3: smt 0, core 3, package 0
ioapic0 at mainbus0: apid 8 pa 0xfec00000, version 20, 24 pins
acpihpet0 at acpi0: 14318179 Hz
acpimcfg0 at acpi0
acpimcfg0: addr 0xf8000000, bus 0-63
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 2 (RP01)
acpiprt2 at acpi0: bus 3 (RP02)
acpiprt3 at acpi0: bus 5 (RP05)
acpiprt4 at acpi0: bus 1 (PEG0)
acpiprt5 at acpi0: bus -1 (PEG1)
acpiprt6 at acpi0: bus -1 (PEG2)
acpiec0 at acpi0: not present
acpicpu0 at acpi0: C2(200@148 mwait.1@0x33), C1(1000@1 mwait.1), PSS
acpicpu1 at acpi0: C2(200@148 mwait.1@0x33), C1(1000@1 mwait.1), PSS
acpicpu2 at acpi0: C2(200@148 mwait.1@0x33), C1(1000@1 mwait.1), PSS
acpicpu3 at acpi0: C2(200@148 mwait.1@0x33), C1(1000@1 mwait.1), PSS
acpitz0 at acpi0: critical temperature is 105 degC
acpitz1 at acpi0: critical temperature is 105 degC
acpipci0 at acpi0 PCI0: 0x00000000 0x00000011 0x00000001
acpicmos0 at acpi0
acpibtn0 at acpi0: PWRB
"PNP0C14" at acpi0 not configured
acpivideo0 at acpi0: GFX0
acpivout0 at acpivideo0: DD1F
cpu0: using VERW MDS workaround (except on vmm entry)
cpu0: Enhanced SpeedStep 3392 MHz: speeds: 3201, 3200, 3000, 2900, 2700, 2500, 2300, 2200, 2000, 1800, 1700, 1500, 1300, 1100, 1000, 800 MHz
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel Xeon E3-1200 v3 Host" rev 0x06
ppb0 at pci0 dev 1 function 0 "Intel Core 4G PCIE" rev 0x06: msi
pci1 at ppb0 bus 1
em0 at pci1 dev 0 function 0 "Intel 82572EI" rev 0x06: apic 8 int 16, address 00:15:17:8e:79:85
inteldrm0 at pci0 dev 2 function 0 "Intel HD Graphics P4600" rev 0x06
drm0 at inteldrm0
inteldrm0: msi, HASWELL, gen 7
xhci0 at pci0 dev 20 function 0 "Intel 8 Series xHCI" rev 0x04: msi, xHCI 1.0
usb0 at xhci0: USB revision 3.0
uhub0 at usb0 configuration 1 interface 0 "Intel xHCI root hub" rev 3.00/1.00 addr 1
"Intel 8 Series MEI" rev 0x04 at pci0 dev 22 function 0 not configured
puc0 at pci0 dev 22 function 3 "Intel 8 Series KT" rev 0x04: ports: 16 com
com4 at puc0 port 0 apic 8 int 19: ns16550a, 16 byte fifo
com4: probed fifo depth: 0 bytes
em1 at pci0 dev 25 function 0 "Intel I217-LM" rev 0x04: msi, address f8:b1:56:ac:32:76
ehci0 at pci0 dev 26 function 0 "Intel 8 Series USB" rev 0x04: apic 8 int 16
usb1 at ehci0: USB revision 2.0
uhub1 at usb1 configuration 1 interface 0 "Intel EHCI root hub" rev 2.00/1.00 addr 1
azalia0 at pci0 dev 27 function 0 "Intel 8 Series HD Audio" rev 0x04: msi
azalia0: codecs: Realtek/0x0280
audio0 at azalia0
ppb1 at pci0 dev 28 function 0 "Intel 8 Series PCIE" rev 0xd4
pci2 at ppb1 bus 2
ppb2 at pci0 dev 28 function 1 "Intel 8 Series PCIE" rev 0xd4: msi
pci3 at ppb2 bus 3
ppb3 at pci3 dev 0 function 0 "TI XIO2001 PCIE-PCI" rev 0x00
pci4 at ppb3 bus 4
ppb4 at pci0 dev 28 function 4 "Intel 8 Series PCIE" rev 0xd4: msi
pci5 at ppb4 bus 5
nvme0 at pci5 dev 0 function 0 "Samsung SM961/PM961 NVMe" rev 0x00: msix, NVMe 1.2
nvme0: SAMSUNG MZVLW256HEHP-000L7, firmware 4L7QCXB7, serial S35ENX0J765205
scsibus1 at nvme0: 2 targets, initiator 0
sd0 at scsibus1 targ 1 lun 0: <NVMe, SAMSUNG MZVLW256, 4L7Q>
sd0: 244198MB, 512 bytes/sector, 500118192 sectors
ehci1 at pci0 dev 29 function 0 "Intel 8 Series USB" rev 0x04: apic 8 int 23
usb2 at ehci1: USB revision 2.0
uhub2 at usb2 configuration 1 interface 0 "Intel EHCI root hub" rev 2.00/1.00 addr 1
pcib0 at pci0 dev 31 function 0 "Intel C226 LPC" rev 0x04
ahci0 at pci0 dev 31 function 2 "Intel 8 Series AHCI" rev 0x04: msi, AHCI 1.3
ahci0: port 0: 6.0Gb/s
scsibus2 at ahci0: 32 targets
sd1 at scsibus2 targ 0 lun 0: <ATA, Samsung SSD 860, RVT0> naa.5002538e9984204d
sd1: 476940MB, 512 bytes/sector, 976773168 sectors, thin
ichiic0 at pci0 dev 31 function 3 "Intel 8 Series SMBus" rev 0x04: apic 8 int 18
iic0 at ichiic0
sdtemp0 at iic0 addr 0x18: stts2002
sdtemp1 at iic0 addr 0x19: stts2002
sdtemp2 at iic0 addr 0x1a: stts2002
sdtemp3 at iic0 addr 0x1b: stts2002
spdmem0 at iic0 addr 0x50: 8GB DDR3 SDRAM ECC PC3-12800 with thermal sensor
spdmem1 at iic0 addr 0x51: 8GB DDR3 SDRAM ECC PC3-12800 with thermal sensor
spdmem2 at iic0 addr 0x52: 8GB DDR3 SDRAM ECC PC3-12800 with thermal sensor
spdmem3 at iic0 addr 0x53: 8GB DDR3 SDRAM ECC PC3-12800 with thermal sensor
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com0: console
pckbc0 at isa0 port 0x60/5 irq 1 irq 12
pckbd0 at pckbc0 (kbd slot)
wskbd0 at pckbd0: console keyboard
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
wbsio0 at isa0 port 0x2e/2 rev 0x00
vmm0 at mainbus0: VMX/EPT
uhub3 at uhub0 port 3 configuration 1 interface 0 "Genesys Logic USB2.0 Hub" rev 2.00/77.64 addr 2
uhub4 at uhub3 port 4 configuration 1 interface 0 "Genesys Logic USB2.0 Hub" rev 2.00/77.64 addr 3
uhub5 at uhub0 port 4 configuration 1 interface 0 "Texas Instruments product 0x8142" rev 2.10/1.00 addr 4
uhub6 at uhub5 port 4 configuration 1 interface 0 "Texas Instruments product 0x8142" rev 2.10/1.00 addr 5
uhub7 at uhub0 port 5 configuration 1 interface 0 "GenesysLogic USB2.0 Hub" rev 2.00/92.24 addr 6
uhub7: device problem, disabling port 1
uaudio0 at uhub7 port 2 configuration 1 interface 1 "GN Netcom GN 9350" rev 2.00/1.00 addr 7
uaudio0: class v1, full-speed, sync, channels: 1 play, 1 rec, 4 ctls
audio1 at uaudio0
uhidev0 at uhub7 port 2 configuration 1 interface 3 "GN Netcom GN 9350" rev 2.00/1.00 addr 7
uhidev0: iclass 3/0
uhid0 at uhidev0: input=2, output=2, feature=0
uhidev1 at uhub7 port 4 configuration 1 interface 0 "Yubico Yubico Yubikey II" rev 2.00/2.12 addr 8
uhidev1: iclass 3/1
ukbd0 at uhidev1: 8 variable keys, 6 key codes
wskbd1 at ukbd0 mux 1
umass0 at uhub0 port 6 configuration 1 interface 0 "Realtek USB3.0 Card Reader" rev 2.10/1.32 addr 9
umass0: using SCSI over Bulk-Only
scsibus3 at umass0: 2 targets, initiator 0
sd2 at scsibus3 targ 1 lun 0: <Generic-, USB3.0 CRW-CF/MD, 1.00> removable
sd3 at scsibus3 targ 1 lun 1: <Generic-, USB3.0 CRW-SM/xD, 1.00> removable
sd4 at scsibus3 targ 1 lun 2: <Generic-, USB3.0 CRW-SD, 1.00> removable
sd5 at scsibus3 targ 1 lun 3: <Generic-, USB3.0 CRW-MS, 1.00> removable
sd6 at scsibus3 targ 1 lun 4: <Generic-, USB3.0 CRW-SD/MS, 1.00> removable
umodem0 at uhub0 port 8 configuration 1 interface 0 "SoftIron, Inc. OverDrive 1000" rev 2.00/1.02 addr 10
umodem0: data interface 1, has no CM over data, has break
umodem0: status change notification available
ucom0 at umodem0
ugen0 at uhub0 port 8 configuration 1 "SoftIron, Inc. OverDrive 1000" rev 2.00/1.02 addr 10
uhidev2 at uhub0 port 10 configuration 1 interface 0 "Lite-On Technology Corp. ThinkPad USB Keyboard with TrackPoint" rev 1.10/1.27 addr 11
uhidev2: iclass 3/1
ukbd1 at uhidev2: 8 variable keys, 6 key codes
wskbd2 at ukbd1 mux 1
uhidev3 at uhub0 port 10 configuration 1 interface 1 "Lite-On Technology Corp. ThinkPad USB Keyboard with TrackPoint" rev 1.10/1.27 addr 11
uhidev3: iclass 3/0, 4 report ids
ums0 at uhidev3 reportid 1: 3 buttons
wsmouse0 at ums0 mux 0
uhid1 at uhidev3 reportid 2: input=1, output=0, feature=0
uhid2 at uhidev3 reportid 3: input=3, output=1, feature=0
uhid3 at uhidev3 reportid 4: input=0, output=0, feature=4
uhub8 at uhub0 port 21 configuration 1 interface 0 "Texas Instruments product 0x8140" rev 3.00/1.00 addr 12
uhub9 at uhub8 port 4 configuration 1 interface 0 "Texas Instruments product 0x8140" rev 3.00/1.00 addr 13
uhub10 at uhub1 port 1 configuration 1 interface 0 "Intel Rate Matching Hub" rev 2.00/0.04 addr 2
uhub11 at uhub2 port 1 configuration 1 interface 0 "Intel Rate Matching Hub" rev 2.00/0.04 addr 2
vscsi0 at root
scsibus4 at vscsi0: 256 targets
softraid0 at root
scsibus5 at softraid0: 256 targets
root on sd1a (2b4432fd9000a5b7.a) swap on sd1b dump on sd1b
inteldrm0: 1920x1200, 32bpp
wsdisplay0 at inteldrm0 mux 1: console (std, vt100 emulation), using wskbd0
wskbd1: connecting to wsdisplay0
wskbd2: connecting to wsdisplay0
wsdisplay0: screen 1-5 added (std, vt100 emulation)
wskbd1: disconnecting from wsdisplay0
wskbd1 detached
ukbd0 detached
uhidev1 detached
arp_rtrequest: bad gateway value: vlan5
uhidev1 at uhub7 port 4 configuration 1 interface 0 "Yubico Yubico Yubikey II" rev 2.00/2.12 addr 8
uhidev1: iclass 3/1
ukbd0 at uhidev1: 8 variable keys, 6 key codes
wskbd1 at ukbd0: console keyboard, using wsdisplay0

Reply | Threaded
Open this post in threaded view
|

Re: X hangs

Stuart Henderson
visa pointed me at sys/kern/kern_event.c r1.132 as a possibility -
I reverted that yesterday and haven't seen the system hang yet.
That's not conclusive but considering how things were going before,
I think it would have happened by now.


On 2020/05/23 23:41, Stuart Henderson wrote:

> I have some hangs with my main workstation running -current if I'm
> running X. I've seen it a couple of times in a day - unsure of anything
> in particular that triggers it (I thought maybe browser-related but
> I've seen it while not actively using a browser too). When it happens
> I can't access DDB via BREAK, and there's nothing on the console
> from an MP_LOCKDEBUG kernel. Nothing stands out with WITNESS either
> (vmmaplk/inode lock order reversal during boot after attaching root,
> but I don't think that's unusual).
>
> I haven't triggered it if X is not running. I've now installed 6.7-release
> files over the top and haven't triggered it yet with that either (12h
> uptime on that now - I was lucky if it got to 3 or 4 hours before).
> (The last couple of times I did trigger it, fsck had to work quite hard
> afterwards).
>
> Any ideas where to poke or of any suspicious commits? I had a look
> through the post-6.7 history but nothing stood out. The window of when
> it started in snapshots seems to be between release and something like
> GENERIC.MP #201: Tue May 19 18:18:29 MDT.
>
>
> OpenBSD 6.7 (GENERIC.MP) #182: Thu May  7 11:11:58 MDT 2020
>     [hidden email]:/usr/src/sys/arch/amd64/compile/GENERIC.MP
> real mem = 34247069696 (32660MB)
> avail mem = 33196531712 (31658MB)
> mpath0 at root
> scsibus0 at mpath0: 256 targets
> mainbus0 at root
> bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xec400 (92 entries)
> bios0: vendor Dell Inc. version "A12" date 05/11/2017
> bios0: Dell Inc. PowerEdge T20
> acpi0 at bios0: ACPI 5.0
> acpi0: sleep states S0 S4 S5
> acpi0: tables DSDT FACP APIC FPDT SLIC LPIT SSDT SSDT SSDT HPET SSDT MCFG SSDT ASF! DMAR
> acpi0: wakeup devices UAR1(S4) RP01(S4) PXSX(S4) RP02(S4) PXSX(S4) PXSX(S4) RP05(S4) PXSX(S4) PXSX(S4) PXSX(S4) PXSX(S4) GLAN(S4) EHC1(S3) EHC2(S3) XHC_(S4) HDEF(S4) [...]
> acpitimer0 at acpi0: 3579545 Hz, 24 bits
> acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
> cpu0 at mainbus0: apid 0 (boot processor)
> cpu0: Intel(R) Xeon(R) CPU E3-1225 v3 @ 3.20GHz, 3392.66 MHz, 06-3c-03
> cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> cpu0: 256KB 64b/line 8-way L2 cache
> cpu0: smt 0, core 0, package 0
> mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
> cpu0: apic clock running at 99MHz
> cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4, IBE
> cpu1 at mainbus0: apid 2 (application processor)
> cpu1: Intel(R) Xeon(R) CPU E3-1225 v3 @ 3.20GHz, 3392.16 MHz, 06-3c-03
> cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> cpu1: 256KB 64b/line 8-way L2 cache
> cpu1: smt 0, core 1, package 0
> cpu2 at mainbus0: apid 4 (application processor)
> cpu2: Intel(R) Xeon(R) CPU E3-1225 v3 @ 3.20GHz, 3392.16 MHz, 06-3c-03
> cpu2: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> cpu2: 256KB 64b/line 8-way L2 cache
> cpu2: smt 0, core 2, package 0
> cpu3 at mainbus0: apid 6 (application processor)
> cpu3: Intel(R) Xeon(R) CPU E3-1225 v3 @ 3.20GHz, 3392.16 MHz, 06-3c-03
> cpu3: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> cpu3: 256KB 64b/line 8-way L2 cache
> cpu3: smt 0, core 3, package 0
> ioapic0 at mainbus0: apid 8 pa 0xfec00000, version 20, 24 pins
> acpihpet0 at acpi0: 14318179 Hz
> acpimcfg0 at acpi0
> acpimcfg0: addr 0xf8000000, bus 0-63
> acpiprt0 at acpi0: bus 0 (PCI0)
> acpiprt1 at acpi0: bus 2 (RP01)
> acpiprt2 at acpi0: bus 3 (RP02)
> acpiprt3 at acpi0: bus 5 (RP05)
> acpiprt4 at acpi0: bus 1 (PEG0)
> acpiprt5 at acpi0: bus -1 (PEG1)
> acpiprt6 at acpi0: bus -1 (PEG2)
> acpiec0 at acpi0: not present
> acpicpu0 at acpi0: C2(200@148 mwait.1@0x33), C1(1000@1 mwait.1), PSS
> acpicpu1 at acpi0: C2(200@148 mwait.1@0x33), C1(1000@1 mwait.1), PSS
> acpicpu2 at acpi0: C2(200@148 mwait.1@0x33), C1(1000@1 mwait.1), PSS
> acpicpu3 at acpi0: C2(200@148 mwait.1@0x33), C1(1000@1 mwait.1), PSS
> acpitz0 at acpi0: critical temperature is 105 degC
> acpitz1 at acpi0: critical temperature is 105 degC
> acpipci0 at acpi0 PCI0: 0x00000000 0x00000011 0x00000001
> acpicmos0 at acpi0
> acpibtn0 at acpi0: PWRB
> "PNP0C14" at acpi0 not configured
> acpivideo0 at acpi0: GFX0
> acpivout0 at acpivideo0: DD1F
> cpu0: using VERW MDS workaround (except on vmm entry)
> cpu0: Enhanced SpeedStep 3392 MHz: speeds: 3201, 3200, 3000, 2900, 2700, 2500, 2300, 2200, 2000, 1800, 1700, 1500, 1300, 1100, 1000, 800 MHz
> pci0 at mainbus0 bus 0
> pchb0 at pci0 dev 0 function 0 "Intel Xeon E3-1200 v3 Host" rev 0x06
> ppb0 at pci0 dev 1 function 0 "Intel Core 4G PCIE" rev 0x06: msi
> pci1 at ppb0 bus 1
> em0 at pci1 dev 0 function 0 "Intel 82572EI" rev 0x06: apic 8 int 16, address 00:15:17:8e:79:85
> inteldrm0 at pci0 dev 2 function 0 "Intel HD Graphics P4600" rev 0x06
> drm0 at inteldrm0
> inteldrm0: msi, HASWELL, gen 7
> xhci0 at pci0 dev 20 function 0 "Intel 8 Series xHCI" rev 0x04: msi, xHCI 1.0
> usb0 at xhci0: USB revision 3.0
> uhub0 at usb0 configuration 1 interface 0 "Intel xHCI root hub" rev 3.00/1.00 addr 1
> "Intel 8 Series MEI" rev 0x04 at pci0 dev 22 function 0 not configured
> puc0 at pci0 dev 22 function 3 "Intel 8 Series KT" rev 0x04: ports: 16 com
> com4 at puc0 port 0 apic 8 int 19: ns16550a, 16 byte fifo
> com4: probed fifo depth: 0 bytes
> em1 at pci0 dev 25 function 0 "Intel I217-LM" rev 0x04: msi, address f8:b1:56:ac:32:76
> ehci0 at pci0 dev 26 function 0 "Intel 8 Series USB" rev 0x04: apic 8 int 16
> usb1 at ehci0: USB revision 2.0
> uhub1 at usb1 configuration 1 interface 0 "Intel EHCI root hub" rev 2.00/1.00 addr 1
> azalia0 at pci0 dev 27 function 0 "Intel 8 Series HD Audio" rev 0x04: msi
> azalia0: codecs: Realtek/0x0280
> audio0 at azalia0
> ppb1 at pci0 dev 28 function 0 "Intel 8 Series PCIE" rev 0xd4
> pci2 at ppb1 bus 2
> ppb2 at pci0 dev 28 function 1 "Intel 8 Series PCIE" rev 0xd4: msi
> pci3 at ppb2 bus 3
> ppb3 at pci3 dev 0 function 0 "TI XIO2001 PCIE-PCI" rev 0x00
> pci4 at ppb3 bus 4
> ppb4 at pci0 dev 28 function 4 "Intel 8 Series PCIE" rev 0xd4: msi
> pci5 at ppb4 bus 5
> nvme0 at pci5 dev 0 function 0 "Samsung SM961/PM961 NVMe" rev 0x00: msix, NVMe 1.2
> nvme0: SAMSUNG MZVLW256HEHP-000L7, firmware 4L7QCXB7, serial S35ENX0J765205
> scsibus1 at nvme0: 2 targets, initiator 0
> sd0 at scsibus1 targ 1 lun 0: <NVMe, SAMSUNG MZVLW256, 4L7Q>
> sd0: 244198MB, 512 bytes/sector, 500118192 sectors
> ehci1 at pci0 dev 29 function 0 "Intel 8 Series USB" rev 0x04: apic 8 int 23
> usb2 at ehci1: USB revision 2.0
> uhub2 at usb2 configuration 1 interface 0 "Intel EHCI root hub" rev 2.00/1.00 addr 1
> pcib0 at pci0 dev 31 function 0 "Intel C226 LPC" rev 0x04
> ahci0 at pci0 dev 31 function 2 "Intel 8 Series AHCI" rev 0x04: msi, AHCI 1.3
> ahci0: port 0: 6.0Gb/s
> scsibus2 at ahci0: 32 targets
> sd1 at scsibus2 targ 0 lun 0: <ATA, Samsung SSD 860, RVT0> naa.5002538e9984204d
> sd1: 476940MB, 512 bytes/sector, 976773168 sectors, thin
> ichiic0 at pci0 dev 31 function 3 "Intel 8 Series SMBus" rev 0x04: apic 8 int 18
> iic0 at ichiic0
> sdtemp0 at iic0 addr 0x18: stts2002
> sdtemp1 at iic0 addr 0x19: stts2002
> sdtemp2 at iic0 addr 0x1a: stts2002
> sdtemp3 at iic0 addr 0x1b: stts2002
> spdmem0 at iic0 addr 0x50: 8GB DDR3 SDRAM ECC PC3-12800 with thermal sensor
> spdmem1 at iic0 addr 0x51: 8GB DDR3 SDRAM ECC PC3-12800 with thermal sensor
> spdmem2 at iic0 addr 0x52: 8GB DDR3 SDRAM ECC PC3-12800 with thermal sensor
> spdmem3 at iic0 addr 0x53: 8GB DDR3 SDRAM ECC PC3-12800 with thermal sensor
> isa0 at pcib0
> isadma0 at isa0
> com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
> com0: console
> pckbc0 at isa0 port 0x60/5 irq 1 irq 12
> pckbd0 at pckbc0 (kbd slot)
> wskbd0 at pckbd0: console keyboard
> pcppi0 at isa0 port 0x61
> spkr0 at pcppi0
> wbsio0 at isa0 port 0x2e/2 rev 0x00
> vmm0 at mainbus0: VMX/EPT
> uhub3 at uhub0 port 3 configuration 1 interface 0 "Genesys Logic USB2.0 Hub" rev 2.00/77.64 addr 2
> uhub4 at uhub3 port 4 configuration 1 interface 0 "Genesys Logic USB2.0 Hub" rev 2.00/77.64 addr 3
> uhub5 at uhub0 port 4 configuration 1 interface 0 "Texas Instruments product 0x8142" rev 2.10/1.00 addr 4
> uhub6 at uhub5 port 4 configuration 1 interface 0 "Texas Instruments product 0x8142" rev 2.10/1.00 addr 5
> uhub7 at uhub0 port 5 configuration 1 interface 0 "GenesysLogic USB2.0 Hub" rev 2.00/92.24 addr 6
> uhub7: device problem, disabling port 1
> uaudio0 at uhub7 port 2 configuration 1 interface 1 "GN Netcom GN 9350" rev 2.00/1.00 addr 7
> uaudio0: class v1, full-speed, sync, channels: 1 play, 1 rec, 4 ctls
> audio1 at uaudio0
> uhidev0 at uhub7 port 2 configuration 1 interface 3 "GN Netcom GN 9350" rev 2.00/1.00 addr 7
> uhidev0: iclass 3/0
> uhid0 at uhidev0: input=2, output=2, feature=0
> uhidev1 at uhub7 port 4 configuration 1 interface 0 "Yubico Yubico Yubikey II" rev 2.00/2.12 addr 8
> uhidev1: iclass 3/1
> ukbd0 at uhidev1: 8 variable keys, 6 key codes
> wskbd1 at ukbd0 mux 1
> umass0 at uhub0 port 6 configuration 1 interface 0 "Realtek USB3.0 Card Reader" rev 2.10/1.32 addr 9
> umass0: using SCSI over Bulk-Only
> scsibus3 at umass0: 2 targets, initiator 0
> sd2 at scsibus3 targ 1 lun 0: <Generic-, USB3.0 CRW-CF/MD, 1.00> removable
> sd3 at scsibus3 targ 1 lun 1: <Generic-, USB3.0 CRW-SM/xD, 1.00> removable
> sd4 at scsibus3 targ 1 lun 2: <Generic-, USB3.0 CRW-SD, 1.00> removable
> sd5 at scsibus3 targ 1 lun 3: <Generic-, USB3.0 CRW-MS, 1.00> removable
> sd6 at scsibus3 targ 1 lun 4: <Generic-, USB3.0 CRW-SD/MS, 1.00> removable
> umodem0 at uhub0 port 8 configuration 1 interface 0 "SoftIron, Inc. OverDrive 1000" rev 2.00/1.02 addr 10
> umodem0: data interface 1, has no CM over data, has break
> umodem0: status change notification available
> ucom0 at umodem0
> ugen0 at uhub0 port 8 configuration 1 "SoftIron, Inc. OverDrive 1000" rev 2.00/1.02 addr 10
> uhidev2 at uhub0 port 10 configuration 1 interface 0 "Lite-On Technology Corp. ThinkPad USB Keyboard with TrackPoint" rev 1.10/1.27 addr 11
> uhidev2: iclass 3/1
> ukbd1 at uhidev2: 8 variable keys, 6 key codes
> wskbd2 at ukbd1 mux 1
> uhidev3 at uhub0 port 10 configuration 1 interface 1 "Lite-On Technology Corp. ThinkPad USB Keyboard with TrackPoint" rev 1.10/1.27 addr 11
> uhidev3: iclass 3/0, 4 report ids
> ums0 at uhidev3 reportid 1: 3 buttons
> wsmouse0 at ums0 mux 0
> uhid1 at uhidev3 reportid 2: input=1, output=0, feature=0
> uhid2 at uhidev3 reportid 3: input=3, output=1, feature=0
> uhid3 at uhidev3 reportid 4: input=0, output=0, feature=4
> uhub8 at uhub0 port 21 configuration 1 interface 0 "Texas Instruments product 0x8140" rev 3.00/1.00 addr 12
> uhub9 at uhub8 port 4 configuration 1 interface 0 "Texas Instruments product 0x8140" rev 3.00/1.00 addr 13
> uhub10 at uhub1 port 1 configuration 1 interface 0 "Intel Rate Matching Hub" rev 2.00/0.04 addr 2
> uhub11 at uhub2 port 1 configuration 1 interface 0 "Intel Rate Matching Hub" rev 2.00/0.04 addr 2
> vscsi0 at root
> scsibus4 at vscsi0: 256 targets
> softraid0 at root
> scsibus5 at softraid0: 256 targets
> root on sd1a (2b4432fd9000a5b7.a) swap on sd1b dump on sd1b
> inteldrm0: 1920x1200, 32bpp
> wsdisplay0 at inteldrm0 mux 1: console (std, vt100 emulation), using wskbd0
> wskbd1: connecting to wsdisplay0
> wskbd2: connecting to wsdisplay0
> wsdisplay0: screen 1-5 added (std, vt100 emulation)
> wskbd1: disconnecting from wsdisplay0
> wskbd1 detached
> ukbd0 detached
> uhidev1 detached
> arp_rtrequest: bad gateway value: vlan5
> uhidev1 at uhub7 port 4 configuration 1 interface 0 "Yubico Yubico Yubikey II" rev 2.00/2.12 addr 8
> uhidev1: iclass 3/1
> ukbd0 at uhidev1: 8 variable keys, 6 key codes
> wskbd1 at ukbd0: console keyboard, using wsdisplay0
>

Reply | Threaded
Open this post in threaded view
|

Re: X hangs

Visa Hankala-2
I have reverted the patch. Let's see if I could somehow reproduce
the problem myself so I can study it.


On Mon, May 25, 2020 at 03:21:01PM +0100, Stuart Henderson wrote:

> visa pointed me at sys/kern/kern_event.c r1.132 as a possibility -
> I reverted that yesterday and haven't seen the system hang yet.
> That's not conclusive but considering how things were going before,
> I think it would have happened by now.
>
>
> On 2020/05/23 23:41, Stuart Henderson wrote:
> > I have some hangs with my main workstation running -current if I'm
> > running X. I've seen it a couple of times in a day - unsure of anything
> > in particular that triggers it (I thought maybe browser-related but
> > I've seen it while not actively using a browser too). When it happens
> > I can't access DDB via BREAK, and there's nothing on the console
> > from an MP_LOCKDEBUG kernel. Nothing stands out with WITNESS either
> > (vmmaplk/inode lock order reversal during boot after attaching root,
> > but I don't think that's unusual).
> >
> > I haven't triggered it if X is not running. I've now installed 6.7-release
> > files over the top and haven't triggered it yet with that either (12h
> > uptime on that now - I was lucky if it got to 3 or 4 hours before).
> > (The last couple of times I did trigger it, fsck had to work quite hard
> > afterwards).
> >
> > Any ideas where to poke or of any suspicious commits? I had a look
> > through the post-6.7 history but nothing stood out. The window of when
> > it started in snapshots seems to be between release and something like
> > GENERIC.MP #201: Tue May 19 18:18:29 MDT.
> >
> >
> > OpenBSD 6.7 (GENERIC.MP) #182: Thu May  7 11:11:58 MDT 2020
> >     [hidden email]:/usr/src/sys/arch/amd64/compile/GENERIC.MP
> > real mem = 34247069696 (32660MB)
> > avail mem = 33196531712 (31658MB)
> > mpath0 at root
> > scsibus0 at mpath0: 256 targets
> > mainbus0 at root
> > bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xec400 (92 entries)
> > bios0: vendor Dell Inc. version "A12" date 05/11/2017
> > bios0: Dell Inc. PowerEdge T20
> > acpi0 at bios0: ACPI 5.0
> > acpi0: sleep states S0 S4 S5
> > acpi0: tables DSDT FACP APIC FPDT SLIC LPIT SSDT SSDT SSDT HPET SSDT MCFG SSDT ASF! DMAR
> > acpi0: wakeup devices UAR1(S4) RP01(S4) PXSX(S4) RP02(S4) PXSX(S4) PXSX(S4) RP05(S4) PXSX(S4) PXSX(S4) PXSX(S4) PXSX(S4) GLAN(S4) EHC1(S3) EHC2(S3) XHC_(S4) HDEF(S4) [...]
> > acpitimer0 at acpi0: 3579545 Hz, 24 bits
> > acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
> > cpu0 at mainbus0: apid 0 (boot processor)
> > cpu0: Intel(R) Xeon(R) CPU E3-1225 v3 @ 3.20GHz, 3392.66 MHz, 06-3c-03
> > cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> > cpu0: 256KB 64b/line 8-way L2 cache
> > cpu0: smt 0, core 0, package 0
> > mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
> > cpu0: apic clock running at 99MHz
> > cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4, IBE
> > cpu1 at mainbus0: apid 2 (application processor)
> > cpu1: Intel(R) Xeon(R) CPU E3-1225 v3 @ 3.20GHz, 3392.16 MHz, 06-3c-03
> > cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> > cpu1: 256KB 64b/line 8-way L2 cache
> > cpu1: smt 0, core 1, package 0
> > cpu2 at mainbus0: apid 4 (application processor)
> > cpu2: Intel(R) Xeon(R) CPU E3-1225 v3 @ 3.20GHz, 3392.16 MHz, 06-3c-03
> > cpu2: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> > cpu2: 256KB 64b/line 8-way L2 cache
> > cpu2: smt 0, core 2, package 0
> > cpu3 at mainbus0: apid 6 (application processor)
> > cpu3: Intel(R) Xeon(R) CPU E3-1225 v3 @ 3.20GHz, 3392.16 MHz, 06-3c-03
> > cpu3: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> > cpu3: 256KB 64b/line 8-way L2 cache
> > cpu3: smt 0, core 3, package 0
> > ioapic0 at mainbus0: apid 8 pa 0xfec00000, version 20, 24 pins
> > acpihpet0 at acpi0: 14318179 Hz
> > acpimcfg0 at acpi0
> > acpimcfg0: addr 0xf8000000, bus 0-63
> > acpiprt0 at acpi0: bus 0 (PCI0)
> > acpiprt1 at acpi0: bus 2 (RP01)
> > acpiprt2 at acpi0: bus 3 (RP02)
> > acpiprt3 at acpi0: bus 5 (RP05)
> > acpiprt4 at acpi0: bus 1 (PEG0)
> > acpiprt5 at acpi0: bus -1 (PEG1)
> > acpiprt6 at acpi0: bus -1 (PEG2)
> > acpiec0 at acpi0: not present
> > acpicpu0 at acpi0: C2(200@148 mwait.1@0x33), C1(1000@1 mwait.1), PSS
> > acpicpu1 at acpi0: C2(200@148 mwait.1@0x33), C1(1000@1 mwait.1), PSS
> > acpicpu2 at acpi0: C2(200@148 mwait.1@0x33), C1(1000@1 mwait.1), PSS
> > acpicpu3 at acpi0: C2(200@148 mwait.1@0x33), C1(1000@1 mwait.1), PSS
> > acpitz0 at acpi0: critical temperature is 105 degC
> > acpitz1 at acpi0: critical temperature is 105 degC
> > acpipci0 at acpi0 PCI0: 0x00000000 0x00000011 0x00000001
> > acpicmos0 at acpi0
> > acpibtn0 at acpi0: PWRB
> > "PNP0C14" at acpi0 not configured
> > acpivideo0 at acpi0: GFX0
> > acpivout0 at acpivideo0: DD1F
> > cpu0: using VERW MDS workaround (except on vmm entry)
> > cpu0: Enhanced SpeedStep 3392 MHz: speeds: 3201, 3200, 3000, 2900, 2700, 2500, 2300, 2200, 2000, 1800, 1700, 1500, 1300, 1100, 1000, 800 MHz
> > pci0 at mainbus0 bus 0
> > pchb0 at pci0 dev 0 function 0 "Intel Xeon E3-1200 v3 Host" rev 0x06
> > ppb0 at pci0 dev 1 function 0 "Intel Core 4G PCIE" rev 0x06: msi
> > pci1 at ppb0 bus 1
> > em0 at pci1 dev 0 function 0 "Intel 82572EI" rev 0x06: apic 8 int 16, address 00:15:17:8e:79:85
> > inteldrm0 at pci0 dev 2 function 0 "Intel HD Graphics P4600" rev 0x06
> > drm0 at inteldrm0
> > inteldrm0: msi, HASWELL, gen 7
> > xhci0 at pci0 dev 20 function 0 "Intel 8 Series xHCI" rev 0x04: msi, xHCI 1.0
> > usb0 at xhci0: USB revision 3.0
> > uhub0 at usb0 configuration 1 interface 0 "Intel xHCI root hub" rev 3.00/1.00 addr 1
> > "Intel 8 Series MEI" rev 0x04 at pci0 dev 22 function 0 not configured
> > puc0 at pci0 dev 22 function 3 "Intel 8 Series KT" rev 0x04: ports: 16 com
> > com4 at puc0 port 0 apic 8 int 19: ns16550a, 16 byte fifo
> > com4: probed fifo depth: 0 bytes
> > em1 at pci0 dev 25 function 0 "Intel I217-LM" rev 0x04: msi, address f8:b1:56:ac:32:76
> > ehci0 at pci0 dev 26 function 0 "Intel 8 Series USB" rev 0x04: apic 8 int 16
> > usb1 at ehci0: USB revision 2.0
> > uhub1 at usb1 configuration 1 interface 0 "Intel EHCI root hub" rev 2.00/1.00 addr 1
> > azalia0 at pci0 dev 27 function 0 "Intel 8 Series HD Audio" rev 0x04: msi
> > azalia0: codecs: Realtek/0x0280
> > audio0 at azalia0
> > ppb1 at pci0 dev 28 function 0 "Intel 8 Series PCIE" rev 0xd4
> > pci2 at ppb1 bus 2
> > ppb2 at pci0 dev 28 function 1 "Intel 8 Series PCIE" rev 0xd4: msi
> > pci3 at ppb2 bus 3
> > ppb3 at pci3 dev 0 function 0 "TI XIO2001 PCIE-PCI" rev 0x00
> > pci4 at ppb3 bus 4
> > ppb4 at pci0 dev 28 function 4 "Intel 8 Series PCIE" rev 0xd4: msi
> > pci5 at ppb4 bus 5
> > nvme0 at pci5 dev 0 function 0 "Samsung SM961/PM961 NVMe" rev 0x00: msix, NVMe 1.2
> > nvme0: SAMSUNG MZVLW256HEHP-000L7, firmware 4L7QCXB7, serial S35ENX0J765205
> > scsibus1 at nvme0: 2 targets, initiator 0
> > sd0 at scsibus1 targ 1 lun 0: <NVMe, SAMSUNG MZVLW256, 4L7Q>
> > sd0: 244198MB, 512 bytes/sector, 500118192 sectors
> > ehci1 at pci0 dev 29 function 0 "Intel 8 Series USB" rev 0x04: apic 8 int 23
> > usb2 at ehci1: USB revision 2.0
> > uhub2 at usb2 configuration 1 interface 0 "Intel EHCI root hub" rev 2.00/1.00 addr 1
> > pcib0 at pci0 dev 31 function 0 "Intel C226 LPC" rev 0x04
> > ahci0 at pci0 dev 31 function 2 "Intel 8 Series AHCI" rev 0x04: msi, AHCI 1.3
> > ahci0: port 0: 6.0Gb/s
> > scsibus2 at ahci0: 32 targets
> > sd1 at scsibus2 targ 0 lun 0: <ATA, Samsung SSD 860, RVT0> naa.5002538e9984204d
> > sd1: 476940MB, 512 bytes/sector, 976773168 sectors, thin
> > ichiic0 at pci0 dev 31 function 3 "Intel 8 Series SMBus" rev 0x04: apic 8 int 18
> > iic0 at ichiic0
> > sdtemp0 at iic0 addr 0x18: stts2002
> > sdtemp1 at iic0 addr 0x19: stts2002
> > sdtemp2 at iic0 addr 0x1a: stts2002
> > sdtemp3 at iic0 addr 0x1b: stts2002
> > spdmem0 at iic0 addr 0x50: 8GB DDR3 SDRAM ECC PC3-12800 with thermal sensor
> > spdmem1 at iic0 addr 0x51: 8GB DDR3 SDRAM ECC PC3-12800 with thermal sensor
> > spdmem2 at iic0 addr 0x52: 8GB DDR3 SDRAM ECC PC3-12800 with thermal sensor
> > spdmem3 at iic0 addr 0x53: 8GB DDR3 SDRAM ECC PC3-12800 with thermal sensor
> > isa0 at pcib0
> > isadma0 at isa0
> > com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
> > com0: console
> > pckbc0 at isa0 port 0x60/5 irq 1 irq 12
> > pckbd0 at pckbc0 (kbd slot)
> > wskbd0 at pckbd0: console keyboard
> > pcppi0 at isa0 port 0x61
> > spkr0 at pcppi0
> > wbsio0 at isa0 port 0x2e/2 rev 0x00
> > vmm0 at mainbus0: VMX/EPT
> > uhub3 at uhub0 port 3 configuration 1 interface 0 "Genesys Logic USB2.0 Hub" rev 2.00/77.64 addr 2
> > uhub4 at uhub3 port 4 configuration 1 interface 0 "Genesys Logic USB2.0 Hub" rev 2.00/77.64 addr 3
> > uhub5 at uhub0 port 4 configuration 1 interface 0 "Texas Instruments product 0x8142" rev 2.10/1.00 addr 4
> > uhub6 at uhub5 port 4 configuration 1 interface 0 "Texas Instruments product 0x8142" rev 2.10/1.00 addr 5
> > uhub7 at uhub0 port 5 configuration 1 interface 0 "GenesysLogic USB2.0 Hub" rev 2.00/92.24 addr 6
> > uhub7: device problem, disabling port 1
> > uaudio0 at uhub7 port 2 configuration 1 interface 1 "GN Netcom GN 9350" rev 2.00/1.00 addr 7
> > uaudio0: class v1, full-speed, sync, channels: 1 play, 1 rec, 4 ctls
> > audio1 at uaudio0
> > uhidev0 at uhub7 port 2 configuration 1 interface 3 "GN Netcom GN 9350" rev 2.00/1.00 addr 7
> > uhidev0: iclass 3/0
> > uhid0 at uhidev0: input=2, output=2, feature=0
> > uhidev1 at uhub7 port 4 configuration 1 interface 0 "Yubico Yubico Yubikey II" rev 2.00/2.12 addr 8
> > uhidev1: iclass 3/1
> > ukbd0 at uhidev1: 8 variable keys, 6 key codes
> > wskbd1 at ukbd0 mux 1
> > umass0 at uhub0 port 6 configuration 1 interface 0 "Realtek USB3.0 Card Reader" rev 2.10/1.32 addr 9
> > umass0: using SCSI over Bulk-Only
> > scsibus3 at umass0: 2 targets, initiator 0
> > sd2 at scsibus3 targ 1 lun 0: <Generic-, USB3.0 CRW-CF/MD, 1.00> removable
> > sd3 at scsibus3 targ 1 lun 1: <Generic-, USB3.0 CRW-SM/xD, 1.00> removable
> > sd4 at scsibus3 targ 1 lun 2: <Generic-, USB3.0 CRW-SD, 1.00> removable
> > sd5 at scsibus3 targ 1 lun 3: <Generic-, USB3.0 CRW-MS, 1.00> removable
> > sd6 at scsibus3 targ 1 lun 4: <Generic-, USB3.0 CRW-SD/MS, 1.00> removable
> > umodem0 at uhub0 port 8 configuration 1 interface 0 "SoftIron, Inc. OverDrive 1000" rev 2.00/1.02 addr 10
> > umodem0: data interface 1, has no CM over data, has break
> > umodem0: status change notification available
> > ucom0 at umodem0
> > ugen0 at uhub0 port 8 configuration 1 "SoftIron, Inc. OverDrive 1000" rev 2.00/1.02 addr 10
> > uhidev2 at uhub0 port 10 configuration 1 interface 0 "Lite-On Technology Corp. ThinkPad USB Keyboard with TrackPoint" rev 1.10/1.27 addr 11
> > uhidev2: iclass 3/1
> > ukbd1 at uhidev2: 8 variable keys, 6 key codes
> > wskbd2 at ukbd1 mux 1
> > uhidev3 at uhub0 port 10 configuration 1 interface 1 "Lite-On Technology Corp. ThinkPad USB Keyboard with TrackPoint" rev 1.10/1.27 addr 11
> > uhidev3: iclass 3/0, 4 report ids
> > ums0 at uhidev3 reportid 1: 3 buttons
> > wsmouse0 at ums0 mux 0
> > uhid1 at uhidev3 reportid 2: input=1, output=0, feature=0
> > uhid2 at uhidev3 reportid 3: input=3, output=1, feature=0
> > uhid3 at uhidev3 reportid 4: input=0, output=0, feature=4
> > uhub8 at uhub0 port 21 configuration 1 interface 0 "Texas Instruments product 0x8140" rev 3.00/1.00 addr 12
> > uhub9 at uhub8 port 4 configuration 1 interface 0 "Texas Instruments product 0x8140" rev 3.00/1.00 addr 13
> > uhub10 at uhub1 port 1 configuration 1 interface 0 "Intel Rate Matching Hub" rev 2.00/0.04 addr 2
> > uhub11 at uhub2 port 1 configuration 1 interface 0 "Intel Rate Matching Hub" rev 2.00/0.04 addr 2
> > vscsi0 at root
> > scsibus4 at vscsi0: 256 targets
> > softraid0 at root
> > scsibus5 at softraid0: 256 targets
> > root on sd1a (2b4432fd9000a5b7.a) swap on sd1b dump on sd1b
> > inteldrm0: 1920x1200, 32bpp
> > wsdisplay0 at inteldrm0 mux 1: console (std, vt100 emulation), using wskbd0
> > wskbd1: connecting to wsdisplay0
> > wskbd2: connecting to wsdisplay0
> > wsdisplay0: screen 1-5 added (std, vt100 emulation)
> > wskbd1: disconnecting from wsdisplay0
> > wskbd1 detached
> > ukbd0 detached
> > uhidev1 detached
> > arp_rtrequest: bad gateway value: vlan5
> > uhidev1 at uhub7 port 4 configuration 1 interface 0 "Yubico Yubico Yubikey II" rev 2.00/2.12 addr 8
> > uhidev1: iclass 3/1
> > ukbd0 at uhidev1: 8 variable keys, 6 key codes
> > wskbd1 at ukbd0: console keyboard, using wsdisplay0
> >
>

Reply | Threaded
Open this post in threaded view
|

Re: X hangs

Stuart Henderson
No idea if related to the hangs I saw earlier, but I've just hit a
uaudio-related kernel crash when starting to play a video in firefox.

RA\xaf\xdeRA\xaf\xdeRA\xaf\xdeRA\xaf\xdeRA\xaf\xdeRA\xaf\xdeRA\xaf\xde: can't set interface
kernel: protection fault trap, code=0
Stopped at      uaudio_stream_close+0x8a:       movzbl  0x8(%r12),%esi
ddb{3}> [-- sthen@localhost attached -- Thu May 28 11:58:19 2020]

Note that RA\xaf\xde is 52 41 af de; case 3 of poison_value() in kern/subr_poison.c
gives us

$ echo '(POISON1 & 0xffff0000) | (~POISON1 & 0x0000ffff)' | sed s/POISON1/0xdeafbead/g | moo
0xdeaf4152

(aren't you glad this is serial console not screenshots :)



[-- MARK -- Thu May 28 11:00:00 2020]
audio1 detached
uaudio0 detached
uhid0 detached
uhidev0 detached
uaudio0 at uhub7 port 2 configuration 1 interface 1 "GN Netcom GN 9350" rev 2.00/1.00 addr 7
uaudio0: class v1, full-speed, sync, channels: 1 play, 1 rec, 4 ctls
audio1 at uaudio0
uhidev0 at uhub7 port 2 configuration 1 interface 3 "GN Netcom GN 9350" rev 2.00/1.00 addr 7
uhidev0: iclass 3/0
uhid0 at uhidev0: input=2, output=2, feature=0
audio1 detached
uaudio0 detached
uhid0 detached
uhidev0 detached
uaudio0 at uhub7 port 2 configuration 1 interface 1 "GN Netcom GN 9350" rev 2.00/1.00 addr 7
uaudio0: failed to get ranges for level control
uaudio0: failed to get ranges for level control
uaudio0: class v1, full-speed, sync, channels: 1 play, 1 rec, 2 ctls
audio1 at uaudio0
uhidev0 at uhub7 port 2 configuration 1 interface 3 "GN Netcom GN 9350" rev 2.00/1.00 addr 7
uhidev0: no report descriptor
audio1 detached
uaudio0 detached
uhidev0 detached
uaudio0 at uhub7 port 2 configuration 1 interface 1 "GN Netcom GN 9350" rev 2.00/1.00 addr 7
uaudio0: class v1, full-speed, sync, channels: 1 play, 1 rec, 4 ctls
audio1 at uaudio0
uhidev0 at uhub7 port 2 configuration 1 interface 3 "GN Netcom GN 9350" rev 2.00/1.00 addr 7
uhidev0: iclass 3/0
uhid0 at uhidev0: input=2, output=2, feature=0
uaudio0: can't reset interface
uaudio0: can't reset interface
audio1 detached
uaudio0 detached
uhid0 detached
uhidev0 detached
RA\xaf\xdeRA\xaf\xdeRA\xaf\xdeRA\xaf\xdeRA\xaf\xdeRA\xaf\xdeRA\xaf\xde: can't set interface
kernel: protection fault trap, code=0
Stopped at      uaudio_stream_close+0x8a:       movzbl  0x8(%r12),%esi
ddb{3}> [-- sthen@localhost attached -- Thu May 28 11:58:19 2020]

ddb{3}>
ddb{3}> tr
uaudio_stream_close(ffff800001dfb000,1) at uaudio_stream_close+0x8a
uaudio_stream_open(ffff800001dfb000,1,ffff8000001e8000,ffff8000001eaa80,2a8,ffffffff816f7630) at uaudio_stream_open+0x761
uaudio_trigger_output(ffff800001dfb000,ffff8000001e8000,ffff8000001eaa80,2a8,ffffffff816f7630,ffff800001e95c00) at uaudio_trigger_output+0x47
audio_start_do(ffff800001e95c00) at audio_start_do+0xb5
audioioctl(2a01,20004126,ffff800035a74470,7,ffff800034fe6750) at audioioctl+0x71
VOP_IOCTL(fffffd867a72e9e0,20004126,ffff800035a74470,7,fffffd84fea6f9c0,ffff800034fe6750) at VOP_IOCTL+0x55
vn_ioctl(fffffd867d490f10,20004126,ffff800035a74470,ffff800034fe6750) at vn_ioctl+0x75
sys_ioctl(ffff800034fe6750,ffff800035a74580,ffff800035a745e0) at sys_ioctl+0x2df
syscall(ffff800035a74650) at syscall+0x389
Xsyscall() at Xsyscall+0x128
end of kernel
end trace frame: 0x7f7fffff4b50, count: -10
ddb{3}> ps
   PID     TID   PPID    UID  S       FLAGS  WAIT          COMMAND
 81832  329115  63766    755  2         0x2                snmpbulkwalk
 29247  373389  18746    755  2         0x2                snmpbulkwalk
 92787   86486    785    667  3    0x1000b2  poll          ping
   785  117710  43947    667  3        0x82  piperd        check_ping
 67767  195190  80498    755  2         0x2                snmpget
 49602  108618  76493    755  3        0x82  select        snmpgetnext
 79359  286268  48050    667  3    0x1000b2  poll          ping
 28903  141903  72867    667  3    0x1000b2  poll          ping
 13991  135834  21132    667  3    0x1000b2  poll          ping
 21132  104090  43947    667  3        0x82  piperd        check_ping
 48050  266973  43947    667  3        0x82  piperd        check_ping
 72867  121151  43947    667  3        0x82  piperd        check_ping
 19194   27887  39235    667  3    0x1000b2  poll          ping6
 39235  146853  43947    667  3        0x82  piperd        check_ping
 90145  177522  43545    667  3    0x1000b2  poll          ping
 43545  424834  43947    667  3        0x82  piperd        check_ping
 77995  496911  25446    667  3    0x1000b2  poll          ping
 25446  446626  43947    667  3        0x82  piperd        check_ping
  1021  516780  38791    667  3    0x1000b2  poll          ping
 38791  392276  43947    667  3        0x82  piperd        check_ping
 63506   75846  85416    732  3    0x200042  netlock       mongod
 63506  259264  85416    732  3   0x420008a  sigwait       mongod
 63506  192300  85416    732  3   0x4200082  fsleep        mongod
 63506  398131  85416    732  3   0x4200082  nanosleep     mongod
 63506  100766  85416    732  3   0x4200082  fsleep        mongod
 63506   18321  85416    732  3   0x4200082  fsleep        mongod
 63506  495524  85416    732  3   0x4200082  fsleep        mongod
 63506   50506  85416    732  3   0x4200082  fsleep        mongod
 63506  206498  85416    732  3   0x4200082  nanosleep     mongod
 63506  280068  85416    732  3   0x4200082  nanosleep     mongod
 63506  134811  85416    732  3   0x4200082  fsleep        mongod
 63506  105081  85416    732  3   0x4200082  nanosleep     mongod
 63506  148396  85416    732  2   0x4200002                mongod
  8135  163358  23211    755  3        0x82  piperd        rrdtool
 29423  260670  23183    755  3        0x82  piperd        rrdtool
 23211  251537  18746    755  3    0x10008a  pause         sh
 23183  419951  18746    755  3    0x10008a  pause         sh
 18746  415909  90380    755  3        0x82  select        php-7.4
 90380  293180  17605    755  3    0x10008a  pause         sh
 63924  321489  83967    755  3        0x82  piperd        rrdtool
 23615  324540  80318    755  3        0x82  piperd        rrdtool
 92797  470038  35189    755  3        0x82  piperd        rrdtool
 80318  175386  80498    755  3    0x10008a  pause         sh
 35189  257627  80498    755  3    0x10008a  pause         sh
 12241  296942  54938    755  3        0x82  piperd        rrdtool
 85621  118238  40623    755  3        0x82  piperd        rrdtool
 80075     777  23366    755  3        0x82  piperd        rrdtool
 54938   25890  63766    755  3    0x10008a  pause         sh
 40623  124220  63766    755  3    0x10008a  pause         sh
 23366  229225  76493    755  3    0x10008a  pause         sh
 83967  520490  76493    755  3    0x10008a  pause         sh
 63766  127004  79019    755  3        0x82  select        php-7.4
 79019  427969  17605    755  3    0x10008a  pause         sh
 80498  408927  39862    755  2       0x482                php-7.4
 76493  477625  77869    755  3        0x82  select        php-7.4
 39862  477017  17605    755  3    0x10008a  pause         sh
 77869   17719  17605    755  3    0x10008a  pause         sh
 17605  192892  44735    755  3    0x200082  fsleep        python2.7
 17605  237454  44735    755  3   0x4200082  wait          python2.7
 17605  143053  44735    755  3   0x4200082  wait          python2.7
 17605  435348  44735    755  3   0x4200082  wait          python2.7
 17605  459470  44735    755  3   0x4200082  wait          python2.7
 17605  113560  44735    755  3   0x4200082  fsleep        python2.7
 44735   56307  46040    755  3    0x10008a  pause         sh
 46040  315109  54939      0  3    0x100090  piperd        cron
 52325  132621  31472      0  3        0x92  kqread        indexer-worker
 78041   45203  31472    518  3        0x92  kqread        indexer
 48798   37324  28752      0  3        0x92  kqread        local
 30103  290854  28752    507  3        0x92  kqread        cleanup
 61942  501886  28752    507  3        0x92  kqread        trivial-rewrite
 74549  104174  28752    507  3        0x92  kqread        smtpd
 37987  218455  28752    507  3        0x92  poll          smtpd
 72654  116900  28752    507  3        0x92  kqread        smtpd
 55069   31658  28752    507  3        0x92  kqread        proxymap
 70191  110500  28752    507  3        0x92  lockf         dnsblog
 23594   12376  28752    507  3        0x92  kqread        dnsblog
 94395  320424  28752    507  3        0x92  kqread        postscreen
 25836  280293  28752    507  3        0x92  kqread        anvil
 65455  498691  31472    518  3        0x92  kqread        auth
 75120  264768      1   1000  3    0x100080  select        ssh
 48413   85099      1   1000  3    0x100080  select        ssh
 63952  194549  12569   1000  3    0x100083  poll          ssh
 67043  484720  98543   1000  2    0x100002                firefox
 67043  167728  98543   1000  3   0x4100082  kqread        firefox
 67043  413283  98543   1000  3   0x4100082  poll          firefox
 67043  161713  98543   1000  3   0x4100082  fsleep        firefox
 67043  357551  98543   1000  3   0x4100082  fsleep        firefox
 67043  105755  98543   1000  3   0x4100082  fsleep        firefox
 67043  434478  98543   1000  3   0x4100082  fsleep        firefox
 67043  231323  98543   1000  3   0x4100082  fsleep        firefox
 67043  216735  98543   1000  3   0x4100082  fsleep        firefox
 67043  105304  98543   1000  3   0x4100082  fsleep        firefox
 67043  214023  98543   1000  3   0x4100082  fsleep        firefox
 67043  260108  98543   1000  3   0x4100082  fsleep        firefox
 67043  468310  98543   1000  3   0x4100082  fsleep        firefox
 67043   73895  98543   1000  3   0x4100082  fsleep        firefox
 67043  397143  98543   1000  3   0x4100082  fsleep        firefox
 67043  404715  98543   1000  3   0x4100082  fsleep        firefox
 67043  281827  98543   1000  3   0x4100082  fsleep        firefox
 67043  321523  98543   1000  3   0x4100082  fsleep        firefox
 67043  512933  98543   1000  3   0x4100082  poll          firefox
 67043  158912  98543   1000  3   0x4100082  poll          firefox
 67043   14627  98543   1000  3   0x4100082  fsleep        firefox
 67043  234070  98543   1000  3   0x4100082  fsleep        firefox
 67043   16325  98543   1000  3   0x4100082  fsleep        firefox
 67043  356419  98543   1000  3   0x4100082  fsleep        firefox
 67043  155223  98543   1000  3   0x4100082  fsleep        firefox
 67043   98583  98543   1000  3   0x4100082  fsleep        firefox
 67043  253351  98543   1000  3   0x4100082  fsleep        firefox
 67043  458021  98543   1000  3   0x4100082  poll          firefox
 67043  378050  98543   1000  3   0x4100082  fsleep        firefox
 67043   60633  98543   1000  3   0x4100082  fsleep        firefox
 67043  332241  98543   1000  3   0x4100082  fsleep        firefox
 67043  283254  98543   1000  2   0x4100002                firefox
 67043   12416  98543   1000  3   0x4100082  fsleep        firefox
 67043   29939  98543   1000  2   0x4100002                firefox
 67043  450907  98543   1000  3   0x4100082  fsleep        firefox
 67043  437629  98543   1000  3   0x4100082  fsleep        firefox
 67043  520064  98543   1000  3   0x4100082  fsleep        firefox
 67043  518743  98543   1000  2   0x4100002                firefox
 67043  363141  98543   1000  3   0x4100082  fsleep        firefox
 67043   93770  98543   1000  3   0x4100082  fsleep        firefox
 67043  242338  98543   1000  3   0x4100082  fsleep        firefox
 67043  290237  98543   1000  3   0x4100082  poll          firefox
 67043  493259  98543   1000  2   0x4100002                firefox
 67043  233050  98543   1000  3   0x4100082  fsleep        firefox
 67043  349246  98543   1000  3   0x4100082  fsleep        firefox
 16893  342352  28752    507  3        0x92  kqread        pickup
 18356   96735  31472   1000  3        0x92  kqread        imap
 13154   28886  31472   1000  3        0x92  kqread        imap
 46657  452299  31472   1000  3        0x92  kqread        imap
 23786  431917  31472    666  3        0x92  kqread        imap-login
 72528  379439  31472    666  3        0x92  kqread        imap-login
 51129  165043  31472    666  3        0x92  kqread        imap-login
 15762   89225  31472   1000  3        0x92  kqread        imap
 46504  310329  31472    666  3        0x92  kqread        imap-login
 79053  186784  60110   1000  3    0x100083  ttyin         ksh
 61758  213963      1   8085  3    0x200080  select        python2.7
 95697  271126  41405     67  3        0x90  kqread        nginx
 41405  455563      1      0  3        0x88  pause         nginx
  6697  154322  60110   1000  3    0x100083  ttyin         ksh
 83834  302117      1   1000  3    0x100080  select        ssh
  6340  187265  88205   1000  3    0x100083  poll          ssh
 88205  342852  49532   1000  3    0x10008b  pause         ksh
 83111  317044  49532   1000  3        0xb0  netio         urxvt
 49532  346810  60054   1000  3        0xb2  select        urxvt
 60054  241898      1   1000  3    0x10008a  pause         sh
 29483  232745  15002    715  3        0x90  lockf         perl
 42038  280383  43710   1000  3        0x83  select        vim
 43710  218879  60110   1000  3    0x10008b  pause         ksh
 12569  365484  21899   1000  3    0x10008b  pause         ksh
 57809   40209  21899   1000  3        0xb0  netio         urxvt
 21899  446487  71764   1000  3        0xb2  select        urxvt
 71764  509773      1   1000  3    0x10008a  pause         sh
 23330  104299  60468   1000  3        0x83  select        vim
 60468  334590  60110   1000  3    0x10008b  pause         ksh
 46690  476322  60110   1000  3    0x100083  ttyin         ksh
 88725  113586  60110   1000  3    0x100083  ttyin         ksh
 82632  286129  29345   1000  3        0x83  select        vim
 29345  306161  60110   1000  3    0x10008b  pause         ksh
  3992  152138  60110   1000  3    0x100083  ttyin         ksh
  4780   94081  15002    715  3        0x90  lockf         perl
 72423  377558  15002    715  3        0x90  lockf         perl
 20940  304777  15002    715  3        0x90  lockf         perl
 56893   14300  15002    715  3        0x90  netcon        perl
 22104  316297  15002    715  3        0x90  lockf         perl
 38259  289191  46594    110  3    0x100091  poll          sndiod
*46594  247023   5931     99  7    0x100013                sndiod
  5931    7365  60110   1000  3    0x10008b  pause         ksh
 95328  386816  60110   1000  3    0x100083  ttyin         ksh
 42495   83304  60110   1000  3    0x100083  ttyin         ksh
 88501   50451  60110   1000  3    0x100083  ttyin         ksh
 72891  135255  60110   1000  3    0x100083  ttyin         ksh
 63870  460682  60110   1000  3    0x100083  ttyin         ksh
 40924  313411  60110   1000  3    0x100083  ttyin         ksh
 87560  433202  60110   1000  3    0x100083  ttyin         ksh
 87543  479862  60110   1000  3    0x100083  ttyin         ksh
 18437  290152  60110   1000  3    0x100083  ttyin         ksh
  1093  275178  60110   1000  3    0x100083  ttyin         ksh
 26316  344192  15002    715  3        0x90  lockf         perl
 43801  238564  15002    715  3        0x90  lockf         perl
 75315  383070  15002    715  3        0x90  lockf         perl
 28081  216595  15002    715  3        0x90  netio         perl
 60722  163293  31472   1000  3        0x92  kqread        imap
 58063  364174  31472    666  3        0x92  kqread        imap-login
 64630   20185      1   1000  3    0x100080  select        ssh
 86873  437772  20382   1000  3    0x100083  poll          ssh
 78679  239696      1   1000  3    0x100080  select        ssh
 24899  424638  77230   1000  3    0x100083  poll          ssh
 37481   32718  31472   1000  3        0x92  kqread        imap
 11585  306043  31472    666  3        0x92  kqread        imap-login
 27986   19573  42117   1000  3    0x100083  poll          mutt
 42117  372855  60110   1000  3    0x10008b  pause         ksh
 60110  413017      1   1000  3    0x100080  kqread        tmux
 65137  275702  34901   1000  3    0x100083  kqread        tmux
 14201   19357  50124    101  3    0x100090  kqread        iked
 60948  460679  50124    101  3    0x100090  kqread        iked
 39631  453105  50124    101  3    0x100090  kqread        iked
 50124  466508      1      0  3    0x100080  kqread        iked
 82586   79598  61165   1000  3    0x100082  select        mupdf-x11
 61165  210293      1   1000  3    0x100088  pause         ksh
 77790  348636  11527   1000  3    0x100083  kqread        cu
 84344  406514  98543   1000  3    0x100082  poll          firefox
 84344  220186  98543   1000  3   0x4100082  kqread        firefox
 84344  292217  98543   1000  3   0x4100082  poll          firefox
 84344  521639  98543   1000  3   0x4100082  fsleep        firefox
 84344  397400  98543   1000  3   0x4100082  fsleep        firefox
 84344  140136  98543   1000  3   0x4100082  fsleep        firefox
 84344  466745  98543   1000  3   0x4100082  fsleep        firefox
 84344  137728  98543   1000  3   0x4100082  fsleep        firefox
 84344  342247  98543   1000  7   0x4100002                firefox
 84344  234094  98543   1000  3   0x4100082  fsleep        firefox
 84344  470028  98543   1000  3   0x4100082  fsleep        firefox
 84344  367338  98543   1000  3   0x4100082  fsleep        firefox
 84344  200452  98543   1000  3   0x4100082  fsleep        firefox
 84344  514895  98543   1000  3   0x4100082  fsleep        firefox
 84344   37173  98543   1000  3   0x4100082  fsleep        firefox
 84344  307475  98543   1000  3   0x4100082  fsleep        firefox
 84344  444626  98543   1000  3   0x4100082  fsleep        firefox
 84344  150326  98543   1000  3   0x4100082  fsleep        firefox
 84344  275687  98543   1000  3   0x4100082  poll          firefox
 84344  153387  98543   1000  3   0x4100082  poll          firefox
 84344  412075  98543   1000  3   0x4100082  fsleep        firefox
 84344  463849  98543   1000  3   0x4100082  fsleep        firefox
 84344  468156  98543   1000  3   0x4100082  fsleep        firefox
 84344  277150  98543   1000  3   0x4100082  fsleep        firefox
 84344  498978  98543   1000  3   0x4100082  fsleep        firefox
 84344   97675  98543   1000  3   0x4100082  fsleep        firefox
 26953  388903      1   1000  3        0x82  poll          gvfsd-metadata
 26953  455393      1   1000  3   0x4000082  fsleep        gvfsd-metadata
 26953  475237      1   1000  3   0x4000082  poll          gvfsd-metadata
 26953  268329      1   1000  3   0x4000082  poll          gvfsd-metadata
 41843  289221      1   1000  3        0x88  select        gpg-agent
 57373  337059      1   1000  3        0x88  select        dirmngr
 74592  435598  98543   1000  3    0x100082  poll          firefox
 74592  188487  98543   1000  3   0x4100082  kqread        firefox
 74592  253921  98543   1000  3   0x4100082  poll          firefox
 74592  177539  98543   1000  3   0x4100082  fsleep        firefox
 74592    2628  98543   1000  3   0x4100082  fsleep        firefox
 74592  340742  98543   1000  3   0x4100082  fsleep        firefox
 74592  145008  98543   1000  3   0x4100082  fsleep        firefox
 74592   89571  98543   1000  3   0x4100082  fsleep        firefox
 74592  273910  98543   1000  2   0x4100482                firefox
 74592  493903  98543   1000  3   0x4100082  fsleep        firefox
 74592  492010  98543   1000  3   0x4100082  fsleep        firefox
 74592  488661  98543   1000  3   0x4100082  fsleep        firefox
 74592  280572  98543   1000  3   0x4100082  fsleep        firefox
 74592  415285  98543   1000  3   0x4100082  fsleep        firefox
 74592  388330  98543   1000  3   0x4100082  fsleep        firefox
 74592  470354  98543   1000  3   0x4100082  fsleep        firefox
 74592  497593  98543   1000  3   0x4100082  fsleep        firefox
 74592   83936  98543   1000  3   0x4100082  fsleep        firefox
 74592   37939  98543   1000  3   0x4100082  poll          firefox
 74592  459654  98543   1000  3   0x4100082  poll          firefox
 74592  274589  98543   1000  3   0x4100082  fsleep        firefox
 74592  321057  98543   1000  3   0x4100082  fsleep        firefox
 74592  168485  98543   1000  3   0x4100082  fsleep        firefox
 74592  461417  98543   1000  3   0x4100082  fsleep        firefox
 74592  154691  98543   1000  3   0x4100082  fsleep        firefox
 74592    7138  98543   1000  3   0x4100082  fsleep        firefox
 74592   45549  98543   1000  3   0x4100082  fsleep        firefox
 74592  136804  98543   1000  3   0x4100082  fsleep        firefox
 74592  173356  98543   1000  3   0x4100082  fsleep        firefox
 74592  432488  98543   1000  3   0x4100082  fsleep        firefox
 11527   40765  43917   1000  3    0x10008b  pause         ksh
  3117  180440  43917   1000  3        0xb0  netio         urxvt
 43917  410711  76003   1000  3        0xb2  select        urxvt
 76003  386280      1   1000  3    0x10008a  pause         sh
 77254  361112  98543   1000  3    0x100082  poll          firefox
 77254   95828  98543   1000  3   0x4100082  kqread        firefox
 77254  499142  98543   1000  3   0x4100082  poll          firefox
 77254  372024  98543   1000  3   0x4100082  fsleep        firefox
 77254  348151  98543   1000  3   0x4100082  fsleep        firefox
 77254  339433  98543   1000  3   0x4100082  fsleep        firefox
 77254  359637  98543   1000  3   0x4100082  fsleep        firefox
 77254   79069  98543   1000  3   0x4100082  fsleep        firefox
 77254    7695  98543   1000  3   0x4100082  fsleep        firefox
 77254  217941  98543   1000  3   0x4100082  fsleep        firefox
 77254  509256  98543   1000  3   0x4100082  fsleep        firefox
 77254  437446  98543   1000  3   0x4100082  fsleep        firefox
 77254  305212  98543   1000  3   0x4100082  fsleep        firefox
 77254   86013  98543   1000  3   0x4100082  fsleep        firefox
 77254  189265  98543   1000  3   0x4100082  fsleep        firefox
 77254  404905  98543   1000  3   0x4100082  fsleep        firefox
 77254  360339  98543   1000  3   0x4100082  fsleep        firefox
 77254  303493  98543   1000  3   0x4100082  fsleep        firefox
 77254     979  98543   1000  3   0x4100082  poll          firefox
 77254  505904  98543   1000  3   0x4100082  poll          firefox
 77254  253692  98543   1000  3   0x4100082  fsleep        firefox
 77254  216092  98543   1000  3   0x4100082  fsleep        firefox
 77254  261131  98543   1000  3   0x4100082  fsleep        firefox
 77254  400501  98543   1000  3   0x4100082  fsleep        firefox
 77254   46057  98543   1000  3   0x4100082  fsleep        firefox
 77254  511491  98543   1000  3   0x4100082  fsleep        firefox
 77254  470373  98543   1000  3   0x4100082  fsleep        firefox
 77254  347646  98543   1000  3   0x4100082  fsleep        firefox
 52860  473199  28752    507  3        0x92  kqread        tlsmgr
 20382  306488  50848   1000  3    0x10008b  pause         ksh
 41351  443791  50848   1000  3        0xb0  netio         urxvt
 50848   62011  48260   1000  3        0xb2  select        urxvt
 48260  455931      1   1000  3    0x10008a  pause         sh
 88378  398492  98543   1000  3    0x100082  poll          firefox
 88378  377862  98543   1000  3   0x4100082  kqread        firefox
 88378  241753  98543   1000  3   0x4100082  poll          firefox
 88378  382868  98543   1000  3   0x4100082  fsleep        firefox
 88378  300880  98543   1000  3   0x4100082  fsleep        firefox
 88378  181612  98543   1000  3   0x4100082  fsleep        firefox
 88378  137055  98543   1000  3   0x4100082  fsleep        firefox
 88378  501064  98543   1000  3   0x4100082  fsleep        firefox
 88378  142487  98543   1000  3   0x4100082  fsleep        firefox
 88378   46187  98543   1000  3   0x4100082  fsleep        firefox
 88378  511369  98543   1000  3   0x4100082  fsleep        firefox
 88378  112401  98543   1000  3   0x4100082  fsleep        firefox
 88378  241365  98543   1000  3   0x4100082  fsleep        firefox
 88378  360548  98543   1000  3   0x4100082  fsleep        firefox
 88378   40129  98543   1000  3   0x4100082  fsleep        firefox
 88378  298191  98543   1000  3   0x4100082  fsleep        firefox
 88378  464997  98543   1000  3   0x4100082  fsleep        firefox
 88378  437551  98543   1000  3   0x4100082  fsleep        firefox
 88378  477709  98543   1000  3   0x4100082  poll          firefox
 88378  446749  98543   1000  3   0x4100082  poll          firefox
 88378  202576  98543   1000  3   0x4100082  fsleep        firefox
 88378  381981  98543   1000  3   0x4100082  fsleep        firefox
 88378  275604  98543   1000  3   0x4100082  fsleep        firefox
 88378  102874  98543   1000  3   0x4100082  fsleep        firefox
 88378  401619  98543   1000  3   0x4100082  fsleep        firefox
 46146   31279  98543   1000  3    0x100082  poll          firefox
 46146   55659  98543   1000  3   0x4100082  kqread        firefox
 46146  466015  98543   1000  3   0x4100082  poll          firefox
 46146  253749  98543   1000  3   0x4100082  fsleep        firefox
 46146  502176  98543   1000  3   0x4100082  fsleep        firefox
 46146   24142  98543   1000  3   0x4100082  fsleep        firefox
 46146   78844  98543   1000  3   0x4100082  fsleep        firefox
 46146   45056  98543   1000  3   0x4100082  fsleep        firefox
 46146  436591  98543   1000  3   0x4100082  fsleep        firefox
 46146  167456  98543   1000  3   0x4100082  fsleep        firefox
 46146   73175  98543   1000  3   0x4100082  fsleep        firefox
 46146  361862  98543   1000  3   0x4100082  fsleep        firefox
 46146  417589  98543   1000  3   0x4100082  fsleep        firefox
 46146  156965  98543   1000  3   0x4100082  fsleep        firefox
 46146  324627  98543   1000  3   0x4100082  fsleep        firefox
 46146  123647  98543   1000  3   0x4100082  fsleep        firefox
 46146   63778  98543   1000  3   0x4100082  fsleep        firefox
 46146  125156  98543   1000  3   0x4100082  fsleep        firefox
 46146  407451  98543   1000  3   0x4100082  poll          firefox
 46146  497018  98543   1000  3   0x4100082  poll          firefox
 46146   87254  98543   1000  3   0x4100082  fsleep        firefox
 46146  137428  98543   1000  3   0x4100082  fsleep        firefox
 46146  273775  98543   1000  3   0x4100082  fsleep        firefox
 46146  306092  98543   1000  3   0x4100082  fsleep        firefox
 46146  319577  98543   1000  3   0x4100082  fsleep        firefox
 46146   92012  98543   1000  3   0x4100082  fsleep        firefox
 46146   18539  98543   1000  3   0x4100082  fsleep        firefox
 46146  162529  98543   1000  3   0x4100082  fsleep        firefox
 46146  263030  98543   1000  3   0x4100082  fsleep        firefox
 67427   42901  98543   1000  3    0x100082  poll          firefox
 67427  330771  98543   1000  3   0x4100082  kqread        firefox
 67427  466467  98543   1000  3   0x4100082  fsleep        firefox
 67427  148343  98543   1000  3   0x4100082  fsleep        firefox
 67427  443094  98543   1000  3   0x4100082  fsleep        firefox
 67427  313633  98543   1000  3   0x4100082  fsleep        firefox
 67427  309233  98543   1000  3   0x4100082  fsleep        firefox
 67427  465653  98543   1000  3   0x4100082  fsleep        firefox
 67427  130738  98543   1000  3   0x4100082  fsleep        firefox
 67427  162532  98543   1000  3   0x4100082  fsleep        firefox
 67427  210476  98543   1000  3   0x4100082  fsleep        firefox
 67427  305739  98543   1000  3   0x4100082  fsleep        firefox
 67427  301994  98543   1000  3   0x4100082  fsleep        firefox
 67427  319878  98543   1000  3   0x4100082  fsleep        firefox
 67427  270809  98543   1000  3   0x4100082  fsleep        firefox
 67427  354586  98543   1000  3   0x4100082  fsleep        firefox
 67427   36610  98543   1000  3   0x4100082  fsleep        firefox
 67427  223250  98543   1000  3   0x4100082  fsleep        firefox
 67427  282538  98543   1000  3   0x4100082  fsleep        firefox
 78988  157435   1123   1000  3        0x82  nanosleep     xstatbar
 33270  328755  80139   1000  3        0x82  nanosleep     xstatbar
 59482  202783      1   1000  3        0x82  poll          gvfsd
 59482  357149      1   1000  3   0x4000082  fsleep        gvfsd
 59482  414592      1   1000  3   0x4000082  poll          gvfsd
 59482  460731      1   1000  3   0x4000082  poll          gvfsd
 34901  197851  68076   1000  3    0x10008b  pause         ksh
  1123   32771      1   1000  3    0x100088  pause         sh
 69067  440816  18842   1000  3        0x82  nanosleep     i3status
 18842   91187  32976   1000  3    0x10008a  pause         sh
 52097  301402  68076   1000  3        0xb0  netio         urxvt
 68076  439685   7590   1000  3        0xb2  select        urxvt
 32976  453621  78135   1000  3        0x82  poll          i3bar
 78135  342297      1   1000  3    0x10008a  pause         sh
 98543  506762  36137   1000  2    0x100002                firefox
 98543  515777  36137   1000  2   0x4100002                firefox
 98543  410431  36137   1000  3   0x4100082  fsleep        firefox
 98543  151459  36137   1000  2   0x4100002                firefox
 98543  493877  36137   1000  3   0x4100082  fsleep        firefox
 98543  494488  36137   1000  3   0x4100082  fsleep        firefox
 98543  438021  36137   1000  3   0x4100082  poll          firefox
 98543  458898  36137   1000  3   0x4100082  poll          firefox
 98543   50146  36137   1000  3   0x4100082  fsleep        firefox
 98543  486894  36137   1000  3   0x4100082  fsleep        firefox
 98543  267659  36137   1000  3   0x4100082  fsleep        firefox
 98543   42272  36137   1000  3   0x4100082  fsleep        firefox
 98543  201026  36137   1000  3   0x4100082  fsleep        firefox
 98543  507937  36137   1000  3   0x4100082  fsleep        firefox
 98543  385654  36137   1000  3   0x4100082  fsleep        firefox
 98543  323435  36137   1000  3   0x4100082  fsleep        firefox
 98543  413940  36137   1000  2   0x4100002                firefox
 98543  517509  36137   1000  3   0x4100082  fsleep        firefox
 98543  317278  36137   1000  3   0x4100082  fsleep        firefox
 98543   84007  36137   1000  3   0x4100082  fsleep        firefox
 98543   29963  36137   1000  2   0x4100002                firefox
 98543  229509  36137   1000  3   0x4100082  fsleep        firefox
 98543  354247  36137   1000  3   0x4100082  fsleep        firefox
 98543  469713  36137   1000  3   0x4100082  fsleep        firefox
 98543   43938  36137   1000  3   0x4100082  fsleep        firefox
 98543  408546  36137   1000  3   0x4100082  fsleep        firefox
 98543  164767  36137   1000  3   0x4100082  fsleep        firefox
 98543  287620  36137   1000  3   0x4100082  fsleep        firefox
 98543  388011  36137   1000  3   0x4100082  fsleep        firefox
 98543  291414  36137   1000  3   0x4100082  fsleep        firefox
 98543  189244  36137   1000  3   0x4100082  fsleep        firefox
 98543  217712  36137   1000  3   0x4100082  fsleep        firefox
 98543  440066  36137   1000  3   0x4100082  poll          firefox
 98543  446075  36137   1000  3   0x4100082  fsleep        firefox
 98543  240182  36137   1000  3   0x4100082  fsleep        firefox
 98543  268816  36137   1000  3   0x4100082  fsleep        firefox
 98543  481278  36137   1000  3   0x4100082  fsleep        firefox
 98543  392910  36137   1000  3   0x4100082  fsleep        firefox
 98543   35158  36137   1000  3   0x4100082  fsleep        firefox
 98543   25239  36137   1000  3   0x4100082  fsleep        firefox
 98543  260495  36137   1000  3   0x4100082  fsleep        firefox
 98543  344459  36137   1000  3   0x4100082  fsleep        firefox
 98543  122897  36137   1000  3   0x4100082  fsleep        firefox
 98543  324315  36137   1000  3   0x4100082  fsleep        firefox
 98543  203849  36137   1000  3   0x4100082  fsleep        firefox
 98543  191563  36137   1000  3   0x4100082  fsleep        firefox
 98543  334855  36137   1000  3   0x4100082  fsleep        firefox
 98543  244871  36137   1000  3   0x4100082  fsleep        firefox
 98543  291080  36137   1000  3   0x4100082  fsleep        firefox
 98543  441550  36137   1000  3   0x4100082  fsleep        firefox
 98543  229415  36137   1000  3   0x4100082  fsleep        firefox
 98543  435821  36137   1000  3   0x4100082  fsleep        firefox
 98543  490444  36137   1000  3   0x4100082  fsleep        firefox
 98543   79695  36137   1000  3   0x4100082  fsleep        firefox
 98543   46604  36137   1000  3   0x4100082  fsleep        firefox
 98543  240835  36137   1000  3   0x4100082  fsleep        firefox
 98543  208264  36137   1000  3   0x4100082  fsleep        firefox
 98543  174161  36137   1000  3   0x4100082  fsleep        firefox
 98543  322067  36137   1000  3   0x4100082  fsleep        firefox
 98543  242321  36137   1000  3   0x4100082  fsleep        firefox
 98543  420147  36137   1000  3   0x4100082  fsleep        firefox
 98543  293113  36137   1000  3   0x4100082  fsleep        firefox
 98543  220251  36137   1000  3   0x4100082  fsleep        firefox
 98543  144088  36137   1000  3   0x4100082  fsleep        firefox
 98543  489277  36137   1000  3   0x4100082  fsleep        firefox
 36137  427608      1   1000  3    0x10008a  pause         sh
  7590  256700      1   1000  3    0x10008a  pause         sh
 45791  438010  86538   1000  3    0x100082  poll          i3
 40347  370257  29935   1000  3    0x100083  ttyin         vi
 77230  484941  19748   1000  3    0x10008b  pause         ksh
 30870  163880  86538   1000  3        0x82  nanosleep     redshift
 80139  486740  86538   1000  3    0x100088  pause         sh
  1016  352965      1   1000  3        0x80  poll          dbus-daemon
 82572  180972  19748   1000  3        0xb0  netio         urxvt
 27846   59966  29935   1000  3        0xb0  netio         urxvt
 77158   43927      1   1000  3        0x80  select        dbus-launch
  2904  181348    200   1000  3        0x82  select        xsettingsd
   200    4635  86538   1000  3    0x100088  pause         sh
 29935  415687  41013   1000  3        0xb2  select        urxvt
 95325  320137  86538   1000  3    0x100082  poll          xidle
 41013  367583  86538   1000  3    0x100088  pause         sh
 19748  418754  86538   1000  3        0xb2  select        urxvt
 41105  282756  86538   1000  3        0x82  poll          xcompmgr
 86538  160043  13215   1000  3    0x10008a  pause         sh
 95061  192403      1   1000  3    0x1000b0  poll          ssh-agent
 13215  223045  20526   1000  3    0x10008a  pause         sh
 20526  500910   3087      0  3    0x100080  wait          xenodm
 37377  423566  67864      0  3    0x100080  netio         Xorg
 80132   22569      1      0  3    0x100083  ttyin         getty
 88385  372872      1      0  3    0x100083  ttyin         getty
 74053  160685      1      0  3    0x100083  ttyin         getty
 67622  402377      1      0  3    0x100083  ttyin         getty
  7959  421034      1      0  3    0x100083  ttyin         getty
 28951  517930      1      0  3    0x100083  ttyin         getty
 67864  137289   3087     35  3        0x92  poll          Xorg
 67864  278470   3087     35  3   0x4000092  fsleep        Xorg
 67864   48733   3087     35  2   0x4000012                Xorg
  3087  150994      1      0  3        0x88  pause         xenodm
 54939  511732      1      0  3    0x100098  poll          cron
 37044   55405      1      0  3    0x100080  nanosleep     sensorsd
 89014  335490      1      0  3        0x80  kqread        apmd
   757  229869  21618    503  3        0x80  poll          postgres
 59690  352857  21618    503  3        0x80  poll          postgres
 21987  360617  21618    503  3        0x80  poll          postgres
 94332   58878  21618    503  3        0x80  poll          postgres
  9536  347516  21618    503  3        0x80  poll          postgres
 30793  327468  21618    503  3        0x80  poll          postgres
 21618  244971      1    503  3        0x82  select        postgres
 35242   27034      1    801  3    0x200083  fsleep        java
 35242  469252      1    801  3   0x4200083  fsleep        java
 35242   67425      1    801  3   0x4200083  fsleep        java
 35242   50874      1    801  3   0x4200083  fsleep        java
 35242  203013      1    801  3   0x4200083  fsleep        java
 35242  438622      1    801  3   0x4200083  fsleep        java
 35242  364063      1    801  3   0x4200083  fsleep        java
 35242  453356      1    801  3   0x4200083  fsleep        java
 35242  354060      1    801  3   0x4200083  fsleep        java
 35242   66787      1    801  3   0x4200083  fsleep        java
 35242   92811      1    801  3   0x4200083  fsleep        java
 35242   28825      1    801  3   0x4200083  fsleep        java
 35242  347285      1    801  3   0x4200083  fsleep        java
 35242   22561      1    801  3   0x4200083  fsleep        java
 35242  305160      1    801  3   0x4200083  fsleep        java
 35242   45684      1    801  3   0x4200083  fsleep        java
 35242  279318      1    801  3   0x4200083  fsleep        java
 35242  184091      1    801  3   0x4200083  fsleep        java
 35242  258878      1    801  3   0x4200083  fsleep        java
 35242   11215      1    801  3   0x4200083  fsleep        java
 35242  255929      1    801  3   0x4200083  fsleep        java
 35242  401579      1    801  3   0x4200083  fsleep        java
 35242  403447      1    801  2   0x4200483                java
 35242  391471      1    801  3   0x4200083  poll          java
 35242  481735      1    801  3   0x4200083  fsleep        java
 35242  453512      1    801  3   0x4200083  fsleep        java
 35242  518675      1    801  3   0x4200083  netcon        java
 35242   39970      1    801  3   0x4200083  kqread        java
 35242  298226      1    801  3   0x4200083  fsleep        java
 35242  156051      1    801  3   0x4200083  kqread        java
 35242  425992      1    801  3   0x4200083  fsleep        java
 35242  239655      1    801  3   0x4200083  fsleep        java
 35242  199637      1    801  3   0x4200083  fsleep        java
 35242  315883      1    801  3   0x4200083  fsleep        java
 35242  487370      1    801  3   0x4200083  fsleep        java
 35242  479940      1    801  3   0x4200083  kqread        java
 35242  247794      1    801  3   0x4200083  fsleep        java
 35242  200379      1    801  3   0x4200083  fsleep        java
 35242   96046      1    801  3   0x4200083  kqread        java
 35242  362174      1    801  3   0x4200083  fsleep        java
 35242   81225      1    801  3   0x4200083  fsleep        java
 35242  432297      1    801  3   0x4200083  fsleep        java
 35242   26994      1    801  3   0x4200083  fsleep        java
 35242   49999      1    801  3   0x4200083  fsleep        java
 35242  417711      1    801  3   0x4200083  fsleep        java
 35242  457669      1    801  3   0x4200083  fsleep        java
 84144  167330  48884    667  3        0x80  nanosleep     icinga2
 84144  516933  48884    667  3   0x4000080  fsleep        icinga2
 84144  506316  48884    667  3   0x4000080  fsleep        icinga2
 84144  424156  48884    667  3   0x4000080  fsleep        icinga2
 84144   60717  48884    667  3   0x4000080  fsleep        icinga2
 84144  365955  48884    667  3   0x4000080  fsleep        icinga2
 84144  124059  48884    667  3   0x4000080  fsleep        icinga2
 84144  299959  48884    667  3   0x4000080  fsleep        icinga2
 84144   67970  48884    667  3   0x4000080  fsleep        icinga2
 84144   50373  48884    667  3   0x4000080  fsleep        icinga2
 84144  380601  48884    667  3   0x4000080  fsleep        icinga2
 84144     432  48884    667  3   0x4000080  fsleep        icinga2
 84144    3145  48884    667  3   0x4000080  kqread        icinga2
 84144  421464  48884    667  3   0x4000080  fsleep        icinga2
 84144   51479  48884    667  3   0x4000080  fsleep        icinga2
 84144  338624  48884    667  3   0x4000080  fsleep        icinga2
 84144  234156  48884    667  3   0x4000080  fsleep        icinga2
 84144  146139  48884    667  3   0x4000080  fsleep        icinga2
 84144  360625  48884    667  3   0x4000080  fsleep        icinga2
 84144   31757  48884    667  3   0x4000080  poll          icinga2
 84144  424413  48884    667  3   0x4000080  poll          icinga2
 84144  517003  48884    667  3   0x4000080  fsleep        icinga2
 84144   98764  48884    667  3   0x4000080  fsleep        icinga2
 84144  171972  48884    667  3   0x4000080  poll          icinga2
 84144   17399  48884    667  3   0x4000080  poll          icinga2
 84144  460364  48884    667  3   0x4000080  poll          icinga2
 84144  286111  48884    667  3   0x4000080  poll          icinga2
 48884  178525      1    667  3        0x80  nanosleep     icinga2
 48884  385723      1    667  3   0x4000080  fsleep        icinga2
 48884  487132      1    667  3   0x4000080  fsleep        icinga2
 48884   68379      1    667  3   0x4000080  fsleep        icinga2
 48884  104147      1    667  3   0x4000080  fsleep        icinga2
 48884    4312      1    667  3   0x4000080  fsleep        icinga2
 48884  241990      1    667  3   0x4000080  fsleep        icinga2
 48884  524109      1    667  3   0x4000080  fsleep        icinga2
 48884  239992      1    667  3   0x4000080  fsleep        icinga2
 43947   23353      1    667  3        0x81  netio         icinga2
 85416  465985      1    732  3    0x200083  fsleep        java
 85416  432983      1    732  3   0x4200083  fsleep        java
 85416  212603      1    732  3   0x4200083  fsleep        java
 85416  117052      1    732  3   0x4200083  fsleep        java
 85416  415605      1    732  3   0x4200083  fsleep        java
 85416  294985      1    732  3   0x4200083  fsleep        java
 85416  280628      1    732  3   0x4200083  fsleep        java
 85416  435584      1    732  3   0x4200083  fsleep        java
 85416  236225      1    732  3   0x4200083  fsleep        java
 85416  321595      1    732  3   0x4200083  fsleep        java
 85416  322202      1    732  3   0x4200083  fsleep        java
 85416  163585      1    732  3   0x4200083  fsleep        java
 85416  150845      1    732  3   0x4200083  fsleep        java
 85416  382683      1    732  3   0x4200083  fsleep        java
 85416  471565      1    732  3   0x4200083  fsleep        java
 85416  476561      1    732  3   0x4200083  fsleep        java
 85416   45395      1    732  3   0x4200083  kqread        java
 85416  280377      1    732  3   0x4200083  kqread        java
 85416  508246      1    732  3   0x4200083  fsleep        java
 85416   89936      1    732  3   0x4200083  piperd        java
 85416  229955      1    732  3   0x4200083  poll          java
 85416  256318      1    732  3   0x4200083  fsleep        java
 85416  487380      1    732  3   0x4200083  fsleep        java
 85416  171577      1    732  3   0x4200083  fsleep        java
 85416  198938      1    732  3   0x4200083  fsleep        java
 85416  504827      1    732  3   0x4200083  fsleep        java
 85416  310971      1    732  3   0x4200083  fsleep        java
 85416  194837      1    732  3   0x4200083  fsleep        java
 85416  116061      1    732  3   0x4200083  netio         java
 85416   99472      1    732  3   0x4200083  fsleep        java
 85416  501716      1    732  3   0x4200083  fsleep        java
 85416   37676      1    732  3   0x4200083  fsleep        java
 85416   54421      1    732  3   0x4200083  fsleep        java
 85416   13391      1    732  3   0x4200083  fsleep        java
 85416  439352      1    732  3   0x4200083  fsleep        java
 85416   18349      1    732  3   0x4200083  netio         java
 85416  261425      1    732  3   0x4200083  fsleep        java
 85416  247620      1    732  3   0x4200083  fsleep        java
 85416  293908      1    732  3   0x4200083  fsleep        java
 85416   34695      1    732  3   0x4200083  fsleep        java
 85416  100711      1    732  3   0x4200083  fsleep        java
 85416  134240      1    732  3   0x4200083  fsleep        java
 85416  430152      1    732  3   0x4200083  fsleep        java
 85416   86107      1    732  3   0x4200083  fsleep        java
 85416  173887      1    732  3   0x4200083  fsleep        java
 85416  151871      1    732  3   0x4200083  fsleep        java
 85416  446134      1    732  3   0x4200083  kqread        java
 85416  397384      1    732  3   0x4200083  kqread        java
 85416  125915      1    732  3   0x4200083  netcon        java
 85416  108025      1    732  3   0x4200083  fsleep        java
 85416  356679      1    732  3   0x4200083  fsleep        java
 85416  259359      1    732  3   0x4200083  kqread        java
 85416  171525      1    732  3   0x4200083  kqread        java
 85416   18719      1    732  3   0x4200083  kqread        java
 85416  510996      1    732  3   0x4200083  netcon        java
 85416  101788      1    732  3   0x4200083  fsleep        java
 85416  277390      1    732  3   0x4200083  kqread        java
 85416  127182      1    732  3   0x4200083  kqread        java
 85416   40791      1    732  3   0x4200083  kqread        java
 85416  188034      1    732  3   0x4200083  netcon        java
 85416   79041      1    732  3   0x4200083  fsleep        java
 85416  316883      1    732  3   0x4200083  kqread        java
 85416  404747      1    732  3   0x4200083  kqread        java
 85416  184335      1    732  3   0x4200083  netcon        java
 85416  349452      1    732  3   0x4200083  fsleep        java
 85416  109034      1    732  3   0x4200083  wait          java
 85416  271792      1    732  3   0x4200083  fsleep        java
 89261  452867  31472    518  3        0x92  kqread        aggregator
 76124  104941  31472      0  3        0x92  kqread        config
 53959  377797  31472    518  3        0x92  kqread        stats
 69299  124137  31472      0  3        0x92  kqread        log
 16380  306101  31472    518  3        0x92  kqread        anvil
 52112  508945  31472      0  3        0x92  kqread        replicator
 31472  234109      1      0  3        0x80  kqread        dovecot
  7934  413437  28752    507  3        0x92  kqread        qmgr
 28752   30448      1      0  3        0x90  kqread        master
 15002  263916  53382    715  3        0x90  select        perl
 67445  258040  53382    715  3        0x90  select        perl
 53382  217706      1    715  3        0x90  nanosleep     perl
 82687  513669   7506    720  3        0x90  kqread        lldpd
  7506  487333      1      0  3        0x80  netio         lldpd
 21588  403008      1      0  3        0x80  poll          nmbd
   860  503698  94726      0  3        0x90  poll          smbd
 98147  166494  94726      0  3        0x80  poll          smbd
  1232  433782  94726      0  3        0x80  poll          smbd
 94726  467878      1      0  3        0x90  poll          smbd
 93550  511359      1    694  3        0x82  poll          colord
 93550  227905      1    694  3   0x4000082  fsleep        colord
 93550  368833      1    694  3   0x4000082  poll          colord
 93550  428109      1    694  3   0x4000082  poll          colord
 86265  384383      1      0  3        0x82  kqread        cupsd
 43299  354167      1    559  3        0x90  kqread        radiusd
 43299  371532      1    559  3   0x4000090  fsleep        radiusd
 43299  483623      1    559  3   0x4000090  fsleep        radiusd
 43299  411012      1    559  3   0x4000090  fsleep        radiusd
 43299  383610      1    559  3   0x4000090  fsleep        radiusd
 43299  502689      1    559  3   0x4000090  fsleep        radiusd
 22425  474615  99021    756  3        0x90  kqread        rspamd
 79544  506401  99021    756  3        0x90  kqread        rspamd
 61313  297277  99021    756  7        0x10                rspamd
 45906  153771  99021    756  3        0x90  kqread        rspamd
 99021   22435      1      0  3        0x80  kqread        rspamd
 83179  215800      1    637  3        0x90  kqread        memcached
 83179   51894      1    637  3   0x4000090  nanosleep     memcached
 83179  518829      1    637  3   0x4000090  kqread        memcached
 83179  448481      1    637  3   0x4000090  kqread        memcached
 83179  203917      1    637  3   0x4000090  kqread        memcached
 83179  103443      1    637  3   0x4000090  kqread        memcached
 83179  266586      1    637  3   0x4000090  fsleep        memcached
 83179  193314      1    637  3   0x4000090  fsleep        memcached
 83179  515914      1    637  3   0x4000090  nanosleep     memcached
 83179  227156      1    637  3   0x4000090  fsleep        memcached
 57603  126951      1     67  3    0x100090  kqread        slowcgi
 29288  165318  20494    502  3        0x93  poll          mysqld
 29288    6188  20494    502  3   0x4000093  fsleep        mysqld
 29288  128765  20494    502  3   0x4000093  fsleep        mysqld
 29288  335296  20494    502  3   0x4000093  fsleep        mysqld
 29288  297309  20494    502  3   0x4000093  fsleep        mysqld
 29288  431416  20494    502  3   0x4000093  fsleep        mysqld
 29288  405662  20494    502  3   0x4000093  fsleep        mysqld
 29288  297212  20494    502  3   0x4000093  fsleep        mysqld
 29288  292526  20494    502  3   0x4000093  fsleep        mysqld
 29288  419341  20494    502  3   0x4000093  fsleep        mysqld
 29288  402339  20494    502  3   0x4000093  fsleep        mysqld
 29288  184716  20494    502  3   0x4000093  fsleep        mysqld
 29288  254414  20494    502  3   0x4000093  fsleep        mysqld
 29288  509345  20494    502  3   0x4000093  fsleep        mysqld
 29288  116073  20494    502  3   0x4000093  fsleep        mysqld
 29288  404595  20494    502  3   0x4000093  fsleep        mysqld
 29288  335134  20494    502  3   0x4000093  fsleep        mysqld
 29288  357240  20494    502  3   0x4000093  fsleep        mysqld
 29288  273294  20494    502  3   0x4000093  fsleep        mysqld
 29288   70863  20494    502  3   0x4000093  nanosleep     mysqld
 29288  404884  20494    502  2   0x4000013                mysqld
 29288  485599  20494    502  3   0x4000093  fsleep        mysqld
 29288  257603  20494    502  3   0x4000093  fsleep        mysqld
 29288  218185  20494    502  3   0x4000093  fsleep        mysqld
 29288  436205  20494    502  3   0x4000093  fsleep        mysqld
 29288  232279  20494    502  3   0x4000093  nanosleep     mysqld
 29288  186284  20494    502  3   0x4000093  fsleep        mysqld
 29288  401848  20494    502  3   0x4000093  fsleep        mysqld
 29288  344714  20494    502  3   0x4000093  fsleep        mysqld
 29288  414608  20494    502  3   0x400009b  sigwait       mysqld
 29288   16895  20494    502  3   0x4000093  fsleep        mysqld
 29288  249021  20494    502  3   0x4000093  poll          mysqld
 29288  252543  20494    502  3   0x4000093  poll          mysqld
 29288  185315  20494    502  3   0x4000093  poll          mysqld
 29288  160742  20494    502  3   0x4000093  poll          mysqld
 29288  321807  20494    502  3   0x4000093  fsleep        mysqld
 29288  493728  20494    502  3   0x4000093  fsleep        mysqld
 29288  108545  20494    502  3   0x4000093  poll          mysqld
 20494   16827      1      0  3    0x10008b  pause         sh
 40147  409371      1    725  3        0x80  poll          rrdcached
 40147  449815      1    725  3   0x4000088  sigwait       rrdcached
 40147  178133      1    725  3   0x4000080  fsleep        rrdcached
 40147  131466      1    725  3   0x4000080  fsleep        rrdcached
 40147  351019      1    725  3   0x4000080  fsleep        rrdcached
 40147   25831      1    725  3   0x4000080  fsleep        rrdcached
 40147   66166      1    725  3   0x4000080  fsleep        rrdcached
 40147   63830      1    725  3   0x4000080  poll          rrdcached
 40147  242127      1    725  2   0x4000480                rrdcached
 40147  365877      1    725  3   0x4000080  poll          rrdcached
 40147  517167      1    725  3   0x4000080  poll          rrdcached
 29590  143208      1    629  3        0x90  poll          avahi-daemon
 27507  409899      1    572  3        0x90  poll          dbus-daemon
 95404  230898      1    686  3        0x80  kqread        redis-server
 95404  270779      1    686  3   0x4000080  fsleep        redis-server
 95404  411177      1    686  3   0x4000080  fsleep        redis-server
 95404  245069      1    686  3   0x4000080  fsleep        redis-server
 14749  304722      1      0  3    0x100080  poll          rpc.bootparamd
 87610  269924      1      0  3    0x100080  poll          rarpd
 76336  396251      1     79  3    0x100090  kqread        tftpd
 34508  257916      1     67  3    0x100090  kqread        slowcgi
 97635  211153      1      0  3    0x100080  kqread        snmpd
 28979  228690      1     91  3    0x100092  kqread        snmpd
 30509  395199      1     91  3        0x92  kqread        snmpd
 35014  304803      1      0  3        0x80  select        sshd
 58781  520833      0      0  3     0x14200  acct          acct
  3887  320770      0      0  3     0x14280  nfsidl        nfsio
 92337  409191      0      0  3     0x14280  nfsidl        nfsio
 11406  417641      0      0  3     0x14280  nfsidl        nfsio
 30071  227540      0      0  3     0x14280  nfsidl        nfsio
 10912  109779      1      0  3        0x80  poll          rpc.statd
 28367  315919      1      0  3        0x80  poll          rpc.lockd
 21344  430606  88190      0  3        0x80  nfsd          nfsd
 34522   13007  88190      0  3        0x80  nfsd          nfsd
 12171  254828  88190      0  3        0x80  nfsd          nfsd
 53557  379783  88190      0  3        0x80  nfsd          nfsd
 56109  199500  88190      0  3        0x80  nfsd          nfsd
 79102  219544  88190      0  3        0x80  nfsd          nfsd
 88190  311924      1      0  3        0x80  netcon        nfsd
 88355  349322  97014      0  3        0x80  poll          mountd
 97014  319681      1      0  3    0x100080  poll          mountd
 71903  142366      1     28  3    0x100090  poll          portmap
 74054  172143      1      0  3    0x100080  poll          ntpd
 91376  395752  84874     83  3    0x100092  poll          ntpd
 84874  512901      1     83  3    0x100092  poll          ntpd
 39359  197924  25053     53  3        0x90  kqread        unbound
 25053  350600      1     53  3        0x90  kqread        unbound
 90108  486928  38513     74  3    0x100092  bpf           pflogd
 38513  346320      1      0  3        0x80  netio         pflogd
 25242  106528  62809     73  2    0x100090                syslogd
 62809  511161      1      0  3    0x100082  netio         syslogd
 30288   57844  77406    115  3    0x100092  kqread        slaacd
 68936  366675  77406    115  3    0x100092  kqread        slaacd
 77406  442935      1      0  3    0x100080  kqread        slaacd
 10002   71553      1      0  3        0x80  mfsidl        mount_mfs
 65093   40656      0      0  3     0x14280  schto         i915/signal:6
 22904  467936      0      0  3     0x14280  schto         i915/signal:2
 24492   36230      0      0  3     0x14280  schto         i915/signal:1
 63389  118459      0      0  3     0x14280  schto         i915/signal:0
 32593  384671      0      0  3     0x14200  bored         i915-userptr-acq
 18124   45838      0      0  3     0x14200  bored         i915_modeset
 44555  369666      0      0  3     0x14200  bored         i915-dp
 66028  131622      0      0  3     0x14200  bored         i915
 30359  439259      0      0  3     0x14200  bored         smr
 85898  482123      0      0  3     0x14200  pgzero        zerothread
 45476  301656      0      0  3     0x14200  aiodoned      aiodoned
 74286  405796      0      0  3     0x14200  syncer        update
 73109  317637      0      0  3     0x14200  cleaner       cleaner
 86807  342727      0      0  3     0x14200  reaper        reaper
 70806  273364      0      0  3     0x14200  pgdaemon      pagedaemon
 56220   61985      0      0  3     0x14200  bored         crynlk
 74594   83224      0      0  3     0x14200  bored         crypto
 21317  157823      0      0  3     0x14200  xhcicmd       usbtask
 98326  356538      0      0  3     0x14200  usbatsk       usbatsk
 38635  492605      0      0  3     0x14200  bored         drmtskl
 74527  227116      0      0  3     0x14200  bored         drmlwq
 47870  516831      0      0  3     0x14200  bored         drmlwq
 34063  178364      0      0  3     0x14200  bored         drmlwq
 48399  391344      0      0  3     0x14200  bored         drmlwq
 17164  463027      0      0  3     0x14200  bored         drmubwq
 70597  389271      0      0  3     0x14200  bored         drmubwq
 87335  405404      0      0  3     0x14200  bored         drmubwq
 96421  324926      0      0  3     0x14200  bored         drmubwq
 69896  119357      0      0  3     0x14200  bored         drmwq
 96088  375547      0      0  3     0x14200  bored         drmwq
  8962  502860      0      0  3     0x14200  bored         drmwq
 33925   58157      0      0  3     0x14200  bored         drmwq
 13826   30395      0      0  3  0x40014200  acpi0         acpi0
 66743  259847      0      0  3  0x40014200                idle3
  7996  338021      0      0  3  0x40014200                idle2
 77045  153926      0      0  3  0x40014200                idle1
 66432  267125      0      0  3     0x14200  bored         sensors
  7646  330918      0      0  7     0x14200                softnet
 85863  316106      0      0  3     0x14200  bored         systqmp
 93264  182571      0      0  3     0x14200  bored         systq
 18419   68744      0      0  3  0x40014200  netlock       softclock
 57732  125761      0      0  3  0x40014200                idle0
     1  501222      0      0  3        0x82  wait          init
     0       0     -1      0  3     0x10200  scheduler     swapper
ddb{3}> mach ddbcpu 2
Stopped at      x86_ipi_db+0x12:        leave
ddb{2}> tr
x86_ipi_db(ffff800022422ff0) at x86_ipi_db+0x12
x86_ipi_handler() at x86_ipi_handler+0x80
Xresume_lapic_ipi() at Xresume_lapic_ipi+0x23
_kernel_lock() at _kernel_lock+0xa2
sorwakeup(fffffd824e8d3050) at sorwakeup+0x78
tcp_input(ffff8000226b91c8,ffff8000226b91d4,6,2) at tcp_input+0x314f
ip_deliver(ffff8000226b91c8,ffff8000226b91d4,6,2) at ip_deliver+0x223
ip_ours(ffff8000226b91c8,ffff8000226b91d4,fffffd80a04d0c00,0) at ip_ours+0x329
ip_input_if(ffff8000226b91c8,ffff8000226b91d4,4,0,ffff80000089c000) at ip_input_if+0x468
ipv4_input(ffff80000089c000,fffffd80a04d0c00) at ipv4_input+0x39
ether_input(ffff80000089c000,fffffd80a04d0c00,0) at ether_input+0x1d3
if_vinput(ffff80000089c000,fffffd80a04d0c00) at if_vinput+0xe5
vlan_input(ffff80000014b048,fffffd80a04d0c00,0) at vlan_input+0x224
if_input_process(ffff80000014b048,ffff8000226b9398) at if_input_process+0xb3
ifiq_process(ffff80000014b400) at ifiq_process+0x69
taskq_thread(ffff80000002f0c0) at taskq_thread+0x4d
end trace frame: 0x0, count: -16
ddb{2}> mach ddbcpu 1
Stopped at      x86_ipi_db+0x12:        leave
ddb{1}> tr
x86_ipi_db(ffff800022411ff0) at x86_ipi_db+0x12
x86_ipi_handler() at x86_ipi_handler+0x80
Xresume_lapic_ipi() at Xresume_lapic_ipi+0x23
_kernel_lock() at _kernel_lock+0xae
pipeselwakeup(fffffd8688212010) at pipeselwakeup+0x6f
pipe_write(fffffd862fe6d900,ffff800034a3eff8,0) at pipe_write+0x3ce
dofilewritev(ffff8000352be5f8,9,ffff800034a3eff8,0,ffff800034a3f0d0) at dofilewritev+0xf9
sys_write(ffff8000352be5f8,ffff800034a3f070,ffff800034a3f0d0) at sys_write+0x51
syscall(ffff800034a3f140) at syscall+0x389
Xsyscall() at Xsyscall+0x128
end of kernel
end trace frame: 0xba71d5d7650, count: -10
ddb{1}> mach ddbcpu 0
Stopped at      x86_ipi_db+0x12:        leave
ddb{0}> tr
x86_ipi_db(ffffffff81f3cff0) at x86_ipi_db+0x12
x86_ipi_handler() at x86_ipi_handler+0x80
Xresume_lapic_ipi() at Xresume_lapic_ipi+0x23
__mp_lock(ffffffff81fe13f8) at __mp_lock+0x62
intr_handler(ffff8000335c6bc0,ffff80000011ad80) at intr_handler+0x44
Xintr_ioapic_edge17_untramp() at Xintr_ioapic_edge17_untramp+0x19f
_kernel_lock() at _kernel_lock+0xa2
Xsoftnet() at Xsoftnet+0x1f
_kernel_lock() at _kernel_lock+0xa9
usertrap(ffff8000335c6e60) at usertrap+0x1b0
recall_trap() at recall_trap+0x8
end of kernel
end trace frame: 0x7f7fffff5190, count: -11
ddb{0}> [-- MARK -- Thu May 28 12:00:00 2020]
bo r
rebooting...


OpenBSD 6.7-current (GENERIC.MP) #213: Sat May 23 20:38:40 MDT 2020
    [hidden email]:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 34247069696 (32660MB)
avail mem = 33196515328 (31658MB)
random: boothowto does not indicate good seed
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xec400 (92 entries)
bios0: vendor Dell Inc. version "A12" date 05/11/2017
bios0: Dell Inc. PowerEdge T20
acpi0 at bios0: ACPI 5.0
acpi0: sleep states S0 S4 S5
acpi0: tables DSDT FACP APIC FPDT SLIC LPIT SSDT SSDT SSDT HPET SSDT MCFG SSDT ASF! DMAR
acpi0: wakeup devices UAR1(S4) RP01(S4) PXSX(S4) RP02(S4) PXSX(S4) PXSX(S4) RP05(S4) PXSX(S4) PXSX(S4) PXSX(S4) PXSX(S4) GLAN(S4) EHC1(S3) EHC2(S3) XHC_(S4) HDEF(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(R) CPU E3-1225 v3 @ 3.20GHz, 3392.58 MHz, 06-3c-03
cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Xeon(R) CPU E3-1225 v3 @ 3.20GHz, 3392.15 MHz, 06-3c-03
cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Xeon(R) CPU E3-1225 v3 @ 3.20GHz, 3392.15 MHz, 06-3c-03
cpu2: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 6 (application processor)
cpu3: Intel(R) Xeon(R) CPU E3-1225 v3 @ 3.20GHz, 3392.15 MHz, 06-3c-03
cpu3: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu3: 256KB 64b/line 8-way L2 cache
cpu3: smt 0, core 3, package 0
ioapic0 at mainbus0: apid 8 pa 0xfec00000, version 20, 24 pins
acpihpet0 at acpi0: 14318179 Hz
acpimcfg0 at acpi0
acpimcfg0: addr 0xf8000000, bus 0-63
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 2 (RP01)
acpiprt2 at acpi0: bus 3 (RP02)
acpiprt3 at acpi0: bus 5 (RP05)
acpiprt4 at acpi0: bus 1 (PEG0)
acpiprt5 at acpi0: bus -1 (PEG1)
acpiprt6 at acpi0: bus -1 (PEG2)
acpiec0 at acpi0: not present
acpicpu0 at acpi0: C2(200@148 mwait.1@0x33), C1(1000@1 mwait.1), PSS
acpicpu1 at acpi0: C2(200@148 mwait.1@0x33), C1(1000@1 mwait.1), PSS
acpicpu2 at acpi0: C2(200@148 mwait.1@0x33), C1(1000@1 mwait.1), PSS
acpicpu3 at acpi0: C2(200@148 mwait.1@0x33), C1(1000@1 mwait.1), PSS
acpitz0 at acpi0: critical temperature is 105 degC
acpitz1 at acpi0: critical temperature is 105 degC
acpipci0 at acpi0 PCI0: 0x00000000 0x00000011 0x00000001
extent `acpipci0 pcibus' (0x0 - 0xff), flags=0
     0x3f - 0xff
extent `acpipci0 pciio' (0x0 - 0xffffffff), flags=0
     0xcf8 - 0xcff
     0x10000 - 0xffffffff
extent `acpipci0 pcimem' (0x0 - 0xffffffffffffffff), flags=0
     0x0 - 0x9ffff
     0xc0000 - 0xd3fff
     0xe8000 - 0xdf1fffff
     0xfeb00000 - 0xffffffffffffffff
acpicmos0 at acpi0
acpibtn0 at acpi0: PWRB
"PNP0C14" at acpi0 not configured
acpivideo0 at acpi0: GFX0
acpivout0 at acpivideo0: DD1F
cpu0: using VERW MDS workaround (except on vmm entry)
cpu0: Enhanced SpeedStep 3392 MHz: speeds: 3201, 3200, 3000, 2900, 2700, 2500, 2300, 2200, 2000, 1800, 1700, 1500, 1300, 1100, 1000, 800 MHz
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel Xeon E3-1200 v3 Host" rev 0x06
ppb0 at pci0 dev 1 function 0 "Intel Core 4G PCIE" rev 0x06: msi
pci1 at ppb0 bus 1
em0 at pci1 dev 0 function 0 "Intel 82572EI" rev 0x06: apic 8 int 16, address 00:15:17:8e:79:85
inteldrm0 at pci0 dev 2 function 0 "Intel HD Graphics P4600" rev 0x06
drm0 at inteldrm0
inteldrm0: msi, HASWELL, gen 7
xhci0 at pci0 dev 20 function 0 "Intel 8 Series xHCI" rev 0x04: msi, xHCI 1.0
usb0 at xhci0: USB revision 3.0
uhub0 at usb0 configuration 1 interface 0 "Intel xHCI root hub" rev 3.00/1.00 addr 1
"Intel 8 Series MEI" rev 0x04 at pci0 dev 22 function 0 not configured
puc0 at pci0 dev 22 function 3 "Intel 8 Series KT" rev 0x04: ports: 16 com
com4 at puc0 port 0 apic 8 int 19: ns16550a, 16 byte fifo
com4: probed fifo depth: 0 bytes
em1 at pci0 dev 25 function 0 "Intel I217-LM" rev 0x04: msi, address f8:b1:56:ac:32:76
ehci0 at pci0 dev 26 function 0 "Intel 8 Series USB" rev 0x04: apic 8 int 16
usb1 at ehci0: USB revision 2.0
uhub1 at usb1 configuration 1 interface 0 "Intel EHCI root hub" rev 2.00/1.00 addr 1
azalia0 at pci0 dev 27 function 0 "Intel 8 Series HD Audio" rev 0x04: msi
azalia0: codecs: Realtek/0x0280
audio0 at azalia0
ppb1 at pci0 dev 28 function 0 "Intel 8 Series PCIE" rev 0xd4
pci2 at ppb1 bus 2
ppb2 at pci0 dev 28 function 1 "Intel 8 Series PCIE" rev 0xd4: msi
pci3 at ppb2 bus 3
ppb3 at pci3 dev 0 function 0 "TI XIO2001 PCIE-PCI" rev 0x00
pci4 at ppb3 bus 4
ppb4 at pci0 dev 28 function 4 "Intel 8 Series PCIE" rev 0xd4: msi
pci5 at ppb4 bus 5
nvme0 at pci5 dev 0 function 0 "Samsung SM961/PM961 NVMe" rev 0x00: msix, NVMe 1.2
nvme0: SAMSUNG MZVLW256HEHP-000L7, firmware 4L7QCXB7, serial S35ENX0J765205
scsibus1 at nvme0: 2 targets, initiator 0
sd0 at scsibus1 targ 1 lun 0: <NVMe, SAMSUNG MZVLW256, 4L7Q>
sd0: 244198MB, 512 bytes/sector, 500118192 sectors
ehci1 at pci0 dev 29 function 0 "Intel 8 Series USB" rev 0x04: apic 8 int 23
usb2 at ehci1: USB revision 2.0
uhub2 at usb2 configuration 1 interface 0 "Intel EHCI root hub" rev 2.00/1.00 addr 1
pcib0 at pci0 dev 31 function 0 "Intel C226 LPC" rev 0x04
ahci0 at pci0 dev 31 function 2 "Intel 8 Series AHCI" rev 0x04: msi, AHCI 1.3
ahci0: port 0: 6.0Gb/s
scsibus2 at ahci0: 32 targets
sd1 at scsibus2 targ 0 lun 0: <ATA, Samsung SSD 860, RVT0> naa.5002538e9984204d
sd1: 476940MB, 512 bytes/sector, 976773168 sectors, thin
ichiic0 at pci0 dev 31 function 3 "Intel 8 Series SMBus" rev 0x04: apic 8 int 18
iic0 at ichiic0
sdtemp0 at iic0 addr 0x18: stts2002
sdtemp1 at iic0 addr 0x19: stts2002
sdtemp2 at iic0 addr 0x1a: stts2002
sdtemp3 at iic0 addr 0x1b: stts2002
spdmem0 at iic0 addr 0x50: 8GB DDR3 SDRAM ECC PC3-12800 with thermal sensor
spdmem1 at iic0 addr 0x51: 8GB DDR3 SDRAM ECC PC3-12800 with thermal sensor
spdmem2 at iic0 addr 0x52: 8GB DDR3 SDRAM ECC PC3-12800 with thermal sensor
spdmem3 at iic0 addr 0x53: 8GB DDR3 SDRAM ECC PC3-12800 with thermal sensor
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com0: console
pckbc0 at isa0 port 0x60/5 irq 1 irq 12
pckbd0 at pckbc0 (kbd slot)
wskbd0 at pckbd0: console keyboard
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
vmm0 at mainbus0: VMX/EPT
uhub3 at uhub0 port 3 configuration 1 interface 0 "Genesys Logic USB2.0 Hub" rev 2.00/77.64 addr 2
uhub4 at uhub3 port 4 configuration 1 interface 0 "Genesys Logic USB2.0 Hub" rev 2.00/77.64 addr 3
uhub5 at uhub0 port 4 configuration 1 interface 0 "Texas Instruments product 0x8142" rev 2.10/1.00 addr 4
uhub6 at uhub5 port 4 configuration 1 interface 0 "Texas Instruments product 0x8142" rev 2.10/1.00 addr 5
uhub7 at uhub0 port 5 configuration 1 interface 0 "GenesysLogic USB2.0 Hub" rev 2.00/92.24 addr 6
uftdi0 at uhub7 port 1 configuration 1 interface 0 "FTDI FT232R USB UART" rev 2.00/6.00 addr 7
ucom0 at uftdi0 portno 1
uaudio0 at uhub7 port 2 configuration 1 interface 1 "GN Netcom GN 9350" rev 2.00/1.00 addr 8
uaudio0: class v1, full-speed, sync, channels: 1 play, 1 rec, 4 ctls
audio1 at uaudio0
uhidev0 at uhub7 port 2 configuration 1 interface 3 "GN Netcom GN 9350" rev 2.00/1.00 addr 8
uhidev0: iclass 3/0
uhid0 at uhidev0: input=2, output=2, feature=0
uhidev1 at uhub7 port 4 configuration 1 interface 0 "Yubico Yubico Yubikey II" rev 2.00/2.12 addr 9
uhidev1: iclass 3/1
ukbd0 at uhidev1: 8 variable keys, 6 key codes
wskbd1 at ukbd0 mux 1
umass0 at uhub0 port 6 configuration 1 interface 0 "Realtek USB3.0 Card Reader" rev 2.10/1.32 addr 10
umass0: using SCSI over Bulk-Only
scsibus3 at umass0: 2 targets, initiator 0
sd2 at scsibus3 targ 1 lun 0: <Generic-, USB3.0 CRW-CF/MD, 1.00> removable
sd3 at scsibus3 targ 1 lun 1: <Generic-, USB3.0 CRW-SM/xD, 1.00> removable
sd4 at scsibus3 targ 1 lun 2: <Generic-, USB3.0 CRW-SD, 1.00> removable
sd5 at scsibus3 targ 1 lun 3: <Generic-, USB3.0 CRW-MS, 1.00> removable
sd6 at scsibus3 targ 1 lun 4: <Generic-, USB3.0 CRW-SD/MS, 1.00> removable
umodem0 at uhub0 port 8 configuration 1 interface 0 "SoftIron, Inc. OverDrive 1000" rev 2.00/1.02 addr 11
umodem0: data interface 1, has no CM over data, has break
umodem0: status change notification available
ucom1 at umodem0
ugen0 at uhub0 port 8 configuration 1 "SoftIron, Inc. OverDrive 1000" rev 2.00/1.02 addr 11
uhidev2 at uhub0 port 10 configuration 1 interface 0 "Lite-On Technology Corp. ThinkPad USB Keyboard with TrackPoint" rev 1.10/1.27 addr 12
uhidev2: iclass 3/1
ukbd1 at uhidev2: 8 variable keys, 6 key codes
wskbd2 at ukbd1 mux 1
uhidev3 at uhub0 port 10 configuration 1 interface 1 "Lite-On Technology Corp. ThinkPad USB Keyboard with TrackPoint" rev 1.10/1.27 addr 12
uhidev3: iclass 3/0, 4 report ids
ums0 at uhidev3 reportid 1: 3 buttons
wsmouse0 at ums0 mux 0
uhid1 at uhidev3 reportid 2: input=1, output=0, feature=0
uhid2 at uhidev3 reportid 3: input=3, output=1, feature=0
uhid3 at uhidev3 reportid 4: input=0, output=0, feature=4
uhub8 at uhub0 port 21 configuration 1 interface 0 "Texas Instruments product 0x8140" rev 3.00/1.00 addr 13
uhub9 at uhub8 port 4 configuration 1 interface 0 "Texas Instruments product 0x8140" rev 3.00/1.00 addr 14
uhub10 at uhub1 port 1 configuration 1 interface 0 "Intel Rate Matching Hub" rev 2.00/0.04 addr 2
uhub11 at uhub2 port 1 configuration 1 interface 0 "Intel Rate Matching Hub" rev 2.00/0.04 addr 2
vscsi0 at root
scsibus4 at vscsi0: 256 targets
softraid0 at root
scsibus5 at softraid0: 256 targets
root on sd1a (2b4432fd9000a5b7.a) swap on sd1b dump on sd1b
WARNING: / was not properly unmounted
inteldrm0: 1920x1200, 32bpp
wsdisplay0 at inteldrm0 mux 1: console (std, vt100 emulation), using wskbd0
wskbd1: connecting to wsdisplay0
wskbd2: connecting to wsdisplay0
wsdisplay0: screen 1-5 added (std, vt100 emulation)
wskbd1: disconnecting from wsdisplay0
wskbd1 detached
ukbd0 detached
uhidev1 detached
arp_rtrequest: bad gateway value: vlan5
uhidev1 at uhub7 port 4 configuration 1 interface 0 "Yubico Yubico Yubikey II" rev 2.00/2.12 addr 9
uhidev1: iclass 3/1
ukbd0 at uhidev1: 8 variable keys, 6 key codes
wskbd1 at ukbd0: console keyboard, using wsdisplay0


Reply | Threaded
Open this post in threaded view
|

Re: X hangs

Visa Hankala-2
On Thu, May 28, 2020 at 01:41:43PM +0100, Stuart Henderson wrote:

> No idea if related to the hangs I saw earlier, but I've just hit a
> uaudio-related kernel crash when starting to play a video in firefox.
>
> RA\xaf\xdeRA\xaf\xdeRA\xaf\xdeRA\xaf\xdeRA\xaf\xdeRA\xaf\xdeRA\xaf\xde: can't set interface
> kernel: protection fault trap, code=0
> Stopped at      uaudio_stream_close+0x8a:       movzbl  0x8(%r12),%esi
> ddb{3}> [-- sthen@localhost attached -- Thu May 28 11:58:19 2020]
>
> Note that RA\xaf\xde is 52 41 af de; case 3 of poison_value() in kern/subr_poison.c
> gives us
>
> $ echo '(POISON1 & 0xffff0000) | (~POISON1 & 0x0000ffff)' | sed s/POISON1/0xdeafbead/g | moo
> 0xdeaf4152
>
> (aren't you glad this is serial console not screenshots :)

Indeed.

> uaudio0 at uhub7 port 2 configuration 1 interface 1 "GN Netcom GN 9350" rev 2.00/1.00 addr 7
> uaudio0: class v1, full-speed, sync, channels: 1 play, 1 rec, 4 ctls
> audio1 at uaudio0
> uhidev0 at uhub7 port 2 configuration 1 interface 3 "GN Netcom GN 9350" rev 2.00/1.00 addr 7
> uhidev0: iclass 3/0
> uhid0 at uhidev0: input=2, output=2, feature=0
> uaudio0: can't reset interface
> uaudio0: can't reset interface
> audio1 detached
> uaudio0 detached
> uhid0 detached
> uhidev0 detached
> RA\xaf\xdeRA\xaf\xdeRA\xaf\xdeRA\xaf\xdeRA\xaf\xdeRA\xaf\xdeRA\xaf\xde: can't set interface
> kernel: protection fault trap, code=0
> Stopped at      uaudio_stream_close+0x8a:       movzbl  0x8(%r12),%esi
> ddb{3}> [-- sthen@localhost attached -- Thu May 28 11:58:19 2020]
>
> ddb{3}>
> ddb{3}> tr
> uaudio_stream_close(ffff800001dfb000,1) at uaudio_stream_close+0x8a
> uaudio_stream_open(ffff800001dfb000,1,ffff8000001e8000,ffff8000001eaa80,2a8,ffffffff816f7630) at uaudio_stream_open+0x761
> uaudio_trigger_output(ffff800001dfb000,ffff8000001e8000,ffff8000001eaa80,2a8,ffffffff816f7630,ffff800001e95c00) at uaudio_trigger_output+0x47
> audio_start_do(ffff800001e95c00) at audio_start_do+0xb5
> audioioctl(2a01,20004126,ffff800035a74470,7,ffff800034fe6750) at audioioctl+0x71
> VOP_IOCTL(fffffd867a72e9e0,20004126,ffff800035a74470,7,fffffd84fea6f9c0,ffff800034fe6750) at VOP_IOCTL+0x55
> vn_ioctl(fffffd867d490f10,20004126,ffff800035a74470,ffff800034fe6750) at vn_ioctl+0x75
> sys_ioctl(ffff800034fe6750,ffff800035a74580,ffff800035a745e0) at sys_ioctl+0x2df
> syscall(ffff800035a74650) at syscall+0x389
> Xsyscall() at Xsyscall+0x128
> end of kernel

I think the above happened because the audio device was not fully
drained of users before it got destroyed. Possibly uaudio_stream_open()
got blocked and then the device was detached. Once uaudio_stream_open()
resumed, the device was gone.

Unfortunately, the kernel has no unified way to do the draining. Maybe
uaudio(4) could adopt more of the tricks that some other USB device
drivers use to manage with detaching devices.

Reply | Threaded
Open this post in threaded view
|

Re: X hangs

Alexandre Ratchov-2
In reply to this post by Stuart Henderson
On Thu, May 28, 2020 at 01:41:43PM +0100, Stuart Henderson wrote:

> uaudio0 at uhub7 port 2 configuration 1 interface 1 "GN Netcom GN 9350" rev 2.00/1.00 addr 7
> uaudio0: class v1, full-speed, sync, channels: 1 play, 1 rec, 4 ctls
> audio1 at uaudio0
> uhidev0 at uhub7 port 2 configuration 1 interface 3 "GN Netcom GN 9350" rev 2.00/1.00 addr 7
> uhidev0: iclass 3/0
> uhid0 at uhidev0: input=2, output=2, feature=0
> uaudio0: can't reset interface
> uaudio0: can't reset interface
> audio1 detached
> uaudio0 detached
> uhid0 detached
> uhidev0 detached
> RA\xaf\xdeRA\xaf\xdeRA\xaf\xdeRA\xaf\xdeRA\xaf\xdeRA\xaf\xdeRA\xaf\xde: can't set interface
> kernel: protection fault trap, code=0
> Stopped at      uaudio_stream_close+0x8a:       movzbl  0x8(%r12),%esi
> ddb{3}> [-- sthen@localhost attached -- Thu May 28 11:58:19 2020]
>
> ddb{3}>
> ddb{3}> tr
> uaudio_stream_close(ffff800001dfb000,1) at uaudio_stream_close+0x8a
> uaudio_stream_open(ffff800001dfb000,1,ffff8000001e8000,ffff8000001eaa80,2a8,ffffffff816f7630) at uaudio_stream_open+0x761
> uaudio_trigger_output(ffff800001dfb000,ffff8000001e8000,ffff8000001eaa80,2a8,ffffffff816f7630,ffff800001e95c00) at uaudio_trigger_output+0x47
> audio_start_do(ffff800001e95c00) at audio_start_do+0xb5
> audioioctl(2a01,20004126,ffff800035a74470,7,ffff800034fe6750) at audioioctl+0x71
> VOP_IOCTL(fffffd867a72e9e0,20004126,ffff800035a74470,7,fffffd84fea6f9c0,ffff800034fe6750) at VOP_IOCTL+0x55
> vn_ioctl(fffffd867d490f10,20004126,ffff800035a74470,ffff800034fe6750) at vn_ioctl+0x75
> sys_ioctl(ffff800034fe6750,ffff800035a74580,ffff800035a745e0) at sys_ioctl+0x2df
> syscall(ffff800035a74650) at syscall+0x389
> Xsyscall() at Xsyscall+0x128
> end of kernel

According to dmesg, audio1 was detached, so we shouldn't enter
audio_start_do().

At this point the DVF_ACTIVE flag is clear; audioioctl() calls
device_lookup() which is supposed to return NULL in this case, so
ioctl() is supposed to return ENXIO, not attempt to start playback.

Reply | Threaded
Open this post in threaded view
|

Re: X hangs

Visa Hankala-2
On Fri, May 29, 2020 at 04:27:46PM +0200, Alexandre Ratchov wrote:

> On Thu, May 28, 2020 at 01:41:43PM +0100, Stuart Henderson wrote:
> > uaudio0 at uhub7 port 2 configuration 1 interface 1 "GN Netcom GN 9350" rev 2.00/1.00 addr 7
> > uaudio0: class v1, full-speed, sync, channels: 1 play, 1 rec, 4 ctls
> > audio1 at uaudio0
> > uhidev0 at uhub7 port 2 configuration 1 interface 3 "GN Netcom GN 9350" rev 2.00/1.00 addr 7
> > uhidev0: iclass 3/0
> > uhid0 at uhidev0: input=2, output=2, feature=0
> > uaudio0: can't reset interface
> > uaudio0: can't reset interface
> > audio1 detached
> > uaudio0 detached
> > uhid0 detached
> > uhidev0 detached
> > RA\xaf\xdeRA\xaf\xdeRA\xaf\xdeRA\xaf\xdeRA\xaf\xdeRA\xaf\xdeRA\xaf\xde: can't set interface
> > kernel: protection fault trap, code=0
> > Stopped at      uaudio_stream_close+0x8a:       movzbl  0x8(%r12),%esi
> > ddb{3}> [-- sthen@localhost attached -- Thu May 28 11:58:19 2020]
> >
> > ddb{3}>
> > ddb{3}> tr
> > uaudio_stream_close(ffff800001dfb000,1) at uaudio_stream_close+0x8a
> > uaudio_stream_open(ffff800001dfb000,1,ffff8000001e8000,ffff8000001eaa80,2a8,ffffffff816f7630) at uaudio_stream_open+0x761
> > uaudio_trigger_output(ffff800001dfb000,ffff8000001e8000,ffff8000001eaa80,2a8,ffffffff816f7630,ffff800001e95c00) at uaudio_trigger_output+0x47
> > audio_start_do(ffff800001e95c00) at audio_start_do+0xb5
> > audioioctl(2a01,20004126,ffff800035a74470,7,ffff800034fe6750) at audioioctl+0x71
> > VOP_IOCTL(fffffd867a72e9e0,20004126,ffff800035a74470,7,fffffd84fea6f9c0,ffff800034fe6750) at VOP_IOCTL+0x55
> > vn_ioctl(fffffd867d490f10,20004126,ffff800035a74470,ffff800034fe6750) at vn_ioctl+0x75
> > sys_ioctl(ffff800034fe6750,ffff800035a74580,ffff800035a745e0) at sys_ioctl+0x2df
> > syscall(ffff800035a74650) at syscall+0x389
> > Xsyscall() at Xsyscall+0x128
> > end of kernel
>
> According to dmesg, audio1 was detached, so we shouldn't enter
> audio_start_do().
>
> At this point the DVF_ACTIVE flag is clear; audioioctl() calls
> device_lookup() which is supposed to return NULL in this case, so
> ioctl() is supposed to return ENXIO, not attempt to start playback.

Lets assume that audio_start_do() started when the device was still
attached to the system. In that case device_lookup() returned a pointer
to a good softc. This is supported by the fact that audio_start_do() did
not crash earlier.

Did usbd_set_interface() block for a moment, letting the detachment
happen? The trace suggests that usbd_set_interface() failed, and when
audio_start_do() resumed, sc pointed to freed memory.

Reply | Threaded
Open this post in threaded view
|

Re: X hangs

Martin Pieuchot
On 29/05/20(Fri) 15:57, Visa Hankala wrote:

> On Fri, May 29, 2020 at 04:27:46PM +0200, Alexandre Ratchov wrote:
> > On Thu, May 28, 2020 at 01:41:43PM +0100, Stuart Henderson wrote:
> > > uaudio0 at uhub7 port 2 configuration 1 interface 1 "GN Netcom GN 9350" rev 2.00/1.00 addr 7
> > > uaudio0: class v1, full-speed, sync, channels: 1 play, 1 rec, 4 ctls
> > > audio1 at uaudio0
> > > uhidev0 at uhub7 port 2 configuration 1 interface 3 "GN Netcom GN 9350" rev 2.00/1.00 addr 7
> > > uhidev0: iclass 3/0
> > > uhid0 at uhidev0: input=2, output=2, feature=0
> > > uaudio0: can't reset interface
> > > uaudio0: can't reset interface
> > > audio1 detached
> > > uaudio0 detached
> > > uhid0 detached
> > > uhidev0 detached
> > > RA\xaf\xdeRA\xaf\xdeRA\xaf\xdeRA\xaf\xdeRA\xaf\xdeRA\xaf\xdeRA\xaf\xde: can't set interface
> > > kernel: protection fault trap, code=0
> > > Stopped at      uaudio_stream_close+0x8a:       movzbl  0x8(%r12),%esi
> > > ddb{3}> [-- sthen@localhost attached -- Thu May 28 11:58:19 2020]
> > >
> > > ddb{3}>
> > > ddb{3}> tr
> > > uaudio_stream_close(ffff800001dfb000,1) at uaudio_stream_close+0x8a
> > > uaudio_stream_open(ffff800001dfb000,1,ffff8000001e8000,ffff8000001eaa80,2a8,ffffffff816f7630) at uaudio_stream_open+0x761
> > > uaudio_trigger_output(ffff800001dfb000,ffff8000001e8000,ffff8000001eaa80,2a8,ffffffff816f7630,ffff800001e95c00) at uaudio_trigger_output+0x47
> > > audio_start_do(ffff800001e95c00) at audio_start_do+0xb5
> > > audioioctl(2a01,20004126,ffff800035a74470,7,ffff800034fe6750) at audioioctl+0x71
> > > VOP_IOCTL(fffffd867a72e9e0,20004126,ffff800035a74470,7,fffffd84fea6f9c0,ffff800034fe6750) at VOP_IOCTL+0x55
> > > vn_ioctl(fffffd867d490f10,20004126,ffff800035a74470,ffff800034fe6750) at vn_ioctl+0x75
> > > sys_ioctl(ffff800034fe6750,ffff800035a74580,ffff800035a745e0) at sys_ioctl+0x2df
> > > syscall(ffff800035a74650) at syscall+0x389
> > > Xsyscall() at Xsyscall+0x128
> > > end of kernel
> >
> > According to dmesg, audio1 was detached, so we shouldn't enter
> > audio_start_do().
> >
> > At this point the DVF_ACTIVE flag is clear; audioioctl() calls
> > device_lookup() which is supposed to return NULL in this case, so
> > ioctl() is supposed to return ENXIO, not attempt to start playback.
>
> Lets assume that audio_start_do() started when the device was still
> attached to the system. In that case device_lookup() returned a pointer
> to a good softc. This is supported by the fact that audio_start_do() did
> not crash earlier.
>
> Did usbd_set_interface() block for a moment, letting the detachment
> happen? The trace suggests that usbd_set_interface() failed, and when
> audio_start_do() resumed, sc pointed to freed memory.

The audio(4) drivers has an unaccounted reference to uaudio(4)'s softc.
So when the USB thread responsible for detaching device kicks in to
clean up the software state of an uaudio(4), it first spins on the
KERNEL_LOCK().  If any of the threads playing/recording audio sleeps
while holding an unaccounted reference to uaudio(4)'s softc, the above
issue can happen.

A way to fix this is to use usbd_ref_incr(9) and its counterpart
usbd_ref_wait(9) in uaudio_detach().

I'm not sure if it's possible for audio(4) to increment the reference
only once.  Is there a place where such increment/decrement can be put?

Otherwise every operation should do the dance.

Reply | Threaded
Open this post in threaded view
|

Re: X hangs

Alexandre Ratchov-2
On Tue, Jun 09, 2020 at 10:53:24AM +0200, Martin Pieuchot wrote:

>
> The audio(4) drivers has an unaccounted reference to uaudio(4)'s softc.
> So when the USB thread responsible for detaching device kicks in to
> clean up the software state of an uaudio(4), it first spins on the
> KERNEL_LOCK().  If any of the threads playing/recording audio sleeps
> while holding an unaccounted reference to uaudio(4)'s softc, the above
> issue can happen.
>
> A way to fix this is to use usbd_ref_incr(9) and its counterpart
> usbd_ref_wait(9) in uaudio_detach().
>
> I'm not sure if it's possible for audio(4) to increment the reference
> only once.  Is there a place where such increment/decrement can be put?
>
> Otherwise every operation should do the dance.

Thanks for the explanation.

- uaudio_{open,close}() could increment/decrement the counter for the
  audio code-paths

- uaudio_{query_devinfo,set_port,get_port} for the controls
  manipulation code-paths. These functions are called directly by the
  kernel, and are not wrapped in uaudio_{open,close} calls (yet).

Resulting diff below

Index: uaudio.c
===================================================================
RCS file: /cvs/src/sys/dev/usb/uaudio.c,v
retrieving revision 1.159
diff -u -p -u -p -r1.159 uaudio.c
--- uaudio.c 4 May 2020 19:19:26 -0000 1.159
+++ uaudio.c 9 Jun 2020 12:07:20 -0000
@@ -3854,6 +3854,8 @@ uaudio_detach(struct device *self, int f
 
  rv = config_detach_children(self, flags);
 
+ usbd_ref_wait(sc->udev);
+
  while ((alt = sc->alts) != NULL) {
  sc->alts = alt->next;
  free(alt, M_DEVBUF, sizeof(struct uaudio_alt));
@@ -3892,6 +3894,8 @@ uaudio_open(void *self, int flags)
  if (usbd_is_dying(sc->udev))
  return EIO;
 
+ usbd_ref_incr(sc->udev);
+
  flags &= (FREAD | FWRITE);
 
  for (p = sc->params_list; p != NULL; p = p->next) {
@@ -3914,6 +3918,7 @@ uaudio_open(void *self, int flags)
  }
  }
 
+ usbd_ref_decr(sc->udev);
  return ENXIO;
 }
 
@@ -3923,6 +3928,7 @@ uaudio_close(void *self)
  struct uaudio_softc *sc = self;
 
  sc->mode = 0;
+ usbd_ref_decr(sc->udev);
 }
 
 int
@@ -4183,9 +4189,8 @@ uaudio_get_props(void *self)
 }
 
 int
-uaudio_get_port(void *arg, struct mixer_ctrl *ctl)
+uaudio_get_port_do(struct uaudio_softc *sc, struct mixer_ctrl *ctl)
 {
- struct uaudio_softc *sc = arg;
  struct uaudio_unit *u;
  struct uaudio_mixent *m;
  unsigned char req_buf[4];
@@ -4253,9 +4258,8 @@ uaudio_get_port(void *arg, struct mixer_
 }
 
 int
-uaudio_set_port(void *arg, struct mixer_ctrl *ctl)
+uaudio_set_port_do(struct uaudio_softc *sc, struct mixer_ctrl *ctl)
 {
- struct uaudio_softc *sc = arg;
  struct uaudio_unit *u;
  struct uaudio_mixent *m;
  unsigned char req_buf[4];
@@ -4312,9 +4316,8 @@ uaudio_set_port(void *arg, struct mixer_
 }
 
 int
-uaudio_query_devinfo(void *arg, struct mixer_devinfo *devinfo)
+uaudio_query_devinfo_do(struct uaudio_softc *sc, struct mixer_devinfo *devinfo)
 {
- struct uaudio_softc *sc = arg;
  struct uaudio_unit *u;
  struct uaudio_mixent *m;
 
@@ -4379,4 +4382,40 @@ uaudio_query_devinfo(void *arg, struct m
  break;
  }
  return 0;
+}
+
+int
+uaudio_get_port(void *arg, struct mixer_ctrl *ctl)
+{
+ struct uaudio_softc *sc = arg;
+ int rc;
+
+ usbd_ref_incr(sc->udev);
+ rc = uaudio_get_port_do(sc, ctl);
+ usbd_ref_decr(sc->udev);
+ return rc;
+}
+
+int
+uaudio_set_port(void *arg, struct mixer_ctrl *ctl)
+{
+ struct uaudio_softc *sc = arg;
+ int rc;
+
+ usbd_ref_incr(sc->udev);
+ rc = uaudio_set_port_do(sc, ctl);
+ usbd_ref_decr(sc->udev);
+ return rc;
+}
+
+int
+uaudio_query_devinfo(void *arg, struct mixer_devinfo *devinfo)
+{
+ struct uaudio_softc *sc = arg;
+ int rc;
+
+ usbd_ref_incr(sc->udev);
+ rc = uaudio_query_devinfo_do(sc, devinfo);
+ usbd_ref_decr(sc->udev);
+ return rc;
 }