Userspace reproducibly hangs for a few minutes

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Userspace reproducibly hangs for a few minutes

Filippo Valsorda
Hello,

Every time I start a certain daemon (Camlistore, written in golang),
during a precise step involving TCP/HTTPS traffic, AES decryption, and
loading data into the heap (loading the encryption metadata of a remote
storage), I experience a long userspace hang.

Sometimes the hang lasts seconds, sometimes minutes. During the hang
everything is unresponsive except the kernel: network traffic drops,
serial console is stuck, ssh is stuck and eventually times out,
sometimes other applications running on the machine notice that "time
jumped forwards", ICMP ping works, trying to ssh in manages to open a
connection but no banner is ever received. Essentially, userspace
doesn't get any CPU time. Most of the times Camlistore fails to start
with a TLS timeout, sometimes it manages to carry on.

Combined output of /var/log/daemon and while(sleep 1; date):

> Mon Mar 20 13:05:26 UTC 2017
> Mon Mar 20 13:05:27 UTC 2017
> Mar 20 13:05:28 7472837 camlistored[92103]: 2017/03/20 13:05:28 Reading encryption metadata...
> Mon Mar 20 13:05:28 UTC 2017
> Mon Mar 20 13:05:38 UTC 2017
> Mon Mar 20 13:05:42 UTC 2017
> Mon Mar 20 13:05:43 UTC 2017
> Mon Mar 20 13:05:44 UTC 2017

ddb breaking from the serial console works, producing variations of the
following two traces. It might be unrelated, but issuing "continue"
often results in a kernel panic. The details of the panic, the ddb ps
and the sendbug context are at the end of the email.

ddb{0}> trace
Debugger() at Debugger+0x9
comintr() at comintr+0x253
intr_handler() at intr_handler+0x67
Xintr_ioapic_edge4() at Xintr_ioapic_edge4+0xc9
--- interrupt ---
Xspllower() at Xspllower+0xc
mtx_leave() at mtx_leave+0x34
uvmpd_scan_inactive() at uvmpd_scan_inactive+0x3e5
uvmpd_scan() at uvmpd_scan+0x2a0
uvm_pageout() at uvm_pageout+0x72
end trace frame: 0x0, count: -9

ddb> trace
Debugger() at Debugger+0x9
comintr() at comintr+0x253
intr_handler() at intr_handler+0x28
Xintr_ioapic_edge4() at Xintr_ioapic_edge4+0xc9
--- interrupt ---
acpicpu_idle() at acpicpu_idle+0x209
cpu_idle_cycle() at cpu_idle_cycle+0x10
end trace frame: 0x0, count: -6

I initially experienced this with 6.0-release, then with 6.0-stable, and
now I reproduced it on the latest snapshot. Can reproduce both with
GENERIC and GENERIC.MP.

I am not sending a bug report (yet?) because I don't have good
reproduction steps to offer, as it only happens when loading a specific
dataset. However, I can't come up with any next step for debugging this,
and it seems like a system issue to me.

Thank you in advance for any pointers.

(Please don't drop me from To/Cc, thanks.)

Filippo


--- ddb ps

ddb{0}> ps
   PID     TID   PPID    UID  S       FLAGS  WAIT          COMMAND
 63722  146561   6308      0  3    0x100001  biowait       ksh
 92103    4793      1   1001  3        0x83  thrsleep      camlistored
 92103  466103      1   1001  3   0x4000083  thrsleep      camlistored
 92103  384503      1   1001  3   0x4000083  thrsleep      camlistored
 92103  319122      1   1001  3   0x4000083  thrsleep      camlistored
 92103  109383      1   1001  7   0x4000003                camlistored
 92103  193551      1   1001  2   0x4000483                camlistored
 92103  171991      1   1001  3   0x4000083  thrsleep      camlistored
 92103  374978      1   1001  3   0x4000083  thrsleep      camlistored
 75111  290822   6308      0  3    0x100083  kqread        tail
 29962  304457      0      0  3     0x14200  bored         srdis
  6308   49434  54236      0  3    0x10008b  pause         ksh
 54236  119780      1      0  3    0x100000  biowait       tmux
 19989  522965  54357      0  3    0x100083  kqread        tmux
 54357   71485  74393      0  3    0x10008b  pause         ksh
 74393  370559  64766   1000  3    0x10008b  pause         ksh
 64766   37272  29806   1000  3        0x10  biowait       sshd
 29806  246367  97042      0  3        0x92  poll          sshd
 11958  234516      1      0  3    0x100003  biowait       getty
 51309  435815      1      0  3    0x100083  ttyin         getty
 10211  322043      1      0  3    0x100083  ttyin         getty
 40280  371749      1      0  3    0x100083  ttyin         getty
 98143  155398      1      0  3    0x100083  ttyin         getty
 95558  393990      1      0  3    0x100083  ttyin         getty
  4996  314592      1      0  3    0x100098  poll          cron
 58632  131899      1    566  3        0x10  biowait       tor
 83136  191438  31021     95  3    0x100092  kqread        smtpd
 60084   28306  31021    103  3    0x100092  kqread        smtpd
  5012  282661  31021     95  3    0x100092  kqread        smtpd
 56987   69124  31021     95  3    0x100092  kqread        smtpd
    44   51012  31021     95  3    0x100092  kqread        smtpd
 96547  397159  31021     95  3    0x100092  kqread        smtpd
 31021   59774      1      0  3    0x100080  kqread        smtpd
 97042   53293      1      0  3        0x80  select        sshd
 66983  373411  88640     83  3    0x100092  poll          ntpd
 88640  105115  41561     83  3    0x100012  biowait       ntpd
 41561   22678      1      0  3    0x100080  poll          ntpd
 64619   80462  23495     74  3    0x100090  bpf           pflogd
 23495   62222      1      0  3        0x80  netio         pflogd
 95812   53321  48627     73  3    0x100010  biowait       syslogd
 48627  100305      1      0  3    0x100082  netio         syslogd
 89444  507883      1     77  3    0x100090  poll          dhclient
 47023  259495      1      0  3        0x80  poll          dhclient
 67587    8267      0      0  3     0x14200  pgzero        zerothread
 34175  505565      0      0  3     0x14200  aiodoned      aiodoned
 45689  324082      0      0  3     0x14200  bqwait        update
 41250  283598      0      0  3     0x14200  cleaner       cleaner
 90072   24208      0      0  3     0x14200  reaper        reaper
*58144  495697      0      0  7     0x14200                pagedaemon
 50007  226008      0      0  3     0x14200  bored         srdis
 29940   86647      0      0  3     0x14200  bored         crynlk
 13961  409675      0      0  3     0x14200  bored         crypto
 42060  471196      0      0  3     0x14200  pftm          pfpurge
 64935  172154      0      0  3     0x14200  usbtsk        usbtask
 66489  450491      0      0  3     0x14200  usbatsk       usbatsk
 79352  330724      0      0  2  0x40014200                acpi0
 57763  356115      0      0  3  0x40014200                idle1
 11190  125915      0      0  3     0x14200  bored         sensors
 54169   38780      0      0  3     0x14200  bored         softnet
 39762  115168      0      0  3     0x14200  bored         systqmp
 61789  216165      0      0  3     0x14200  bored         systq
 65388  516936      0      0  3  0x40014200  bored         softclock
  5065   96329      0      0  3  0x40014200                idle0
 15105  260699      0      0  3     0x14200  bored         sbar
     1  396118      0      0  3        0x82  wait          init
     0       0     -1      0  3     0x10200  scheduler     swapper


--- continue panic information

ddb{0}> continue
panic: init died (signal 4, exit 0)
Stopped at      Debugger+0x9:   leave
    TID    PID    UID     PRFLAGS     PFLAGS  CPU  COMMAND
  24208  90072      0     0x14000      0x200    1  reaper
*396118      1      0       0x802     0x2000    0  init
Debugger() at Debugger+0x9
panic() at panic+0xfe
exit1() at exit1+0x592
sendsig() at sendsig+0x152
trapsignal() at trapsignal+0x174
trap() at trap+0x6c0
--- trap (number 6) ---
end of kernel
end trace frame: 0x7f7ffffd5bd8, count: 9
0x1417a9413c2c:
https://www.openbsd.org/ddb.html describes the minimum info required in
bug reports.  Insufficient info makes it difficult to find and fix bugs.
ddb{0}> trace
Debugger() at Debugger+0x9
panic() at panic+0xfe
exit1() at exit1+0x592
sendsig() at sendsig+0x152
trapsignal() at trapsignal+0x174
trap() at trap+0x6c0
--- trap (number 6) ---
end of kernel
end trace frame: 0x7f7ffffd5bd8, count: -6
0x1417a9413c2c:
ddb{0}> machine ddbcpu 1
Stopped at      Debugger+0x9:   leave
Debugger() at Debugger+0x9
x86_ipi_handler() at x86_ipi_handler+0x76
Xresume_lapic_ipi() at Xresume_lapic_ipi+0x1c
--- interrupt ---
__mp_lock() at __mp_lock+0x42
uvm_unmap_detach() at uvm_unmap_detach+0x2b
uvm_map_teardown() at uvm_map_teardown+0x19d
uvmspace_free() at uvmspace_free+0x36
uvm_exit() at uvm_exit+0x18
reaper() at reaper+0xdb
end trace frame: 0x0, count: 6


--- sendbug context

>Environment:
        System      : OpenBSD 6.1
        Details     : OpenBSD 6.1-beta (GENERIC.MP) #37: Fri Mar 17
        07:46:47 MDT 2017
                         [hidden email]:/usr/src/sys/arch/amd64/compile/GENERIC.MP

        Architecture: OpenBSD.amd64
        Machine     : amd64

dmesg:
OpenBSD 6.1-beta (GENERIC.MP) #37: Fri Mar 17 07:46:47 MDT 2017
    [hidden email]:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 2094940160 (1997MB)
avail mem = 2026835968 (1932MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0x7fbdb000 (63 entries)
bios0: vendor HP version "J06" date 06/06/2014
bios0: HP ProLiant MicroServer Gen8
acpi0 at bios0: rev 2
acpi0: sleep states S0 S4 S5
acpi0: tables DSDT FACP SPCR MCFG HPET FFFF SPMI ERST APIC FFFF BERT
HEST FFFF FFFF SSDT SSDT SSDT SSDT SSDT
acpi0: wakeup devices PCI0(S4)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimcfg0 at acpi0 addr 0xf4000000, bus 0-63
acpihpet0 at acpi0: 14318179 Hz
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Celeron(R) CPU G1610T @ 2.30GHz, 2295.13 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,POPCNT,DEADLINE,XSAVE,NXE,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: TSC frequency 2295126240 Hz
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.1, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Celeron(R) CPU G1610T @ 2.30GHz, 2294.79 MHz
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2cpu1:
failed to identify
,SSSE3ioapic0 at mainbus0,CX16,xTPR: apid 8 pa
0xfec00000,PDCM,PCID,SSE4.1,SSE4.2,POPCNT,DEADLINE,XSAVE, version 20, 24
pins
,NXE,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
acpiprt0 at acpi0: bus 13 (IPT1)
acpiprt1 at acpi0: bus -1 (IPT2)
acpiprt2 at acpi0: bus -1 (IPT3)
acpiprt3 at acpi0: bus -1 (IPT4)
acpiprt4 at acpi0: bus 3 (IPT5)
acpiprt5 at acpi0: bus -1 (IPT6)
acpiprt6 at acpi0: bus 4 (IPT7)
acpiprt7 at acpi0: bus 1 (IPT8)
acpiprt8 at acpi0: bus 7 (PT02)
acpiprt9 at acpi0: bus -1 (PT03)
acpiprt10 at acpi0: bus 2 (PT05)
acpiprt11 at acpi0: bus 0 (PCI0)
acpicpu0 at acpi0: C2(350@96 mwait.1@0x20), C1(1000@1 mwait.1)
acpicpu1 at acpi0: C2(350@96 mwait.1@0x20), C1(1000@1 mwait.1)
acpitz0 at acpi0: critical temperature is 31 degC
"IPI0001" at acpi0 not configured
"PNP0A06" at acpi0 not configured
"PNP0303" at acpi0 not configured
"PNP0F13" at acpi0 not configured
"ACPI000D" at acpi0 not configured
ipmi at mainbus0 not configured
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel Core 3G Host" rev 0x09
ppb0 at pci0 dev 1 function 0 "Intel Core 3G PCIE" rev 0x09: msi
pci1 at ppb0 bus 7
ppb1 at pci0 dev 6 function 0 "Intel Core 3G PCIE" rev 0x09: msi
pci2 at ppb1 bus 2
ehci0 at pci0 dev 26 function 0 "Intel 6 Series USB" rev 0x05: apic 8
int 21
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 configuration 1 interface 0 "Intel EHCI root hub" rev
2.00/1.00 addr 1
ppb2 at pci0 dev 28 function 0 "Intel 6 Series PCIE" rev 0xb5
pci3 at ppb2 bus 13
ppb3 at pci0 dev 28 function 4 "Intel 6 Series PCIE" rev 0xb5
pci4 at ppb3 bus 3
bge0 at pci4 dev 0 function 0 "Broadcom BCM5720" rev 0x00, BCM5720 A0
(0x5720000), APE firmware NCSI 1.2.46.0: msi, address d0:bf:9c:46:95:44
brgphy0 at bge0 phy 1: BCM5720C 10/100/1000baseT PHY, rev. 0
bge1 at pci4 dev 0 function 1 "Broadcom BCM5720" rev 0x00, BCM5720 A0
(0x5720000), APE firmware NCSI 1.2.46.0: msi, address d0:bf:9c:46:95:45
brgphy1 at bge1 phy 2: BCM5720C 10/100/1000baseT PHY, rev. 0
ppb4 at pci0 dev 28 function 6 "Intel 6 Series PCIE" rev 0xb5
pci5 at ppb4 bus 4
xhci0 at pci5 dev 0 function 0 "Renesas uPD720201 xHCI" rev 0x03: msi
usb1 at xhci0: USB revision 3.0
uhub1 at usb1 configuration 1 interface 0 "Renesas xHCI root hub" rev
3.00/1.00 addr 1
ppb5 at pci0 dev 28 function 7 "Intel 6 Series PCIE" rev 0xb5
pci6 at ppb5 bus 1
"Hewlett-Packard iLO3 Slave" rev 0x05 at pci6 dev 0 function 0 not
configured
vga1 at pci6 dev 0 function 1 "Matrox MGA G200eH" rev 0x00
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
"Hewlett-Packard iLO3 Management" rev 0x05 at pci6 dev 0 function 2 not
configured
uhci0 at pci6 dev 0 function 4 "Hewlett-Packard USB" rev 0x02: apic 8
int 16
usb2 at uhci0: USB revision 1.0
uhub2 at usb2 configuration 1 interface 0 "Hewlett-Packard UHCI root
hub" rev 1.00/1.00 addr 1
ehci1 at pci0 dev 29 function 0 "Intel 6 Series USB" rev 0x05: apic 8
int 20
usb3 at ehci1: USB revision 2.0
uhub3 at usb3 configuration 1 interface 0 "Intel EHCI root hub" rev
2.00/1.00 addr 1
ppb6 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev 0xa5
pci7 at ppb6 bus 20
pcib0 at pci0 dev 31 function 0 "Intel C204 LPC" rev 0x05
ahci0 at pci0 dev 31 function 2 "Intel 6 Series AHCI" rev 0x05: msi,
AHCI 1.3
ahci0: port 0: 6.0Gb/s
ahci0: port 1: 6.0Gb/s
ahci0: port 2: 3.0Gb/s
ahci0: port 3: 3.0Gb/s
scsibus1 at ahci0: 32 targets
sd0 at scsibus1 targ 0 lun 0: <ATA, WDC WD30EFRX-68E, 82.0> SCSI3
0/direct fixed naa.50014ee20c774af4
sd0: 2861588MB, 512 bytes/sector, 5860533168 sectors
sd1 at scsibus1 targ 1 lun 0: <ATA, WDC WD30EFRX-68E, 82.0> SCSI3
0/direct fixed naa.50014ee20c75f5a5
sd1: 2861588MB, 512 bytes/sector, 5860533168 sectors
sd2 at scsibus1 targ 2 lun 0: <ATA, WDC WD30EFRX-68E, 82.0> SCSI3
0/direct fixed naa.50014ee263232d72
sd2: 2861588MB, 512 bytes/sector, 5860533168 sectors
sd3 at scsibus1 targ 3 lun 0: <ATA, WDC WD30EFRX-68E, 82.0> SCSI3
0/direct fixed naa.50014ee20cdaaf8b
sd3: 2861588MB, 512 bytes/sector, 5860533168 sectors
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com0: console
pckbc0 at isa0 port 0x60/5 irq 1 irq 12
pckbd0 at pckbc0 (kbd slot)
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
vmm0 at mainbus0: VMX/EPT
uhub4 at uhub0 port 1 configuration 1 interface 0 "Intel Rate Matching
Hub" rev 2.00/0.00 addr 2
umass0 at uhub1 port 2 configuration 1 interface 0 "Seagate Expansion
Desk" rev 3.00/1.00 addr 2
umass0: using SCSI over Bulk-Only
scsibus2 at umass0: 2 targets, initiator 0
sd4 at scsibus2 targ 1 lun 0: <Seagate, Expansion Desk, 9401> SCSI4
0/direct fixed
sd4: 2861588MB, 512 bytes/sector, 5860533167 sectors
umass1 at uhub1 port 5 configuration 1 interface 0 "Seagate Expansion
Desk" rev 2.10/9.11 addr 3
umass1: using SCSI over Bulk-Only
scsibus3 at umass1: 2 targets, initiator 0
sd5 at scsibus3 targ 1 lun 0: <Seagate, Expansion Desk, 0911> SCSI4
0/direct fixed
sd5: 4769307MB, 512 bytes/sector, 9767541167 sectors
uhub5 at uhub3 port 1 configuration 1 interface 0 "Intel Rate Matching
Hub" rev 2.00/0.00 addr 2
uhub6 at uhub5 port 3 configuration 1 interface 0 "Standard Microsystems
product 0x2660" rev 2.00/8.01 addr 3
umass2 at uhub6 port 1 configuration 1 interface 0 "Generic Ultra Fast
Media Reader" rev 2.00/2.09 addr 4
umass2: using SCSI over Bulk-Only
scsibus4 at umass2: 2 targets, initiator 0
sd6 at scsibus4 targ 1 lun 0: <HP iLO, Internal SD-CARD, 2.09> SCSI0
0/direct fixed serial.04244030000002660A01
sd6: 14804MB, 512 bytes/sector, 30318592 sectors
vscsi0 at root
scsibus5 at vscsi0: 256 targets
softraid0 at root
scsibus6 at softraid0: 256 targets
sd7 at scsibus6 targ 1 lun 0: <OPENBSD, SR RAID 1, 006> SCSI2 0/direct
fixed
sd7: 2861588MB, 512 bytes/sector, 5860532513 sectors
root on sd6a (b8519dcc12fa73b0.a) swap on sd6b dump on sd6b

usbdevs:
Controller /dev/usb0:
addr 1: high speed, self powered, config 1, EHCI root hub(0x0000),
Intel(0x8086), rev 1.00
 port 1 addr 2: high speed, self powered, config 1, Rate Matching
 Hub(0x0024), Intel(0x8087), rev 0.00
  port 1 powered
  port 2 powered
  port 3 powered
  port 4 powered
  port 5 powered
  port 6 powered
 port 2 powered
Controller /dev/usb1:
addr 1: super speed, self powered, config 1, xHCI root hub(0x0000),
Renesas(0x1912), rev 1.00
 port 1 disabled
 port 2 addr 2: super speed, self powered, config 1, Expansion
 Desk(0x3322), Seagate(0x0bc2), rev 1.00, iSerialNumber NA8FEB73
 port 3 disabled
 port 4 disabled
 port 5 addr 3: high speed, self powered, config 1, Expansion
 Desk(0x331a), Seagate(0x0bc2), rev 9.11, iSerialNumber NA8FFSE4
 port 6 disabled
Controller /dev/usb2:
addr 1: full speed, self powered, config 1, UHCI root hub(0x0000),
Hewlett-Packard(0x103c), rev 1.00
 port 1 powered
 port 2 powered
Controller /dev/usb3:
addr 1: high speed, self powered, config 1, EHCI root hub(0x0000),
Intel(0x8086), rev 1.00
 port 1 addr 2: high speed, self powered, config 1, Rate Matching
 Hub(0x0024), Intel(0x8087), rev 0.00
  port 1 powered
  port 2 powered
  port 3 addr 3: high speed, self powered, config 1, product
  0x2660(0x2660), Standard Microsystems(0x0424), rev 8.01
   port 1 addr 4: high speed, power 96 mA, config 1, Ultra Fast Media
   Reader(0x4030), Generic(0x0424), rev 2.09, iSerialNumber 000002660A01
   port 2 powered
  port 4 powered
  port 5 powered
  port 6 powered
 port 2 powered

pcidump:
Domain /dev/pci0:
 0:0:0: Intel Core 3G Host
        0x0000: Vendor ID: 8086 Product ID: 0150
        0x0004: Command: 0146 Status: 2090
        0x0008: Class: 06 Subclass: 00 Interface: 00 Revision: 09
        0x000c: BIST: 00 Header Type: 00 Latency Timer: 00 Cache Line
        Size: 00
        0x0010: BAR empty (00000000)
        0x0014: BAR empty (00000000)
        0x0018: BAR empty (00000000)
        0x001c: BAR empty (00000000)
        0x0020: BAR empty (00000000)
        0x0024: BAR empty (00000000)
        0x0028: Cardbus CIS: 00000000
        0x002c: Subsystem Vendor ID: 0000 Product ID: 0000
        0x0030: Expansion ROM Base Address: 00000000
        0x0038: 00000000
        0x003c: Interrupt Pin: 00 Line: 00 Min Gnt: 00 Max Lat: 00
        0x00e0: Capability 0x09: Vendor Specific
        0x0000: 01508086 20900146 06000009 00000000
        0x0010: 00000000 00000000 00000000 00000000
        0x0020: 00000000 00000000 00000000 00000000
        0x0030: 00000000 000000e0 00000000 00000000
        0x0040: fed19001 00000000 fed10001 00000000
        0x0050: 00000002 00002009 00000000 7e000000
        0x0060: f4000005 00000000 fed18001 00000000
        0x0070: fff00000 0000007f 00000000 00000000
        0x0080: 33333330 00333333 00000002 00000000
        0x0090: fff00000 0000000f 00000000 00000000
        0x00a0: 80000000 00000000 80000000 00000000
        0x00b0: 80000000 80000000 7e000001 80000000
        0x00c0: 00000000 00000000 00000000 00000002
        0x00d0: 00000000 00000000 00000000 00000000
        0x00e0: 010c0009 61808892 021000e0 00000000
        0x00f0: 00000000 00000000 00090fc8 00000000
 0:1:0: Intel Core 3G PCIE
        0x0000: Vendor ID: 8086 Product ID: 0151
        0x0004: Command: 0147 Status: 0010
        0x0008: Class: 06 Subclass: 04 Interface: 00 Revision: 09
        0x000c: BIST: 00 Header Type: 81 Latency Timer: 00 Cache Line
        Size: 10
        0x0010: 00000000
        0x0014: 00000000
        0x0018: Primary Bus: 0 Secondary Bus: 7 Subordinate Bus: 7
                Secondary Latency Timer: 00
        0x001c: I/O Base: f0 I/O Limit: 00 Secondary Status: 0000
        0x0020: Memory Base: fff0 Memory Limit: 0000
        0x0024: Prefetch Memory Base: fff1 Prefetch Memory Limit: 0001
        0x0028: Prefetch Memory Base Upper 32 Bits: 00000000
        0x002c: Prefetch Memory Limit Upper 32 Bits: 00000000
        0x0030: I/O Base Upper 16 Bits: 0000 I/O Limit Upper 16 Bits:
        0000
        0x0038: Expansion ROM Base Address: 00000000
        0x003c: Interrupt Pin: 01 Line: ff Bridge Control: 0003
        0x0088: Capability 0x0d: PCI-PCI
        0x0080: Capability 0x01: Power Management
                State: D0 PME# enabled
        0x0090: Capability 0x05: Message Signalled Interrupts (MSI)
        0x00a0: Capability 0x10: PCI Express
                Link Speed: 2.5 / 5.0 GT/s Link Width: x0 / x16
        0x0100: Enhanced Capability 0x02: Virtual Channel Capability
        0x0140: Enhanced Capability 0x05: Root Complex Link Declaration
        0x0000: 01518086 00100147 06040009 00810010
        0x0010: 00000000 00000000 00070700 000000f0
        0x0020: 0000fff0 0001fff1 00000000 00000000
        0x0030: 00000000 00000088 00000000 000301ff
        0x0040: 00000000 00000000 00000000 00000000
        0x0050: 00000000 00000000 00000000 00000000
        0x0060: 00000000 00000000 00000000 00000000
        0x0070: 00000000 00000000 00000000 0a000000
        0x0080: c8039001 00000108 0000800d 330b103c
        0x0090: 0001a005 fee00000 00000060 00000000
        0x00a0: 01420010 00008001 00000026 0261cd03
        0x00b0: 18010210 00040000 00000000 00000006
        0x00c0: 00000000 00000000 00000000 00000006
        0x00d0: 00000003 00000000 00000000 00000000
        0x00e0: 00000000 00000000 00000000 00000000
        0x00f0: 00000000 00010000 00000000 00100001
 0:6:0: Intel Core 3G PCIE
        0x0000: Vendor ID: 8086 Product ID: 015d
        0x0004: Command: 0147 Status: 0010
        0x0008: Class: 06 Subclass: 04 Interface: 00 Revision: 09
        0x000c: BIST: 00 Header Type: 81 Latency Timer: 00 Cache Line
        Size: 10
        0x0010: 00000000
        0x0014: 00000000
        0x0018: Primary Bus: 0 Secondary Bus: 2 Subordinate Bus: 2
                Secondary Latency Timer: 00
        0x001c: I/O Base: f0 I/O Limit: 00 Secondary Status: 0000
        0x0020: Memory Base: fff0 Memory Limit: 0000
        0x0024: Prefetch Memory Base: fff1 Prefetch Memory Limit: 0001
        0x0028: Prefetch Memory Base Upper 32 Bits: 00000000
        0x002c: Prefetch Memory Limit Upper 32 Bits: 00000000
        0x0030: I/O Base Upper 16 Bits: 0000 I/O Limit Upper 16 Bits:
        0000
        0x0038: Expansion ROM Base Address: 00000000
        0x003c: Interrupt Pin: 01 Line: ff Bridge Control: 0003
        0x0088: Capability 0x0d: PCI-PCI
        0x0080: Capability 0x01: Power Management
                State: D0 PME# enabled
        0x0090: Capability 0x05: Message Signalled Interrupts (MSI)
        0x00a0: Capability 0x10: PCI Express
                Link Speed: 5.0 / 5.0 GT/s Link Width: x1 / x4
        0x0100: Enhanced Capability 0x02: Virtual Channel Capability
        0x0140: Enhanced Capability 0x05: Root Complex Link Declaration
        0x0000: 015d8086 00100147 06040009 00810010
        0x0010: 00000000 00000000 00020200 000000f0
        0x0020: 0000fff0 0001fff1 00000000 00000000
        0x0030: 00000000 00000088 00000000 000301ff
        0x0040: 00000000 00000000 00000000 00000000
        0x0050: 00000000 00000000 00000000 00000000
        0x0060: 00000000 00000000 00000000 00000000
        0x0070: 00000000 00000000 00000000 0a000000
        0x0080: c8039001 00000108 0000800d 330b103c
        0x0090: 0001a005 fee00000 00000061 00000000
        0x00a0: 01420010 00008000 00000006 0561cc42
        0x00b0: 50120010 00040000 00400000 00000006
        0x00c0: 00000000 00000000 00000000 00000006
        0x00d0: 00000002 00000000 00000000 00000000
        0x00e0: 00000000 00000000 00000000 00000000
        0x00f0: 00000000 00010000 00000000 00100001
 0:26:0: Intel 6 Series USB
        0x0000: Vendor ID: 8086 Product ID: 1c2d
        0x0004: Command: 0146 Status: 0290
        0x0008: Class: 0c Subclass: 03 Interface: 20 Revision: 05
        0x000c: BIST: 00 Header Type: 00 Latency Timer: 00 Cache Line
        Size: 00
        0x0010: BAR mem 32bit addr: 0xfacf0000/0x00000400
        0x0014: BAR empty (00000000)
        0x0018: BAR empty (00000000)
        0x001c: BAR empty (00000000)
        0x0020: BAR empty (00000000)
        0x0024: BAR empty (00000000)
        0x0028: Cardbus CIS: 00000000
        0x002c: Subsystem Vendor ID: 103c Product ID: 330d
        0x0030: Expansion ROM Base Address: 00000000
        0x0038: 00000000
        0x003c: Interrupt Pin: 01 Line: 03 Min Gnt: 00 Max Lat: 00
        0x0050: Capability 0x01: Power Management
                State: D0 PME# enabled
        0x0058: Capability 0x0a: Debug Port
        0x0098: Capability 0x13: PCI Advanced Features
        0x0000: 1c2d8086 02900146 0c032005 00000000
        0x0010: facf0000 00000000 00000000 00000000
        0x0020: 00000000 00000000 00000000 330d103c
        0x0030: 00000000 00000050 00000000 00000103
        0x0040: 00000000 00000000 00000000 00000000
        0x0050: c9c25801 00000100 20a0980a 00000000
        0x0060: 07ff2020 00000000 01000001 c0082000
        0x0070: 3fdf0000 00000000 00000000 00000000
        0x0080: 00800000 930c8811 24000d30 00000000
        0x0090: 00000000 00000000 03060013 00000000
        0x00a0: 00000000 00000000 00000000 00000000
        0x00b0: 00000000 00000000 00000000 00000000
        0x00c0: 00000000 00000000 00000000 00000000
        0x00d0: 00000000 00ffaa00 00000000 00000000
        0x00e0: 00000000 22010008 26300a09 7d56b004
        0x00f0: 00000000 00808588 08070f87 205b1708
 0:28:0: Intel 6 Series PCIE
        0x0000: Vendor ID: 8086 Product ID: 1c10
        0x0004: Command: 0147 Status: 0010
        0x0008: Class: 06 Subclass: 04 Interface: 00 Revision: b5
        0x000c: BIST: 00 Header Type: 81 Latency Timer: 00 Cache Line
        Size: 10
        0x0010: 00000000
        0x0014: 00000000
        0x0018: Primary Bus: 0 Secondary Bus: 13 Subordinate Bus: 13
                Secondary Latency Timer: 00
        0x001c: I/O Base: f0 I/O Limit: 00 Secondary Status: 2000
        0x0020: Memory Base: fff0 Memory Limit: 0000
        0x0024: Prefetch Memory Base: fff1 Prefetch Memory Limit: 0001
        0x0028: Prefetch Memory Base Upper 32 Bits: 00000000
        0x002c: Prefetch Memory Limit Upper 32 Bits: 00000000
        0x0030: I/O Base Upper 16 Bits: 0000 I/O Limit Upper 16 Bits:
        0000
        0x0038: Expansion ROM Base Address: 00000000
        0x003c: Interrupt Pin: 01 Line: ff Bridge Control: 0003
        0x0040: Capability 0x10: PCI Express
                Link Speed: 2.5 / 5.0 GT/s Link Width: x0 / x4
        0x0100: Enhanced Capability 0x00: Unknown
        0x0080: Capability 0x05: Message Signalled Interrupts (MSI)
        0x0090: Capability 0x0d: PCI-PCI
        0x00a0: Capability 0x01: Power Management
                State: D0 PME# enabled
        0x0000: 1c108086 00100147 060400b5 00810010
        0x0010: 00000000 00000000 000d0d00 200000f0
        0x0020: 0000fff0 0001fff1 00000000 00000000
        0x0030: 00000000 00000040 00000000 000301ff
        0x0040: 00428010 00008000 00100006 01114c42
        0x0050: 18010000 00040060 00400000 00000006
        0x0060: 00000000 00000016 00000000 00000000
        0x0070: 00010002 00000000 00000000 00000000
        0x0080: 00009005 00000000 00000000 00000000
        0x0090: 0000a00d 330d103c 00000000 00000000
        0x00a0: c8020001 00000100 00000000 00000000
        0x00b0: 00000000 00000000 00000000 00000000
        0x00c0: 00000000 00000000 00000000 00000000
        0x00d0: 01000000 00000b02 01110000 00000000
        0x00e0: 00000000 00000000 00000000 00000000
        0x00f0: 00000000 00000000 08070f87 00000000
 0:28:4: Intel 6 Series PCIE
        0x0000: Vendor ID: 8086 Product ID: 1c18
        0x0004: Command: 0147 Status: 0010
        0x0008: Class: 06 Subclass: 04 Interface: 00 Revision: b5
        0x000c: BIST: 00 Header Type: 81 Latency Timer: 00 Cache Line
        Size: 10
        0x0010: 00000000
        0x0014: 00000000
        0x0018: Primary Bus: 0 Secondary Bus: 3 Subordinate Bus: 3
                Secondary Latency Timer: 00
        0x001c: I/O Base: f0 I/O Limit: 00 Secondary Status: 2000
        0x0020: Memory Base: fc00 Memory Limit: fdf0
        0x0024: Prefetch Memory Base: fab1 Prefetch Memory Limit: fab1
        0x0028: Prefetch Memory Base Upper 32 Bits: 00000000
        0x002c: Prefetch Memory Limit Upper 32 Bits: 00000000
        0x0030: I/O Base Upper 16 Bits: 0000 I/O Limit Upper 16 Bits:
        0000
        0x0038: Expansion ROM Base Address: 00000000
        0x003c: Interrupt Pin: 01 Line: ff Bridge Control: 0003
        0x0040: Capability 0x10: PCI Express
                Link Speed: 5.0 / 5.0 GT/s Link Width: x1 / x1
        0x0100: Enhanced Capability 0x00: Unknown
        0x0080: Capability 0x05: Message Signalled Interrupts (MSI)
        0x0090: Capability 0x0d: PCI-PCI
        0x00a0: Capability 0x01: Power Management
                State: D0 PME# enabled
        0x0000: 1c188086 00100147 060400b5 00810010
        0x0010: 00000000 00000000 00030300 200000f0
        0x0020: fdf0fc00 fab1fab1 00000000 00000000
        0x0030: 00000000 00000040 00000000 000301ff
        0x0040: 00428010 00008000 00100006 05114c12
        0x0050: 70120000 00040060 01400000 00000006
        0x0060: 00000000 00000016 00000000 00000000
        0x0070: 00000002 00000000 00000000 00000000
        0x0080: 00009005 00000000 00000000 00000000
        0x0090: 0000a00d 330d103c 00000000 00000000
        0x00a0: c8020001 00000100 00000000 00000000
        0x00b0: 00000000 00000000 00000000 00000000
        0x00c0: 00000000 00000000 00000000 00000000
        0x00d0: 01000000 00000b02 01110000 00000000
        0x00e0: 00000000 00000000 00000000 00000000
        0x00f0: 00000000 00000000 08070f87 00000000
 0:28:6: Intel 6 Series PCIE
        0x0000: Vendor ID: 8086 Product ID: 1c1c
        0x0004: Command: 0147 Status: 0010
        0x0008: Class: 06 Subclass: 04 Interface: 00 Revision: b5
        0x000c: BIST: 00 Header Type: 81 Latency Timer: 00 Cache Line
        Size: 10
        0x0010: 00000000
        0x0014: 00000000
        0x0018: Primary Bus: 0 Secondary Bus: 4 Subordinate Bus: 4
                Secondary Latency Timer: 00
        0x001c: I/O Base: f0 I/O Limit: 00 Secondary Status: 0000
        0x0020: Memory Base: fbf0 Memory Limit: fbf0
        0x0024: Prefetch Memory Base: fff1 Prefetch Memory Limit: 0001
        0x0028: Prefetch Memory Base Upper 32 Bits: 00000000
        0x002c: Prefetch Memory Limit Upper 32 Bits: 00000000
        0x0030: I/O Base Upper 16 Bits: 0000 I/O Limit Upper 16 Bits:
        0000
        0x0038: Expansion ROM Base Address: 00000000
        0x003c: Interrupt Pin: 03 Line: ff Bridge Control: 0003
        0x0040: Capability 0x10: PCI Express
                Link Speed: 5.0 / 5.0 GT/s Link Width: x1 / x1
        0x0100: Enhanced Capability 0x00: Unknown
        0x0080: Capability 0x05: Message Signalled Interrupts (MSI)
        0x0090: Capability 0x0d: PCI-PCI
        0x00a0: Capability 0x01: Power Management
                State: D0 PME# enabled
        0x0000: 1c1c8086 00100147 060400b5 00810010
        0x0010: 00000000 00000000 00040400 000000f0
        0x0020: fbf0fbf0 0001fff1 00000000 00000000
        0x0030: 00000000 00000040 00000000 000303ff
        0x0040: 00428010 00008000 00100006 07114c12
        0x0050: 70120000 00040060 01400000 00000006
        0x0060: 00000000 00000016 00000000 00000000
        0x0070: 00000002 00000000 00000000 00000000
        0x0080: 00009005 00000000 00000000 00000000
        0x0090: 0000a00d 330d103c 00000000 00000000
        0x00a0: c8020001 00000100 00000000 00000000
        0x00b0: 00000000 00000000 00000000 00000000
        0x00c0: 00000000 00000000 00000000 00000000
        0x00d0: 01000000 00000b00 01110000 00000000
        0x00e0: 00000000 00000000 00000000 00000000
        0x00f0: 00000000 00000000 08070f87 00000000
 0:28:7: Intel 6 Series PCIE
        0x0000: Vendor ID: 8086 Product ID: 1c1e
        0x0004: Command: 0147 Status: 0010
        0x0008: Class: 06 Subclass: 04 Interface: 00 Revision: b5
        0x000c: BIST: 00 Header Type: 81 Latency Timer: 00 Cache Line
        Size: 10
        0x0010: 00000000
        0x0014: 00000000
        0x0018: Primary Bus: 0 Secondary Bus: 1 Subordinate Bus: 1
                Secondary Latency Timer: 00
        0x001c: I/O Base: 30 I/O Limit: 30 Secondary Status: 2000
        0x0020: Memory Base: fad0 Memory Limit: fbe0
        0x0024: Prefetch Memory Base: f901 Prefetch Memory Limit: f9f1
        0x0028: Prefetch Memory Base Upper 32 Bits: 00000000
        0x002c: Prefetch Memory Limit Upper 32 Bits: 00000000
        0x0030: I/O Base Upper 16 Bits: 0000 I/O Limit Upper 16 Bits:
        0000
        0x0038: Expansion ROM Base Address: 00000000
        0x003c: Interrupt Pin: 04 Line: ff Bridge Control: 000b
        0x0040: Capability 0x10: PCI Express
                Link Speed: 2.5 / 5.0 GT/s Link Width: x1 / x1
        0x0100: Enhanced Capability 0x00: Unknown
        0x0080: Capability 0x05: Message Signalled Interrupts (MSI)
        0x0090: Capability 0x0d: PCI-PCI
        0x00a0: Capability 0x01: Power Management
                State: D0 PME# enabled
        0x0000: 1c1e8086 00100147 060400b5 00810010
        0x0010: 00000000 00000000 00010100 20003030
        0x0020: fbe0fad0 f9f1f901 00000000 00000000
        0x0030: 00000000 00000040 00000000 000b04ff
        0x0040: 00428010 00008000 00100006 08114c12
        0x0050: 70110000 00040060 01400000 00000006
        0x0060: 00000000 00000016 00000000 00000000
        0x0070: 00010002 00000000 00000000 00000000
        0x0080: 00009005 00000000 00000000 00000000
        0x0090: 0000a00d 330d103c 00000000 00000000
        0x00a0: c8020001 00000100 00000000 00000000
        0x00b0: 00000000 00000000 00000000 00000000
        0x00c0: 00000000 00000000 00000000 00000000
        0x00d0: 01000000 00000b00 01110000 00000000
        0x00e0: 00000000 00000000 00000000 00000000
        0x00f0: 00000000 00000000 08070f87 00000000
 0:29:0: Intel 6 Series USB
        0x0000: Vendor ID: 8086 Product ID: 1c26
        0x0004: Command: 0146 Status: 0290
        0x0008: Class: 0c Subclass: 03 Interface: 20 Revision: 05
        0x000c: BIST: 00 Header Type: 00 Latency Timer: 00 Cache Line
        Size: 00
        0x0010: BAR mem 32bit addr: 0xface0000/0x00000400
        0x0014: BAR empty (00000000)
        0x0018: BAR empty (00000000)
        0x001c: BAR empty (00000000)
        0x0020: BAR empty (00000000)
        0x0024: BAR empty (00000000)
        0x0028: Cardbus CIS: 00000000
        0x002c: Subsystem Vendor ID: 103c Product ID: 330d
        0x0030: Expansion ROM Base Address: 00000000
        0x0038: 00000000
        0x003c: Interrupt Pin: 01 Line: 05 Min Gnt: 00 Max Lat: 00
        0x0050: Capability 0x01: Power Management
                State: D0 PME# enabled
        0x0058: Capability 0x0a: Debug Port
        0x0098: Capability 0x13: PCI Advanced Features
        0x0000: 1c268086 02900146 0c032005 00000000
        0x0010: face0000 00000000 00000000 00000000
        0x0020: 00000000 00000000 00000000 330d103c
        0x0030: 00000000 00000050 00000000 00000105
        0x0040: 00000000 00000000 00000000 00000000
        0x0050: c9c25801 00000100 20a0980a 00000000
        0x0060: 07ff2020 00000000 01000001 c0002000
        0x0070: 3fdf0000 00000000 00000000 00000000
        0x0080: 00800000 930c8811 24000d30 00000000
        0x0090: 00000000 00000000 03060013 00000100
        0x00a0: 00000000 00000000 00000000 00000000
        0x00b0: 00000000 00000000 00000000 00000000
        0x00c0: 00000000 00000000 00000000 00000000
        0x00d0: 00000000 00ffaa00 00000000 00000000
        0x00e0: 00000000 00000000 00000000 7d4ee004
        0x00f0: 00000000 00808588 08070f87 205b1708
 0:30:0: Intel 82801BA Hub-to-PCI
        0x0000: Vendor ID: 8086 Product ID: 244e
        0x0004: Command: 0147 Status: 0010
        0x0008: Class: 06 Subclass: 04 Interface: 01 Revision: a5
        0x000c: BIST: 00 Header Type: 01 Latency Timer: 00 Cache Line
        Size: 00
        0x0010: 00000000
        0x0014: 00000000
        0x0018: Primary Bus: 0 Secondary Bus: 20 Subordinate Bus: 20
                Secondary Latency Timer: 20
        0x001c: I/O Base: f0 I/O Limit: 00 Secondary Status: 2280
        0x0020: Memory Base: fff0 Memory Limit: 0000
        0x0024: Prefetch Memory Base: fff1 Prefetch Memory Limit: 0001
        0x0028: Prefetch Memory Base Upper 32 Bits: 00000000
        0x002c: Prefetch Memory Limit Upper 32 Bits: 00000000
        0x0030: I/O Base Upper 16 Bits: 0000 I/O Limit Upper 16 Bits:
        0000
        0x0038: Expansion ROM Base Address: 00000000
        0x003c: Interrupt Pin: 00 Line: ff Bridge Control: 0003
        0x0050: Capability 0x0d: PCI-PCI
        0x0000: 244e8086 00100147 060401a5 00010000
        0x0010: 00000000 00000000 20141400 228000f0
        0x0020: 0000fff0 0001fff1 00000000 00000000
        0x0030: 00000000 00000050 00000000 000300ff
        0x0040: 00000000 00000000 00000000 10001200
        0x0050: 0000000d 330d103c 00000000 00000000
        0x0060: 00000000 00000000 00000000 00000000
        0x0070: 00000000 00000000 00000000 00000000
        0x0080: 00000000 00000000 00000000 00000000
        0x0090: 00000000 00000000 00000000 00000000
        0x00a0: 00000000 00000000 00000000 00000000
        0x00b0: 00000000 00000000 00000000 00000000
        0x00c0: 00000000 00000000 00000000 00000000
        0x00d0: 00000000 00000000 00000000 00000000
        0x00e0: 00000000 00000000 00000000 00000000
        0x00f0: 00000000 00000000 08070f87 00000000
 0:31:0: Intel C204 LPC
        0x0000: Vendor ID: 8086 Product ID: 1c54
        0x0004: Command: 0147 Status: 0210
        0x0008: Class: 06 Subclass: 01 Interface: 00 Revision: 05
        0x000c: BIST: 00 Header Type: 80 Latency Timer: 00 Cache Line
        Size: 00
        0x0010: BAR empty (00000000)
        0x0014: BAR empty (00000000)
        0x0018: BAR empty (00000000)
        0x001c: BAR empty (00000000)
        0x0020: BAR empty (00000000)
        0x0024: BAR empty (00000000)
        0x0028: Cardbus CIS: 00000000
        0x002c: Subsystem Vendor ID: 0000 Product ID: 0000
        0x0030: Expansion ROM Base Address: 00000000
        0x0038: 00000000
        0x003c: Interrupt Pin: 00 Line: 00 Min Gnt: 00 Max Lat: 00
        0x00e0: Capability 0x09: Vendor Specific
        0x0000: 1c548086 02100147 06010005 00800000
        0x0010: 00000000 00000000 00000000 00000000
        0x0020: 00000000 00000000 00000000 00000000
        0x0030: 00000000 000000e0 00000000 00000000
        0x0040: 00000901 00000080 00000881 00000010
        0x0050: 000000f8 00000000 00000000 00000000
        0x0060: 0a0b070b 000000d0 80800305 000000f1
        0x0070: 00f800f8 00f800f8 00f800f8 00f800f8
        0x0080: 340a0000 00fc0801 000c0ca1 000c0cd1
        0x0090: 001c0311 00000f30 00000000 00000000
        0x00a0: 00300a08 00000200 00004700 40000300
        0x00b0: 00000000 00000000 00004000 00000000
        0x00c0: 00000000 00000000 00000000 00000000
        0x00d0: 00000000 00004567 0000ffcf 00000020
        0x00e0: 100c0009 00000000 0c640691 00000000
        0x00f0: fed1c001 00000000 08070f87 00000000
 0:31:2: Intel 6 Series AHCI
        0x0000: Vendor ID: 8086 Product ID: 1c02
        0x0004: Command: 0047 Status: 02b0
        0x0008: Class: 01 Subclass: 06 Interface: 01 Revision: 05
        0x000c: BIST: 00 Header Type: 00 Latency Timer: 00 Cache Line
        Size: 00
        0x0010: BAR io addr: 0x000010c0/0x0008
        0x0014: BAR io addr: 0x000010c8/0x0004
        0x0018: BAR io addr: 0x000010d0/0x0008
        0x001c: BAR io addr: 0x000010d8/0x0004
        0x0020: BAR io addr: 0x000010e0/0x0020
        0x0024: BAR mem 32bit addr: 0xfacd0000/0x00000800
        0x0028: Cardbus CIS: 00000000
        0x002c: Subsystem Vendor ID: 103c Product ID: 330d
        0x0030: Expansion ROM Base Address: 00000000
        0x0038: 00000000
        0x003c: Interrupt Pin: 02 Line: 07 Min Gnt: 00 Max Lat: 00
        0x0080: Capability 0x05: Message Signalled Interrupts (MSI)
        0x0070: Capability 0x01: Power Management
                State: D0
        0x00a8: Capability 0x12: SATA
        0x00b0: Capability 0x13: PCI Advanced Features
        0x0000: 1c028086 02b00047 01060105 00000000
        0x0010: 000010c1 000010c9 000010d1 000010d9
        0x0020: 000010e1 facd0000 00000000 330d103c
        0x0030: 00000000 00000080 00000000 00000207
        0x0040: 00000000 00000000 00000000 00000000
        0x0050: 00000000 00000000 00000000 00000000
        0x0060: 00000000 00000000 00000000 00000000
        0x0070: 4003a801 00000008 00000000 00000000
        0x0080: 00017005 fee00000 00000066 00000000
        0x0090: 8f3f0060 00000183 015c4208 00000080
        0x00a0: 000000d4 10000000 0010b012 00000048
        0x00b0: 03060013 00000000 00000000 00000000
        0x00c0: 00000000 00000000 00000000 00000000
        0x00d0: 00000000 00000000 00000000 00000000
        0x00e0: 00000000 00000000 00000000 00000000
        0x00f0: 00000000 00000000 08070f87 00000000
 1:0:0: Hewlett-Packard iLO3 Slave
        0x0000: Vendor ID: 103c Product ID: 3306
        0x0004: Command: 0007 Status: 0010
        0x0008: Class: 08 Subclass: 80 Interface: 00 Revision: 05
        0x000c: BIST: 00 Header Type: 80 Latency Timer: 00 Cache Line
        Size: 10
        0x0010: BAR io addr: 0x00003000/0x0100
        0x0014: BAR mem 32bit addr: 0xfbef0000/0x00000200
        0x0018: BAR io addr: 0x00003400/0x0100
        0x001c: BAR empty (00000000)
        0x0020: BAR empty (00000000)
        0x0024: BAR empty (00000000)
        0x0028: Cardbus CIS: 00000000
        0x002c: Subsystem Vendor ID: 103c Product ID: 3381
        0x0030: Expansion ROM Base Address: 00000000
        0x0038: 00000000
        0x003c: Interrupt Pin: 01 Line: 0a Min Gnt: 00 Max Lat: 00
        0x0078: Capability 0x01: Power Management
                State: D0
        0x00b0: Capability 0x05: Message Signalled Interrupts (MSI)
        0x00c0: Capability 0x10: PCI Express
                Link Speed: 2.5 / 2.5 GT/s Link Width: x1 / x1
        0x0100: Enhanced Capability 0x00: Unknown
        0x0000: 3306103c 00100007 08800005 00800010
        0x0010: 00003001 fbef0000 00003401 00000000
        0x0020: 00000000 00000000 00000000 3381103c
        0x0030: 00000000 00000078 00000000 0000010a
        0x0040: 00fedf27 00002011 00000000 000d010d
        0x0050: 33c80004 02000000 77fcfeef 5a000001
        0x0060: 00000008 00000000 00000000 00000000
        0x0070: 80000000 00460000 0003b001 00000008
        0x0080: 00000000 3a008100 03c20014 00515043
        0x0090: 00000001 5f3f0002 00030000 20000000
        0x00a0: 00000000 0002fe50 0000002e 00000000
        0x00b0: 0080c005 00000000 00000000 00000000
        0x00c0: 00110010 00008fc0 00080006 00016411
        0x00d0: 00110000 00000000 00000000 00000000
        0x00e0: 00000000 00000000 00000000 00000000
        0x00f0: 00000000 00000000 00000000 00000000
 1:0:1: Matrox MGA G200eH
        0x0000: Vendor ID: 102b Product ID: 0533
        0x0004: Command: 0047 Status: 0010
        0x0008: Class: 03 Subclass: 00 Interface: 00 Revision: 00
        0x000c: BIST: 00 Header Type: 80 Latency Timer: 00 Cache Line
        Size: 10
        0x0010: BAR mem prefetchable 32bit addr: 0xf9000000/0x01000000
        0x0014: BAR mem 32bit addr: 0xfbee0000/0x00004000
        0x0018: BAR mem 32bit addr: 0xfb000000/0x00800000
        0x001c: BAR empty (00000000)
        0x0020: BAR empty (00000000)
        0x0024: BAR empty (00000000)
        0x0028: Cardbus CIS: 00000000
        0x002c: Subsystem Vendor ID: 103c Product ID: 3381
        0x0030: Expansion ROM Base Address: 00000000
        0x0038: 00000000
        0x003c: Interrupt Pin: 02 Line: 0b Min Gnt: 00 Max Lat: 00
        0x00a8: Capability 0x01: Power Management
                State: D0
        0x00b0: Capability 0x05: Message Signalled Interrupts (MSI)
        0x00c0: Capability 0x10: PCI Express
                Link Speed: 2.5 / 2.5 GT/s Link Width: x1 / x1
        0x0100: Enhanced Capability 0x00: Unknown
        0x0000: 0533102b 00100047 03000000 00800010
        0x0010: f9000008 fbee0000 fb000000 00000000
        0x0020: 00000000 00000000 00000000 3381103c
        0x0030: 00000000 000000a8 00000000 0000020b
        0x0040: 00000120 00003c00 00ff0000 00000000
        0x0050: 00003000 00000000 00000000 00000000
        0x0060: 00000000 00000000 00000000 00000000
        0x0070: 00000000 00000000 00000000 00000000
        0x0080: 00000000 0008007b 00000000 00000300
        0x0090: 00000000 00000000 00000000 00000000
        0x00a0: f0000000 0f000003 0023b001 00000008
        0x00b0: 0080c005 00000000 00000000 00000000
        0x00c0: 00110010 00008fc0 00080006 00016411
        0x00d0: 00110000 00000000 00000000 00000000
        0x00e0: 00000000 00000000 00000000 00000000
        0x00f0: 00000000 00000000 00000000 00000000
 1:0:2: Hewlett-Packard iLO3 Management
        0x0000: Vendor ID: 103c Product ID: 3307
        0x0004: Command: 0007 Status: 0010
        0x0008: Class: 08 Subclass: 80 Interface: 00 Revision: 05
        0x000c: BIST: 00 Header Type: 80 Latency Timer: 00 Cache Line
        Size: 10
        0x0010: BAR io addr: 0x00003800/0x0100
        0x0014: BAR mem 32bit addr: 0xfaff0000/0x00000100
        0x0018: BAR mem 32bit addr: 0xfae00000/0x00100000
        0x001c: BAR mem 32bit addr: 0xfad80000/0x00080000
        0x0020: BAR mem 32bit addr: 0xfad70000/0x00008000
        0x0024: BAR mem 32bit addr: 0xfad60000/0x00008000
        0x0028: Cardbus CIS: 00000000
        0x002c: Subsystem Vendor ID: 103c Product ID: 3381
        0x0030: Expansion ROM Base Address: 00000000
        0x0038: 00000000
        0x003c: Interrupt Pin: 02 Line: 0b Min Gnt: 00 Max Lat: 00
        0x0078: Capability 0x01: Power Management
                State: D0 PME# enabled
        0x00b0: Capability 0x05: Message Signalled Interrupts (MSI)
        0x00c0: Capability 0x10: PCI Express
                Link Speed: 2.5 / 2.5 GT/s Link Width: x1 / x1
        0x0100: Enhanced Capability 0x00: Unknown
        0x0000: 3307103c 00100007 08800005 00800010
        0x0010: 00003801 faff0000 fae00000 fad80000
        0x0020: fad70000 fad60000 00000000 3381103c
        0x0030: 00000000 00000078 00000000 0000020b
        0x0040: 00000000 00000000 00000000 00000000
        0x0050: 00000000 00000000 00000000 00000000
        0x0060: 00000000 00000000 00000000 00000000
        0x0070: 00000000 00000000 c803b001 00000108
        0x0080: 00000000 00000000 00000000 00000000
        0x0090: 00000000 00000000 00000000 00000000
        0x00a0: 00050115 00000000 00000000 00000000
        0x00b0: 0080c005 00000000 00000000 00000000
        0x00c0: 00110010 00008fc0 00080006 00016411
        0x00d0: 00110000 00000000 00000000 00000000
        0x00e0: 00000000 00000000 00000000 00000000
        0x00f0: 00000000 00000000 00000000 00000000
 1:0:4: Hewlett-Packard USB
        0x0000: Vendor ID: 103c Product ID: 3300
        0x0004: Command: 0045 Status: 0010
        0x0008: Class: 0c Subclass: 03 Interface: 00 Revision: 02
        0x000c: BIST: 00 Header Type: 80 Latency Timer: 00 Cache Line
        Size: 10
        0x0010: BAR empty (00000000)
        0x0014: BAR empty (00000000)
        0x0018: BAR empty (00000000)
        0x001c: BAR empty (00000000)
        0x0020: BAR io addr: 0x00003c00/0x0020
        0x0024: BAR empty (00000000)
        0x0028: Cardbus CIS: 00000000
        0x002c: Subsystem Vendor ID: 103c Product ID: 3381
        0x0030: Expansion ROM Base Address: 00000000
        0x0038: 00000000
        0x003c: Interrupt Pin: 02 Line: 0b Min Gnt: 00 Max Lat: 00
        0x0070: Capability 0x05: Message Signalled Interrupts (MSI)
        0x0080: Capability 0x10: PCI Express
                Link Speed: 2.5 / 2.5 GT/s Link Width: x1 / x1
        0x0100: Enhanced Capability 0x00: Unknown
        0x00f0: Capability 0x01: Power Management
                State: D0
        0x0000: 3300103c 00100045 0c030002 00800010
        0x0010: 00000000 00000000 00000000 00000000
        0x0020: 00003c01 00000000 00000000 3381103c
        0x0030: 00000000 00000070 00000000 0000020b
        0x0040: 00000000 00000000 00000000 00000000
        0x0050: 00000000 00000000 00000000 00000000
        0x0060: 00000010 00000000 00000000 00000000
        0x0070: 00808005 00000000 00000000 00000000
        0x0080: 0011f010 00008fc0 00080006 00016411
        0x0090: 00110000 00000000 00000000 00000000
        0x00a0: 00000000 00000000 00000000 00000000
        0x00b0: 00000000 00000000 00000000 00000000
        0x00c0: 00002000 00000000 00000000 00000000
        0x00d0: 00000000 00000000 00000000 00000000
        0x00e0: 00000000 00000000 00000000 00000000
        0x00f0: 00030001 00000008 00000000 00000000
 3:0:0: Broadcom BCM5720
        0x0000: Vendor ID: 14e4 Product ID: 165f
        0x0004: Command: 0046 Status: 0010
        0x0008: Class: 02 Subclass: 00 Interface: 00 Revision: 00
        0x000c: BIST: 00 Header Type: 80 Latency Timer: 00 Cache Line
        Size: 10
        0x0010: BAR mem prefetchable 64bit addr:
        0x00000000fabf0000/0x00010000
        0x0018: BAR mem prefetchable 64bit addr:
        0x00000000fabe0000/0x00010000
        0x0020: BAR mem prefetchable 64bit addr:
        0x00000000fabd0000/0x00010000
        0x0028: Cardbus CIS: 00000000
        0x002c: Subsystem Vendor ID: 103c Product ID: 2133
        0x0030: Expansion ROM Base Address: 00000000
        0x0038: 00000000
        0x003c: Interrupt Pin: 01 Line: 0b Min Gnt: 00 Max Lat: 00
        0x0048: Capability 0x01: Power Management
                State: D0 PME# enabled
        0x0050: Capability 0x03: Vital Product Data (VPD)
        0x0058: Capability 0x05: Message Signalled Interrupts (MSI)
        0x00a0: Capability 0x11: Extended Message Signalled Interrupts
        (MSI-X)
        0x00ac: Capability 0x10: PCI Express
                Link Speed: 5.0 / 5.0 GT/s Link Width: x1 / x2
        0x0100: Enhanced Capability 0x01: Advanced Error Reporting
        0x013c: Enhanced Capability 0x03: Device Serial Number
        0x0150: Enhanced Capability 0x04: Power Budgeting
        0x0160: Enhanced Capability 0x02: Virtual Channel Capability
        0x0000: 165f14e4 00100046 02000000 00800010
        0x0010: fabf000c 00000000 fabe000c 00000000
        0x0020: fabd000c 00000000 00000000 2133103c
        0x0030: 00000000 00000048 00000000 0000010b
        0x0040: 00000000 85000000 c8035001 64002108
        0x0050: 00005803 00000000 0087a005 fee00000
        0x0060: 00000000 00000072 f0000289 76380080
        0x0070: 00000000 ffffebe1 00007020 00000000
        0x0080: 00000000 0000001e 00000000 000000bc
        0x0090: 00000000 00000000 00000000 000000ce
        0x00a0: 0010ac11 00000004 00001004 00020010
        0x00b0: 10008d81 00104406 00055c22 10120000
        0x00c0: 00000000 00000000 00000000 00000000
        0x00d0: 0000001f 00000000 00000000 00000001
        0x00e0: 00000000 00000000 00000000 00000000
        0x00f0: 00000000 05720000 00000000 a1000000
 3:0:1: Broadcom BCM5720
        0x0000: Vendor ID: 14e4 Product ID: 165f
        0x0004: Command: 0046 Status: 0010
        0x0008: Class: 02 Subclass: 00 Interface: 00 Revision: 00
        0x000c: BIST: 00 Header Type: 80 Latency Timer: 00 Cache Line
        Size: 10
        0x0010: BAR mem prefetchable 64bit addr:
        0x00000000fabc0000/0x00010000
        0x0018: BAR mem prefetchable 64bit addr:
        0x00000000fabb0000/0x00010000
        0x0020: BAR mem prefetchable 64bit addr:
        0x00000000faba0000/0x00010000
        0x0028: Cardbus CIS: 00000000
        0x002c: Subsystem Vendor ID: 103c Product ID: 2133
        0x0030: Expansion ROM Base Address: 00000000
        0x0038: 00000000
        0x003c: Interrupt Pin: 02 Line: 07 Min Gnt: 00 Max Lat: 00
        0x0048: Capability 0x01: Power Management
                State: D0 PME# enabled
        0x0050: Capability 0x03: Vital Product Data (VPD)
        0x0058: Capability 0x05: Message Signalled Interrupts (MSI)
        0x00a0: Capability 0x11: Extended Message Signalled Interrupts
        (MSI-X)
        0x00ac: Capability 0x10: PCI Express
                Link Speed: 5.0 / 5.0 GT/s Link Width: x1 / x2
        0x0100: Enhanced Capability 0x01: Advanced Error Reporting
        0x013c: Enhanced Capability 0x03: Device Serial Number
        0x0150: Enhanced Capability 0x04: Power Budgeting
        0x0160: Enhanced Capability 0x02: Virtual Channel Capability
        0x0000: 165f14e4 00100046 02000000 00800010
        0x0010: fabc000c 00000000 fabb000c 00000000
        0x0020: faba000c 00000000 00000000 2133103c
        0x0030: 00000000 00000048 00000000 00000207
        0x0040: 00000000 00000000 c8035001 64002108
        0x0050: 00005803 00000000 0087a005 fee00000
        0x0060: 00000000 00000073 f000028b 76380080
        0x0070: 00000000 facaffd1 0001400c 00000000
        0x0080: 00000300 00000000 00000000 00000000
        0x0090: 00000000 00000000 00000000 00000000
        0x00a0: 0010ac11 00000004 00001004 00020010
        0x00b0: 10008d81 00194406 00055c22 10120000
        0x00c0: 00000000 00000000 00000000 00000000
        0x00d0: 0000001f 00000000 00000000 00000000
        0x00e0: 00000000 00000000 00000000 00000000
        0x00f0: 00000000 05720000 00000000 00000000
 4:0:0: Renesas uPD720201 xHCI
        0x0000: Vendor ID: 1912 Product ID: 0014
        0x0004: Command: 0046 Status: 0010
        0x0008: Class: 0c Subclass: 03 Interface: 30 Revision: 03
        0x000c: BIST: 00 Header Type: 00 Latency Timer: 00 Cache Line
        Size: 10
        0x0010: BAR mem 64bit addr: 0x00000000fbff0000/0x00002000
        0x0018: BAR empty (00000000)
        0x001c: BAR empty (00000000)
        0x0020: BAR empty (00000000)
        0x0024: BAR empty (00000000)
        0x0028: Cardbus CIS: 00000000
        0x002c: Subsystem Vendor ID: 103c Product ID: 1996
        0x0030: Expansion ROM Base Address: 00000000
        0x0038: 00000000
        0x003c: Interrupt Pin: 01 Line: 0b Min Gnt: 00 Max Lat: 00
        0x0050: Capability 0x01: Power Management
                State: D0 PME# enabled
        0x0070: Capability 0x05: Message Signalled Interrupts (MSI)
        0x0090: Capability 0x11: Extended Message Signalled Interrupts
        (MSI-X)
        0x00a0: Capability 0x10: PCI Express
                Link Speed: 5.0 / 5.0 GT/s Link Width: x1 / x1
        0x0100: Enhanced Capability 0x01: Advanced Error Reporting
        0x0150: Enhanced Capability 0x18: Latency Tolerance Reporting
        0x0000: 00141912 00100046 0c033003 00000010
        0x0010: fbff0004 00000000 00000000 00000000
        0x0020: 00000000 00000000 00000000 1996103c
        0x0030: 00000000 00000050 00000000 0000010b
        0x0040: 00000000 00000000 00000000 00000000
        0x0050: c9c37001 00000108 00000000 00000000
        0x0060: 00002030 00000000 00000000 00202409
        0x0070: 00879005 fee00000 00000000 00000063
        0x0080: 00000000 00000000 00000000 00000000
        0x0090: 0007a011 00001000 00001080 00000000
        0x00a0: 00020010 00008fc0 00105806 0007ec12
        0x00b0: 10120000 00000000 00000000 00000000
        0x00c0: 00000000 00000810 00000000 00000000
        0x00d0: 00000002 00000000 00000000 00000000
        0x00e0: 00000000 00550000 05050000 00000000
        0x00f0: 00000000 00000010 0000ba9c 000f0256

acpidump:
begin-base64 644 APIC.9
QVBJQ1ICAAABS0hQICAgIFByb0xpYW50AgAAAAAAAAAAAAAAAADg/gEAAAAACAAAAQAAAAAIAgIB
AAAAAAgAAAAAAAAACAAAAAAAAAAIAAAAAAAAAAgAAAAAAAAACAAAAAAAAAAIAAAAAAAAAAgAAAAA
AAAACAAAAAAAAAAIAAAAAAAAAAgAAAAAAAAACAAAAAAAAAAIAAAAAAAAAAgAAAAAAAAACAAAAAAA
AAAIAAAAAAAAAAgAAAAAAAAACAAAAAAAAAAIAAAAAAAAAAgAAAAAAAAACAAAAAAAAAAIAAAAAAAA
AAgAAAAAAAAACAAAAAAAAAAIAAAAAAAAAAgAAAAAAAAACAAAAAAAAAAIAAAAAAAAAAgAAAAAAAAA
CAAAAAAAAAAIAAAAAAAAAAgAAAAAAAAACAAAAAAAAAAIAAAAAAAAAAgAAAAAAAAACAAAAAAAAAAI
AAAAAAAAAAgAAAAAAAAACAAAAAAAAAAIAAAAAAAAAAgAAAAAAAAACAAAAAAAAAAIAAAAAAAAAAgA
AAAAAAAACAAAAAAAAAAIAAAAAAAAAAgAAAAAAAAACAAAAAAAAAAIAAAAAAAAAAgAAAAAAAAACAAA
AAAAAAAIAAAAAAAAAAgAAAAAAAAACAAAAAAAAAAIAAAAAAAAAAgAAAAAAAAACAAAAAAAAAAIAAAA
AAAAAAgAAAAAAAAACAAAAAAAAAAIAAAAAAAAAAgAAAAAAAAACAAAAAAAAAEMCAAAAMD+AAAAAAIK
AAACAAAABQACCgAJCQAAAA0ABAb/AAAB
====
begin-base64 644 BERT.11
QkVSVDAAAAABu0hQICAgIFByb0xpYW50AQAAANIEAAAuFgAAQAAAAADw3n0AAAAA
====
begin-base64 644 DSDT.2
RFNEVBocAAABWkhQICAgIERTRFQAAAAAAQAAAElOVEwoAgMgCF9TMF8SBgQAAAAACF9TNF8SBwQK
BgAAAAhfUzVfEgcECgcAAAAQTglfUFJfW4MRQ1AwMAAAAAAAAAhfUFhNAFuDEUNQMDEBAAAAAAAI
X1BYTQBbgxFDUDAyAgAAAAAACF9QWE0AW4MRQ1AwMwMAAAAAAAhfUFhNAFuDEUNQMDQEAAAAAAAI
X1BYTQBbgxFDUDA1BQAAAAAACF9QWE0AW4MRQ1AwNgYAAAAAAAhfUFhNAFuDEUNQMDcHAAAAAAAI
X1BYTQAQTQZfU0JfCFVJREEREwoQ0DfJ5VM1ek2RF+pNGcNDTRRNBFBEU00FoD6TaFVJREGgNpNp
CgKgCZNqAKQRAwGBoSagJJNqCgcIUkVUMBICAnBriFJFVDAAAHBsiFJFVDABAKRSRVQwoQakEQMB
AFuCSBQuX1NCX1BDSTAIX0hJRAxB0AoICF9DSUQMQdAKAwhfQURSAAhfUFJXEgYCCgsKBBRLBV9P
U0MEimsAQ0RXMYprCgRDRFcyimsKCENEVzNwQ0RXM2CgFJB7SU9WUwEAk1NJT1YBe2AKFWChBXtg
AGCgE5KTYENEVzN9Q0RXMQoQQ0RXMXBgQ0RXM6RrW4IPSVBUMQhfQURSDAAAHABbgg9JUFQyCF9B
RFIMAQAcAFuCD0lQVDMIX0FEUgwCABwAW4IPSVBUNAhfQURSDAMAHABbgg9JUFQ1CF9BRFIMBAAc
AFuCD0lQVDYIX0FEUgwFABwAW4IPSVBUNwhfQURSDAYAHABbgg9JUFQ4CF9BRFIMBwAcAFuCD1BU
MDIIX0FEUgwAAAEAW4IPUFQwMwhfQURSDAEAAQBbgg9QVDA1CF9BRFIMAAAGABBCCV9TQl8IU1VQ
UAAISU9WUwAUTwdfT1NDBIprAENEVzGKawoEQ0RXMqApk2gREwoQCWjvjFCDoEapRvaHTt/8G4pr
AElPVjFwSU9WMUlPVlOka6E0oDKTaBETChAMXoXtkGy/R6YqJt4Pxa1ccENEVzJTVVBQoBGQe1NV
UFABAJNJTE9XAaRrfUNEVzEKBACka1uCNC8DX1NCX1BDSTBJQlJHCF9BRFIMAAAfAFuAUE1DRgIA
CwABW4EOUE1DRgAAQFJBRkczCFuCSiUvBF9TQl9QQ0kwSUJSR01PTUIIX0hJRAxB0AwCCF9VSUQA
CENSUzERRg0K0kcBcABwAAAIRwEIBAgEAAhHAdAE0AQAAkcBIAAgAAAgRwGgAKAAACBHAZAAkAAA
EEcBUABQAAAERwEQAxADAARHARYDFgMAAkcBAAcABwAgRwGACIAIAIBHAQAJAAkAgEcBEAAQAAAQ
RwHUDNQMAARHAdAM0AwABEcBUA9QDwAJRwHwAPAAAAFHAaAMoAwAAkcBpAykDAAChgkAAQAAAAAA
AAAAhgkAAQAAAP4AAMAAhgkAAQAAAAAAAAAAhgkAAQAAAAAAAAAARwH4AvgCAAh5AItDUlMxCspW
TUlOi0NSUzEKzFZNQViMQ1JTMQrPVkxFTopDUlMxCpxYQlNFikNSUzEKoFhMRU6KQ1JTMQq0VjFC
U4pDUlMxCrhWMUxOikNSUzEKwFYyQlOKQ1JTMQrEVjJMThRGD19DUlMAcFBFWFRYQlNFcFBMRU5Y
TEVOcFYxQlJhoBqRYQB7YQwA4P//YXBhVjFCU3ALACBWMUxOcFYyQlJhoCCRYQBwVjJCUmF7YQwA
4P//YXBhVjJCU3ALACBWMkxOcFZQUlRge2AKB2BwAFZNSU5wAFZNQVhwAFZMRU6gG5NgAXAL+ANW
TUlOcAv4A1ZNQVhwCghWTEVOoByTYAoCcAv4AlZNSU5wC/gCVk1BWHAKCFZMRU6gHJNgCgNwC+gD
Vk1JTnAL6ANWTUFYcAoIVkxFTqAck2AKBHAL6AJWTUlOcAvoAlZNQVhwCghWTEVOpENSUzFbgkYG
LwRfU0JfUENJMElCUkdNSTBfCF9ISUQMJgkAAQhfU1RSERUKEkkAUABNAEkAXwBLAEMAUwAAAAhf
VUlEAAhfQ1JTEQ0KCkcBogyiDAACeQAUCF9JRlQApAEUCl9TUlYApAsAAluCNC8EX1NCX1BDSTBJ
QlJHVElNRQhfSElEDEHQAQAIX0NSUxERCg4jAQABRwFAAEAAAQR5AFuCQQUvBF9TQl9QQ0kwSUJS
R0hQRVQIX0hJRAxB0AEDCF9VSUQACEJVRjAREQoOhgkAAQAA0P4ABAAAeQAUCV9TVEEApAoPFAtf
Q1JTCKRCVUYwW4JEBC8EX1NCX1BDSTBJQlJHRE1BMAhfSElEDEHQAgAIX0NSUxEgCh0qgAJHAQAA
AAABEEcBgACAAAEQRwHAAMAAASB5AFuCMC8EX1NCX1BDSTBJQlJHQkVFUAhfSElEDEHQCAAIX0NS
UxENCgpHAWEAYQABAXkAEE0ELwNfU0JfUENJMElCUkdbgElDSF8CCmAKDFuBMElDSF8BUElSQQhQ
SVJCCFBJUkMIUElSRAgAIFBJUkUIUElSRghQSVJHCFBJUkgIEEQILl9TQl9QQ0kwW4BJTlRQAQtE
MAoEW4ESSU5UUAFJTlQxEElOVDMIAAhbgElOVEwBC8gwCgJbgQtJTlRMAUlSUUwQW4BOVlJNAQsA
MAsAAVuBKk5WUk0AAEBFQVVUTwEATwRHU1RBCAAwTlYxQghOVjFDCABIBE5WUEcQWwFMQ0sxABAm
LwNfU0JfUENJMElQVDhbghVBU01ECF9BRFIAFAlfU1RBAKQKCxBGDS8DX1NCX1BDSTBJQlJHW4JD
CVNJT18IX0hJRAxB0AoGCF9DUlMRDQoKRwEuAC4AAAJ5AFuATlNJTwEKLgoCW4EQTlNJTwFJTkRY
CERBVEEIW4ZKBElORFhEQVRBAQAQQ0ZHXwgAIExETl8IAEAUQUNUUggASBdJT0FICElPQUwIAEAH
SU5UUggAGERNQ0gIAEg9Q0ZHMQhDRkcyCFsBTENLMgBbgi9SVEMwCF9ISUQMQdALAAhfRklYEgcB
DEHQCwAIX0NSUxENCgpHAXAAcAABAnkAED8vA19TQl9QQ0kwSUJSR1uCLktCRF8IX0hJRAxB0AMD
CF9DUlMRGQoWRwFgAGAAAQFHAWQAZAABASMCAAF5ABA4LwNfU0JfUENJMElCUkdbgidQUzJNCF9I
SUQMQdAPEwhfQ0lEDEHQDw4IX0NSUxEICgUiABB5ABBHJy8EX1NCX1BDSTBJQlJHU0lPX1uCQSZD
T01BCF9ISUQMQdAFAQhfQ0lEDEHQBQAIX0RETg1DT00xAAhfVUlEAAhDUlMwERAKDSIQAEcB+AP4
AwAIeQCLQ1JTMAFJUlFGjENSUzAKBUlPTFCMQ1JTMAoGSU9IUIxDUlMwCgdJT0xSjENSUzAKCElP
SFKMQ1JTMAoKSUxFThRCBF9TVEEAWyNMQ0sy//9wCgJMRE5foA5BQ1RSWydMQ0sypAoPoRygD5NJ
TlRSAFsnTENLMqQAoQpbJ0xDSzKkCg0URwhfQ1JTAFsjTENLMv//cAoCTEROX6Ark0lOVFIAcABJ
UlFGcABJT0xQcABJT0hQcABJT0xScABJT0hScABJTEVOoTlwAWB5YElOVFJJUlFGcElPQUxJT0xQ
cElPQUhJT0hQcElPQUxJT0xScElPQUhJT0hScAoISUxFTlsnTENLMqRDUlMwFCFfRElTAFsjTENL
Mv//cAoCTEROX3AAQUNUUlsnTENLMhRHCF9QUlMAWyNMQ0sy//9wCgJMRE5foCuTSU5UUgBwAElS
UUZwAElPTFBwAElPSFBwAElPTFJwAElPSFJwAElMRU6hOXABYHlgSU5UUklSUUZwSU9BTElPTFBw
SU9BSElPSFBwSU9BTElPTFJwSU9BSElPSFJwCghJTEVOWydMQ0sypENSUzAUSQZfU1JTAVsjTENL
Mv//cAoCTEROX4toAUlSUVeMaAoFSU9MT4xoCgZJT0hJcElPTE9JT0FMcElPSElJT0FIoA2TSVJR
VwBwAElOVFKhD4FJUlFXYHZgcGBJTlRScAFBQ1RSWydMQ0syEE4JX1RaXwhcVEVNUAv/CluFTQhU
SE0wCF9UU1AKPAhfVEMxCgQIX1RDMgoECF9QU0wSDAFcLl9QUl9DUDAwFBpfUFNWAHANX1BTViBN
ZXRob2QAWzGkCw4LFA1fU0NQAYZUSE0wCoEUG19UTVAAcA1fVE1QIE1ldGhvZABbMaRURU1QFBpf
Q1JUAHANX0NSVCBNZXRob2QAWzGkC+ULCFBJQ00AFAxfUElDAXBoUElDTRBNb19TQl9bgkwNTE5L
QQhfSElEDEHQDA8IX1VJRAEIQlVGQREJCgYjAAgYeQCLQlVGQQFJUkEwFCZfQ1JTAHteXi8DUENJ
MElCUkdQSVJBCg9geQFgSVJBMKRCVUZBFCVfU1RBAHteXi8DUENJMElCUkdQSVJBCoBgoAVgpAoJ
oQSkCgsUKV9ESVMAfV5eLwNQQ0kwSUJSR1BJUkEKgF5eLwNQQ0kwSUJSR1BJUkEIX1BSUxEJCgYj
oAwYeQAUJl9TUlMBi2gBSVJBX4JJUkFfYHRgAV5eLwNQQ0kwSUJSR1BJUkFbgk0NTE5LQghfSElE
DEHQDA8IX1VJRAoCCEJVRkIRCQoGIwAIGHkAi0JVRkIBSVJBMBQmX0NSUwB7Xl4vA1BDSTBJQlJH
UElSQgoPYHkBYElSQTCkQlVGQhQlX1NUQQB7Xl4vA1BDSTBJQlJHUElSQgqAYKAFYKQKCaEEpAoL
FClfRElTAH1eXi8DUENJMElCUkdQSVJCCoBeXi8DUENJMElCUkdQSVJCCF9QUlMRCQoGI6AMGHkA
FCZfU1JTAYtoAUlSQV+CSVJBX2B0YAFeXi8DUENJMElCUkdQSVJCW4JNDUxOS0MIX0hJRAxB0AwP
CF9VSUQKAwhCVUZDEQkKBiMACBh5AItCVUZDAUlSQTAUJl9DUlMAe15eLwNQQ0kwSUJSR1BJUkMK
D2B5AWBJUkEwpEJVRkMUJV9TVEEAe15eLwNQQ0kwSUJSR1BJUkMKgGCgBWCkCgmhBKQKCxQpX0RJ
UwB9Xl4vA1BDSTBJQlJHUElSQwqAXl4vA1BDSTBJQlJHUElSQwhfUFJTEQkKBiOgDBh5ABQmX1NS
UwGLaAFJUkFfgklSQV9gdGABXl4vA1BDSTBJQlJHUElSQ1uCTQ1MTktECF9ISUQMQdAMDwhfVUlE
CgQIQlVGRBEJCgYjAAgYeQCLQlVGRAFJUkEwFCZfQ1JTAHteXi8DUENJMElCUkdQSVJECg9geQFg
SVJBMKRCVUZEFCVfU1RBAHteXi8DUENJMElCUkdQSVJECoBgoAVgpAoJoQSkCgsUKV9ESVMAfV5e
LwNQQ0kwSUJSR1BJUkQKgF5eLwNQQ0kwSUJSR1BJUkQIX1BSUxEJCgYjoAwYeQAUJl9TUlMBi2gB
SVJBX4JJUkFfYHRgAV5eLwNQQ0kwSUJSR1BJUkRbgk0NTE5LRQhfSElEDEHQDA8IX1VJRAoFCEJV
RkURCQoGIwAIGHkAi0JVRkUBSVJBMBQmX0NSUwB7Xl4vA1BDSTBJQlJHUElSRQoPYHkBYElSQTCk
QlVGRRQlX1NUQQB7Xl4vA1BDSTBJQlJHUElSRQqAYKAFYKQKCaEEpAoLFClfRElTAH1eXi8DUENJ
MElCUkdQSVJFCoBeXi8DUENJMElCUkdQSVJFCF9QUlMRCQoGI6AMGHkAFCZfU1JTAYtoAUlSQV+C
SVJBX2B0YAFeXi8DUENJMElCUkdQSVJFW4JNDUxOS0YIX0hJRAxB0AwPCF9VSUQKBghCVUZGEQkK
BiMACBh5AItCVUZGAUlSQTAUJl9DUlMAe15eLwNQQ0kwSUJSR1BJUkYKD2B5AWBJUkEwpEJVRkYU
JV9TVEEAe15eLwNQQ0kwSUJSR1BJUkYKgGCgBWCkCgmhBKQKCxQpX0RJUwB9Xl4vA1BDSTBJQlJH
UElSRgqAXl4vA1BDSTBJQlJHUElSRghfUFJTEQkKBiOgDBh5ABQmX1NSUwGLaAFJUkFfgklSQV9g
dGABXl4vA1BDSTBJQlJHUElSRluCTQ1MTktHCF9ISUQMQdAMDwhfVUlECgcIQlVGRxEJCgYjAAgY
eQCLQlVGRwFJUkEwFCZfQ1JTAHteXi8DUENJMElCUkdQSVJHCg9geQFgSVJBMKRCVUZHFCVfU1RB
AHteXi8DUENJMElCUkdQSVJHCoBgoAVgpAoJoQSkCgsUKV9ESVMAfV5eLwNQQ0kwSUJSR1BJUkcK
gF5eLwNQQ0kwSUJSR1BJUkcIX1BSUxEJCgYjoAwYeQAUJl9TUlMBi2gBSVJBX4JJUkFfYHRgAV5e
LwNQQ0kwSUJSR1BJUkdbgk0NTE5LSAhfSElEDEHQDA8IX1VJRAoICEJVRkgRCQoGIwAIGHkAi0JV
RkgBSVJBMBQmX0NSUwB7Xl4vA1BDSTBJQlJHUElSSAoPYHkBYElSQTCkQlVGSBQlX1NUQQB7Xl4v
A1BDSTBJQlJHUElSSAqAYKAFYKQKCaEEpAoLFClfRElTAH1eXi8DUENJMElCUkdQSVJICoBeXi8D
UENJMElCUkdQSVJICF9QUlMRCQoGI6AMGHkAFCZfU1JTAYtoAUlSQV+CSVJBX2B0YAFeXi8DUENJ
MElCUkdQSVJIFE0XLwNfU0JfUENJMF9QUlQAoEsKk1BJQ00BpBJBCg0SCQQL//8AAAoQEgsEDP//
AQAAAAoQEgsEDP//AQABAAoREgwEDP//AQAKAgAKEhIMBAz//wEACgMAChMSCwQM//8GAAAAChAS
CwQM//8cAAAAChASCwQM//8cAAEAChESDAQM//8cAAoCAAoSEgwEDP//HAAKAwAKExILBAz//x0A
AAAKFBILBAz//xoAAAAKFRILBAz//x8AAQAKEaFPC6QSSwsNEgsEC///AExOS0EAEg0EDP//AQAA
TE5LQQASDQQM//8BAAFMTktCABIOBAz//wEACgJMTktDABIOBAz//wEACgNMTktEABINBAz//wYA
AExOS0EAEg0EDP//HAAATE5LQQASDQQM//8cAAFMTktCABIOBAz//xwACgJMTktDABIOBAz//xwA
CgNMTktEABINBAz//x0AAExOS0UAEg0EDP//GgAATE5LRgASDQQM//8fAAFMTktCABRDCC8EX1NC
X1BDSTBJUFQxX1BSVACgNZNQSUNNAaQSLAQSCQQL//8AAAoQEgkEC///AQAKERIKBAv//woCAAoS
EgoEC///CgMAChOhN6QSNAQSCwQL//8ATE5LQQASCwQL//8BTE5LQgASDAQL//8KAkxOS0MAEgwE
C///CgNMTktEABRDCC8EX1NCX1BDSTBJUFQyX1BSVACgNZNQSUNNAaQSLAQSCQQL//8AAAoREgkE
C///AQAKEhIKBAv//woCAAoTEgoEC///CgMAChChN6QSNAQSCwQL//8ATE5LQgASCwQL//8BTE5L
QwASDAQL//8KAkxOS0QAEgwEC///CgNMTktBABRDCC8EX1NCX1BDSTBJUFQzX1BSVACgNZNQSUNN
AaQSLAQSCQQL//8AAAoSEgkEC///AQAKExIKBAv//woCAAoQEgoEC///CgMAChGhN6QSNAQSCwQL
//8ATE5LQwASCwQL//8BTE5LRAASDAQL//8KAkxOS0EAEgwEC///CgNMTktCABRDCC8EX1NCX1BD
STBJUFQ0X1BSVACgNZNQSUNNAaQSLAQSCQQL//8AAAoTEgkEC///AQAKEBIKBAv//woCAAoREgoE
C///CgMAChKhN6QSNAQSCwQL//8ATE5LRAASCwQL//8BTE5LQQASDAQL//8KAkxOS0IAEgwEC///
CgNMTktDABRDCC8EX1NCX1BDSTBJUFQ1X1BSVACgNZNQSUNNAaQSLAQSCQQL//8AAAoQEgkEC///
AQAKERIKBAv//woCAAoSEgoEC///CgMAChOhN6QSNAQSCwQL//8ATE5LQQASCwQL//8BTE5LQgAS
DAQL//8KAkxOS0MAEgwEC///CgNMTktEABRDCC8EX1NCX1BDSTBJUFQ2X1BSVACgNZNQSUNNAaQS
LAQSCQQL//8AAAoREgkEC///AQAKEhIKBAv//woCAAoTEgoEC///CgMAChChN6QSNAQSCwQL//8A
TE5LQgASCwQL//8BTE5LQwASDAQL//8KAkxOS0QAEgwEC///CgNMTktBABRDCC8EX1NCX1BDSTBJ
UFQ3X1BSVACgNZNQSUNNAaQSLAQSCQQL//8AAAoSEgkEC///AQAKExIKBAv//woCAAoQEgoEC///
CgMAChGhN6QSNAQSCwQL//8ATE5LQwASCwQL//8BTE5LRAASDAQL//8KAkxOS0EAEgwEC///CgNM
TktCABRDCC8EX1NCX1BDSTBJUFQ4X1BSVACgNZNQSUNNAaQSLAQSCQQL//8AAAoTEgkEC///AQAK
EBIKBAv//woCAAoREgoEC///CgMAChKhN6QSNAQSCwQL//8ATE5LRAASCwQL//8BTE5LQQASDAQL
//8KAkxOS0IAEgwEC///CgNMTktDABRDCC8EX1NCX1BDSTBQVDAyX1BSVACgNZNQSUNNAaQSLAQS
CQQL//8AAAoQEgkEC///AQAKERIKBAv//woCAAoSEgoEC///CgMAChOhN6QSNAQSCwQL//8ATE5L
QQASCwQL//8BTE5LQgASDAQL//8KAkxOS0MAEgwEC///CgNMTktEABRDCC8EX1NCX1BDSTBQVDAz
X1BSVACgNZNQSUNNAaQSLAQSCQQL//8AAAoQEgkEC///AQAKERIKBAv//woCAAoSEgoEC///CgMA
ChOhN6QSNAQSCwQL//8ATE5LQQASCwQL//8BTE5LQgASDAQL//8KAkxOS0MAEgwEC///CgNMTktE
ABRDCC8EX1NCX1BDSTBQVDA1X1BSVACgNZNQSUNNAaQSLAQSCQQL//8AAAoQEgkEC///AQAKERIK
BAv//woCAAoSEgoEC///CgMAChOhN6QSNAQSCwQL//8ATE5LQQASCwQL//8BTE5LQgASDAQL//8K
AkxOS0MAEgwEC///CgNMTktEAFuASU1FTQAMUIAOAAFbgQtJTUVNAElMT1cIW4BBTUVNAAyggQ4A
AVuBC0FNRU0AVlBSVAhbgExNRU0ADKSBDgABW4ELTE1FTQBMUFQxCFuAWE1FTQAMAIIOAAoIW4EQ
WE1FTQBQRVhUIFBMRU4gW4BWVE1NAAwIgg4ACghbgRBWVE1NAFYxQlIgVjJCUiBbgFZNRU0ADBSC
DgABW4ELVk1FTQFTSU9WCFuAU0JQTQELAAkKBFuBEFNCUE0BU1BNUxBTUE1FEFuAU0dQRQELIAkK
EFuBGlNHUEUBR1BTMCBHUFMxIEdQRTAgR1BFMSBbgFNZU1MBCrIBW4ELU1lTUwFTTUlQCBRJBF9Q
VFMBcAv//1NQTVNw/0dQUzBw/0dQUzFwXC8EX1NCX1BDSTBJQlJHQUZHM2B9YAFgcGBcLwRfU0Jf
UENJMElCUkdBRkcz
====
begin-base64 644 ERST.8
RVJTVDACAAABF0hQICAgIFByb0xpYW50AQAAANIEAAAuFgAADAAAAAAAAAAQAAAAAAMAAAAIAAEk
gA4AAAAAAAAAAAAAAAAA/wAAAAAAAAABAwAAAAgAASSADgAAAAAAAQAAAAAAAAD/AAAAAAAAAAID
AAAACAABJIAOAAAAAAACAAAAAAAAAP8AAAAAAAAAAwMAAAAIAAEkgA4AAAAAAP8AAAAAAAAA/wAA
AAAAAAAEAgAAAEAABCiADgAAAAAAAAAAAAAAAAD//wAAAAAAAAUDAAAACAABJYAOAAAAAAABAAAA
AAAAAP8AAAAAAAAABQMAAAEIAAGyAAAAAAAAABAAAAAAAAAA/wAAAAAAAAAGAQAAAAgAASWADgAA
AAAAAQAAAAAAAAD/AAAAAAAAAAcAAAAACAABJoAOAAAAAAAAAAAAAAAAAP8AAAAAAAAACAAAAABA
AAQwgA4AAAAAAAAAAAAAAAAA//////////8JAgAAAEAABDCADgAAAAAAAAAAAAAAAAD/////////
/woAAAAACAABJ4AOAAAAAAAAAAAAAAAAAP8AAAAAAAAACwMAAAAIAAEkgA4AAAAAAAsAAAAAAAAA
/wAAAAAAAAANAAAAAEAABDiADgAAAAAAAAAAAAAAAAD//////////w4AAAAAQAAEQIAOAAAAAAAA
AAAAAAAAAP//////////DwAAAABAAARIgA4AAAAAAAAAAAAAAAAA//////////8=
====
begin-base64 644 FACP.1
RkFDUHQAAAABqUhQICAgIFByb0xpYW50AgAAANIEAAAuFgAAQEHefUBm3n0BAAkAsgAAAAEAAAAA
CQAAAAAAAAQJAAAAAAAAUAkAAAgJAAAgCQAAAAAAAAQCAQQQAAAAZQDpAwAAAAAAAAAAABcAAKUA
AAA=
====
begin-base64 644 FFFF.10
RkZGRnYBAAABFEhQICAgIFByb0xpYW50AQAAANIEAAAuFgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
====
begin-base64 644 FFFF.13
RkZGRhQZAAABNUhQICAgIFByb0xpYW50AQAAANIEAAAuFgAAJgMAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
====
begin-base64 644 FFFF.14
RkZGRi0AAAABbUhQICAgIFByb0xpYW50AQAAAAAAAAAAAAAAAQAAAAAAAAAK
====
begin-base64 644 FFFF.6
RkZGRmQAAAACzUhQICAgIFByb0xpYW50AgAAANIEAAAuFgAAAQAAAAAAAQAAAAAAAGAAAAAAAAAg
AQAAAAAAAAAAAAAAAAADAADU/gAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==
====
begin-base64 644 HEST.12
SEVTVLwAAAABHEhQICAgIFByb0xpYW50AQAAANIEAAAuFgAAAwAAAAYABgAAAAIAAQAAAAEAAAAA
AAAAAAAAAFYIAAAgABAAEfAXAMERAAAAAAAABgAAAAcABwAAAAIAAQAAAAEAAAAAAAAAAAAAAFYI
AAAgABAAEfAXAMERAAAAAAAACAAIAAAAAgABAAAAAQAAAAAAAAAAAAAABgAAACAAEAAR8BcAwREA
AAAAAAAgABAAEfAXAAAAAAA=
====
begin-base64 644 HPET.5
SFBFVDgAAAABwkhQICAgIFByb0xpYW50AgAAANIEAAAuFgAAAaKGgAAAAAAAAND+AAAAAAAAAAA=
====
begin-base64 644 MCFG.4
TUNGRzwAAAABMUhQICAgIFByb0xpYW50AQAAAAAAAAAAAAAAAAAAAAAAAAAAAAD0AAAAAAAAAD8A
AAAA
====
begin-base64 644 RSDT.0
UlNEVGwAAAABn0hQICAgIFByb0xpYW50AgAAANIEAAAuFgAAwEDefYBB3n0AQt59QELefYBC3n0A
Q959QEPefYBF3n0ASN59gEnefcBJ3n2ASt59wGPefYCC3n3Ag959QIXefUCH3n3Ait59
====
begin-base64 644 SPCR.3
U1BDUlAAAAAByEhQICAgIFNQQ1JSQlNVAQAAANIEAAAuFgAAAAAAAAAAAAAAAAAAAAAAAAEEAAAA
AAMAAQMAAP////8AAAAAAAAAAAAAAAA=
====
begin-base64 644 SPMI.7
U1BNSUAAAAAFYkhQICAgIFByb0xpYW50AQAAANIEAAAuFgAAAQEAAgAAAAAAAAAAARAAAaIMAAAA
AAAAAAAAAA==
====
begin-base64 644 SSDT.15
U1NEVDcBAAADU0hQAAAAAENSU1BDSTAAAgAAAEhQAAABAAAACFwvA19TQl9QQ0kwX0NSUxFODQra
iA0AAgwAAAAAABQAAAAVAIcXAAAMAQAAAAAAAAD4////+wAAAAAAAAAEiA0AAQwDAAAAEP//AAAA
8IgNAAEMAwAAAACvAwAAsAOIDQABDAMAAOAD9wwAABgJiA0AAQwDAAAADf8PAAAAA4cXAAAMAQAA
AAAAAND+/z/Q/gAAAAAAQAAAhxcAAAwBAAAAAAAA0P7/T9T+AAAAAABQBACIDQABDAMAALADuwMA
AAwAiA0AAQwDAADAA98DAAAgAIcXAAAMAQAAAAAAAAoA//8LAAAAAAAAAAIAeQAIXC8DX1NCX1BD
STBfQkJOCgAIXC8DX1NCX1BDSTBfUFhNCgA=
====
begin-base64 644 SSDT.16
U1NEVHcBAAADckhQAAAAAHJpc2VyMAAAAgAAAElOVEwoAgMgEEIVLwNfU0JfUENJMFBUMDIIU0xU
TgEIREVWQwBbgiRTTFQwFBJfQURSAHBERVZDYH1gAGGkYRQLX1NVTgCkU0xUTluCJFNMVDEUEl9B
RFIAcERFVkNgfWABYaRhFAtfU1VOAKRTTFROW4IlU0xUMhQTX0FEUgBwREVWQ2B9YAoCYaRhFAtf
U1VOAKRTTFROW4IlU0xUMxQTX0FEUgBwREVWQ2B9YAoDYaRhFAtfU1VOAKRTTFROW4IlU0xUNBQT
X0FEUgBwREVWQ2B9YAoEYaRhFAtfU1VOAKRTTFROW4IlU0xUNRQTX0FEUgBwREVWQ2B9YAoFYaRh
FAtfU1VOAKRTTFROW4IlU0xUNhQTX0FEUgBwREVWQ2B9YAoGYaRhFAtfU1VOAKRTTFROW4IlU0xU
NxQTX0FEUgBwREVWQ2B9YAoHYaRhFAtfU1VOAKRTTFRO
====
begin-base64 644 SSDT.17
U1NEVOEBAAABgUhQAAAAAHBjYwAAAAAAAQAAAElOVEwlBgkgEEwFXF9TQl8IUENDSBJPBAQRMwow
iisAAAwDAAAAAAAAAAAAYP59AAAAAP9f/30AAAAAAAAAAAAAAAAAAAEAAAAAAHkAERQKEYIMAAEI
AACyAAAAAAAAAHkAAAogEClcLl9QUl9DUDAwCFBDQ1ASBgIKRApICF9QU0QSDAESCQUKBQAACv4B
EClcLl9QUl9DUDAxCFBDQ1ASBgIKhAqICF9QU0QSDAESCQUKBQABCv4BECpcLl9QUl9DUDAyCFBD
Q1ASBgIKxArICF9QU0QSDQESCgUKBQAKAgr+ARAsXC5fUFJfQ1AwMwhQQ0NQEggCCwQBCwgBCF9Q
U0QSDQESCgUKBQAKAwr+ARAsXC5fUFJfQ1AwNAhQQ0NQEggCC0QBC0gBCF9QU0QSDQESCgUKBQAK
BAr+ARAsXC5fUFJfQ1AwNQhQQ0NQEggCC4QBC4gBCF9QU0QSDQESCgUKBQAKBQr+ARAsXC5fUFJf
Q1AwNghQQ0NQEggCC8QBC8gBCF9QU0QSDQESCgUKBQAKBgr+ARAsXC5fUFJfQ1AwNwhQQ0NQEggC
CwQCCwgCCF9QU0QSDQESCgUKBQAKBwr+AQ==
====
begin-base64 644 SSDT.18
U1NEVHcDAAAB0khQAAAAAHBtYWIAAAAAAQAAAElOVEwlBgkgEEI1XF9TQl9bgkk0UE1JMAhfSElE
DUFDUEkwMDBEAFuAUE9XUgcLAiwLAAFbgRNQT1dSFQEFAUdQT1cIR0NBUAgIUFZBTAAIUEFWRwzg
kwQACEhMSU0AFEoNX1BNQwAIUkVUMBICDnAKBYhSRVQwAABwAIhSRVQwAQBwAIhSRVQwCgIAcAyQ
XwEAiFJFVDAKAwBwC/QBiFJFVDAKBABwDOCTBACIUkVUMAoFAHAM4JMEAIhSRVQwCgYAcP+IUkVU
MAoHAHAAiFJFVDAKCABw/4hSRVQwCgkAcP+IUkVUMAoKAAhNRExfDV9Nb2RlbAAIU0VSXw1fU2Vy
aWFsAAhPRU1fDUhQAHBNRExfiFJFVDAKCwBwU0VSX4hSRVQwCgwAcE9FTV+IUkVUMAoNAKRSRVQw
FBhfUE1EAAhSRVQwEgcBXF9TQl+kUkVUMBRODF9QTU0ACEJVRkYRAwogjEJVRkYAU1RBVIxCVUZG
AUxFTkdwCgRMRU5HjEJVRkYKAkVYVElwCtxFWFRJjEJVRkYKA01PREVwAU1PREWLQlVGRgoEVU5V
U4xCVUZGCgJDTVBDjEJVRkYKA0VYVE+LQlVGRgoEQ1VSUItCVUZGCgZNSU5Qi0JVRkYKCE1BWFCL
QlVGRgoKQVBPV3BwQlVGRl5HUE9XQlVGRqAhkJNTVEFUAJNDTVBDAHdBUE9XC+gDXlBWQUykXlBW
QUyk/xQMX0dBSQCkXlBBVkcUJl9QQUkBoBuQkpVoDOCTBACSlGgM4JMEAJ1oXlBBVkekAKEDpAEU
QgtfR0hMAAhCVUZGEQMKIIxCVUZGAFNUQVSMQlVGRgFMRU5HcAoDTEVOR4xCVUZGCgJFWFRJcArc
RVhUSYtCVUZGCgRVTlUxjEJVRkYKAkNNUEOMQlVGRgoDRVhUT4tCVUZGCgRVTlUyjEJVRkYKBVVO
VTOLQlVGRgoHSENBUHBwQlVGRl5HQ0FQQlVGRqAhkJNTVEFUAJNDTVBDAHdIQ0FQC+gDXkhMSU2k
XkhMSU2k/xRBBV9TVEEAcFxfT1NfYJ5gAAoJYaA3k2ENTWljcm9zb2Z0AKAkWxJfT1NJYqAXXF9P
U0kNV2luZG93cyAyMDA5AKQKD6EDpAChA6QAoQSkCg8=
====
begin-base64 644 SSDT.19
U1NEVAQFAAABhUlOVEVMIFBQTSBSQ00gAQAAgElOVEwJEQYgEE8mXC5fUFJfQ1AwMAhUWVBFDAAA
AAAIUFNFTgwAAAAACENTRU4MAQAAAAhNV09TDAEAAAAIVFNFTgwAAAAAFEkHX1BEQwGKaAoIQ0FQ
QXBDQVBBVFlQRYpoCgBSRVZTimgKBFNJWkVwh2hgcHRgCggAYVsTaApAd2EKCABURU1QCFNUUzAR
BwoEAAAAAHNTVFMwVEVNUGJfT1NDERMKEBamd0AMKb5Hnr3YcFhxOVNSRVZTU0laRWIUTg9fT1ND
BIprCgRDQVBBcENBUEFUWVBFimsKAFNUUzCKawoEQ0FQMIpoCgBJSUQwimgKBElJRDGKaAoISUlE
MopoCgxJSUQzCFVJRDAREwoQFqZ3QAwpvkeevdhwWHE5U4poCgBFSUQwimgKBEVJRDGKaAoIRUlE
MopoCgxFSUQzoEcDkpCQk0lJRDBFSUQwk0lJRDFFSUQxkJNJSUQyRUlEMpNJSUQzRUlEM3AKBohT
VFMwCgAApGugRAGSk2kKAXAKCohTVFMwCgAApGugSAF7U1RTMAoBAHtDQVAwC/8LQ0FQMKRre0NB
UDAL/wtDQVAwcENBUDBUWVBFpGsURwtfQ1NUAKBNCkNTRU6gSwWQTVdPU3tUWVBFCwACAKQSSQQD
CgISIQQRFAoRggwAfwECAQAAAAAAAAAAeQAKAQsBAAzoAwAAEiEEERQKEYIMAH8BAgEgAAAAAAAA
AHkACgILYAAMXgEAAKQSSQQDCgISIQQRFAoRggwAfwAAAAAAAAAAAAAAeQAKAQsBAAzoAwAAEiEE
ERQKEYIMAAEIAAAVCQAAAAAAAHkACgILYAAMXgEAAKQAEE8mXC5fUFJfQ1AwMghUWVBFDAAAAAAI
UFNFTgwAAAAACENTRU4MAQAAAAhNV09TDAEAAAAIVFNFTgwAAAAAFEkHX1BEQwGKaAoIQ0FQQXBD
QVBBVFlQRYpoCgBSRVZTimgKBFNJWkVwh2hgcHRgCggAYVsTaApAd2EKCABURU1QCFNUUzARBwoE
AAAAAHNTVFMwVEVNUGJfT1NDERMKEBamd0AMKb5Hnr3YcFhxOVNSRVZTU0laRWIUTg9fT1NDBIpr
CgRDQVBBcENBUEFUWVBFimsKAFNUUzCKawoEQ0FQMIpoCgBJSUQwimgKBElJRDGKaAoISUlEMopo
CgxJSUQzCFVJRDAREwoQFqZ3QAwpvkeevdhwWHE5U4poCgBFSUQwimgKBEVJRDGKaAoIRUlEMopo
CgxFSUQzoEcDkpCQk0lJRDBFSUQwk0lJRDFFSUQxkJNJSUQyRUlEMpNJSUQzRUlEM3AKBohTVFMw
CgAApGugRAGSk2kKAXAKCohTVFMwCgAApGugSAF7U1RTMAoBAHtDQVAwC/8LQ0FQMKRre0NBUDAL
/wtDQVAwcENBUDBUWVBFpGsURwtfQ1NUAKBNCkNTRU6gSwWQTVdPU3tUWVBFCwACAKQSSQQDCgIS
IQQRFAoRggwAfwECAQAAAAAAAAAAeQAKAQsBAAzoAwAAEiEEERQKEYIMAH8BAgEgAAAAAAAAAHkA
CgILYAAMXgEAAKQSSQQDCgISIQQRFAoRggwAfwAAAAAAAAAAAAAAeQAKAQsBAAzoAwAAEiEEERQK
EYIMAAEIAAAVCQAAAAAAAHkACgILYAAMXgEAAKQA
====
begin-base64 644 headers
ClJTRCBQVFI6IENoZWNrc3VtPTQ0LCBPRU1JRD1IUCwgUnNkdEFkZHJlc3M9MHg3ZGRlNDAwMAoK
ClJTRFQ6IExlbmd0aD0xMDgsIFJldmlzaW9uPTEsIENoZWNrc3VtPTE1OSwKCU9FTUlEPUhQLCBP
RU0gVGFibGUgSUQ9UHJvTGlhbnQsIE9FTSBSZXZpc2lvbj0weDIsCglDcmVhdG9yIElEPdIELCBD
cmVhdG9yIFJldmlzaW9uPTB4MTYyZQoKCglFbnRyaWVzPXsgMHg3ZGRlNDBjMCwgMHg3ZGRlNDE4
MCwgMHg3ZGRlNDIwMCwgMHg3ZGRlNDI0MCwgMHg3ZGRlNDI4MCwgMHg3ZGRlNDMwMCwgMHg3ZGRl
NDM0MCwgMHg3ZGRlNDU4MCwgMHg3ZGRlNDgwMCwgMHg3ZGRlNDk4MCwgMHg3ZGRlNDljMCwgMHg3
ZGRlNGE4MCwgMHg3ZGRlNjNjMCwgMHg3ZGRlODI4MCwgMHg3ZGRlODNjMCwgMHg3ZGRlODU0MCwg
MHg3ZGRlODc0MCwgMHg3ZGRlOGFjMCB9CgoKCURTRFQ9MHg3ZGRlNjY0MAoJSU5UX01PREVMPUFQ
SUMKCVNDSV9JTlQ9OQoJU01JX0NNRD0weGIyLCBBQ1BJX0VOQUJMRT0weDEsIEFDUElfRElTQUJM
RT0weDAsIFM0QklPU19SRVE9MHgwCglQTTFhX0VWVF9CTEs9MHg5MDAtMHg5MDMKCVBNMWFfQ05U
X0JMSz0weDkwNC0weDkwNQoJUE0yX0NOVF9CTEs9MHg5NTAtMHg5NTAKCVBNMl9UTVJfQkxLPTB4
OTA4LTB4OTBiCglQTTJfR1BFMF9CTEs9MHg5MjAtMHg5MmYKCVBfTFZMMl9MQVQ9MTAxbXMsIFBf
TFZMM19MQVQ9MTAwMW1zCglGTFVTSF9TSVpFPTAsIEZMVVNIX1NUUklERT0wCglEVVRZX09GRlNF
VD0wLCBEVVRZX1dJRFRIPTAKCURBWV9BTFJNPTAsIE1PTl9BTFJNPTAsIENFTlRVUlk9MAoJRmxh
Z3M9e1dCSU5WRCxQUk9DX0MxLFNMUF9CVVRUT04sUlRDX1M0fQoKCkRTRFQ6IExlbmd0aD03MTk0
LCBSZXZpc2lvbj0xLCBDaGVja3N1bT05MCwKCU9FTUlEPUhQLCBPRU0gVGFibGUgSUQ9RFNEVCwg
T0VNIFJldmlzaW9uPTB4MSwKCUNyZWF0b3IgSUQ9SU5UTCwgQ3JlYXRvciBSZXZpc2lvbj0weDIw
MDMwMjI4CgoKU1BDUjogTGVuZ3RoPTgwLCBSZXZpc2lvbj0xLCBDaGVja3N1bT0yMDAsCglPRU1J
RD1IUCwgT0VNIFRhYmxlIElEPVNQQ1JSQlNVLCBPRU0gUmV2aXNpb249MHgxLAoJQ3JlYXRvciBJ
RD3SBCwgQ3JlYXRvciBSZXZpc2lvbj0weDE2MmUKCgpNQ0ZHOiBMZW5ndGg9NjAsIFJldmlzaW9u
PTEsIENoZWNrc3VtPTQ5LAoJT0VNSUQ9SFAsIE9FTSBUYWJsZSBJRD1Qcm9MaWFudCwgT0VNIFJl
dmlzaW9uPTB4MSwKCUNyZWF0b3IgSUQ9LCBDcmVhdG9yIFJldmlzaW9uPTB4MAoKCkhQRVQ6IExl
bmd0aD01NiwgUmV2aXNpb249MSwgQ2hlY2tzdW09MTk0LAoJT0VNSUQ9SFAsIE9FTSBUYWJsZSBJ
RD1Qcm9MaWFudCwgT0VNIFJldmlzaW9uPTB4MiwKCUNyZWF0b3IgSUQ90gQsIENyZWF0b3IgUmV2
aXNpb249MHgxNjJlCgoKRkZGRjogTGVuZ3RoPTEwMCwgUmV2aXNpb249MiwgQ2hlY2tzdW09MjA1
LAoJT0VNSUQ9SFAsIE9FTSBUYWJsZSBJRD1Qcm9MaWFudCwgT0VNIFJldmlzaW9uPTB4MiwKCUNy
ZWF0b3IgSUQ90gQsIENyZWF0b3IgUmV2aXNpb249MHgxNjJlCgoKU1BNSTogTGVuZ3RoPTY0LCBS
ZXZpc2lvbj01LCBDaGVja3N1bT05OCwKCU9FTUlEPUhQLCBPRU0gVGFibGUgSUQ9UHJvTGlhbnQs
IE9FTSBSZXZpc2lvbj0weDEsCglDcmVhdG9yIElEPdIELCBDcmVhdG9yIFJldmlzaW9uPTB4MTYy
ZQoKCkVSU1Q6IExlbmd0aD01NjAsIFJldmlzaW9uPTEsIENoZWNrc3VtPTIzLAoJT0VNSUQ9SFAs
IE9FTSBUYWJsZSBJRD1Qcm9MaWFudCwgT0VNIFJldmlzaW9uPTB4MSwKCUNyZWF0b3IgSUQ90gQs
IENyZWF0b3IgUmV2aXNpb249MHgxNjJlCgoKQVBJQzogTGVuZ3RoPTU5NCwgUmV2aXNpb249MSwg
Q2hlY2tzdW09NzUsCglPRU1JRD1IUCwgT0VNIFRhYmxlIElEPVByb0xpYW50LCBPRU0gUmV2aXNp
b249MHgyLAoJQ3JlYXRvciBJRD0sIENyZWF0b3IgUmV2aXNpb249MHgwCgoKRkZGRjogTGVuZ3Ro
PTM3NCwgUmV2aXNpb249MSwgQ2hlY2tzdW09MjAsCglPRU1JRD1IUCwgT0VNIFRhYmxlIElEPVBy
b0xpYW50LCBPRU0gUmV2aXNpb249MHgxLAoJQ3JlYXRvciBJRD3SBCwgQ3JlYXRvciBSZXZpc2lv
bj0weDE2MmUKCgpCRVJUOiBMZW5ndGg9NDgsIFJldmlzaW9uPTEsIENoZWNrc3VtPTE4NywKCU9F
TUlEPUhQLCBPRU0gVGFibGUgSUQ9UHJvTGlhbnQsIE9FTSBSZXZpc2lvbj0weDEsCglDcmVhdG9y
IElEPdIELCBDcmVhdG9yIFJldmlzaW9uPTB4MTYyZQoKCkhFU1Q6IExlbmd0aD0xODgsIFJldmlz
aW9uPTEsIENoZWNrc3VtPTI4LAoJT0VNSUQ9SFAsIE9FTSBUYWJsZSBJRD1Qcm9MaWFudCwgT0VN
IFJldmlzaW9uPTB4MSwKCUNyZWF0b3IgSUQ90gQsIENyZWF0b3IgUmV2aXNpb249MHgxNjJlCgoK
RkZGRjogTGVuZ3RoPTY0MjAsIFJldmlzaW9uPTEsIENoZWNrc3VtPTUzLAoJT0VNSUQ9SFAsIE9F
TSBUYWJsZSBJRD1Qcm9MaWFudCwgT0VNIFJldmlzaW9uPTB4MSwKCUNyZWF0b3IgSUQ90gQsIENy
ZWF0b3IgUmV2aXNpb249MHgxNjJlCgoKRkZGRjogTGVuZ3RoPTQ1LCBSZXZpc2lvbj0xLCBDaGVj
a3N1bT0xMDksCglPRU1JRD1IUCwgT0VNIFRhYmxlIElEPVByb0xpYW50LCBPRU0gUmV2aXNpb249
MHgxLAoJQ3JlYXRvciBJRD0sIENyZWF0b3IgUmV2aXNpb249MHgwCgoKU1NEVDogTGVuZ3RoPTMx
MSwgUmV2aXNpb249MywgQ2hlY2tzdW09ODMsCglPRU1JRD1IUCwgT0VNIFRhYmxlIElEPUNSU1BD
STAsIE9FTSBSZXZpc2lvbj0weDIsCglDcmVhdG9yIElEPUhQLCBDcmVhdG9yIFJldmlzaW9uPTB4
MQoKClNTRFQ6IExlbmd0aD0zNzUsIFJldmlzaW9uPTMsIENoZWNrc3VtPTExNCwKCU9FTUlEPUhQ
LCBPRU0gVGFibGUgSUQ9cmlzZXIwLCBPRU0gUmV2aXNpb249MHgyLAoJQ3JlYXRvciBJRD1JTlRM
LCBDcmVhdG9yIFJldmlzaW9uPTB4MjAwMzAyMjgKCgpTU0RUOiBMZW5ndGg9NDgxLCBSZXZpc2lv
bj0xLCBDaGVja3N1bT0xMjksCglPRU1JRD1IUCwgT0VNIFRhYmxlIElEPXBjYywgT0VNIFJldmlz
aW9uPTB4MSwKCUNyZWF0b3IgSUQ9SU5UTCwgQ3JlYXRvciBSZXZpc2lvbj0weDIwMDkwNjI1CgoK
U1NEVDogTGVuZ3RoPTg4NywgUmV2aXNpb249MSwgQ2hlY2tzdW09MjEwLAoJT0VNSUQ9SFAsIE9F
TSBUYWJsZSBJRD1wbWFiLCBPRU0gUmV2aXNpb249MHgxLAoJQ3JlYXRvciBJRD1JTlRMLCBDcmVh
dG9yIFJldmlzaW9uPTB4MjAwOTA2MjUKCgpTU0RUOiBMZW5ndGg9MTI4NCwgUmV2aXNpb249MSwg
Q2hlY2tzdW09MTMzLAoJT0VNSUQ9SU5URUwsIE9FTSBUYWJsZSBJRD1QUE0gUkNNLCBPRU0gUmV2
aXNpb249MHg4MDAwMDAwMSwKCUNyZWF0b3IgSUQ9SU5UTCwgQ3JlYXRvciBSZXZpc2lvbj0weDIw
MDYxMTA5Cgo=
====

Reply | Threaded
Open this post in threaded view
|

Re: Userspace reproducibly hangs for a few minutes

Ted Unangst-6
Filippo Valsorda wrote:
> Sometimes the hang lasts seconds, sometimes minutes. During the hang
> everything is unresponsive except the kernel: network traffic drops,
> serial console is stuck, ssh is stuck and eventually times out,
> sometimes other applications running on the machine notice that "time
> jumped forwards", ICMP ping works, trying to ssh in manages to open a
> connection but no banner is ever received. Essentially, userspace
> doesn't get any CPU time. Most of the times Camlistore fails to start
> with a TLS timeout, sometimes it manages to carry on.

ktrace -t cnst may be interesting. kdump -TR should make it easy to spot the
syscalls around the pause.

Reply | Threaded
Open this post in threaded view
|

Re: Userspace reproducibly hangs for a few minutes

Philip Guenther-2
In reply to this post by Filippo Valsorda
On Mon, 20 Mar 2017, Filippo Valsorda wrote:

> Every time I start a certain daemon (Camlistore, written in golang),
> during a precise step involving TCP/HTTPS traffic, AES decryption, and
> loading data into the heap (loading the encryption metadata of a remote
> storage), I experience a long userspace hang.
>
> Sometimes the hang lasts seconds, sometimes minutes. During the hang
> everything is unresponsive except the kernel: network traffic drops,
> serial console is stuck, ssh is stuck and eventually times out,
> sometimes other applications running on the machine notice that "time
> jumped forwards", ICMP ping works, trying to ssh in manages to open a
> connection but no banner is ever received. Essentially, userspace
> doesn't get any CPU time. Most of the times Camlistore fails to start
> with a TLS timeout, sometimes it manages to carry on.
...
> ddb breaking from the serial console works, producing variations of the
> following two traces.
...
> --- interrupt ---
> Xspllower() at Xspllower+0xc
> mtx_leave() at mtx_leave+0x34
> uvmpd_scan_inactive() at uvmpd_scan_inactive+0x3e5
> uvmpd_scan() at uvmpd_scan+0x2a0
> uvm_pageout() at uvm_pageout+0x72
> end trace frame: 0x0, count: -9
...
> dmesg:
> OpenBSD 6.1-beta (GENERIC.MP) #37: Fri Mar 17 07:46:47 MDT 2017
>     [hidden email]:/usr/src/sys/arch/amd64/compile/GENERIC.MP
> real mem = 2094940160 (1997MB)
> avail mem = 2026835968 (1932MB)

In the interest of helping you get work done, it sounds like you're
running a task that has a (much?) larger resident set requirement than
this box has physical memory available to userspace, resulting in constant
paging that drags it into the bowels of the UVM subsystem.  We could
handle that better...

"You're going to need a bigger boat."


Philip Guenther

Reply | Threaded
Open this post in threaded view
|

Re: Userspace reproducibly hangs for a few minutes

Filippo Valsorda-2
Thanks everyone for the pointers, I'm back with more logs and traces.
This is the context, as captured in a tmux after running rcctl start and
ktrace in another window. SSH dropped halfway through.

> $ while true; do sleep 1; date; done & tail -f /var/log/daemon
> Sat Mar 25 23:42:16 UTC 2017
> Sat Mar 25 23:42:17 UTC 2017
> Mar 25 23:42:18 7472837 camlistored[7531]: 2017/03/25 23:42:18 Reading
encryption metadata...

> Sat Mar 25 23:42:18 UTC 2017
> Sat Mar 25 23:42:19 UTC 2017
> Sat Mar 25 23:42:20 UTC 2017
> Sat Mar 25 23:42:21 UTC 2017
> [... intermittent gaps ...]
> Sat Mar 25 23:45:49 UTC 2017
> Sat Mar 25 23:46:05 UTC 2017
> Sat Mar 25 23:46:08 UTC 2017
> Sat Mar 25 23:46:09 UTC 2017
> Mar 25 23:46:08 7472837 Tor[40093]: Your system clock just jumped 175
seconds forward; assuming established circuits no longer work.

> Sat Mar 25 23:46:10 UTC 2017
> Sat Mar 25 23:46:11 UTC 2017
> Sat Mar 25 23:46:13 UTC 2017
> Sat Mar 25 23:46:14 UTC 2017
> [... intermittent gaps ...]
> Sat Mar 25 23:49:24 UTC 2017
> Sat Mar 25 23:49:25 UTC 2017
> Sat Mar 25 23:49:26 UTC 2017
> Sat Mar 25 23:49:41 UTC 2017
> Mar 25 23:49:43 7472837 ntpd[64718]: IMSG_CONSTRAINT_KILL for invalid id 2
> Sat Mar 25 23:49:44 UTC 2017
> Sat Mar 25 23:49:46 UTC 2017
> Sat Mar 25 23:49:56 UTC 2017
> Sat Mar 25 23:49:57 UTC 2017
> [... intermittent gaps ...]
> Sat Mar 25 23:54:40 UTC 2017
> Sat Mar 25 23:54:41 UTC 2017
> Sat Mar 25 23:54:42 UTC 2017
> Sat Mar 25 23:54:51 UTC 2017
> Sat Mar 25 23:54:56 UTC 2017
> Sat Mar 25 23:54:57 UTC 2017
> Sat Mar 25 23:54:58 UTC 2017
> Sat Mar 25 23:54:59 UTC 2017
> Sat Mar 25 23:55:22 UTC 2017
> Mar 25 23:55:23 7472837 camlistored[7531]: 2017/03/25 23:55:23 Caught panic
installer handlers: error instantiating storage for prefix "/enc-b2/", type
"encrypt": error scanning metadata on start-up: error with meta blob sha1-xxx:
fetch failed: Get https://f001.backblazeb2.com/file/xxx/meta/sha1-xxx:
net/http: TLS handshake timeout
> Mar 25 23:55:23 7472837 Tor[40093]: Your system clock just jumped 178
seconds forward; assuming established circuits no longer work.
> Sat Mar 25 23:55:27 UTC 2017
> Mar 25 23:55:27 7472837 Tor[40093]: Tor has successfully opened a circuit.
Looks like client functionality is working.
> Sat Mar 25 23:55:28 UTC 2017
> Sat Mar 25 23:55:29 UTC 2017

And here's the ktrace -di -p which is... essentially empty. This is an
excerpt from deep into the freeze.

>   7531 camlistored 1490486112.921212 STRU  struct timespec {
459373.989343598 }
>   7531 camlistored 1490486112.921215 RET   clock_gettime 0
>   7531 camlistored 1490486112.921217 CALL 
clock_gettime(CLOCK_REALTIME,0xc420031ef8)
>   7531 camlistored 1490486112.921220 STRU  struct timespec {
1490486112<"Mar 25 23:55:12 2017">.921219705 }
>   7531 camlistored 1490486112.921222 RET   clock_gettime 0
>   7531 camlistored 1490486112.921225 CALL 
kevent(5,0,0,0xc420031730,64,0xc420031718)
>   7531 camlistored 1490486112.921228 STRU  struct timespec { 0 }
>   7531 camlistored 1490486112.921231 RET   kevent 0
>   7531 camlistored 1490486112.921234 CALL  nanosleep(0xc420031f20,0)
>   7531 camlistored 1490486112.921237 STRU  struct timespec { 0.010000000
}
>   7531 camlistored 1490486112.941202 RET   nanosleep 0
>   7531 camlistored 1490486112.941211 CALL 
clock_gettime(CLOCK_MONOTONIC,0xc420031f20)
>   7531 camlistored 1490486112.941216 STRU  struct timespec {
459374.009347523 }
>   7531 camlistored 1490486112.941219 RET   clock_gettime 0
>   7531 camlistored 1490486112.941221 CALL 
clock_gettime(CLOCK_REALTIME,0xc420031ef8)
>   7531 camlistored 1490486112.941224 STRU  struct timespec {
1490486112<"Mar 25 23:55:12 2017">.941223979 }
>   7531 camlistored 1490486112.941226 RET   clock_gettime 0
>   7531 camlistored 1490486112.941229 CALL 
kevent(5,0,0,0xc420031730,64,0xc420031718)
>   7531 camlistored 1490486112.941232 STRU  struct timespec { 0 }
>   7531 camlistored 1490486112.941235 RET   kevent 0
>   7531 camlistored 1490486112.941238 CALL  nanosleep(0xc420031f20,0)
>   7531 camlistored 1490486112.941240 STRU  struct timespec { 0.010000000
}
>   7531 camlistored 1490486112.961205 RET   nanosleep 0
>   7531 camlistored 1490486112.961214 CALL 
clock_gettime(CLOCK_MONOTONIC,0xc420031f20)
>   7531 camlistored 1490486112.961220 STRU  struct timespec {
459374.029350819 }
>   7531 camlistored 1490486112.961222 RET   clock_gettime 0
>   7531 camlistored 1490486112.961224 CALL 
clock_gettime(CLOCK_REALTIME,0xc420031ef8)
>   7531 camlistored 1490486112.961227 STRU  struct timespec {
1490486112<"Mar 25 23:55:12 2017">.961227136 }
>   7531 camlistored 1490486112.961229 RET   clock_gettime 0
>   7531 camlistored 1490486112.961232 CALL 
kevent(5,0,0,0xc420031730,64,0xc420031718)
>   7531 camlistored 1490486112.961235 STRU  struct timespec { 0 }
>   7531 camlistored 1490486112.961238 RET   kevent 0
>   7531 camlistored 1490486112.961241 CALL  nanosleep(0xc420031f20,0)
>   7531 camlistored 1490486112.961244 STRU  struct timespec { 0.010000000
}
>   7531 camlistored 1490486112.981207 RET   nanosleep 0
>   7531 camlistored 1490486112.981215 CALL 
clock_gettime(CLOCK_MONOTONIC,0xc420031f20)
>   7531 camlistored 1490486112.981221 STRU  struct timespec {
459374.049352090 }
>   7531 camlistored 1490486112.981223 RET   clock_gettime 0
>   7531 camlistored 1490486112.981225 CALL 
clock_gettime(CLOCK_REALTIME,0xc420031ef8)
>   7531 camlistored 1490486112.981229 STRU  struct timespec {
1490486112<"Mar 25 23:55:12 2017">.981228616 }
>   7531 camlistored 1490486112.981231 RET   clock_gettime 0
>   7531 camlistored 1490486112.981234 CALL 
kevent(5,0,0,0xc420031730,64,0xc420031718)
>   7531 camlistored 1490486112.981237 STRU  struct timespec { 0 }
>   7531 camlistored 1490486112.981239 RET   kevent 0
>   7531 camlistored 1490486112.981242 CALL  nanosleep(0xc420031f20,0)
>   7531 camlistored 1490486112.981245 STRU  struct timespec { 0.010000000
}

The only two other events happened in the surrounding 10 seconds are
these.

>   7531 camlistored 1490486111.961230 STRU  struct kevent { ident=11,
filter=EVFILT_READ, flags=0x8021<EV_ADD|EV_CLEAR|EV_EOF>, fflags=0<>,
data=4175, udata=0x2280f96c0 }
>   7531 camlistored 1490486111.961232 RET   kevent 1
>   7531 camlistored 1490486111.981238 STRU  struct kevent { ident=85,
filter=EVFILT_READ, flags=0x8021<EV_ADD|EV_CLEAR|EV_EOF>, fflags=0<>,
data=4175, udata=0x2280f9480 }
>   7531 camlistored 1490486111.981241 RET   kevent 1

Something I noticed is that generating I/O during that step, either to
syslog or to stderr (by increasing verbosity), significantly increases
the chances of the process making it through and seems to reduce the
pauses in other processes.

Drilling a little more into memory utilization with a gctrace seems to
suggest that Go does obtain from the OS a bigger allocation than there
is resident memory, but never really uses all of it, so I'd expect most
hot pages to fit in memory.

> gc 1 @0.511s 3%: 18+7.6+0.049 ms clock, 37+4.0/0.23/5.3+0.098 ms cpu,
4->4->1 MB, 5 MB goal, 2 P
> gc 2 @0.551s 3%: 0.013+4.1+0.046 ms clock, 0.026+0/0.010/4.0+0.092 ms cpu,
4->4->2 MB, 5 MB goal, 2 P
> gc 3 @0.574s 3%: 0.013+1.6+0.051 ms clock, 0.027+0.24/0.10/1.4+0.10 ms cpu,
5->6->4 MB, 6 MB goal, 2 P
> gc 4 @2.518s 0%: 0.015+5.4+0.045 ms clock, 0.030+0.39/0.67/5.4+0.090 ms cpu,
11->11->8 MB, 12 MB goal, 2 P
> gc 5 @2.531s 1%: 8.9+2.6+0.026 ms clock, 17+0.18/0.65/2.1+0.052 ms cpu,
12->12->8 MB, 17 MB goal, 2 P
> gc 6 @2.663s 1%: 0.015+2.9+0.038 ms clock, 0.031+0.23/1.0/2.1+0.077 ms cpu,
17->17->12 MB, 18 MB goal, 2 P
> gc 7 @3.257s 0%: 0.017+3.0+0.048 ms clock, 0.035+0.26/1.2/1.7+0.096 ms cpu,
1038->1038->1037 MB, 1039 MB goal, 2 P
> gc 8 @10.316s 0%: 0.011+2.0+0.040 ms clock, 0.023+0/1.9/1.9+0.080 ms cpu,
1038->1038->13 MB, 2074 MB goal, 2 P
> gc 9 @10.619s 0%: 10+5.8+0.037 ms clock, 20+0.41/5.7/0+0.074 ms cpu,
20->21->17 MB, 27 MB goal, 2 P
> gc 10 @10.732s 0%: 7.4+3.6+0.040 ms clock, 14+0.11/3.6/0+0.080 ms cpu,
31->31->19 MB, 35 MB goal, 2 P
> gc 11 @10.811s 0%: 0.014+4.2+0.28 ms clock, 0.028+0.34/2.0/1.7+0.56 ms cpu,
35->35->19 MB, 38 MB goal, 2 P
> gc 12 @10.978s 0%: 0.015+4.4+0.042 ms clock, 0.031+0.28/3.6/0.49+0.084 ms
cpu, 37->37->23 MB, 38 MB goal, 2 P
> gc 13 @11.211s 0%: 0.019+5.6+0.040 ms clock, 0.038+0.28/4.8/0.53+0.081 ms
cpu, 46->46->29 MB, 47 MB goal, 2 P
> gc 14 @11.482s 0%: 0.015+4.8+0.038 ms clock, 0.031+0.85/4.0/0.38+0.077 ms
cpu, 56->57->29 MB, 58 MB goal, 2 P
> gc 15 @11.752s 0%: 0.19+4.2+0.17 ms clock, 0.39+1.7/2.6/0+0.34 ms cpu,
59->59->30 MB, 60 MB goal, 2 P
> gc 16 @12.002s 0%: 0.020+5.2+0.045 ms clock, 0.041+0.28/0.53/4.6+0.091 ms
cpu, 60->60->36 MB, 61 MB goal, 2 P
> gc 17 @12.525s 0%: 0.018+4.0+0.053 ms clock, 0.036+0.37/0.19/3.4+0.10 ms
cpu, 71->72->35 MB, 73 MB goal, 2 P
> gc 18 @12.965s 0%: 0.016+8.8+1.2 ms clock, 0.032+0.28/7.6/0.89+2.4 ms cpu,
70->70->54 MB, 71 MB goal, 2 P
> gc 19 @13.527s 0%: 0.017+9.6+0.055 ms clock, 0.034+0.69/9.1/0.31+0.11 ms
cpu, 112->112->63 MB, 113 MB goal, 2 P
> gc 20 @14.193s 0%: 0.017+11+0.044 ms clock, 0.035+0.79/10/0.010+0.088 ms
cpu, 136->137->74 MB, 137 MB goal, 2 P
> gc 21 @15.072s 0%: 0.015+12+0.047 ms clock, 0.031+0.74/12/0.42+0.095 ms cpu,
166->166->89 MB, 167 MB goal, 2 P
> gc 22 @17.645s 0%: 0.019+3.5+0.61 ms clock, 0.039+0.32/1.2/3.0+1.2 ms cpu,
150->155->72 MB, 179 MB goal, 2 P
> gc 23 @19.914s 2%: 353+5.0+0.037 ms clock, 706+0.53/4.9/0+0.074 ms cpu,
1131->1131->1112 MB, 1132 MB goal, 2 P
> gc 24 @21.064s 2%: 0.016+4.4+0.034 ms clock, 0.032+0.40/3.4/0.93+0.068 ms
cpu, 1113->1113->1112 MB, 2225 MB goal, 2 P
> gc 25 @111.711s 12%: 2881+28410+4.7 ms clock, 5763+11089/18939/638+9.5 ms
cpu, 1688->1704->188 MB, 2225 MB goal, 2 P
> scvg0: inuse: 1706, idle: 362, sys: 2069, released: 0, consumed: 2069 (MB)
> gc 26 @237.001s 22%: 92+17694+40738 ms clock, 185+1294/15152/0+81477 ms cpu,
339->343->215 MB, 377 MB goal, 2 P
> scvg1: inuse: 230, idle: 1838, sys: 2069, released: 0, consumed: 2069 (MB)
> GC forced
> gc 27 @426.241s 15%: 0.89+7.9+0.049 ms clock, 1.7+0.93/6.0/0.046+0.099 ms
cpu, 241->241->165 MB, 430 MB goal, 2 P

(https://github.com/golang/go/blob/f8ed4539e/src/runtime/extern.go#L74)

The interesting gc rounds are 7 and 8, which go smooth and see the heap
grow to 1G and shrink back down, and rounds 23+ which seem to handle
something similar but go spectacularly bad, with 80s (!!) STW phases.
The peak of the Go heap size there is 1704M, which is very close to my
free resident memory. (Note that these are all Go-internal numbers. The
scvg rounds tell us that Go asked the OS for 2G and never released any
of it.)

Admittedly this task seems to require (barely) more heap than I have
free memory (I am indeed going to need a bigger boat!), so the scheduler
getting stuck deep into paging routines seems likely, with the GC sweep
making everything worse by bringing all pages in to follow the pointers,
and I guess it would also explain I/O giving it some breathing space.
However, such a small over-use of resources maybe shouldn't be this
catastrophic, and probably shouldn't freeze the rest of userspace, but I
accept more expert opinions.

Thanks again.