witness report

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

witness report

Christophe Prevotaux-2
Hello,

This a witness report I got on boot with snapshot Jun 1st amd64

root on sd0a (9b49e3196b9bfae8.a) swap on sd0b dump on sd0b
lock order reversal:
 1st 0xffffff021cdac180 vmmaplk (&map->lock) @ /usr/src/sys/uvm/uvm_map.c:4433
 2nd 0xffffff01dc5f71a8 inode (&ip->i_lock)
@ /usr/src/sys/ufs/ufs/ufs_vnops.c:1559 lock order "&ip->i_lock"(rrwlock) ->
"&map->lock"(rwlock) first seen at: #0  witness_checkorder+0x494
#1  _rw_enter_read+0x49
#2  uvmfault_lookup+0x7d
#3  uvm_fault+0x65
#4  trap+0x509
#5  Xalltraps_untramp+0xec
#6  copyout+0x48
#7  ffs_read+0x1e0
#8  VOP_READ+0x3c
#9  vn_read+0xba
#10 dofilereadv+0x21c
#11 sys_read+0x75
#12 syscall+0x31d
#13 Xsyscall_untramp+0xc0
lock order "&map->lock"(rwlock) -> "&ip->i_lock"(rrwlock) first seen at:
#0  witness_checkorder+0x494
#1  _rw_enter+0x56
#2  _rrw_enter+0x32
#3  VOP_LOCK+0x30
#4  vn_lock+0x24
#5  uvn_io+0x1a5
#6  uvm_pager_put+0xf9
#7  uvn_flush+0x414
#8  uvm_map_clean+0x3c7
#9  syscall+0x31d
#10 Xsyscall_untramp+0xc0


--
Christophe Prévotaux

Reply | Threaded
Open this post in threaded view
|

Re: witness report

Philip Guenther
On Sat, 2 Jun 2018, Christophe Prévotaux wrote:
> This a witness report I got on boot with snapshot Jun 1st amd64
>
> root on sd0a (9b49e3196b9bfae8.a) swap on sd0b dump on sd0b
> lock order reversal:
>  1st 0xffffff021cdac180 vmmaplk (&map->lock) @ /usr/src/sys/uvm/uvm_map.c:4433
>  2nd 0xffffff01dc5f71a8 inode (&ip->i_lock)

I believe uvm and the vnode layer handle this correctly, with lock tries
that fall back to releasing the other lock and retrying so progress is
made.  The fix for WITNESS complaining is to mark vmmaplk as a vnode lock.

ok?

Index: uvm/uvm_map.c
===================================================================
RCS file: /data/src/openbsd/src/sys/uvm/uvm_map.c,v
retrieving revision 1.237
diff -u -p -r1.237 uvm_map.c
--- uvm/uvm_map.c 18 Apr 2018 16:05:21 -0000 1.237
+++ uvm/uvm_map.c 20 Apr 2018 02:22:26 -0000
@@ -2552,7 +2552,7 @@ uvm_map_setup(struct vm_map *map, vaddr_
  map->s_start = map->s_end = 0; /* Empty stack area by default. */
  map->flags = flags;
  map->timestamp = 0;
- rw_init_flags(&map->lock, "vmmaplk", RWL_DUPOK);
+ rw_init_flags(&map->lock, "vmmaplk", RWL_DUPOK | RWL_IS_VNODE);
  mtx_init(&map->mtx, IPL_VM);
  mtx_init(&map->flags_lock, IPL_VM);
 

Reply | Threaded
Open this post in threaded view
|

Re: witness report

Visa Hankala-2
On Sat, Jun 02, 2018 at 03:08:14PM -0700, Philip Guenther wrote:

> On Sat, 2 Jun 2018, Christophe Prévotaux wrote:
> > This a witness report I got on boot with snapshot Jun 1st amd64
> >
> > root on sd0a (9b49e3196b9bfae8.a) swap on sd0b dump on sd0b
> > lock order reversal:
> >  1st 0xffffff021cdac180 vmmaplk (&map->lock) @ /usr/src/sys/uvm/uvm_map.c:4433
> >  2nd 0xffffff01dc5f71a8 inode (&ip->i_lock)
>
> I believe uvm and the vnode layer handle this correctly, with lock tries
> that fall back to releasing the other lock and retrying so progress is
> made.  The fix for WITNESS complaining is to mark vmmaplk as a vnode lock.

I think there is an actual issue because the locking calls are
unconditional. FreeBSD appears to work around the problem by unlocking
the vm_map when calling the pager. The diff below adapts that logic
to OpenBSD.

Because the temporary unlocking may allow another thread to change the
vm_map, the code has to check if the map has been altered since the
unlocking, and if so, handle the case somehow. The patch uses a best
effort approach where the code proceeds from the vm_map entry indicated
by the end address of the current vm_map entry. The sanity checks that
are done at the start of uvm_map_clean() are not rerun.

The system call that triggers the reversal is msync(2), and the
reversal can be reproduced with the sys/kern/mmap regression test.
sys/kern/mmap3 shows that there is another similar reversal with
mlock(2) which is not covered by the patch.

Index: uvm/uvm_map.c
===================================================================
RCS file: src/sys/uvm/uvm_map.c,v
retrieving revision 1.237
diff -u -p -r1.237 uvm_map.c
--- uvm/uvm_map.c 18 Apr 2018 16:05:21 -0000 1.237
+++ uvm/uvm_map.c 3 Jun 2018 10:42:59 -0000
@@ -4420,8 +4420,11 @@ uvm_map_clean(struct vm_map *map, vaddr_
  struct vm_page *pg;
  struct uvm_object *uobj;
  vaddr_t cp_start, cp_end;
+ vaddr_t ent_start, ent_end;
+ voff_t ent_offset;
  int refs;
  int error;
+ unsigned int timestamp;
  boolean_t rv;
 
  KASSERT((flags & (PGO_FREE|PGO_DEACTIVATE)) !=
@@ -4450,8 +4453,7 @@ uvm_map_clean(struct vm_map *map, vaddr_
  }
 
  error = 0;
- for (entry = first; entry != NULL && entry->start < end;
-    entry = RBT_NEXT(uvm_map_addr, entry)) {
+ for (entry = first; entry != NULL && entry->start < end; ) {
  amap = entry->aref.ar_amap; /* top layer */
  if (UVM_ET_ISOBJ(entry))
  uobj = entry->object.uvm_obj;
@@ -4546,13 +4548,27 @@ flush_object:
     ((flags & PGO_FREE) == 0 ||
      ((entry->max_protection & PROT_WRITE) != 0 &&
       (entry->etype & UVM_ET_COPYONWRITE) == 0))) {
+ ent_start = entry->start;
+ ent_end = entry->end;
+ ent_offset = entry->offset;
+ timestamp = map->timestamp;
+ vm_map_unlock_read(map);
+
  rv = uobj->pgops->pgo_flush(uobj,
-    cp_start - entry->start + entry->offset,
-    cp_end - entry->start + entry->offset, flags);
+    cp_start - ent_start + ent_offset,
+    cp_end - ent_start + ent_offset, flags);
 
  if (rv == FALSE)
  error = EFAULT;
- }
+
+ vm_map_lock_read(map);
+ if (timestamp != map->timestamp)
+ entry = uvm_map_entrybyaddr(&map->addr,
+    ent_end);
+ else
+ entry = RBT_NEXT(uvm_map_addr, entry);
+ } else
+ entry = RBT_NEXT(uvm_map_addr, entry);
  }
 
  vm_map_unlock_read(map);

Reply | Threaded
Open this post in threaded view
|

Re: witness report

Mark Kettenis
In reply to this post by Philip Guenther
> Date: Sat, 2 Jun 2018 15:08:14 -0700
> From: Philip Guenther <[hidden email]>
>
> On Sat, 2 Jun 2018, Christophe Prévotaux wrote:
> > This a witness report I got on boot with snapshot Jun 1st amd64
> >
> > root on sd0a (9b49e3196b9bfae8.a) swap on sd0b dump on sd0b
> > lock order reversal:
> >  1st 0xffffff021cdac180 vmmaplk (&map->lock) @ /usr/src/sys/uvm/uvm_map.c:4433
> >  2nd 0xffffff01dc5f71a8 inode (&ip->i_lock)
>
> I believe uvm and the vnode layer handle this correctly, with lock tries
> that fall back to releasing the other lock and retrying so progress is
> made.  The fix for WITNESS complaining is to mark vmmaplk as a vnode lock.
>
> ok?

I don't think so.  While we have UVM_FLAG_TRYLOCK, I don't think it is
used in the code paths shown in the original report.

This may still be a false positive though.  Non-kernel maps will
experience different locking patterns and their locks should probably
be considered to be a different class than the lock associated with
the kernel map.

That said I think the kernel could end up mapping files into kernel
address space as well so I think this lock order reversal is real.  A
way to avoid this is to change the pool allocator used for the
dirhash.  The pool entries are smaller than a page, so I think
pool_allocator_single is the appropriate choice here.



Index: ufs/ufs/ufs_dirhash.c
===================================================================
RCS file: /cvs/src/sys/ufs/ufs/ufs_dirhash.c,v
retrieving revision 1.40
diff -u -p -r1.40 ufs_dirhash.c
--- ufs/ufs/ufs_dirhash.c 26 Oct 2017 02:38:54 -0000 1.40
+++ ufs/ufs/ufs_dirhash.c 3 Jun 2018 11:50:11 -0000
@@ -1053,7 +1053,7 @@ void
 ufsdirhash_init(void)
 {
  pool_init(&ufsdirhash_pool, DH_NBLKOFF * sizeof(doff_t), 0, IPL_NONE,
-    PR_WAITOK, "dirhash", NULL);
+    PR_WAITOK, "dirhash", &pool_allocator_single);
  rw_init(&ufsdirhash_mtx, "dirhash_list");
  arc4random_buf(&ufsdirhash_key, sizeof(ufsdirhash_key));
  TAILQ_INIT(&ufsdirhash_list);


Reply | Threaded
Open this post in threaded view
|

Re: witness report

Mark Kettenis
> Date: Sun, 3 Jun 2018 13:51:19 +0200 (CEST)
> From: Mark Kettenis <[hidden email]>
>
> > Date: Sat, 2 Jun 2018 15:08:14 -0700
> > From: Philip Guenther <[hidden email]>
> >
> > On Sat, 2 Jun 2018, Christophe Prévotaux wrote:
> > > This a witness report I got on boot with snapshot Jun 1st amd64
> > >
> > > root on sd0a (9b49e3196b9bfae8.a) swap on sd0b dump on sd0b
> > > lock order reversal:
> > >  1st 0xffffff021cdac180 vmmaplk (&map->lock) @ /usr/src/sys/uvm/uvm_map.c:4433
> > >  2nd 0xffffff01dc5f71a8 inode (&ip->i_lock)
> >
> > I believe uvm and the vnode layer handle this correctly, with lock tries
> > that fall back to releasing the other lock and retrying so progress is
> > made.  The fix for WITNESS complaining is to mark vmmaplk as a vnode lock.
> >
> > ok?
>
> I don't think so.  While we have UVM_FLAG_TRYLOCK, I don't think it is
> used in the code paths shown in the original report.
>
> This may still be a false positive though.  Non-kernel maps will
> experience different locking patterns and their locks should probably
> be considered to be a different class than the lock associated with
> the kernel map.
>
> That said I think the kernel could end up mapping files into kernel
> address space as well so I think this lock order reversal is real.  A
> way to avoid this is to change the pool allocator used for the
> dirhash.  The pool entries are smaller than a page, so I think
> pool_allocator_single is the appropriate choice here.

But this doesn't fix the lock order reversal triggered by
regress/sys/kern/mmap, so maybe ignore this for now.

> Index: ufs/ufs/ufs_dirhash.c
> ===================================================================
> RCS file: /cvs/src/sys/ufs/ufs/ufs_dirhash.c,v
> retrieving revision 1.40
> diff -u -p -r1.40 ufs_dirhash.c
> --- ufs/ufs/ufs_dirhash.c 26 Oct 2017 02:38:54 -0000 1.40
> +++ ufs/ufs/ufs_dirhash.c 3 Jun 2018 11:50:11 -0000
> @@ -1053,7 +1053,7 @@ void
>  ufsdirhash_init(void)
>  {
>   pool_init(&ufsdirhash_pool, DH_NBLKOFF * sizeof(doff_t), 0, IPL_NONE,
> -    PR_WAITOK, "dirhash", NULL);
> +    PR_WAITOK, "dirhash", &pool_allocator_single);
>   rw_init(&ufsdirhash_mtx, "dirhash_list");
>   arc4random_buf(&ufsdirhash_key, sizeof(ufsdirhash_key));
>   TAILQ_INIT(&ufsdirhash_list);
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: witness report

Hrvoje Popovski
In reply to this post by Visa Hankala-2
On 3.6.2018. 13:38, Visa Hankala wrote:

> On Sat, Jun 02, 2018 at 03:08:14PM -0700, Philip Guenther wrote:
>> On Sat, 2 Jun 2018, Christophe Prévotaux wrote:
>>> This a witness report I got on boot with snapshot Jun 1st amd64
>>>
>>> root on sd0a (9b49e3196b9bfae8.a) swap on sd0b dump on sd0b
>>> lock order reversal:
>>>  1st 0xffffff021cdac180 vmmaplk (&map->lock) @ /usr/src/sys/uvm/uvm_map.c:4433
>>>  2nd 0xffffff01dc5f71a8 inode (&ip->i_lock)
>> I believe uvm and the vnode layer handle this correctly, with lock tries
>> that fall back to releasing the other lock and retrying so progress is
>> made.  The fix for WITNESS complaining is to mark vmmaplk as a vnode lock.
> I think there is an actual issue because the locking calls are
> unconditional. FreeBSD appears to work around the problem by unlocking
> the vm_map when calling the pager. The diff below adapts that logic
> to OpenBSD.
>
> Because the temporary unlocking may allow another thread to change the
> vm_map, the code has to check if the map has been altered since the
> unlocking, and if so, handle the case somehow. The patch uses a best
> effort approach where the code proceeds from the vm_map entry indicated
> by the end address of the current vm_map entry. The sanity checks that
> are done at the start of uvm_map_clean() are not rerun.
>
> The system call that triggers the reversal is msync(2), and the
> reversal can be reproduced with the sys/kern/mmap regression test.
> sys/kern/mmap3 shows that there is another similar reversal with
> mlock(2) which is not covered by the patch.

Hi all,

with WITNESS I'm getting that similar log and with visa@ diff it's gone.


WITNESS log:

lock order reversal:
 1st 0xffffff01ef4eb2f0 vmmaplk (&map->lock) @
/usr/src/sys/uvm/uvm_map.c:4435
 2nd 0xffffff020f58f700 inode (&ip->i_lock) @
/usr/src/sys/ufs/ufs/ufs_vnops.c:1544
lock order "&ip->i_lock"(rrwlock) -> "&map->lock"(rwlock) first seen at:
#0  witness_checkorder+0x4c0
#1  _rw_enter+0x68
#2  vm_map_lock_ln+0xbc
#3  uvm_map+0x1a1
#4  km_alloc+0x16a
#5  pool_multi_alloc_ni+0xbb
#6  pool_p_alloc+0x56
#7  pool_do_get+0xe4
#8  pool_get+0xaf
#9  ufsdirhash_build+0x31e
#10 ufs_lookup+0x19d
#11 VOP_LOOKUP+0x4f
#12 vfs_lookup+0x2cf
#13 namei+0x2e3
#14 start_init+0xb2
lock order "&map->lock"(rwlock) -> "&ip->i_lock"(rrwlock) first seen at:
#0  witness_checkorder+0x4c0
#1  _rw_enter+0x68
#2  _rrw_enter+0x3e
#3  VOP_LOCK+0x3d
#4  vn_lock+0x34
#5  uvn_io+0x1b8
#6  uvm_pager_put+0x109
#7  uvn_flush+0x424
#8  uvm_map_clean+0x3e7
#9  syscall+0x32a
#10 Xsyscall+0x128


OpenBSD 6.4-current (GENERIC.MP) #7: Mon Nov  5 22:09:07 CET 2018
    [hidden email]:/sys/arch/amd64/compile/GENERIC.MP
real mem = 8456089600 (8064MB)
avail mem = 8121876480 (7745MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xe87b1 (86 entries)
bios0: vendor Hewlett-Packard version "J01 v02.29" date 04/04/2016
bios0: Hewlett-Packard HP Compaq 8200 Elite CMT PC
acpi0 at bios0: rev 2
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP APIC SSDT MCFG HPET SSDT SLIC TCPA
acpi0: wakeup devices PS2K(S3) PS2M(S3) BR20(S4) EUSB(S3) USBE(S3)
PEX0(S4) PEX1(S4) PEX2(S4) PEX3(S4) PEX4(S4) PEX5(S4) PEX6(S4) PEX7(S4)
P0P1(S4) P0P2(S4) P0P3(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM) i5-2500 CPU @ 3.30GHz, 3293.52 MHz, 06-2a-07
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.1, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Core(TM) i5-2500 CPU @ 3.30GHz, 3292.56 MHz, 06-2a-07
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Core(TM) i5-2500 CPU @ 3.30GHz, 3292.56 MHz, 06-2a-07
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 6 (application processor)
cpu3: Intel(R) Core(TM) i5-2500 CPU @ 3.30GHz, 3292.56 MHz, 06-2a-07
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu3: 256KB 64b/line 8-way L2 cache
cpu3: smt 0, core 3, package 0
ioapic0 at mainbus0: apid 0 pa 0xfec00000, version 20, 24 pins
acpimcfg0 at acpi0
acpimcfg0: addr 0xe0000000, bus 0-255
acpihpet0 at acpi0: 14318179 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 5 (BR20)
acpiprt2 at acpi0: bus 1 (PEX0)
acpiprt3 at acpi0: bus -1 (PEX1)
acpiprt4 at acpi0: bus -1 (PEX2)
acpiprt5 at acpi0: bus -1 (PEX3)
acpiprt6 at acpi0: bus 2 (PEX4)
acpiprt7 at acpi0: bus -1 (PEX5)
acpiprt8 at acpi0: bus 3 (PEX6)
acpiprt9 at acpi0: bus 4 (PEX7)
acpiprt10 at acpi0: bus -1 (P0P1)
acpiprt11 at acpi0: bus -1 (P0P2)
acpiprt12 at acpi0: bus -1 (P0P3)
acpiprt13 at acpi0: bus -1 (P0P4)
acpicpu0 at acpi0: C1(1000@1 halt), PSS
acpicpu1 at acpi0: C1(1000@1 halt), PSS
acpicpu2 at acpi0: C1(1000@1 halt), PSS
acpicpu3 at acpi0: C1(1000@1 halt), PSS
acpipci0 at acpi0 PCI0: 0x00000010 0x00000011 0x00000000
acpicmos0 at acpi0
tpm0 at acpi0: TPM_ addr 0xfed40000/0x5000: Infineon SLB9635 1.2 rev 0x10
acpibtn0 at acpi0: PWRB
"PNP0C14" at acpi0 not configured
ipmi at mainbus0 not configured
cpu0: Enhanced SpeedStep 3293 MHz: speeds: 3301, 3300, 3100, 2900, 2700,
2500, 2300, 2100, 1900, 1700, 1600 MHz
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel Core 2G Host" rev 0x09
inteldrm0 at pci0 dev 2 function 0 "Intel HD Graphics 2000" rev 0x09
drm0 at inteldrm0
inteldrm0: msi
inteldrm0: 1024x768, 32bpp
wsdisplay0 at inteldrm0 mux 1: console (std, vt100 emulation)
wsdisplay0: screen 1-5 added (std, vt100 emulation)
em0 at pci0 dev 25 function 0 "Intel 82579LM" rev 0x04: msi, address
78:ac:c0:ba:4a:7c
ehci0 at pci0 dev 26 function 0 "Intel 6 Series USB" rev 0x04: apic 0 int 16
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 configuration 1 interface 0 "Intel EHCI root hub" rev
2.00/1.00 addr 1
azalia0 at pci0 dev 27 function 0 "Intel 6 Series HD Audio" rev 0x04: msi
azalia0: codecs: Realtek ALC662, Intel/0x2805, using Realtek ALC662
audio0 at azalia0
ppb0 at pci0 dev 28 function 0 "Intel 6 Series PCIE" rev 0xb4: msi
pci1 at ppb0 bus 1
ppb1 at pci0 dev 28 function 4 "Intel 6 Series PCIE" rev 0xb4: msi
pci2 at ppb1 bus 2
ppb2 at pci0 dev 28 function 6 "Intel 6 Series PCIE" rev 0xb4: msi
pci3 at ppb2 bus 3
ppb3 at pci0 dev 28 function 7 "Intel 6 Series PCIE" rev 0xb4: msi
pci4 at ppb3 bus 4
ehci1 at pci0 dev 29 function 0 "Intel 6 Series USB" rev 0x04: apic 0 int 23
usb1 at ehci1: USB revision 2.0
uhub1 at usb1 configuration 1 interface 0 "Intel EHCI root hub" rev
2.00/1.00 addr 1
ppb4 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev 0xa4
pci5 at ppb4 bus 5
pcib0 at pci0 dev 31 function 0 "Intel Q67 LPC" rev 0x04
ahci0 at pci0 dev 31 function 2 "Intel 6 Series AHCI" rev 0x04: msi,
AHCI 1.3
ahci0: port 0: 6.0Gb/s
ahci0: port 1: 6.0Gb/s
ahci0: port 2: 1.5Gb/s
ahci0: port 3: 3.0Gb/s
scsibus1 at ahci0: 32 targets
sd0 at scsibus1 targ 0 lun 0: <ATA, WDC WD1000DHTZ-0, 04.0> SCSI3
0/direct fixed naa.50014ee7aaada488
sd0: 953869MB, 512 bytes/sector, 1953525168 sectors
sd1 at scsibus1 targ 1 lun 0: <ATA, WDC WD30EZRX-00D, 80.0> SCSI3
0/direct fixed naa.50014ee25e87ef8e
sd1: 2861588MB, 512 bytes/sector, 5860533168 sectors
cd0 at scsibus1 targ 2 lun 0: <hp, DVD A DH16ABLH, 3HD9> ATAPI 5/cdrom
removable
sd2 at scsibus1 targ 3 lun 0: <ATA, ST3500413AS, HP61> SCSI3 0/direct
fixed naa.5000c5002db8bf5e
sd2: 476940MB, 512 bytes/sector, 976773168 sectors
ichiic0 at pci0 dev 31 function 3 "Intel 6 Series SMBus" rev 0x04: apic
0 int 18
iic0 at ichiic0
spdmem0 at iic0 addr 0x50: 2GB DDR3 SDRAM PC3-10600
spdmem1 at iic0 addr 0x51: 2GB DDR3 SDRAM PC3-10600
spdmem2 at iic0 addr 0x52: 2GB DDR3 SDRAM PC3-10600
spdmem3 at iic0 addr 0x53: 2GB DDR3 SDRAM PC3-10600
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com0: console
pckbc0 at isa0 port 0x60/5 irq 1 irq 12
pckbd0 at pckbc0 (kbd slot)
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
uhub2 at uhub0 port 1 configuration 1 interface 0 "Intel Rate Matching
Hub" rev 2.00/0.00 addr 2
uhub3 at uhub1 port 1 configuration 1 interface 0 "Intel Rate Matching
Hub" rev 2.00/0.00 addr 2
vscsi0 at root
scsibus2 at vscsi0: 256 targets
softraid0 at root
scsibus3 at softraid0: 256 targets
root on sd2a (9fc0acceac93c3a4.a) swap on sd2b dump on sd2b