Small/Mini 10Gbe Router Recommendation

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Small/Mini 10Gbe Router Recommendation

Daniel Melameth
Looking to finally part with my legacy OpenBSD router and upgrade to
something that can push more than 2Gbps out of a single port.  Since
my switching equipment is still only 1Gbe, I also want something that
has, at least, two Gbe ports.

Any recommendations that work well with OpenBSD?  I am currently
thinking https://www.supermicro.com/en/products/system/Mini-ITX/SYS-E300-8D.cfm,
but would like other opinions.

Thank you.

Reply | Threaded
Open this post in threaded view
|

Re: Small/Mini 10Gbe Router Recommendation

Stuart Henderson
On 2021-04-07, Daniel Melameth <[hidden email]> wrote:
> Looking to finally part with my legacy OpenBSD router and upgrade to
> something that can push more than 2Gbps out of a single port.  Since
> my switching equipment is still only 1Gbe, I also want something that
> has, at least, two Gbe ports.
>
> Any recommendations that work well with OpenBSD?  I am currently
> thinking https://www.supermicro.com/en/products/system/Mini-ITX/SYS-E300-8D.cfm,
> but would like other opinions.

I have several routers using that same motherboard (been using them for
3-4 years), they work nicely and have a useful selection of NICs. dmesg below -
the onboard SFP+ are ix0/1, the ixl(4) in there are a PCIE card. DOM works ok
on the fibre ports ("ifconfig ix0 sff" etc).

Mine aren't in a good place on the network to test the speed of packet
forwarding (and those will vary quite a bit depending on configuration)
but should be good for what you're asking.

Note that the BMC defaults to sharing em0 if it doesn't have link on the
separate management port, you may want to change that to dedicated, it
can be done in config (or IIRC you can also change that setting by
poking at it with ipmitool/freeipmi if you enable ipmi in kernel config;
that also gets you additional sensors in hw.sensors rather than just
the cpu temperature).

(FWIW that one is in a nice 1U chassis with front-facing ports and dual
PSU - https://www.supermicro.com/en/products/chassis/1U/515/SC515-R407 -
not quite so mini but some of these are an 8h round trip away and sometimes
need to get remote hands to plug things in so these are good features for
my use).

Might not be an issue for your use but be aware the 40x28mm fans in
CSE-E300 are pretty whiny. You can change the power management profile
in bios config which helps, and the noctua 40x20 fans can be made to
work if that's not enough (though it's a bit of a faff and you will
need to find screws that work, noctua's usual rubber mounts won't
fit and their screw holes are weird sizes) but even with those changes
it's not the best chassis for a noise-sensitive location. The 1Ux19"
chassis aren't really quieter but the noise profile is more pleasant.

OpenBSD 6.8-current (GENERIC.MP) #284: Wed Jan 20 02:40:03 MST 2021
    [hidden email]:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 8464850944 (8072MB)
avail mem = 8192978944 (7813MB)
random: good seed from bootblocks
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.8 @ 0xed9b0 (46 entries)
bios0: vendor American Megatrends Inc. version "1.0c" date 10/31/2017
bios0: Supermicro X10SDV-TP8F
acpi0 at bios0: ACPI 5.0
acpi0: sleep states S0 S4 S5
acpi0: tables DSDT FACP APIC FPDT FIDT SPMI MCFG UEFI DBG2 HPET WDDT SSDT SSDT SSDT PRAD DMAR HEST BERT ERST EINJ
acpi0: wakeup devices IP2P(S4) EHC1(S4) EHC2(S4) RP07(S4) RP08(S4) BR1A(S4) BR1B(S4) BR2A(S4) BR2B(S4) BR2C(S4) BR2D(S4) BR3A(S4) BR3B(S4) BR3C(S4) BR3D(S4) RP01(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz, 2200.26 MHz, 06-56-03
cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 100MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.2, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz, 2200.02 MHz, 06-56-03
cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz, 2200.02 MHz, 06-56-03
cpu2: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 6 (application processor)
cpu3: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz, 2200.02 MHz, 06-56-03
cpu3: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu3: 256KB 64b/line 8-way L2 cache
cpu3: smt 0, core 3, package 0
cpu4 at mainbus0: apid 1 (application processor)
cpu4: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz, 2200.02 MHz, 06-56-03
cpu4: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu4: 256KB 64b/line 8-way L2 cache
cpu4: smt 1, core 0, package 0
cpu5 at mainbus0: apid 3 (application processor)
cpu5: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz, 2200.02 MHz, 06-56-03
cpu5: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu5: 256KB 64b/line 8-way L2 cache
cpu5: smt 1, core 1, package 0
cpu6 at mainbus0: apid 5 (application processor)
cpu6: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz, 2200.02 MHz, 06-56-03
cpu6: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu6: 256KB 64b/line 8-way L2 cache
cpu6: smt 1, core 2, package 0
cpu7 at mainbus0: apid 7 (application processor)
cpu7: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz, 2200.02 MHz, 06-56-03
cpu7: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu7: 256KB 64b/line 8-way L2 cache
cpu7: smt 1, core 3, package 0
ioapic0 at mainbus0: apid 8 pa 0xfec00000, version 20, 24 pins
acpimcfg0 at acpi0
acpimcfg0: addr 0x80000000, bus 0-255
acpihpet0 at acpi0: 14318179 Hz
acpiprt0 at acpi0: bus 255 (UNC0)
acpiprt1 at acpi0: bus 0 (PCI0)
acpiprt2 at acpi0: bus 1 (BR1A)
acpiprt3 at acpi0: bus 2 (BR1B)
acpiprt4 at acpi0: bus 4 (BR2C)
acpiprt5 at acpi0: bus 5 (BR3A)
acpiprt6 at acpi0: bus 7 (RP01)
acpiprt7 at acpi0: bus 8 (RP02)
acpiprt8 at acpi0: bus 9 (RP04)
acpiprt9 at acpi0: bus 10 (BR3A)
acpiprt10 at acpi0: bus 11 (RP05)
acpipci0 at acpi0 UNC0: 0x00000010 0x00000011 0x00000000
"ACPI0004" at acpi0 not configured
"ACPI0007" at acpi0 not configured
"ACPI0007" at acpi0 not configured
"ACPI0007" at acpi0 not configured
"ACPI0007" at acpi0 not configured
"ACPI0007" at acpi0 not configured
"ACPI0007" at acpi0 not configured
"ACPI0007" at acpi0 not configured
"ACPI0007" at acpi0 not configured
"PNP0C33" at acpi0 not configured
"PNP0C14" at acpi0 not configured
acpipci1 at acpi0 PCI0: 0x00000010 0x00000011 0x00000000
"PNP0003" at acpi0 not configured
"PNP0C14" at acpi0 not configured
acpicmos0 at acpi0
"IPI0001" at acpi0 not configured
acpibtn0 at acpi0: PWRB
acpicpu0 at acpi0: C1(1000@1 halt), PSS
acpicpu1 at acpi0: C1(1000@1 halt), PSS
acpicpu2 at acpi0: C1(1000@1 halt), PSS
acpicpu3 at acpi0: C1(1000@1 halt), PSS
acpicpu4 at acpi0: C1(1000@1 halt), PSS
acpicpu5 at acpi0: C1(1000@1 halt), PSS
acpicpu6 at acpi0: C1(1000@1 halt), PSS
acpicpu7 at acpi0: C1(1000@1 halt), PSS
ipmi at mainbus0 not configured
cpu0: using VERW MDS workaround (except on vmm entry)
cpu0: Enhanced SpeedStep 2200 MHz: speeds: 2200, 2100, 2000, 1900, 1800, 1700, 1600, 1500, 1400, 1300, 1200, 1100, 1000, 900, 800 MHz
pci0 at mainbus0 bus 255
"Intel Xeon-D QPI Link" rev 0x03 at pci0 dev 11 function 0 not configured
"Intel Xeon-D QPI Link" rev 0x03 at pci0 dev 11 function 1 not configured
"Intel Xeon-D QPI Link" rev 0x03 at pci0 dev 11 function 2 not configured
"Intel Xeon-D QPI Debug" rev 0x03 at pci0 dev 11 function 3 not configured
"Intel Xeon-D Cache" rev 0x03 at pci0 dev 12 function 0 not configured
"Intel Xeon-D Cache" rev 0x03 at pci0 dev 12 function 1 not configured
"Intel Xeon-D Cache" rev 0x03 at pci0 dev 12 function 2 not configured
"Intel Xeon-D Cache" rev 0x03 at pci0 dev 12 function 3 not configured
"Intel Xeon-D Cache" rev 0x03 at pci0 dev 15 function 0 not configured
"Intel Xeon-D Cache" rev 0x03 at pci0 dev 15 function 4 not configured
"Intel Xeon-D Cache" rev 0x03 at pci0 dev 15 function 5 not configured
"Intel Xeon-D Cache" rev 0x03 at pci0 dev 15 function 6 not configured
"Intel Xeon-D PCIE" rev 0x03 at pci0 dev 16 function 0 not configured
"Intel E5 v4 R2PCIe Agent" rev 0x03 at pci0 dev 16 function 1 not configured
"Intel Xeon-D Ubox" rev 0x03 at pci0 dev 16 function 5 not configured
"Intel Xeon-D Ubox" rev 0x03 at pci0 dev 16 function 6 not configured
"Intel Xeon-D Ubox" rev 0x03 at pci0 dev 16 function 7 not configured
"Intel Xeon-D Home Agent" rev 0x03 at pci0 dev 18 function 0 not configured
"Intel Xeon-D Home Agent" rev 0x03 at pci0 dev 18 function 1 not configured
"Intel Xeon-D Memory" rev 0x03 at pci0 dev 19 function 0 not configured
"Intel Xeon-D Memory" rev 0x03 at pci0 dev 19 function 1 not configured
"Intel Xeon-D Memory" rev 0x03 at pci0 dev 19 function 2 not configured
"Intel Xeon-D Memory" rev 0x03 at pci0 dev 19 function 3 not configured
"Intel Xeon-D Memory" rev 0x03 at pci0 dev 19 function 4 not configured
"Intel Xeon-D Memory" rev 0x03 at pci0 dev 19 function 5 not configured
"Intel Xeon-D Memory" rev 0x03 at pci0 dev 19 function 6 not configured
"Intel Xeon-D Memory" rev 0x03 at pci0 dev 19 function 7 not configured
"Intel Xeon-D Memory" rev 0x03 at pci0 dev 20 function 0 not configured
"Intel Xeon-D Memory" rev 0x03 at pci0 dev 20 function 1 not configured
"Intel Xeon-D Memory" rev 0x03 at pci0 dev 20 function 2 not configured
"Intel Xeon-D Memory" rev 0x03 at pci0 dev 20 function 3 not configured
"Intel Xeon-D Memory" rev 0x03 at pci0 dev 20 function 4 not configured
"Intel Xeon-D Memory" rev 0x03 at pci0 dev 20 function 5 not configured
"Intel Xeon-D Memory" rev 0x03 at pci0 dev 20 function 6 not configured
"Intel Xeon-D Memory" rev 0x03 at pci0 dev 20 function 7 not configured
"Intel Xeon-D Memory" rev 0x03 at pci0 dev 21 function 0 not configured
"Intel Xeon-D Memory" rev 0x03 at pci0 dev 21 function 1 not configured
"Intel Xeon-D Memory" rev 0x03 at pci0 dev 21 function 2 not configured
"Intel Xeon-D Memory" rev 0x03 at pci0 dev 21 function 3 not configured
"Intel Xeon-D PCU" rev 0x03 at pci0 dev 30 function 0 not configured
"Intel Xeon-D PCU" rev 0x03 at pci0 dev 30 function 1 not configured
"Intel Xeon-D PCU" rev 0x03 at pci0 dev 30 function 2 not configured
"Intel Xeon-D PCU" rev 0x03 at pci0 dev 30 function 3 not configured
"Intel Xeon-D PCU" rev 0x03 at pci0 dev 30 function 4 not configured
"Intel Xeon-D PCU" rev 0x03 at pci0 dev 31 function 0 not configured
"Intel Xeon-D PCU" rev 0x03 at pci0 dev 31 function 2 not configured
pci1 at mainbus0 bus 0
pchb0 at pci1 dev 0 function 0 "Intel Xeon-D Host" rev 0x03
ppb0 at pci1 dev 1 function 0 "Intel Xeon-D PCIE" rev 0x03
pci2 at ppb0 bus 1
ppb1 at pci1 dev 1 function 1 "Intel Xeon-D PCIE" rev 0x03
pci3 at ppb1 bus 2
ppb2 at pci1 dev 2 function 0 "Intel Xeon-D PCIE" rev 0x03
pci4 at ppb2 bus 3
"Intel Xeon-D QuickData" rev 0x00 at pci4 dev 0 function 0 not configured
"Intel Xeon-D QuickData" rev 0x00 at pci4 dev 0 function 1 not configured
"Intel Xeon-D QuickData" rev 0x00 at pci4 dev 0 function 2 not configured
"Intel Xeon-D QuickData" rev 0x00 at pci4 dev 0 function 3 not configured
ppb3 at pci1 dev 2 function 2 "Intel Xeon-D PCIE" rev 0x03
pci5 at ppb3 bus 4
ix0 at pci5 dev 0 function 0 "Intel X552 SFP+" rev 0x00, msix, 4 queues, address ac:1f:6b:1d:fb:a4
ix1 at pci5 dev 0 function 1 "Intel X552 SFP+" rev 0x00, msix, 4 queues, address ac:1f:6b:1d:fb:a5
ppb4 at pci1 dev 3 function 0 "Intel Xeon-D PCIE" rev 0x03: msi
pci6 at ppb4 bus 5
ixl0 at pci6 dev 0 function 0 "Intel X710 SFP+" rev 0x02: port 1, FW 7.0.50775 API 1.8, msix, 4 queues, address 00:e0:ed:75:a5:60
ixl1 at pci6 dev 0 function 1 "Intel X710 SFP+" rev 0x02: port 0, FW 7.0.50775 API 1.8, msix, 4 queues, address 00:e0:ed:75:a5:61
ixl2 at pci6 dev 0 function 2 "Intel X710 SFP+" rev 0x02: port 2, FW 7.0.50775 API 1.8, msix, 4 queues, address 00:e0:ed:75:a5:62
ixl3 at pci6 dev 0 function 3 "Intel X710 SFP+" rev 0x02: port 3, FW 7.0.50775 API 1.8, msix, 4 queues, address 00:e0:ed:75:a5:63
ppb5 at pci1 dev 3 function 2 "Intel Xeon-D PCIE" rev 0x03
pci7 at ppb5 bus 6
"Intel Xeon-D Address Map" rev 0x03 at pci1 dev 5 function 0 not configured
"Intel Xeon-D Hot Plug" rev 0x03 at pci1 dev 5 function 1 not configured
"Intel Xeon-D RAS" rev 0x03 at pci1 dev 5 function 2 not configured
xhci0 at pci1 dev 20 function 0 "Intel 8 Series xHCI" rev 0x05: msi, xHCI 1.0
usb0 at xhci0: USB revision 3.0
uhub0 at usb0 configuration 1 interface 0 "Intel xHCI root hub" rev 3.00/1.00 addr 1
"Intel 8 Series MEI" rev 0x04 at pci1 dev 22 function 0 not configured
"Intel 8 Series MEI" rev 0x04 at pci1 dev 22 function 1 not configured
ehci0 at pci1 dev 26 function 0 "Intel 8 Series USB" rev 0x05: apic 8 int 18
usb1 at ehci0: USB revision 2.0
uhub1 at usb1 configuration 1 interface 0 "Intel EHCI root hub" rev 2.00/1.00 addr 1
ppb6 at pci1 dev 28 function 0 "Intel 8 Series PCIE" rev 0xd5: msi
pci8 at ppb6 bus 7
em0 at pci8 dev 0 function 0 "Intel I210" rev 0x03: msi, address ac:1f:6b:1d:f6:a8
ppb7 at pci1 dev 28 function 1 "Intel 8 Series PCIE" rev 0xd5: msi
pci9 at ppb7 bus 8
em1 at pci9 dev 0 function 0 "Intel I210" rev 0x03: msi, address ac:1f:6b:1d:f6:a9
ppb8 at pci1 dev 28 function 3 "Intel 8 Series PCIE" rev 0xd5: msi
pci10 at ppb8 bus 9
ppb9 at pci10 dev 0 function 0 "ASPEED Technology AST1150 PCI" rev 0x03
pci11 at ppb9 bus 10
vga1 at pci11 dev 0 function 0 "ASPEED Technology AST2000" rev 0x30
wsdisplay at vga1 not configured
ppb10 at pci1 dev 28 function 4 "Intel 8 Series PCIE" rev 0xd5: msi
pci12 at ppb10 bus 11
em2 at pci12 dev 0 function 0 "Intel I350" rev 0x01: msi, address ac:1f:6b:1d:f6:aa
em3 at pci12 dev 0 function 1 "Intel I350" rev 0x01: msi, address ac:1f:6b:1d:f6:ab
em4 at pci12 dev 0 function 2 "Intel I350" rev 0x01: msi, address ac:1f:6b:1d:f6:ac
em5 at pci12 dev 0 function 3 "Intel I350" rev 0x01: msi, address ac:1f:6b:1d:f6:ad
ehci1 at pci1 dev 29 function 0 "Intel 8 Series USB" rev 0x05: apic 8 int 18
usb2 at ehci1: USB revision 2.0
uhub2 at usb2 configuration 1 interface 0 "Intel EHCI root hub" rev 2.00/1.00 addr 1
pcib0 at pci1 dev 31 function 0 "Intel C224 LPC" rev 0x05
ahci0 at pci1 dev 31 function 2 "Intel 8 Series AHCI" rev 0x05: msi, AHCI 1.3
ahci0: port 4: 6.0Gb/s
scsibus1 at ahci0: 32 targets
sd0 at scsibus1 targ 4 lun 0: <ATA, PLEXTOR PX-128S3, 1.04> naa.50023031011e7e4f
sd0: 122104MB, 512 bytes/sector, 250069680 sectors, thin
ichiic0 at pci1 dev 31 function 3 "Intel 8 Series SMBus" rev 0x05: apic 8 int 18
iic0 at ichiic0
"Intel 8 Series Thermal" rev 0x05 at pci1 dev 31 function 6 not configured
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com0: console
com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
vmm0 at mainbus0: VMX/EPT
uhub3 at uhub0 port 3 configuration 1 interface 0 "Genesys Logic USB2.0 Hub" rev 2.00/32.98 addr 2
uhub4 at uhub3 port 2 configuration 1 interface 0 "Genesys Logic USB2.0 Hub" rev 2.00/32.98 addr 3
uhub5 at uhub0 port 4 configuration 1 interface 0 "ATEN International product 0x7000" rev 2.00/0.00 addr 4
uhidev0 at uhub5 port 1 configuration 1 interface 0 "ATEN International product 0x2419" rev 1.10/1.00 addr 5
uhidev0: iclass 3/1
ukbd0 at uhidev0: 8 variable keys, 6 key codes
wskbd0 at ukbd0 mux 1
uhidev1 at uhub5 port 1 configuration 1 interface 1 "ATEN International product 0x2419" rev 1.10/1.00 addr 5
uhidev1: iclass 3/1
ums0 at uhidev1: 3 buttons, Z dir
wsmouse0 at ums0 mux 0
uhub6 at uhub1 port 1 configuration 1 interface 0 "Intel Rate Matching Hub" rev 2.00/0.05 addr 2
uhub7 at uhub2 port 1 configuration 1 interface 0 "Intel Rate Matching Hub" rev 2.00/0.05 addr 2
vscsi0 at root
scsibus2 at vscsi0: 256 targets
softraid0 at root
scsibus3 at softraid0: 256 targets
root on sd0a (025251186f4f78cc.a) swap on sd0b dump on sd0b


Reply | Threaded
Open this post in threaded view
|

Re: Small/Mini 10Gbe Router Recommendation

Daniel Melameth
On Thu, Apr 8, 2021 at 3:57 AM Stuart Henderson <[hidden email]> wrote:

> On 2021-04-07, Daniel Melameth <[hidden email]> wrote:
> > Looking to finally part with my legacy OpenBSD router and upgrade to
> > something that can push more than 2Gbps out of a single port.  Since
> > my switching equipment is still only 1Gbe, I also want something that
> > has, at least, two Gbe ports.
> >
> > Any recommendations that work well with OpenBSD?  I am currently
> > thinking https://www.supermicro.com/en/products/system/Mini-ITX/SYS-E300-8D.cfm,
> > but would like other opinions.
>
> I have several routers using that same motherboard (been using them for
> 3-4 years), they work nicely and have a useful selection of NICs. dmesg below -
> the onboard SFP+ are ix0/1, the ixl(4) in there are a PCIE card. DOM works ok
> on the fibre ports ("ifconfig ix0 sff" etc).

Wonderful--and the dmesg is even better.

> Note that the BMC defaults to sharing em0 if it doesn't have link on the
> separate management port, you may want to change that to dedicated, it
> can be done in config (or IIRC you can also change that setting by
> poking at it with ipmitool/freeipmi if you enable ipmi in kernel config;
> that also gets you additional sensors in hw.sensors rather than just
> the cpu temperature).

Appreciate this detail--will definitely be using the ipmi and didn't
know about the added sensors.

> Might not be an issue for your use but be aware the 40x28mm fans in
> CSE-E300 are pretty whiny. You can change the power management profile
> in bios config which helps, and the noctua 40x20 fans can be made to
> work if that's not enough (though it's a bit of a faff and you will
> need to find screws that work, noctua's usual rubber mounts won't
> fit and their screw holes are weird sizes) but even with those changes
> it's not the best chassis for a noise-sensitive location. The 1Ux19"
> chassis aren't really quieter but the noise profile is more pleasant.

Yes.  I've heard these are loud and appreciate the extra detail on
other available options in case I go down this route, but this will be
sitting in an unfinished space behind a door so I expect I'll be okay.

Reply | Threaded
Open this post in threaded view
|

Re: Small/Mini 10Gbe Router Recommendation

Hrvoje Popovski
On 8.4.2021. 20:56, Daniel Melameth wrote:

> On Thu, Apr 8, 2021 at 3:57 AM Stuart Henderson <[hidden email]> wrote:
>> On 2021-04-07, Daniel Melameth <[hidden email]> wrote:
>>> Looking to finally part with my legacy OpenBSD router and upgrade to
>>> something that can push more than 2Gbps out of a single port.  Since
>>> my switching equipment is still only 1Gbe, I also want something that
>>> has, at least, two Gbe ports.
>>>
>>> Any recommendations that work well with OpenBSD?  I am currently
>>> thinking https://www.supermicro.com/en/products/system/Mini-ITX/SYS-E300-8D.cfm,
>>> but would like other opinions.
>>
>> I have several routers using that same motherboard (been using them for
>> 3-4 years), they work nicely and have a useful selection of NICs. dmesg below -
>> the onboard SFP+ are ix0/1, the ixl(4) in there are a PCIE card. DOM works ok
>> on the fibre ports ("ifconfig ix0 sff" etc).
>
> Wonderful--and the dmesg is even better.
>

my thinking is that if you want to push 10G traffic you'll need at least
8 faster cores ..
for now you won't be using them, but when multiqueue RSS forwarding is
unlocked you will be happy ...

this is vmstat -iz from 12 core box with ixl, mcx and ix

irq114/ixl0                        27        0
irq115/ixl0:0                       4        0
irq116/ixl0:1                       0        0
irq117/ixl0:2                       0        0
irq118/ixl0:3                       0        0
irq119/ixl0:4                       0        0
irq120/ixl0:5                       0        0
irq121/ixl0:6                       0        0
irq122/ixl0:7                       8        0
irq123/ixl1                        27        0
irq124/ixl1:0                       4        0
irq125/ixl1:1                       0        0
irq126/ixl1:2                       0        0
irq127/ixl1:3                       0        0
irq128/ixl1:4                       0        0
irq129/ixl1:5                       0        0
irq130/ixl1:6                       0        0
irq131/ixl1:7                       8        0
irq132/mcx0                        35        0
irq133/mcx0:0                      11        0
irq134/mcx0:1                       0        0
irq135/mcx0:2                       0        0
irq136/mcx0:3                       0        0
irq137/mcx0:4                       0        0
irq138/mcx0:5                       0        0
irq139/mcx0:6                       0        0
irq140/mcx0:7                       0        0
irq141/mcx1                        39        0
irq142/mcx1:0                      11        0
irq143/mcx1:1                       0        0
irq144/mcx1:2                       0        0
irq145/mcx1:3                       0        0
irq146/mcx1:4                       0        0
irq147/mcx1:5                       0        0
irq148/mcx1:6                       0        0
irq149/mcx1:7                       0        0
irq150/ix0:0                       13        0
irq151/ix0:1                        0        0
irq152/ix0:2                        0        0
irq153/ix0:3                        0        0
irq154/ix0:4                        2        0
irq155/ix0:5                        0        0
irq156/ix0:6                        2        0
irq157/ix0:7                        0        0
irq158/ix0:8                        0        0
irq159/ix0:9                        0        0
irq160/ix0:10                       0        0
irq161/ix0:11                       0        0
irq162/ix0                          0        0
irq163/ix1:0                       13        0
irq164/ix1:1                        0        0
irq165/ix1:2                        2        0
irq166/ix1:3                        0        0
irq167/ix1:4                        2        0
irq168/ix1:5                        0        0
irq169/ix1:6                        0        0
irq170/ix1:7                        0        0
irq171/ix1:8                        0        0
irq172/ix1:9                        0        0
irq173/ix1:10                       0        0
irq174/ix1:11                       0        0
irq175/ix1                          0        0




dmesg for this one:
https://www.supermicro.com/en/products/system/1U/5018/SYS-5018D-FN8T.cfm


OpenBSD 6.8-current (GENERIC.MP) #120: Sun Oct 18 09:31:14 MDT 2020
    [hidden email]:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 17054588928 (16264MB)
avail mem = 16522625024 (15757MB)
random: good seed from bootblocks
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.8 @ 0xed9b0 (47 entries)
bios0: vendor American Megatrends Inc. version "2.1" date 11/08/2019
bios0: Supermicro Super Server
acpi0 at bios0: ACPI 5.0
acpi0: sleep states S0 S4 S5
acpi0: tables DSDT FACP APIC FPDT FIDT SPMI MCFG UEFI DBG2 HPET WDDT
SSDT SSDT SSDT PRAD DMAR HEST BERT ERST EINJ
acpi0: wakeup devices IP2P(S4) EHC1(S4) EHC2(S4) RP07(S4) RP08(S4)
BR1A(S4) BR1B(S4) BR2A(S4) BR2B(S4) BR2C(S4) BR2D(S4) BR3A(S4) BR3B(S4)
BR3C(S4) BR3D(S4) RP01(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz, 2200.30 MHz, 06-56-03
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.2, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz, 2200.01 MHz, 06-56-03
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz, 2200.01 MHz, 06-56-03
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 6 (application processor)
cpu3: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz, 2200.01 MHz, 06-56-03
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu3: 256KB 64b/line 8-way L2 cache
cpu3: smt 0, core 3, package 0
cpu4 at mainbus0: apid 1 (application processor)
cpu4: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz, 2200.01 MHz, 06-56-03
cpu4:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu4: 256KB 64b/line 8-way L2 cache
cpu4: smt 1, core 0, package 0
cpu5 at mainbus0: apid 3 (application processor)
cpu5: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz, 2200.01 MHz, 06-56-03
cpu5:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu5: 256KB 64b/line 8-way L2 cache
cpu5: smt 1, core 1, package 0
cpu6 at mainbus0: apid 5 (application processor)
cpu6: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz, 2200.01 MHz, 06-56-03
cpu6:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu6: 256KB 64b/line 8-way L2 cache
cpu6: smt 1, core 2, package 0
cpu7 at mainbus0: apid 7 (application processor)
cpu7: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz, 2200.01 MHz, 06-56-03
cpu7:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu7: 256KB 64b/line 8-way L2 cache
cpu7: smt 1, core 3, package 0
ioapic0 at mainbus0: apid 8 pa 0xfec00000, version 20, 24 pins
acpimcfg0 at acpi0
acpimcfg0: addr 0x80000000, bus 0-255
acpihpet0 at acpi0: 14318179 Hz
acpiprt0 at acpi0: bus 255 (UNC0)
acpiprt1 at acpi0: bus 0 (PCI0)
acpiprt2 at acpi0: bus 1 (BR1A)
acpiprt3 at acpi0: bus 2 (BR1B)
acpiprt4 at acpi0: bus 4 (BR2C)
acpiprt5 at acpi0: bus 5 (BR3A)
acpiprt6 at acpi0: bus 7 (RP01)
acpiprt7 at acpi0: bus 8 (RP02)
acpiprt8 at acpi0: bus 9 (RP04)
acpiprt9 at acpi0: bus 10 (BR3A)
acpiprt10 at acpi0: bus 11 (RP05)
acpipci0 at acpi0 UNC0: 0x00000010 0x00000011 0x00000000
"ACPI0004" at acpi0 not configured
"ACPI0007" at acpi0 not configured
"ACPI0007" at acpi0 not configured
"ACPI0007" at acpi0 not configured
"ACPI0007" at acpi0 not configured
"ACPI0007" at acpi0 not configured
"ACPI0007" at acpi0 not configured
"ACPI0007" at acpi0 not configured
"ACPI0007" at acpi0 not configured
"PNP0C33" at acpi0 not configured
"PNP0C14" at acpi0 not configured
acpipci1 at acpi0 PCI0: 0x00000010 0x00000011 0x00000000
"PNP0003" at acpi0 not configured
"PNP0C14" at acpi0 not configured
acpicmos0 at acpi0
"IPI0001" at acpi0 not configured
acpibtn0 at acpi0: PWRB
acpicpu0 at acpi0: C1(1000@1 halt), PSS
acpicpu1 at acpi0: C1(1000@1 halt), PSS
acpicpu2 at acpi0: C1(1000@1 halt), PSS
acpicpu3 at acpi0: C1(1000@1 halt), PSS
acpicpu4 at acpi0: C1(1000@1 halt), PSS
acpicpu5 at acpi0: C1(1000@1 halt), PSS
acpicpu6 at acpi0: C1(1000@1 halt), PSS
acpicpu7 at acpi0: C1(1000@1 halt), PSS
ipmi at mainbus0 not configured
cpu0: using VERW MDS workaround (except on vmm entry)
cpu0: Enhanced SpeedStep 2200 MHz: speeds: 2200, 2100, 2000, 1900, 1800,
1700, 1600, 1500, 1400, 1300, 1200, 1100, 1000, 900, 800 MHz
pci0 at mainbus0 bus 255
"Intel Xeon-D QPI Link" rev 0x03 at pci0 dev 11 function 0 not configured
"Intel Xeon-D QPI Link" rev 0x03 at pci0 dev 11 function 1 not configured
"Intel Xeon-D QPI Link" rev 0x03 at pci0 dev 11 function 2 not configured
"Intel Xeon-D QPI Debug" rev 0x03 at pci0 dev 11 function 3 not configured
"Intel Xeon-D Cache" rev 0x03 at pci0 dev 12 function 0 not configured
"Intel Xeon-D Cache" rev 0x03 at pci0 dev 12 function 1 not configured
"Intel Xeon-D Cache" rev 0x03 at pci0 dev 12 function 2 not configured
"Intel Xeon-D Cache" rev 0x03 at pci0 dev 12 function 3 not configured
"Intel Xeon-D Cache" rev 0x03 at pci0 dev 15 function 0 not configured
"Intel Xeon-D Cache" rev 0x03 at pci0 dev 15 function 4 not configured
"Intel Xeon-D Cache" rev 0x03 at pci0 dev 15 function 5 not configured
"Intel Xeon-D Cache" rev 0x03 at pci0 dev 15 function 6 not configured
"Intel Xeon-D PCIE" rev 0x03 at pci0 dev 16 function 0 not configured
"Intel E5 v4 R2PCIe Agent" rev 0x03 at pci0 dev 16 function 1 not configured
"Intel Xeon-D Ubox" rev 0x03 at pci0 dev 16 function 5 not configured
"Intel Xeon-D Ubox" rev 0x03 at pci0 dev 16 function 6 not configured
"Intel Xeon-D Ubox" rev 0x03 at pci0 dev 16 function 7 not configured
"Intel Xeon-D Home Agent" rev 0x03 at pci0 dev 18 function 0 not configured
"Intel Xeon-D Home Agent" rev 0x03 at pci0 dev 18 function 1 not configured
"Intel Xeon-D Memory" rev 0x03 at pci0 dev 19 function 0 not configured
"Intel Xeon-D Memory" rev 0x03 at pci0 dev 19 function 1 not configured
"Intel Xeon-D Memory" rev 0x03 at pci0 dev 19 function 2 not configured
"Intel Xeon-D Memory" rev 0x03 at pci0 dev 19 function 3 not configured
"Intel Xeon-D Memory" rev 0x03 at pci0 dev 19 function 4 not configured
"Intel Xeon-D Memory" rev 0x03 at pci0 dev 19 function 5 not configured
"Intel Xeon-D Memory" rev 0x03 at pci0 dev 19 function 6 not configured
"Intel Xeon-D Memory" rev 0x03 at pci0 dev 19 function 7 not configured
"Intel Xeon-D Memory" rev 0x03 at pci0 dev 20 function 0 not configured
"Intel Xeon-D Memory" rev 0x03 at pci0 dev 20 function 1 not configured
"Intel Xeon-D Memory" rev 0x03 at pci0 dev 20 function 2 not configured
"Intel Xeon-D Memory" rev 0x03 at pci0 dev 20 function 3 not configured
"Intel Xeon-D Memory" rev 0x03 at pci0 dev 20 function 4 not configured
"Intel Xeon-D Memory" rev 0x03 at pci0 dev 20 function 5 not configured
"Intel Xeon-D Memory" rev 0x03 at pci0 dev 20 function 6 not configured
"Intel Xeon-D Memory" rev 0x03 at pci0 dev 20 function 7 not configured
"Intel Xeon-D Memory" rev 0x03 at pci0 dev 21 function 0 not configured
"Intel Xeon-D Memory" rev 0x03 at pci0 dev 21 function 1 not configured
"Intel Xeon-D Memory" rev 0x03 at pci0 dev 21 function 2 not configured
"Intel Xeon-D Memory" rev 0x03 at pci0 dev 21 function 3 not configured
"Intel Xeon-D PCU" rev 0x03 at pci0 dev 30 function 0 not configured
"Intel Xeon-D PCU" rev 0x03 at pci0 dev 30 function 1 not configured
"Intel Xeon-D PCU" rev 0x03 at pci0 dev 30 function 2 not configured
"Intel Xeon-D PCU" rev 0x03 at pci0 dev 30 function 3 not configured
"Intel Xeon-D PCU" rev 0x03 at pci0 dev 30 function 4 not configured
"Intel Xeon-D PCU" rev 0x03 at pci0 dev 31 function 0 not configured
"Intel Xeon-D PCU" rev 0x03 at pci0 dev 31 function 2 not configured
pci1 at mainbus0 bus 0
pchb0 at pci1 dev 0 function 0 "Intel Xeon-D Host" rev 0x03
ppb0 at pci1 dev 1 function 0 "Intel Xeon-D PCIE" rev 0x03
pci2 at ppb0 bus 1
ppb1 at pci1 dev 1 function 1 "Intel Xeon-D PCIE" rev 0x03
pci3 at ppb1 bus 2
ppb2 at pci1 dev 2 function 0 "Intel Xeon-D PCIE" rev 0x03
pci4 at ppb2 bus 3
"Intel Xeon-D QuickData" rev 0x00 at pci4 dev 0 function 0 not configured
"Intel Xeon-D QuickData" rev 0x00 at pci4 dev 0 function 1 not configured
"Intel Xeon-D QuickData" rev 0x00 at pci4 dev 0 function 2 not configured
"Intel Xeon-D QuickData" rev 0x00 at pci4 dev 0 function 3 not configured
ppb3 at pci1 dev 2 function 2 "Intel Xeon-D PCIE" rev 0x03
pci5 at ppb3 bus 4
ix0 at pci5 dev 0 function 0 "Intel X552 SFP+" rev 0x00, msix, 4 queues,
address ac:1f:6b:1c:db:9a
ix1 at pci5 dev 0 function 1 "Intel X552 SFP+" rev 0x00, msix, 4 queues,
address ac:1f:6b:1c:db:9b
ppb4 at pci1 dev 3 function 0 "Intel Xeon-D PCIE" rev 0x03: msi
pci6 at ppb4 bus 5
ppb5 at pci1 dev 3 function 2 "Intel Xeon-D PCIE" rev 0x03
pci7 at ppb5 bus 6
"Intel Xeon-D Address Map" rev 0x03 at pci1 dev 5 function 0 not configured
"Intel Xeon-D Hot Plug" rev 0x03 at pci1 dev 5 function 1 not configured
"Intel Xeon-D RAS" rev 0x03 at pci1 dev 5 function 2 not configured
xhci0 at pci1 dev 20 function 0 "Intel 8 Series xHCI" rev 0x05: msi,
xHCI 1.0
usb0 at xhci0: USB revision 3.0
uhub0 at usb0 configuration 1 interface 0 "Intel xHCI root hub" rev
3.00/1.00 addr 1
"Intel 8 Series MEI" rev 0x04 at pci1 dev 22 function 0 not configured
"Intel 8 Series MEI" rev 0x04 at pci1 dev 22 function 1 not configured
ehci0 at pci1 dev 26 function 0 "Intel 8 Series USB" rev 0x05: apic 8 int 18
usb1 at ehci0: USB revision 2.0
uhub1 at usb1 configuration 1 interface 0 "Intel EHCI root hub" rev
2.00/1.00 addr 1
ppb6 at pci1 dev 28 function 0 "Intel 8 Series PCIE" rev 0xd5: msi
pci8 at ppb6 bus 7
em0 at pci8 dev 0 function 0 "Intel I210" rev 0x03: msi, address
ac:1f:6b:1c:d9:6a
ppb7 at pci1 dev 28 function 1 "Intel 8 Series PCIE" rev 0xd5: msi
pci9 at ppb7 bus 8
em1 at pci9 dev 0 function 0 "Intel I210" rev 0x03: msi, address
ac:1f:6b:1c:d9:6b
ppb8 at pci1 dev 28 function 3 "Intel 8 Series PCIE" rev 0xd5: msi
pci10 at ppb8 bus 9
ppb9 at pci10 dev 0 function 0 "ASPEED Technology AST1150 PCI" rev 0x03
pci11 at ppb9 bus 10
vga1 at pci11 dev 0 function 0 "ASPEED Technology AST2000" rev 0x30
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
ppb10 at pci1 dev 28 function 4 "Intel 8 Series PCIE" rev 0xd5: msi
pci12 at ppb10 bus 11
em2 at pci12 dev 0 function 0 "Intel I350" rev 0x01: msi, address
ac:1f:6b:1c:d9:6c
em3 at pci12 dev 0 function 1 "Intel I350" rev 0x01: msi, address
ac:1f:6b:1c:d9:6d
em4 at pci12 dev 0 function 2 "Intel I350" rev 0x01: msi, address
ac:1f:6b:1c:d9:6e
em5 at pci12 dev 0 function 3 "Intel I350" rev 0x01: msi, address
ac:1f:6b:1c:d9:6f
ehci1 at pci1 dev 29 function 0 "Intel 8 Series USB" rev 0x05: apic 8 int 18
usb2 at ehci1: USB revision 2.0
uhub2 at usb2 configuration 1 interface 0 "Intel EHCI root hub" rev
2.00/1.00 addr 1
pcib0 at pci1 dev 31 function 0 "Intel C224 LPC" rev 0x05
ahci0 at pci1 dev 31 function 2 "Intel 8 Series AHCI" rev 0x05: msi,
AHCI 1.3
ahci0: port 4: 6.0Gb/s
scsibus1 at ahci0: 32 targets
sd0 at scsibus1 targ 4 lun 0: <ATA, INTEL SSDSCKKW25, LHF>
naa.55cd2e414de3a866
sd0: 244198MB, 512 bytes/sector, 500118192 sectors, thin
ichiic0 at pci1 dev 31 function 3 "Intel 8 Series SMBus" rev 0x05: apic
8 int 18
iic0 at ichiic0
"Intel 8 Series Thermal" rev 0x05 at pci1 dev 31 function 6 not configured
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5 irq 1 irq 12
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
vmm0 at mainbus0: VMX/EPT
uhub3 at uhub0 port 3 configuration 1 interface 0 "Genesys Logic USB2.0
Hub" rev 2.00/32.98 addr 2
uhub4 at uhub3 port 2 configuration 1 interface 0 "Genesys Logic USB2.0
Hub" rev 2.00/32.98 addr 3
uhub5 at uhub0 port 4 configuration 1 interface 0 "ATEN International
product 0x7000" rev 2.00/0.00 addr 4
uhidev0 at uhub5 port 1 configuration 1 interface 0 "ATEN International
product 0x2419" rev 1.10/1.00 addr 5
uhidev0: iclass 3/1
ukbd0 at uhidev0: 8 variable keys, 6 key codes
wskbd0 at ukbd0: console keyboard, using wsdisplay0
uhidev1 at uhub5 port 1 configuration 1 interface 1 "ATEN International
product 0x2419" rev 1.10/1.00 addr 5
uhidev1: iclass 3/1
ums0 at uhidev1: 3 buttons, Z dir
wsmouse0 at ums0 mux 0
uhub6 at uhub1 port 1 configuration 1 interface 0 "Intel Rate Matching
Hub" rev 2.00/0.05 addr 2
uhub7 at uhub2 port 1 configuration 1 interface 0 "Intel Rate Matching
Hub" rev 2.00/0.05 addr 2
vscsi0 at root
scsibus2 at vscsi0: 256 targets
softraid0 at root
scsibus3 at softraid0: 256 targets
root on sd0a (1837f33daf10d330.a) swap on sd0b dump on sd0b

Reply | Threaded
Open this post in threaded view
|

Re: Small/Mini 10Gbe Router Recommendation

Daniel Melameth
On Thu, Apr 8, 2021 at 1:52 PM Hrvoje Popovski <[hidden email]> wrote:

> On 8.4.2021. 20:56, Daniel Melameth wrote:
> > On Thu, Apr 8, 2021 at 3:57 AM Stuart Henderson <[hidden email]> wrote:
> >> On 2021-04-07, Daniel Melameth <[hidden email]> wrote:
> >>> Looking to finally part with my legacy OpenBSD router and upgrade to
> >>> something that can push more than 2Gbps out of a single port.  Since
> >>> my switching equipment is still only 1Gbe, I also want something that
> >>> has, at least, two Gbe ports.
> >>>
> >>> Any recommendations that work well with OpenBSD?  I am currently
> >>> thinking https://www.supermicro.com/en/products/system/Mini-ITX/SYS-E300-8D.cfm,
> >>> but would like other opinions.
>
> my thinking is that if you want to push 10G traffic you'll need at least
> 8 faster cores ..
> for now you won't be using them, but when multiqueue RSS forwarding is
> unlocked you will be happy ...
>
> this is vmstat -iz from 12 core box with ixl, mcx and ix

The dmesg you noted below is for a box with 4 cores, and I was hoping
to future proof a bit with that.  If I understand you correctly, you
are saying I'll need 12 cores to do 10Gbps eventually?  What bandwidth
are you getting out of the box with the dmesg below?

Thank you.

> irq114/ixl0                        27        0
> irq115/ixl0:0                       4        0
> irq116/ixl0:1                       0        0
> irq117/ixl0:2                       0        0
> irq118/ixl0:3                       0        0
> irq119/ixl0:4                       0        0
> irq120/ixl0:5                       0        0
> irq121/ixl0:6                       0        0
> irq122/ixl0:7                       8        0
> irq123/ixl1                        27        0
> irq124/ixl1:0                       4        0
> irq125/ixl1:1                       0        0
> irq126/ixl1:2                       0        0
> irq127/ixl1:3                       0        0
> irq128/ixl1:4                       0        0
> irq129/ixl1:5                       0        0
> irq130/ixl1:6                       0        0
> irq131/ixl1:7                       8        0
> irq132/mcx0                        35        0
> irq133/mcx0:0                      11        0
> irq134/mcx0:1                       0        0
> irq135/mcx0:2                       0        0
> irq136/mcx0:3                       0        0
> irq137/mcx0:4                       0        0
> irq138/mcx0:5                       0        0
> irq139/mcx0:6                       0        0
> irq140/mcx0:7                       0        0
> irq141/mcx1                        39        0
> irq142/mcx1:0                      11        0
> irq143/mcx1:1                       0        0
> irq144/mcx1:2                       0        0
> irq145/mcx1:3                       0        0
> irq146/mcx1:4                       0        0
> irq147/mcx1:5                       0        0
> irq148/mcx1:6                       0        0
> irq149/mcx1:7                       0        0
> irq150/ix0:0                       13        0
> irq151/ix0:1                        0        0
> irq152/ix0:2                        0        0
> irq153/ix0:3                        0        0
> irq154/ix0:4                        2        0
> irq155/ix0:5                        0        0
> irq156/ix0:6                        2        0
> irq157/ix0:7                        0        0
> irq158/ix0:8                        0        0
> irq159/ix0:9                        0        0
> irq160/ix0:10                       0        0
> irq161/ix0:11                       0        0
> irq162/ix0                          0        0
> irq163/ix1:0                       13        0
> irq164/ix1:1                        0        0
> irq165/ix1:2                        2        0
> irq166/ix1:3                        0        0
> irq167/ix1:4                        2        0
> irq168/ix1:5                        0        0
> irq169/ix1:6                        0        0
> irq170/ix1:7                        0        0
> irq171/ix1:8                        0        0
> irq172/ix1:9                        0        0
> irq173/ix1:10                       0        0
> irq174/ix1:11                       0        0
> irq175/ix1                          0        0
>
> dmesg for this one:
> https://www.supermicro.com/en/products/system/1U/5018/SYS-5018D-FN8T.cfm
>
> OpenBSD 6.8-current (GENERIC.MP) #120: Sun Oct 18 09:31:14 MDT 2020
>     [hidden email]:/usr/src/sys/arch/amd64/compile/GENERIC.MP
> real mem = 17054588928 (16264MB)
> avail mem = 16522625024 (15757MB)
> random: good seed from bootblocks
> mpath0 at root
> scsibus0 at mpath0: 256 targets
> mainbus0 at root
> bios0 at mainbus0: SMBIOS rev. 2.8 @ 0xed9b0 (47 entries)
> bios0: vendor American Megatrends Inc. version "2.1" date 11/08/2019
> bios0: Supermicro Super Server
> acpi0 at bios0: ACPI 5.0
> acpi0: sleep states S0 S4 S5
> acpi0: tables DSDT FACP APIC FPDT FIDT SPMI MCFG UEFI DBG2 HPET WDDT
> SSDT SSDT SSDT PRAD DMAR HEST BERT ERST EINJ
> acpi0: wakeup devices IP2P(S4) EHC1(S4) EHC2(S4) RP07(S4) RP08(S4)
> BR1A(S4) BR1B(S4) BR2A(S4) BR2B(S4) BR2C(S4) BR2D(S4) BR3A(S4) BR3B(S4)
> BR3C(S4) BR3D(S4) RP01(S4) [...]
> acpitimer0 at acpi0: 3579545 Hz, 24 bits
> acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
> cpu0 at mainbus0: apid 0 (boot processor)
> cpu0: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz, 2200.30 MHz, 06-56-03
> cpu0:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> cpu0: 256KB 64b/line 8-way L2 cache
> cpu0: smt 0, core 0, package 0
> mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
> cpu0: apic clock running at 99MHz
> cpu0: mwait min=64, max=64, C-substates=0.2.1.2, IBE
> cpu1 at mainbus0: apid 2 (application processor)
> cpu1: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz, 2200.01 MHz, 06-56-03
> cpu1:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> cpu1: 256KB 64b/line 8-way L2 cache
> cpu1: smt 0, core 1, package 0
> cpu2 at mainbus0: apid 4 (application processor)
> cpu2: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz, 2200.01 MHz, 06-56-03
> cpu2:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> cpu2: 256KB 64b/line 8-way L2 cache
> cpu2: smt 0, core 2, package 0
> cpu3 at mainbus0: apid 6 (application processor)
> cpu3: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz, 2200.01 MHz, 06-56-03
> cpu3:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> cpu3: 256KB 64b/line 8-way L2 cache
> cpu3: smt 0, core 3, package 0
> cpu4 at mainbus0: apid 1 (application processor)
> cpu4: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz, 2200.01 MHz, 06-56-03
> cpu4:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> cpu4: 256KB 64b/line 8-way L2 cache
> cpu4: smt 1, core 0, package 0
> cpu5 at mainbus0: apid 3 (application processor)
> cpu5: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz, 2200.01 MHz, 06-56-03
> cpu5:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> cpu5: 256KB 64b/line 8-way L2 cache
> cpu5: smt 1, core 1, package 0
> cpu6 at mainbus0: apid 5 (application processor)
> cpu6: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz, 2200.01 MHz, 06-56-03
> cpu6:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> cpu6: 256KB 64b/line 8-way L2 cache
> cpu6: smt 1, core 2, package 0
> cpu7 at mainbus0: apid 7 (application processor)
> cpu7: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz, 2200.01 MHz, 06-56-03
> cpu7:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> cpu7: 256KB 64b/line 8-way L2 cache
> cpu7: smt 1, core 3, package 0
> ioapic0 at mainbus0: apid 8 pa 0xfec00000, version 20, 24 pins
> acpimcfg0 at acpi0
> acpimcfg0: addr 0x80000000, bus 0-255
> acpihpet0 at acpi0: 14318179 Hz
> acpiprt0 at acpi0: bus 255 (UNC0)
> acpiprt1 at acpi0: bus 0 (PCI0)
> acpiprt2 at acpi0: bus 1 (BR1A)
> acpiprt3 at acpi0: bus 2 (BR1B)
> acpiprt4 at acpi0: bus 4 (BR2C)
> acpiprt5 at acpi0: bus 5 (BR3A)
> acpiprt6 at acpi0: bus 7 (RP01)
> acpiprt7 at acpi0: bus 8 (RP02)
> acpiprt8 at acpi0: bus 9 (RP04)
> acpiprt9 at acpi0: bus 10 (BR3A)
> acpiprt10 at acpi0: bus 11 (RP05)
> acpipci0 at acpi0 UNC0: 0x00000010 0x00000011 0x00000000
> "ACPI0004" at acpi0 not configured
> "ACPI0007" at acpi0 not configured
> "ACPI0007" at acpi0 not configured
> "ACPI0007" at acpi0 not configured
> "ACPI0007" at acpi0 not configured
> "ACPI0007" at acpi0 not configured
> "ACPI0007" at acpi0 not configured
> "ACPI0007" at acpi0 not configured
> "ACPI0007" at acpi0 not configured
> "PNP0C33" at acpi0 not configured
> "PNP0C14" at acpi0 not configured
> acpipci1 at acpi0 PCI0: 0x00000010 0x00000011 0x00000000
> "PNP0003" at acpi0 not configured
> "PNP0C14" at acpi0 not configured
> acpicmos0 at acpi0
> "IPI0001" at acpi0 not configured
> acpibtn0 at acpi0: PWRB
> acpicpu0 at acpi0: C1(1000@1 halt), PSS
> acpicpu1 at acpi0: C1(1000@1 halt), PSS
> acpicpu2 at acpi0: C1(1000@1 halt), PSS
> acpicpu3 at acpi0: C1(1000@1 halt), PSS
> acpicpu4 at acpi0: C1(1000@1 halt), PSS
> acpicpu5 at acpi0: C1(1000@1 halt), PSS
> acpicpu6 at acpi0: C1(1000@1 halt), PSS
> acpicpu7 at acpi0: C1(1000@1 halt), PSS
> ipmi at mainbus0 not configured
> cpu0: using VERW MDS workaround (except on vmm entry)
> cpu0: Enhanced SpeedStep 2200 MHz: speeds: 2200, 2100, 2000, 1900, 1800,
> 1700, 1600, 1500, 1400, 1300, 1200, 1100, 1000, 900, 800 MHz
> pci0 at mainbus0 bus 255
> "Intel Xeon-D QPI Link" rev 0x03 at pci0 dev 11 function 0 not configured
> "Intel Xeon-D QPI Link" rev 0x03 at pci0 dev 11 function 1 not configured
> "Intel Xeon-D QPI Link" rev 0x03 at pci0 dev 11 function 2 not configured
> "Intel Xeon-D QPI Debug" rev 0x03 at pci0 dev 11 function 3 not configured
> "Intel Xeon-D Cache" rev 0x03 at pci0 dev 12 function 0 not configured
> "Intel Xeon-D Cache" rev 0x03 at pci0 dev 12 function 1 not configured
> "Intel Xeon-D Cache" rev 0x03 at pci0 dev 12 function 2 not configured
> "Intel Xeon-D Cache" rev 0x03 at pci0 dev 12 function 3 not configured
> "Intel Xeon-D Cache" rev 0x03 at pci0 dev 15 function 0 not configured
> "Intel Xeon-D Cache" rev 0x03 at pci0 dev 15 function 4 not configured
> "Intel Xeon-D Cache" rev 0x03 at pci0 dev 15 function 5 not configured
> "Intel Xeon-D Cache" rev 0x03 at pci0 dev 15 function 6 not configured
> "Intel Xeon-D PCIE" rev 0x03 at pci0 dev 16 function 0 not configured
> "Intel E5 v4 R2PCIe Agent" rev 0x03 at pci0 dev 16 function 1 not configured
> "Intel Xeon-D Ubox" rev 0x03 at pci0 dev 16 function 5 not configured
> "Intel Xeon-D Ubox" rev 0x03 at pci0 dev 16 function 6 not configured
> "Intel Xeon-D Ubox" rev 0x03 at pci0 dev 16 function 7 not configured
> "Intel Xeon-D Home Agent" rev 0x03 at pci0 dev 18 function 0 not configured
> "Intel Xeon-D Home Agent" rev 0x03 at pci0 dev 18 function 1 not configured
> "Intel Xeon-D Memory" rev 0x03 at pci0 dev 19 function 0 not configured
> "Intel Xeon-D Memory" rev 0x03 at pci0 dev 19 function 1 not configured
> "Intel Xeon-D Memory" rev 0x03 at pci0 dev 19 function 2 not configured
> "Intel Xeon-D Memory" rev 0x03 at pci0 dev 19 function 3 not configured
> "Intel Xeon-D Memory" rev 0x03 at pci0 dev 19 function 4 not configured
> "Intel Xeon-D Memory" rev 0x03 at pci0 dev 19 function 5 not configured
> "Intel Xeon-D Memory" rev 0x03 at pci0 dev 19 function 6 not configured
> "Intel Xeon-D Memory" rev 0x03 at pci0 dev 19 function 7 not configured
> "Intel Xeon-D Memory" rev 0x03 at pci0 dev 20 function 0 not configured
> "Intel Xeon-D Memory" rev 0x03 at pci0 dev 20 function 1 not configured
> "Intel Xeon-D Memory" rev 0x03 at pci0 dev 20 function 2 not configured
> "Intel Xeon-D Memory" rev 0x03 at pci0 dev 20 function 3 not configured
> "Intel Xeon-D Memory" rev 0x03 at pci0 dev 20 function 4 not configured
> "Intel Xeon-D Memory" rev 0x03 at pci0 dev 20 function 5 not configured
> "Intel Xeon-D Memory" rev 0x03 at pci0 dev 20 function 6 not configured
> "Intel Xeon-D Memory" rev 0x03 at pci0 dev 20 function 7 not configured
> "Intel Xeon-D Memory" rev 0x03 at pci0 dev 21 function 0 not configured
> "Intel Xeon-D Memory" rev 0x03 at pci0 dev 21 function 1 not configured
> "Intel Xeon-D Memory" rev 0x03 at pci0 dev 21 function 2 not configured
> "Intel Xeon-D Memory" rev 0x03 at pci0 dev 21 function 3 not configured
> "Intel Xeon-D PCU" rev 0x03 at pci0 dev 30 function 0 not configured
> "Intel Xeon-D PCU" rev 0x03 at pci0 dev 30 function 1 not configured
> "Intel Xeon-D PCU" rev 0x03 at pci0 dev 30 function 2 not configured
> "Intel Xeon-D PCU" rev 0x03 at pci0 dev 30 function 3 not configured
> "Intel Xeon-D PCU" rev 0x03 at pci0 dev 30 function 4 not configured
> "Intel Xeon-D PCU" rev 0x03 at pci0 dev 31 function 0 not configured
> "Intel Xeon-D PCU" rev 0x03 at pci0 dev 31 function 2 not configured
> pci1 at mainbus0 bus 0
> pchb0 at pci1 dev 0 function 0 "Intel Xeon-D Host" rev 0x03
> ppb0 at pci1 dev 1 function 0 "Intel Xeon-D PCIE" rev 0x03
> pci2 at ppb0 bus 1
> ppb1 at pci1 dev 1 function 1 "Intel Xeon-D PCIE" rev 0x03
> pci3 at ppb1 bus 2
> ppb2 at pci1 dev 2 function 0 "Intel Xeon-D PCIE" rev 0x03
> pci4 at ppb2 bus 3
> "Intel Xeon-D QuickData" rev 0x00 at pci4 dev 0 function 0 not configured
> "Intel Xeon-D QuickData" rev 0x00 at pci4 dev 0 function 1 not configured
> "Intel Xeon-D QuickData" rev 0x00 at pci4 dev 0 function 2 not configured
> "Intel Xeon-D QuickData" rev 0x00 at pci4 dev 0 function 3 not configured
> ppb3 at pci1 dev 2 function 2 "Intel Xeon-D PCIE" rev 0x03
> pci5 at ppb3 bus 4
> ix0 at pci5 dev 0 function 0 "Intel X552 SFP+" rev 0x00, msix, 4 queues,
> address ac:1f:6b:1c:db:9a
> ix1 at pci5 dev 0 function 1 "Intel X552 SFP+" rev 0x00, msix, 4 queues,
> address ac:1f:6b:1c:db:9b
> ppb4 at pci1 dev 3 function 0 "Intel Xeon-D PCIE" rev 0x03: msi
> pci6 at ppb4 bus 5
> ppb5 at pci1 dev 3 function 2 "Intel Xeon-D PCIE" rev 0x03
> pci7 at ppb5 bus 6
> "Intel Xeon-D Address Map" rev 0x03 at pci1 dev 5 function 0 not configured
> "Intel Xeon-D Hot Plug" rev 0x03 at pci1 dev 5 function 1 not configured
> "Intel Xeon-D RAS" rev 0x03 at pci1 dev 5 function 2 not configured
> xhci0 at pci1 dev 20 function 0 "Intel 8 Series xHCI" rev 0x05: msi,
> xHCI 1.0
> usb0 at xhci0: USB revision 3.0
> uhub0 at usb0 configuration 1 interface 0 "Intel xHCI root hub" rev
> 3.00/1.00 addr 1
> "Intel 8 Series MEI" rev 0x04 at pci1 dev 22 function 0 not configured
> "Intel 8 Series MEI" rev 0x04 at pci1 dev 22 function 1 not configured
> ehci0 at pci1 dev 26 function 0 "Intel 8 Series USB" rev 0x05: apic 8 int 18
> usb1 at ehci0: USB revision 2.0
> uhub1 at usb1 configuration 1 interface 0 "Intel EHCI root hub" rev
> 2.00/1.00 addr 1
> ppb6 at pci1 dev 28 function 0 "Intel 8 Series PCIE" rev 0xd5: msi
> pci8 at ppb6 bus 7
> em0 at pci8 dev 0 function 0 "Intel I210" rev 0x03: msi, address
> ac:1f:6b:1c:d9:6a
> ppb7 at pci1 dev 28 function 1 "Intel 8 Series PCIE" rev 0xd5: msi
> pci9 at ppb7 bus 8
> em1 at pci9 dev 0 function 0 "Intel I210" rev 0x03: msi, address
> ac:1f:6b:1c:d9:6b
> ppb8 at pci1 dev 28 function 3 "Intel 8 Series PCIE" rev 0xd5: msi
> pci10 at ppb8 bus 9
> ppb9 at pci10 dev 0 function 0 "ASPEED Technology AST1150 PCI" rev 0x03
> pci11 at ppb9 bus 10
> vga1 at pci11 dev 0 function 0 "ASPEED Technology AST2000" rev 0x30
> wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
> wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
> ppb10 at pci1 dev 28 function 4 "Intel 8 Series PCIE" rev 0xd5: msi
> pci12 at ppb10 bus 11
> em2 at pci12 dev 0 function 0 "Intel I350" rev 0x01: msi, address
> ac:1f:6b:1c:d9:6c
> em3 at pci12 dev 0 function 1 "Intel I350" rev 0x01: msi, address
> ac:1f:6b:1c:d9:6d
> em4 at pci12 dev 0 function 2 "Intel I350" rev 0x01: msi, address
> ac:1f:6b:1c:d9:6e
> em5 at pci12 dev 0 function 3 "Intel I350" rev 0x01: msi, address
> ac:1f:6b:1c:d9:6f
> ehci1 at pci1 dev 29 function 0 "Intel 8 Series USB" rev 0x05: apic 8 int 18
> usb2 at ehci1: USB revision 2.0
> uhub2 at usb2 configuration 1 interface 0 "Intel EHCI root hub" rev
> 2.00/1.00 addr 1
> pcib0 at pci1 dev 31 function 0 "Intel C224 LPC" rev 0x05
> ahci0 at pci1 dev 31 function 2 "Intel 8 Series AHCI" rev 0x05: msi,
> AHCI 1.3
> ahci0: port 4: 6.0Gb/s
> scsibus1 at ahci0: 32 targets
> sd0 at scsibus1 targ 4 lun 0: <ATA, INTEL SSDSCKKW25, LHF>
> naa.55cd2e414de3a866
> sd0: 244198MB, 512 bytes/sector, 500118192 sectors, thin
> ichiic0 at pci1 dev 31 function 3 "Intel 8 Series SMBus" rev 0x05: apic
> 8 int 18
> iic0 at ichiic0
> "Intel 8 Series Thermal" rev 0x05 at pci1 dev 31 function 6 not configured
> isa0 at pcib0
> isadma0 at isa0
> com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
> com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
> pckbc0 at isa0 port 0x60/5 irq 1 irq 12
> pcppi0 at isa0 port 0x61
> spkr0 at pcppi0
> vmm0 at mainbus0: VMX/EPT
> uhub3 at uhub0 port 3 configuration 1 interface 0 "Genesys Logic USB2.0
> Hub" rev 2.00/32.98 addr 2
> uhub4 at uhub3 port 2 configuration 1 interface 0 "Genesys Logic USB2.0
> Hub" rev 2.00/32.98 addr 3
> uhub5 at uhub0 port 4 configuration 1 interface 0 "ATEN International
> product 0x7000" rev 2.00/0.00 addr 4
> uhidev0 at uhub5 port 1 configuration 1 interface 0 "ATEN International
> product 0x2419" rev 1.10/1.00 addr 5
> uhidev0: iclass 3/1
> ukbd0 at uhidev0: 8 variable keys, 6 key codes
> wskbd0 at ukbd0: console keyboard, using wsdisplay0
> uhidev1 at uhub5 port 1 configuration 1 interface 1 "ATEN International
> product 0x2419" rev 1.10/1.00 addr 5
> uhidev1: iclass 3/1
> ums0 at uhidev1: 3 buttons, Z dir
> wsmouse0 at ums0 mux 0
> uhub6 at uhub1 port 1 configuration 1 interface 0 "Intel Rate Matching
> Hub" rev 2.00/0.05 addr 2
> uhub7 at uhub2 port 1 configuration 1 interface 0 "Intel Rate Matching
> Hub" rev 2.00/0.05 addr 2
> vscsi0 at root
> scsibus2 at vscsi0: 256 targets
> softraid0 at root
> scsibus3 at softraid0: 256 targets
> root on sd0a (1837f33daf10d330.a) swap on sd0b dump on sd0b

Reply | Threaded
Open this post in threaded view
|

Re: Small/Mini 10Gbe Router Recommendation

Hrvoje Popovski
On 8.4.2021. 22:16, Daniel Melameth wrote:

> On Thu, Apr 8, 2021 at 1:52 PM Hrvoje Popovski <[hidden email]> wrote:
>> On 8.4.2021. 20:56, Daniel Melameth wrote:
>>> On Thu, Apr 8, 2021 at 3:57 AM Stuart Henderson <[hidden email]> wrote:
>>>> On 2021-04-07, Daniel Melameth <[hidden email]> wrote:
>>>>> Looking to finally part with my legacy OpenBSD router and upgrade to
>>>>> something that can push more than 2Gbps out of a single port.  Since
>>>>> my switching equipment is still only 1Gbe, I also want something that
>>>>> has, at least, two Gbe ports.
>>>>>
>>>>> Any recommendations that work well with OpenBSD?  I am currently
>>>>> thinking https://www.supermicro.com/en/products/system/Mini-ITX/SYS-E300-8D.cfm,
>>>>> but would like other opinions.
>>
>> my thinking is that if you want to push 10G traffic you'll need at least
>> 8 faster cores ..
>> for now you won't be using them, but when multiqueue RSS forwarding is
>> unlocked you will be happy ...
>>
>> this is vmstat -iz from 12 core box with ixl, mcx and ix
>
> The dmesg you noted below is for a box with 4 cores, and I was hoping
> to future proof a bit with that.  

dmesg below is from SYS-5018D-FN8T which is basically same box as
SYS-E300-8D only rackmount ..

If I understand you correctly, you
> are saying I'll need 12 cores to do 10Gbps eventually?  What bandwidth
> are you getting out of the box with the dmesg below?

no no, i'm not saying that ... i'm saying that if you want some 10G
router/firewall in the future, you will need more than 4 core, actually
i would suggest 8 or more faster cores with 1 NUMA domain

this vmstat -iz output shows that on 12 cores box only ix is using all
12 queues while mcx and ixl are using 8 queues ... there is explanation
of why 8 queues but i can't remember it .. power of 2 something
something multiqueue :)

regarding forwarding performance of SYS-5018D-FN8T, i can't test it
right now but i remember it was something around 800 or 900 kpps of
plain forwaring and half of that when pf is enabled ..


>> irq114/ixl0                        27        0
>> irq115/ixl0:0                       4        0
>> irq116/ixl0:1                       0        0
>> irq117/ixl0:2                       0        0
>> irq118/ixl0:3                       0        0
>> irq119/ixl0:4                       0        0
>> irq120/ixl0:5                       0        0
>> irq121/ixl0:6                       0        0
>> irq122/ixl0:7                       8        0
>> irq123/ixl1                        27        0
>> irq124/ixl1:0                       4        0
>> irq125/ixl1:1                       0        0
>> irq126/ixl1:2                       0        0
>> irq127/ixl1:3                       0        0
>> irq128/ixl1:4                       0        0
>> irq129/ixl1:5                       0        0
>> irq130/ixl1:6                       0        0
>> irq131/ixl1:7                       8        0
>> irq132/mcx0                        35        0
>> irq133/mcx0:0                      11        0
>> irq134/mcx0:1                       0        0
>> irq135/mcx0:2                       0        0
>> irq136/mcx0:3                       0        0
>> irq137/mcx0:4                       0        0
>> irq138/mcx0:5                       0        0
>> irq139/mcx0:6                       0        0
>> irq140/mcx0:7                       0        0
>> irq141/mcx1                        39        0
>> irq142/mcx1:0                      11        0
>> irq143/mcx1:1                       0        0
>> irq144/mcx1:2                       0        0
>> irq145/mcx1:3                       0        0
>> irq146/mcx1:4                       0        0
>> irq147/mcx1:5                       0        0
>> irq148/mcx1:6                       0        0
>> irq149/mcx1:7                       0        0
>> irq150/ix0:0                       13        0
>> irq151/ix0:1                        0        0
>> irq152/ix0:2                        0        0
>> irq153/ix0:3                        0        0
>> irq154/ix0:4                        2        0
>> irq155/ix0:5                        0        0
>> irq156/ix0:6                        2        0
>> irq157/ix0:7                        0        0
>> irq158/ix0:8                        0        0
>> irq159/ix0:9                        0        0
>> irq160/ix0:10                       0        0
>> irq161/ix0:11                       0        0
>> irq162/ix0                          0        0
>> irq163/ix1:0                       13        0
>> irq164/ix1:1                        0        0
>> irq165/ix1:2                        2        0
>> irq166/ix1:3                        0        0
>> irq167/ix1:4                        2        0
>> irq168/ix1:5                        0        0
>> irq169/ix1:6                        0        0
>> irq170/ix1:7                        0        0
>> irq171/ix1:8                        0        0
>> irq172/ix1:9                        0        0
>> irq173/ix1:10                       0        0
>> irq174/ix1:11                       0        0
>> irq175/ix1                          0        0
>>
>> dmesg for this one:
>> https://www.supermicro.com/en/products/system/1U/5018/SYS-5018D-FN8T.cfm
>>
>> OpenBSD 6.8-current (GENERIC.MP) #120: Sun Oct 18 09:31:14 MDT 2020
>>     [hidden email]:/usr/src/sys/arch/amd64/compile/GENERIC.MP
>> real mem = 17054588928 (16264MB)
>> avail mem = 16522625024 (15757MB)
>> random: good seed from bootblocks
>> mpath0 at root
>> scsibus0 at mpath0: 256 targets
>> mainbus0 at root
>> bios0 at mainbus0: SMBIOS rev. 2.8 @ 0xed9b0 (47 entries)
>> bios0: vendor American Megatrends Inc. version "2.1" date 11/08/2019
>> bios0: Supermicro Super Server
>> acpi0 at bios0: ACPI 5.0
>> acpi0: sleep states S0 S4 S5
>> acpi0: tables DSDT FACP APIC FPDT FIDT SPMI MCFG UEFI DBG2 HPET WDDT
>> SSDT SSDT SSDT PRAD DMAR HEST BERT ERST EINJ
>> acpi0: wakeup devices IP2P(S4) EHC1(S4) EHC2(S4) RP07(S4) RP08(S4)
>> BR1A(S4) BR1B(S4) BR2A(S4) BR2B(S4) BR2C(S4) BR2D(S4) BR3A(S4) BR3B(S4)
>> BR3C(S4) BR3D(S4) RP01(S4) [...]
>> acpitimer0 at acpi0: 3579545 Hz, 24 bits
>> acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
>> cpu0 at mainbus0: apid 0 (boot processor)
>> cpu0: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz, 2200.30 MHz, 06-56-03
>> cpu0:
>> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
>> cpu0: 256KB 64b/line 8-way L2 cache
>> cpu0: smt 0, core 0, package 0
>> mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
>> cpu0: apic clock running at 99MHz
>> cpu0: mwait min=64, max=64, C-substates=0.2.1.2, IBE
>> cpu1 at mainbus0: apid 2 (application processor)
>> cpu1: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz, 2200.01 MHz, 06-56-03
>> cpu1:
>> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
>> cpu1: 256KB 64b/line 8-way L2 cache
>> cpu1: smt 0, core 1, package 0
>> cpu2 at mainbus0: apid 4 (application processor)
>> cpu2: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz, 2200.01 MHz, 06-56-03
>> cpu2:
>> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
>> cpu2: 256KB 64b/line 8-way L2 cache
>> cpu2: smt 0, core 2, package 0
>> cpu3 at mainbus0: apid 6 (application processor)
>> cpu3: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz, 2200.01 MHz, 06-56-03
>> cpu3:
>> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
>> cpu3: 256KB 64b/line 8-way L2 cache
>> cpu3: smt 0, core 3, package 0
>> cpu4 at mainbus0: apid 1 (application processor)
>> cpu4: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz, 2200.01 MHz, 06-56-03
>> cpu4:
>> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
>> cpu4: 256KB 64b/line 8-way L2 cache
>> cpu4: smt 1, core 0, package 0
>> cpu5 at mainbus0: apid 3 (application processor)
>> cpu5: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz, 2200.01 MHz, 06-56-03
>> cpu5:
>> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
>> cpu5: 256KB 64b/line 8-way L2 cache
>> cpu5: smt 1, core 1, package 0
>> cpu6 at mainbus0: apid 5 (application processor)
>> cpu6: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz, 2200.01 MHz, 06-56-03
>> cpu6:
>> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
>> cpu6: 256KB 64b/line 8-way L2 cache
>> cpu6: smt 1, core 2, package 0
>> cpu7 at mainbus0: apid 7 (application processor)
>> cpu7: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz, 2200.01 MHz, 06-56-03
>> cpu7:
>> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
>> cpu7: 256KB 64b/line 8-way L2 cache
>> cpu7: smt 1, core 3, package 0
>> ioapic0 at mainbus0: apid 8 pa 0xfec00000, version 20, 24 pins
>> acpimcfg0 at acpi0
>> acpimcfg0: addr 0x80000000, bus 0-255
>> acpihpet0 at acpi0: 14318179 Hz
>> acpiprt0 at acpi0: bus 255 (UNC0)
>> acpiprt1 at acpi0: bus 0 (PCI0)
>> acpiprt2 at acpi0: bus 1 (BR1A)
>> acpiprt3 at acpi0: bus 2 (BR1B)
>> acpiprt4 at acpi0: bus 4 (BR2C)
>> acpiprt5 at acpi0: bus 5 (BR3A)
>> acpiprt6 at acpi0: bus 7 (RP01)
>> acpiprt7 at acpi0: bus 8 (RP02)
>> acpiprt8 at acpi0: bus 9 (RP04)
>> acpiprt9 at acpi0: bus 10 (BR3A)
>> acpiprt10 at acpi0: bus 11 (RP05)
>> acpipci0 at acpi0 UNC0: 0x00000010 0x00000011 0x00000000
>> "ACPI0004" at acpi0 not configured
>> "ACPI0007" at acpi0 not configured
>> "ACPI0007" at acpi0 not configured
>> "ACPI0007" at acpi0 not configured
>> "ACPI0007" at acpi0 not configured
>> "ACPI0007" at acpi0 not configured
>> "ACPI0007" at acpi0 not configured
>> "ACPI0007" at acpi0 not configured
>> "ACPI0007" at acpi0 not configured
>> "PNP0C33" at acpi0 not configured
>> "PNP0C14" at acpi0 not configured
>> acpipci1 at acpi0 PCI0: 0x00000010 0x00000011 0x00000000
>> "PNP0003" at acpi0 not configured
>> "PNP0C14" at acpi0 not configured
>> acpicmos0 at acpi0
>> "IPI0001" at acpi0 not configured
>> acpibtn0 at acpi0: PWRB
>> acpicpu0 at acpi0: C1(1000@1 halt), PSS
>> acpicpu1 at acpi0: C1(1000@1 halt), PSS
>> acpicpu2 at acpi0: C1(1000@1 halt), PSS
>> acpicpu3 at acpi0: C1(1000@1 halt), PSS
>> acpicpu4 at acpi0: C1(1000@1 halt), PSS
>> acpicpu5 at acpi0: C1(1000@1 halt), PSS
>> acpicpu6 at acpi0: C1(1000@1 halt), PSS
>> acpicpu7 at acpi0: C1(1000@1 halt), PSS
>> ipmi at mainbus0 not configured
>> cpu0: using VERW MDS workaround (except on vmm entry)
>> cpu0: Enhanced SpeedStep 2200 MHz: speeds: 2200, 2100, 2000, 1900, 1800,
>> 1700, 1600, 1500, 1400, 1300, 1200, 1100, 1000, 900, 800 MHz
>> pci0 at mainbus0 bus 255
>> "Intel Xeon-D QPI Link" rev 0x03 at pci0 dev 11 function 0 not configured
>> "Intel Xeon-D QPI Link" rev 0x03 at pci0 dev 11 function 1 not configured
>> "Intel Xeon-D QPI Link" rev 0x03 at pci0 dev 11 function 2 not configured
>> "Intel Xeon-D QPI Debug" rev 0x03 at pci0 dev 11 function 3 not configured
>> "Intel Xeon-D Cache" rev 0x03 at pci0 dev 12 function 0 not configured
>> "Intel Xeon-D Cache" rev 0x03 at pci0 dev 12 function 1 not configured
>> "Intel Xeon-D Cache" rev 0x03 at pci0 dev 12 function 2 not configured
>> "Intel Xeon-D Cache" rev 0x03 at pci0 dev 12 function 3 not configured
>> "Intel Xeon-D Cache" rev 0x03 at pci0 dev 15 function 0 not configured
>> "Intel Xeon-D Cache" rev 0x03 at pci0 dev 15 function 4 not configured
>> "Intel Xeon-D Cache" rev 0x03 at pci0 dev 15 function 5 not configured
>> "Intel Xeon-D Cache" rev 0x03 at pci0 dev 15 function 6 not configured
>> "Intel Xeon-D PCIE" rev 0x03 at pci0 dev 16 function 0 not configured
>> "Intel E5 v4 R2PCIe Agent" rev 0x03 at pci0 dev 16 function 1 not configured
>> "Intel Xeon-D Ubox" rev 0x03 at pci0 dev 16 function 5 not configured
>> "Intel Xeon-D Ubox" rev 0x03 at pci0 dev 16 function 6 not configured
>> "Intel Xeon-D Ubox" rev 0x03 at pci0 dev 16 function 7 not configured
>> "Intel Xeon-D Home Agent" rev 0x03 at pci0 dev 18 function 0 not configured
>> "Intel Xeon-D Home Agent" rev 0x03 at pci0 dev 18 function 1 not configured
>> "Intel Xeon-D Memory" rev 0x03 at pci0 dev 19 function 0 not configured
>> "Intel Xeon-D Memory" rev 0x03 at pci0 dev 19 function 1 not configured
>> "Intel Xeon-D Memory" rev 0x03 at pci0 dev 19 function 2 not configured
>> "Intel Xeon-D Memory" rev 0x03 at pci0 dev 19 function 3 not configured
>> "Intel Xeon-D Memory" rev 0x03 at pci0 dev 19 function 4 not configured
>> "Intel Xeon-D Memory" rev 0x03 at pci0 dev 19 function 5 not configured
>> "Intel Xeon-D Memory" rev 0x03 at pci0 dev 19 function 6 not configured
>> "Intel Xeon-D Memory" rev 0x03 at pci0 dev 19 function 7 not configured
>> "Intel Xeon-D Memory" rev 0x03 at pci0 dev 20 function 0 not configured
>> "Intel Xeon-D Memory" rev 0x03 at pci0 dev 20 function 1 not configured
>> "Intel Xeon-D Memory" rev 0x03 at pci0 dev 20 function 2 not configured
>> "Intel Xeon-D Memory" rev 0x03 at pci0 dev 20 function 3 not configured
>> "Intel Xeon-D Memory" rev 0x03 at pci0 dev 20 function 4 not configured
>> "Intel Xeon-D Memory" rev 0x03 at pci0 dev 20 function 5 not configured
>> "Intel Xeon-D Memory" rev 0x03 at pci0 dev 20 function 6 not configured
>> "Intel Xeon-D Memory" rev 0x03 at pci0 dev 20 function 7 not configured
>> "Intel Xeon-D Memory" rev 0x03 at pci0 dev 21 function 0 not configured
>> "Intel Xeon-D Memory" rev 0x03 at pci0 dev 21 function 1 not configured
>> "Intel Xeon-D Memory" rev 0x03 at pci0 dev 21 function 2 not configured
>> "Intel Xeon-D Memory" rev 0x03 at pci0 dev 21 function 3 not configured
>> "Intel Xeon-D PCU" rev 0x03 at pci0 dev 30 function 0 not configured
>> "Intel Xeon-D PCU" rev 0x03 at pci0 dev 30 function 1 not configured
>> "Intel Xeon-D PCU" rev 0x03 at pci0 dev 30 function 2 not configured
>> "Intel Xeon-D PCU" rev 0x03 at pci0 dev 30 function 3 not configured
>> "Intel Xeon-D PCU" rev 0x03 at pci0 dev 30 function 4 not configured
>> "Intel Xeon-D PCU" rev 0x03 at pci0 dev 31 function 0 not configured
>> "Intel Xeon-D PCU" rev 0x03 at pci0 dev 31 function 2 not configured
>> pci1 at mainbus0 bus 0
>> pchb0 at pci1 dev 0 function 0 "Intel Xeon-D Host" rev 0x03
>> ppb0 at pci1 dev 1 function 0 "Intel Xeon-D PCIE" rev 0x03
>> pci2 at ppb0 bus 1
>> ppb1 at pci1 dev 1 function 1 "Intel Xeon-D PCIE" rev 0x03
>> pci3 at ppb1 bus 2
>> ppb2 at pci1 dev 2 function 0 "Intel Xeon-D PCIE" rev 0x03
>> pci4 at ppb2 bus 3
>> "Intel Xeon-D QuickData" rev 0x00 at pci4 dev 0 function 0 not configured
>> "Intel Xeon-D QuickData" rev 0x00 at pci4 dev 0 function 1 not configured
>> "Intel Xeon-D QuickData" rev 0x00 at pci4 dev 0 function 2 not configured
>> "Intel Xeon-D QuickData" rev 0x00 at pci4 dev 0 function 3 not configured
>> ppb3 at pci1 dev 2 function 2 "Intel Xeon-D PCIE" rev 0x03
>> pci5 at ppb3 bus 4
>> ix0 at pci5 dev 0 function 0 "Intel X552 SFP+" rev 0x00, msix, 4 queues,
>> address ac:1f:6b:1c:db:9a
>> ix1 at pci5 dev 0 function 1 "Intel X552 SFP+" rev 0x00, msix, 4 queues,
>> address ac:1f:6b:1c:db:9b
>> ppb4 at pci1 dev 3 function 0 "Intel Xeon-D PCIE" rev 0x03: msi
>> pci6 at ppb4 bus 5
>> ppb5 at pci1 dev 3 function 2 "Intel Xeon-D PCIE" rev 0x03
>> pci7 at ppb5 bus 6
>> "Intel Xeon-D Address Map" rev 0x03 at pci1 dev 5 function 0 not configured
>> "Intel Xeon-D Hot Plug" rev 0x03 at pci1 dev 5 function 1 not configured
>> "Intel Xeon-D RAS" rev 0x03 at pci1 dev 5 function 2 not configured
>> xhci0 at pci1 dev 20 function 0 "Intel 8 Series xHCI" rev 0x05: msi,
>> xHCI 1.0
>> usb0 at xhci0: USB revision 3.0
>> uhub0 at usb0 configuration 1 interface 0 "Intel xHCI root hub" rev
>> 3.00/1.00 addr 1
>> "Intel 8 Series MEI" rev 0x04 at pci1 dev 22 function 0 not configured
>> "Intel 8 Series MEI" rev 0x04 at pci1 dev 22 function 1 not configured
>> ehci0 at pci1 dev 26 function 0 "Intel 8 Series USB" rev 0x05: apic 8 int 18
>> usb1 at ehci0: USB revision 2.0
>> uhub1 at usb1 configuration 1 interface 0 "Intel EHCI root hub" rev
>> 2.00/1.00 addr 1
>> ppb6 at pci1 dev 28 function 0 "Intel 8 Series PCIE" rev 0xd5: msi
>> pci8 at ppb6 bus 7
>> em0 at pci8 dev 0 function 0 "Intel I210" rev 0x03: msi, address
>> ac:1f:6b:1c:d9:6a
>> ppb7 at pci1 dev 28 function 1 "Intel 8 Series PCIE" rev 0xd5: msi
>> pci9 at ppb7 bus 8
>> em1 at pci9 dev 0 function 0 "Intel I210" rev 0x03: msi, address
>> ac:1f:6b:1c:d9:6b
>> ppb8 at pci1 dev 28 function 3 "Intel 8 Series PCIE" rev 0xd5: msi
>> pci10 at ppb8 bus 9
>> ppb9 at pci10 dev 0 function 0 "ASPEED Technology AST1150 PCI" rev 0x03
>> pci11 at ppb9 bus 10
>> vga1 at pci11 dev 0 function 0 "ASPEED Technology AST2000" rev 0x30
>> wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
>> wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
>> ppb10 at pci1 dev 28 function 4 "Intel 8 Series PCIE" rev 0xd5: msi
>> pci12 at ppb10 bus 11
>> em2 at pci12 dev 0 function 0 "Intel I350" rev 0x01: msi, address
>> ac:1f:6b:1c:d9:6c
>> em3 at pci12 dev 0 function 1 "Intel I350" rev 0x01: msi, address
>> ac:1f:6b:1c:d9:6d
>> em4 at pci12 dev 0 function 2 "Intel I350" rev 0x01: msi, address
>> ac:1f:6b:1c:d9:6e
>> em5 at pci12 dev 0 function 3 "Intel I350" rev 0x01: msi, address
>> ac:1f:6b:1c:d9:6f
>> ehci1 at pci1 dev 29 function 0 "Intel 8 Series USB" rev 0x05: apic 8 int 18
>> usb2 at ehci1: USB revision 2.0
>> uhub2 at usb2 configuration 1 interface 0 "Intel EHCI root hub" rev
>> 2.00/1.00 addr 1
>> pcib0 at pci1 dev 31 function 0 "Intel C224 LPC" rev 0x05
>> ahci0 at pci1 dev 31 function 2 "Intel 8 Series AHCI" rev 0x05: msi,
>> AHCI 1.3
>> ahci0: port 4: 6.0Gb/s
>> scsibus1 at ahci0: 32 targets
>> sd0 at scsibus1 targ 4 lun 0: <ATA, INTEL SSDSCKKW25, LHF>
>> naa.55cd2e414de3a866
>> sd0: 244198MB, 512 bytes/sector, 500118192 sectors, thin
>> ichiic0 at pci1 dev 31 function 3 "Intel 8 Series SMBus" rev 0x05: apic
>> 8 int 18
>> iic0 at ichiic0
>> "Intel 8 Series Thermal" rev 0x05 at pci1 dev 31 function 6 not configured
>> isa0 at pcib0
>> isadma0 at isa0
>> com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
>> com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
>> pckbc0 at isa0 port 0x60/5 irq 1 irq 12
>> pcppi0 at isa0 port 0x61
>> spkr0 at pcppi0
>> vmm0 at mainbus0: VMX/EPT
>> uhub3 at uhub0 port 3 configuration 1 interface 0 "Genesys Logic USB2.0
>> Hub" rev 2.00/32.98 addr 2
>> uhub4 at uhub3 port 2 configuration 1 interface 0 "Genesys Logic USB2.0
>> Hub" rev 2.00/32.98 addr 3
>> uhub5 at uhub0 port 4 configuration 1 interface 0 "ATEN International
>> product 0x7000" rev 2.00/0.00 addr 4
>> uhidev0 at uhub5 port 1 configuration 1 interface 0 "ATEN International
>> product 0x2419" rev 1.10/1.00 addr 5
>> uhidev0: iclass 3/1
>> ukbd0 at uhidev0: 8 variable keys, 6 key codes
>> wskbd0 at ukbd0: console keyboard, using wsdisplay0
>> uhidev1 at uhub5 port 1 configuration 1 interface 1 "ATEN International
>> product 0x2419" rev 1.10/1.00 addr 5
>> uhidev1: iclass 3/1
>> ums0 at uhidev1: 3 buttons, Z dir
>> wsmouse0 at ums0 mux 0
>> uhub6 at uhub1 port 1 configuration 1 interface 0 "Intel Rate Matching
>> Hub" rev 2.00/0.05 addr 2
>> uhub7 at uhub2 port 1 configuration 1 interface 0 "Intel Rate Matching
>> Hub" rev 2.00/0.05 addr 2
>> vscsi0 at root
>> scsibus2 at vscsi0: 256 targets
>> softraid0 at root
>> scsibus3 at softraid0: 256 targets
>> root on sd0a (1837f33daf10d330.a) swap on sd0b dump on sd0b
>