Virtualize or bare-metal?

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Virtualize or bare-metal?

LeviaComm Networks NOC
I have recently inherited a set of high-spec machines that I intend to
use for OpenBSD.  I am planning on using these machines for DNS, HTTP,
mail, LDAP, netboot, build system for following -stable, etc.  So my
question is, is it recommended to load all these
services on a single instance OpenBSD running on bare metal or to
virtualize and use much smaller OpenBSD virtual machines?

If the recommendation is to virtualize, what platform should I use?







The dmesg of the systems I'll be using:

OpenBSD 5.4 (GENERIC.MP) #41: Tue Jul 30 15:30:02 MDT 2013
     [hidden email]:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 34314244096 (32724MB)
avail mem = 33393082368 (31846MB)
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xeb4c0 (57 entries)
bios0: vendor American Megatrends Inc. version "2.0b" date 09/17/2012
bios0: Supermicro X9SCD
acpi0 at bios0: rev 2
acpi0: sleep states S0 S1 S4 S5
acpi0: tables DSDT FACP APIC FPDT MCFG HPET SSDT PRAD SPMI SSDT SSDT DMAR
acpi0: wakeup devices PS2K(S4) PS2M(S4) UAR1(S4) P0P1(S4) USB1(S4)
USB2(S4) USB3(S4) USB4(S4) USB5(S4) USB6(S4) USB7(S4) PXSX(S4) RP01(S4)
PXSX(S4) RP02(S4) PXSX(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz, 3300.56 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
cpu0: apic clock running at 100MHz
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz, 3300.02 MHz
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz, 3300.02 MHz
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 6 (application processor)
cpu3: Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz, 3300.02 MHz
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS
cpu3: 256KB 64b/line 8-way L2 cache
cpu3: smt 0, core 3, package 0
cpu4 at mainbus0: apid 1 (application processor)
cpu4: Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz, 3300.02 MHz
cpu4:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS
cpu4: 256KB 64b/line 8-way L2 cache
cpu4: smt 1, core 0, package 0
cpu5 at mainbus0: apid 3 (application processor)
cpu5: Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz, 3300.02 MHz
cpu5:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS
cpu5: 256KB 64b/line 8-way L2 cache
cpu5: smt 1, core 1, package 0
cpu6 at mainbus0: apid 5 (application processor)
cpu6: Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz, 3300.02 MHz
cpu6:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS
cpu6: 256KB 64b/line 8-way L2 cache
cpu6: smt 1, core 2, package 0
cpu7 at mainbus0: apid 7 (application processor)
cpu7: Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz, 3300.02 MHz
cpu7:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS
cpu7: 256KB 64b/line 8-way L2 cache
cpu7: smt 1, core 3, package 0
ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 20, 24 pins
acpimcfg0 at acpi0 addr 0xe0000000, bus 0-255
acpihpet0 at acpi0: 14318179 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 2 (P0P1)
acpiprt2 at acpi0: bus -1 (RP01)
acpiprt3 at acpi0: bus -1 (RP02)
acpiprt4 at acpi0: bus -1 (RP03)
acpiprt5 at acpi0: bus -1 (RP04)
acpiprt6 at acpi0: bus -1 (RP05)
acpiprt7 at acpi0: bus -1 (RP06)
acpiprt8 at acpi0: bus -1 (RP07)
acpiprt9 at acpi0: bus -1 (RP08)
acpiprt10 at acpi0: bus 1 (PEG0)
acpiprt11 at acpi0: bus -1 (PEG1)
acpiprt12 at acpi0: bus -1 (PEG2)
acpiprt13 at acpi0: bus -1 (PEG3)
acpiec0 at acpi0: Failed to read resource settings
acpicpu0 at acpi0: C3, C1, PSS
acpicpu1 at acpi0: C3, C1, PSS
acpicpu2 at acpi0: C3, C1, PSS
acpicpu3 at acpi0: C3, C1, PSS
acpicpu4 at acpi0: C3, C1, PSS
acpicpu5 at acpi0: C3, C1, PSS
acpicpu6 at acpi0: C3, C1, PSS
acpicpu7 at acpi0: C3, C1, PSS
acpipwrres0 at acpi0: FN00
acpipwrres1 at acpi0: FN01
acpipwrres2 at acpi0: FN02
acpipwrres3 at acpi0: FN03
acpipwrres4 at acpi0: FN04
acpitz0 at acpi0: critical temperature is 106 degC
acpitz1 at acpi0: critical temperature is 106 degC
acpibat0 at acpi0: BAT0 not present
acpibat1 at acpi0: BAT1 not present
acpibat2 at acpi0: BAT2 not present
acpibtn0 at acpi0: PWRB
acpibtn1 at acpi0: LID0
acpivideo0 at acpi0: GFX0
acpivout0 at acpivideo0: DD02
ipmi at mainbus0 not configured
cpu0: Enhanced SpeedStep 3300 MHz: speeds: 3301, 3300, 3200, 3100, 2900,
2800, 2700, 2600, 2400, 2300, 2200, 2100, 2000, 1800, 1700, 1600 MHz
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 vendor "Intel", unknown product 0x0158
rev 0x09
ppb0 at pci0 dev 1 function 0 "Intel Xeon E3-1200v2 PCIE" rev 0x09: msi
pci1 at ppb0 bus 1
em0 at pci1 dev 0 function 0 "Intel 82580" rev 0x01: msi, address
00:25:90:c1:90:86
em1 at pci1 dev 0 function 1 "Intel 82580" rev 0x01: msi, address
00:25:90:c1:90:87
"Intel 6 Series MEI" rev 0x04 at pci0 dev 22 function 0 not configured
vendor "Intel", unknown product 0x1c3b (class communications subclass
miscellaneous, rev 0x04) at pci0 dev 22 function 1 not configured
ehci0 at pci0 dev 26 function 0 "Intel 6 Series USB" rev 0x05: apic 2 int 16
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 "Intel EHCI root hub" rev 2.00/1.00 addr 1
ehci1 at pci0 dev 29 function 0 "Intel 6 Series USB" rev 0x05: apic 2 int 23
usb1 at ehci1: USB revision 2.0
uhub1 at usb1 "Intel EHCI root hub" rev 2.00/1.00 addr 1
ppb1 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev 0xa5
pci2 at ppb1 bus 2
vga1 at pci2 dev 3 function 0 "Matrox MGA G200eW" rev 0x0a
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
pcib0 at pci0 dev 31 function 0 "Intel C204 LPC" rev 0x05
ahci0 at pci0 dev 31 function 2 "Intel 6 Series AHCI" rev 0x05: msi,
AHCI 1.3
scsibus0 at ahci0: 32 targets
sd0 at scsibus0 targ 0 lun 0: <ATA, ST1000NM0011, SN03> SCSI3 0/direct
fixed naa.5000c5004ffde646
sd0: 953869MB, 512 bytes/sector, 1953525168 sectors
sd1 at scsibus0 targ 1 lun 0: <ATA, ST1000NM0011, SN03> SCSI3 0/direct
fixed naa.5000c5004ffdcc10
sd1: 953869MB, 512 bytes/sector, 1953525168 sectors
ichiic0 at pci0 dev 31 function 3 "Intel 6 Series SMBus" rev 0x05: apic
2 int 18
iic0 at ichiic0
sdtemp0 at iic0 addr 0x18: stts2002
sdtemp1 at iic0 addr 0x19: stts2002
sdtemp2 at iic0 addr 0x1a: stts2002
sdtemp3 at iic0 addr 0x1b: stts2002
lm1 at iic0 addr 0x2c: W83627DHG
spdmem0 at iic0 addr 0x50: 8GB DDR3 SDRAM ECC PC3-12800 with thermal sensor
spdmem1 at iic0 addr 0x51: 8GB DDR3 SDRAM ECC PC3-12800 with thermal sensor
spdmem2 at iic0 addr 0x52: 8GB DDR3 SDRAM ECC PC3-12800 with thermal sensor
spdmem3 at iic0 addr 0x53: 8GB DDR3 SDRAM ECC PC3-12800 with thermal sensor
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
wbsio0 at isa0 port 0x2e/2: NCT6776F rev 0x33
lm2 at wbsio0 port 0xa30/8: NCT6776F
mtrr: Pentium Pro MTRR support
uhub2 at uhub0 port 1 "Intel Rate Matching Hub" rev 2.00/0.00 addr 2
uhidev0 at uhub2 port 2 configuration 1 interface 0 "Winbond Electronics
Corp Hermon USB hidmouse Device" rev 1.10/0.01 addr 3
uhidev0: iclass 3/1
ums0 at uhidev0: 3 buttons, Z dir
wsmouse0 at ums0 mux 0
uhidev1 at uhub2 port 2 configuration 1 interface 1 "Winbond Electronics
Corp Hermon USB hidmouse Device" rev 1.10/0.01 addr 3
uhidev1: iclass 3/1
ukbd0 at uhidev1: 8 variable keys, 6 key codes
wskbd1 at ukbd0 mux 1
wskbd1: connecting to wsdisplay0
uhub3 at uhub1 port 1 "Intel Rate Matching Hub" rev 2.00/0.00 addr 2
uhub4 at uhub3 port 2 "ATEN International product 0x7000" rev 1.10/1.00
addr 3
uhidev2 at uhub4 port 1 configuration 1 interface 0 "ATEN CS-1734A
V4.2.418" rev 1.10/1.00 addr 4
uhidev2: iclass 3/1
ukbd1 at uhidev2: 8 variable keys, 6 key codes
wskbd2 at ukbd1 mux 1
wskbd2: connecting to wsdisplay0
uhidev3 at uhub4 port 1 configuration 1 interface 1 "ATEN CS-1734A
V4.2.418" rev 1.10/1.00 addr 4
uhidev3: iclass 3/1
ums1 at uhidev3: 5 buttons, Z dir
wsmouse1 at ums1 mux 0
vscsi0 at root
scsibus1 at vscsi0: 256 targets
softraid0 at root
scsibus2 at softraid0: 256 targets
root on sd0a (d2322e15199ee423.a) swap on sd0b dump on sd0b

Reply | Threaded
Open this post in threaded view
|

Re: Virtualize or bare-metal?

L. V. Lammert
On Mon, 13 Jan 2014, Christopher Ahrens wrote:

> I have recently inherited a set of high-spec machines that I intend to
> use for OpenBSD.  I am planning on using these machines for DNS, HTTP,
> mail, LDAP, netboot, build system for following -stable, etc.  So my
> question is, is it recommended to load all these
> services on a single instance OpenBSD running on bare metal or to
> virtualize and use much smaller OpenBSD virtual machines?
>
It would be much better to use a set of small machines (we use older
Compaq 386s & 486s) for most of those servers, .. save the 'big iron' for
a web server where it might be beneficial.

Virtualization does not make sense for core services - higher chance of a
single failure taking down multiple services and security can be a
problem.

        Lee

Reply | Threaded
Open this post in threaded view
|

Re: Virtualize or bare-metal?

LeviaComm Networks NOC
L. V. Lammert wrote:

> On Mon, 13 Jan 2014, Christopher Ahrens wrote:
>
>> I have recently inherited a set of high-spec machines that I intend to
>> use for OpenBSD.  I am planning on using these machines for DNS, HTTP,
>> mail, LDAP, netboot, build system for following -stable, etc.  So my
>> question is, is it recommended to load all these
>> services on a single instance OpenBSD running on bare metal or to
>> virtualize and use much smaller OpenBSD virtual machines?
>>
> It would be much better to use a set of small machines (we use older
> Compaq 386s & 486s) for most of those servers, .. save the 'big iron' for
> a web server where it might be beneficial.
>
> Virtualization does not make sense for core services - higher chance of a
> single failure taking down multiple services and security can be a
> problem.
>
> Lee
>
>

Wish I could split everything off to physical, but all I have for space
for is a mini-rack that fits under my desk in my apartment. (hosting
services around here are insane, especially the $200+ per incident costs
if you need to do something.

Since I have 8 of these machines, I was planning on setting up
duplicated machines on each Virtualization host and carp across them.

Reply | Threaded
Open this post in threaded view
|

Re: Virtualize or bare-metal?

Jack Woehr-2
Christopher Ahrens wrote:
>
> Wish I could split everything off to physical, but all I have for space for is a mini-rack that fits under my desk in
> my apartment

Sounds like you have answered your own question!

--
Jack Woehr               # "We commonly say we have no time when,
Box 51, Golden CO 80402  #  of course, we have all that there is."
http://www.softwoehr.com # - James Mason, _The Art of Chess_, 1905

Reply | Threaded
Open this post in threaded view
|

Re: Virtualize or bare-metal?

LeviaComm Networks NOC
Jack Woehr wrote:
> Christopher Ahrens wrote:
>>
>> Wish I could split everything off to physical, but all I have for
>> space for is a mini-rack that fits under my desk in my apartment
>
> Sounds like you have answered your own question!
>

What I meant by bare-metal was if I should run a bunch of services on
the same installation of OpenBSD.

Reply | Threaded
Open this post in threaded view
|

Re: Virtualize or bare-metal?

Matthew Weigel
On 1/13/2014 9:11 PM, Christopher Ahrens wrote:

> Jack Woehr wrote:
>> Christopher Ahrens wrote:
>>>
>>> Wish I could split everything off to physical, but all I have for
>>> space for is a mini-rack that fits under my desk in my apartment
>>
>> Sounds like you have answered your own question!
>>
>
> What I meant by bare-metal was if I should run a bunch of services on the same
> installation of OpenBSD.

Well, hardware failures on a small pool of machines are still hardware
failures on a small pool of machines, whether you have virtual servers or not.

For security, chroot (especially with privilege separation) accomplishes a lot
of what virtualization claims to offer, with a much longer history of auditing
and better understood weaknesses.

It is usually easier, in my experience, to manage one system running many
services in individual chroot environments than to manage many (virtual)
systems.  Files in chroot environments will sometimes need to be updated when
you change the main system, but in my experience this is a much easier task to
identify and manage than applying those changes en masse to a collection of
virtual hosts.  Plus, there will be plenty of system updates to the main
system that don't need to trickle down to the chroot environments, but will
almost always need to be applied individually to each virtual host.

You may still want to physically separate some concerns if you have enough
machines (e.g., build machines vs. service machines, spreading out
disk-intensive services, etc.), but in general I don't think virtualization
will particularly help you.
--
 Matthew Weigel
 hacker
 unique & idempot . ent

Reply | Threaded
Open this post in threaded view
|

Re: Virtualize or bare-metal?

Giancarlo Razzolini-3
In reply to this post by LeviaComm Networks NOC
Em 14-01-2014 01:11, Christopher Ahrens escreveu:
>
> What I meant by bare-metal was if I should run a bunch of services on
> the same installation of OpenBSD.
>

I've run in the same physical space issue with my company servers and
didn't think twice to use virtualization. But, as pointed by others, you
could easily accommodate all your services into one openbsd server with
chroot's. But I disagree when they compare chroot directly with a vm
hypervisor, because there are many things it can do, that a chroot
can't. I've been using linux with qemu/kvm. Lots of pci devices
passthrough's that work like a charm (there are potential security
issues, worth noting). I believe that the other obvious choice is Xen. I
would not go with virtualbox. And Vmware is expensive. Qemu/kvm tights
nicely into the system so it's my choice. You should make your own choice.

Cheers,

--
Giancarlo Razzolini
GPG: 4096R/77B981BC

Reply | Threaded
Open this post in threaded view
|

Re: Virtualize or bare-metal?

LeviaComm Networks NOC
In reply to this post by Matthew Weigel
Matthew Weigel wrote:

> On 1/13/2014 9:11 PM, Christopher Ahrens wrote:
>> Jack Woehr wrote:
>>> Christopher Ahrens wrote:
>>>>
>>>> Wish I could split everything off to physical, but all I have for
>>>> space for is a mini-rack that fits under my desk in my apartment
>>>
>>> Sounds like you have answered your own question!
>>>
>>
>> What I meant by bare-metal was if I should run a bunch of services on the same
>> installation of OpenBSD.
>
> Well, hardware failures on a small pool of machines are still hardware
> failures on a small pool of machines, whether you have virtual servers or not.
>
> For security, chroot (especially with privilege separation) accomplishes a lot
> of what virtualization claims to offer, with a much longer history of auditing
> and better understood weaknesses.
>
> It is usually easier, in my experience, to manage one system running many
> services in individual chroot environments than to manage many (virtual)
> systems.  Files in chroot environments will sometimes need to be updated when
> you change the main system, but in my experience this is a much easier task to
> identify and manage than applying those changes en masse to a collection of
> virtual hosts.  Plus, there will be plenty of system updates to the main
> system that don't need to trickle down to the chroot environments, but will
> almost always need to be applied individually to each virtual host.
>
> You may still want to physically separate some concerns if you have enough
> machines (e.g., build machines vs. service machines, spreading out
> disk-intensive services, etc.), but in general I don't think virtualization
> will particularly help you.
>


OK, I think I'll try loading multiple services onto single machines, I'm
thinking that I could always just attach a bunch of carp interfaces (one
for each service) to the machine then if I want to move that service to
another machine (Virtual or physical) I just destroy the carp interface
and recreate it on the new one.

At this point my plan is to use a pair of machines for a specific
category (to allow for a machine failure or allow for update cycles with
no downtime), each pair would handle one of Public internet services
(external-facing DNS, Public Web server, SMTP filtering), internal
services (Internal DNS, LDAP, CA), or business applications (Wiki, Mail
Store / IMAP access, source code control).  The last two boxes used as
spare and to test virtualization options.

I am just not using a single machine for multiple roles (I cut my teeth
on Windows 2000/2003 and picked up some bad habits and obsolete advice)

Reply | Threaded
Open this post in threaded view
|

Re: Virtualize or bare-metal?

Chris M-2
I personally wouldn't advise using a single bare-metal machine just for
dhcp, a separate one for dns, a separate one for sendmail etc. Seems like a
huge waste of resources to me. My opinion is that you would fare better, as
was suggested earlier, to use some of the other bare-metal machines for
more intensive tasks like Apache. And, I always like to have a spare box or
two to experiment with different things on, so I would keep one just for
that if it were me. Virtualizing is great for testing and experimenting,
but sometimes you can't beat a real machine for that.


On Tue, Jan 14, 2014 at 12:50 AM, Christopher Ahrens <[hidden email]>wrote:

> Matthew Weigel wrote:
>
>> On 1/13/2014 9:11 PM, Christopher Ahrens wrote:
>>
>>> Jack Woehr wrote:
>>>
>>>> Christopher Ahrens wrote:
>>>>
>>>>>
>>>>> Wish I could split everything off to physical, but all I have for
>>>>> space for is a mini-rack that fits under my desk in my apartment
>>>>>
>>>>
>>>> Sounds like you have answered your own question!
>>>>
>>>>
>>> What I meant by bare-metal was if I should run a bunch of services on
>>> the same
>>> installation of OpenBSD.
>>>
>>
>> Well, hardware failures on a small pool of machines are still hardware
>> failures on a small pool of machines, whether you have virtual servers or
>> not.
>>
>> For security, chroot (especially with privilege separation) accomplishes
>> a lot
>> of what virtualization claims to offer, with a much longer history of
>> auditing
>> and better understood weaknesses.
>>
>> It is usually easier, in my experience, to manage one system running many
>> services in individual chroot environments than to manage many (virtual)
>> systems.  Files in chroot environments will sometimes need to be updated
>> when
>> you change the main system, but in my experience this is a much easier
>> task to
>> identify and manage than applying those changes en masse to a collection
>> of
>> virtual hosts.  Plus, there will be plenty of system updates to the main
>> system that don't need to trickle down to the chroot environments, but
>> will
>> almost always need to be applied individually to each virtual host.
>>
>> You may still want to physically separate some concerns if you have enough
>> machines (e.g., build machines vs. service machines, spreading out
>> disk-intensive services, etc.), but in general I don't think
>> virtualization
>> will particularly help you.
>>
>>
>
> OK, I think I'll try loading multiple services onto single machines, I'm
> thinking that I could always just attach a bunch of carp interfaces (one
> for each service) to the machine then if I want to move that service to
> another machine (Virtual or physical) I just destroy the carp interface and
> recreate it on the new one.
>
> At this point my plan is to use a pair of machines for a specific category
> (to allow for a machine failure or allow for update cycles with no
> downtime), each pair would handle one of Public internet services
> (external-facing DNS, Public Web server, SMTP filtering), internal services
> (Internal DNS, LDAP, CA), or business applications (Wiki, Mail Store / IMAP
> access, source code control).  The last two boxes used as spare and to test
> virtualization options.
>
> I am just not using a single machine for multiple roles (I cut my teeth on
> Windows 2000/2003 and picked up some bad habits and obsolete advice)

Reply | Threaded
Open this post in threaded view
|

Re: Virtualize or bare-metal?

Renaud Allard-2
In reply to this post by Giancarlo Razzolini-3
On 01/14/2014 05:49 AM, Giancarlo Razzolini wrote:

> Em 14-01-2014 01:11, Christopher Ahrens escreveu:
>>
>> What I meant by bare-metal was if I should run a bunch of services on
>> the same installation of OpenBSD.
>>
>
> I've run in the same physical space issue with my company servers and
> didn't think twice to use virtualization. But, as pointed by others, you
> could easily accommodate all your services into one openbsd server with
> chroot's. But I disagree when they compare chroot directly with a vm
> hypervisor, because there are many things it can do, that a chroot
> can't. I've been using linux with qemu/kvm. Lots of pci devices
> passthrough's that work like a charm (there are potential security
> issues, worth noting). I believe that the other obvious choice is Xen. I
> would not go with virtualbox. And Vmware is expensive. Qemu/kvm tights
> nicely into the system so it's my choice. You should make your own choice.
>
> Cheers,
>

To be fair, virtualizing stuff without a common shared storage is a
little bit useless. The biggest power of virtualization is to be able to
move VMs between physical hosts or even powering on physical hosts when
you need more power.

But security wise, just to cite Theo:
x86 virtualization is about basically placing another nearly full
kernel, full of new bugs, on top of a nasty x86 architecture which
barely has correct page protection. Then running your operating system
on the other side of this brand new pile of shit.

You are absolutely deluded, if not stupid, if you think that a worldwide
collection of software engineers who can't write operating systems or
applications without security holes, can then turn around and suddenly
write virtualization layers without security holes.

Reply | Threaded
Open this post in threaded view
|

Re: Virtualize or bare-metal?

Giancarlo Razzolini-3
Em 14-01-2014 06:49, Renaud Allard escreveu:

>
> To be fair, virtualizing stuff without a common shared storage is a
> little bit useless. The biggest power of virtualization is to be able
> to move VMs between physical hosts or even powering on physical hosts
> when you need more power.
>
> But security wise, just to cite Theo:
> x86 virtualization is about basically placing another nearly full
> kernel, full of new bugs, on top of a nasty x86 architecture which
> barely has correct page protection. Then running your operating system
> on the other side of this brand new pile of shit.
>
> You are absolutely deluded, if not stupid, if you think that a
> worldwide collection of software engineers who can't write operating
> systems or applications without security holes, can then turn around
> and suddenly write virtualization layers without security holes.
I've never said that virtualization is secure. Some recent work on the
field even prove that virtualization is almost impossible to do in a
secure way. See the Gal Diskin talk on 30c3. But the demand is ever
rising and power and cooling costs go along. That's why I use
virtualization, not only it provides a better usage of resources, but it
reduces the power bill. And in a world that is going to face more and
more blackouts in the near future, that is a great thing. I've reduced
my 10 server farm to just two, using 1/5 of the power that I used
before, and faster. You can easily improve the hardware of two machines.
But try improving the hardware of 10, spending the same amount of money.
That's why I didn't blink when choosing to virtualize everything.

Cheers,

--
Giancarlo Razzolini
GPG: 4096R/77B981BC