clocksource tsc sometimes not available within debian vm on OpenBSD 6.4

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

clocksource tsc sometimes not available within debian vm on OpenBSD 6.4

Robert Paschedag
Hi misc,

I'm running OpenBSD 6.4 stable on my notebook and wanted to play around
with docker within debian VMs.

1 created 3 debian (stretch) VMs (2nd and 3rd are just copies and
modified later).

I had some trouble getting docker (containerd) running. It was always
using 100% CPU and I didn't know, why. Found out it made many
"clock_gettime" syscalls and found a thread, that setting the
clocksource to "tsc" within the VM would improve performance.

And, indeed, setting the clocksource to "tsc" fixes the containerd problem.

But the main problem is, that sometimes the "tsc" clocksource is not
available within the VM.

Until now, only 2 of the 3 debian VMs have the "tsc" clocksource
available, if all 3 are running. If I stop one of the 2 VMs that have
"tsc" available and restart the one that did not have it, that one
*then* has the "tsc" available. But not always. So this looks somehow
pretty random to me.

So here is an example, when "tsc" is not available

root@debian2-vm:~# dmesg | grep tsc
[    0.000000] tsc: Fast TSC calibration failed
[    0.000000] tsc: Unable to calibrate against PIT
[    0.000000] tsc: No reference (HPET/PMTIMER) available
[    0.000000] tsc: Marking TSC unstable due to could not calculate TSC khz
root@debian2-vm:~#

And here it is (on the same VM after one or more reboots)

root@debian2-vm:~# dmesg | grep tsc
[    0.000000] tsc: Fast TSC calibration failed
[    0.000000] tsc: Using PIT calibration value
[    0.000000] tsc: Detected 15521.837 MHz processor
[    1.948708] clocksource: tsc: mask: 0xffffffffffffffff max_cycles:
0x1bf7a091b1e1, max_idle_ns: 881591361965 ns
[    1.948709] clocksource: Switched to clocksource tsc
[    2.797715] clocksource: timekeeping watchdog on CPU0: Marking
clocksource 'tsc' as unstable because the skew is too large:
[    2.797716] clocksource:                       'tsc' cs_now:
2cd3e49b586 cs_last: 2cbb8a34d2c mask: ffffffffffffffff
root@debian2-vm:~#

Is this known and possibly fixed in current?

Thanks,
Robert


Here is the dmesg

OpenBSD 6.4 (GENERIC.MP) #3: Thu Dec 20 19:19:32 CET 2018

[hidden email]:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 16708087808 (15934MB)
avail mem = 16192450560 (15442MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xdae9d000 (68 entries)
bios0: vendor LENOVO version "HEET34WW (1.15 )" date 07/02/2013
bios0: LENOVO 68852HG
acpi0 at bios0: rev 2
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP SSDT SSDT ASF! HPET APIC MCFG FPDT SSDT SSDT
UEFI UEFI POAT UEFI DBG2
acpi0: wakeup devices P0P1(S4) EHC1(S3) EHC2(S3) XHC_(S3) HDEF(S3)
PXSX(S4) RP01(S4) PXSX(S4) RP02(S4) PXSX(S4) RP03(S4) PXSX(S4) RP04(S4)
PXSX(S4) RP05(S4) PXSX(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpihpet0 at acpi0: 14318179 Hz
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM) i5-3230M CPU @ 2.60GHz, 2594.54 MHz, 06-3a-09
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.1.2, IBE
cpu1 at mainbus0: apid 1 (application processor)
cpu1: Intel(R) Core(TM) i5-3230M CPU @ 2.60GHz, 2594.12 MHz, 06-3a-09
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 1, core 0, package 0
cpu2 at mainbus0: apid 2 (application processor)
cpu2: Intel(R) Core(TM) i5-3230M CPU @ 2.60GHz, 2594.12 MHz, 06-3a-09
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 1, package 0
cpu3 at mainbus0: apid 3 (application processor)
cpu3: Intel(R) Core(TM) i5-3230M CPU @ 2.60GHz, 2594.12 MHz, 06-3a-09
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu3: 256KB 64b/line 8-way L2 cache
cpu3: smt 1, core 1, package 0
ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 20, 24 pins
acpimcfg0 at acpi0
acpimcfg0: addr 0xf8000000, bus 0-63
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus -1 (P0P1)
acpiprt2 at acpi0: bus 3 (RP01)
acpiprt3 at acpi0: bus 4 (RP02)
acpiprt4 at acpi0: bus -1 (RP03)
acpiprt5 at acpi0: bus 5 (RP04)
acpiprt6 at acpi0: bus -1 (RP05)
acpiprt7 at acpi0: bus -1 (RP06)
acpiprt8 at acpi0: bus -1 (RP07)
acpiprt9 at acpi0: bus -1 (RP08)
acpiprt10 at acpi0: bus -1 (PEG0)
acpiprt11 at acpi0: bus -1 (PEG1)
acpiprt12 at acpi0: bus -1 (PEG2)
acpiprt13 at acpi0: bus -1 (PEG3)
acpiec0 at acpi0
acpicpu0 at acpi0: C2(350@80 mwait.1@0x20), C1(1000@1 mwait.1), PSS
acpicpu1 at acpi0: C2(350@80 mwait.1@0x20), C1(1000@1 mwait.1), PSS
acpicpu2 at acpi0: C2(350@80 mwait.1@0x20), C1(1000@1 mwait.1), PSS
acpicpu3 at acpi0: C2(350@80 mwait.1@0x20), C1(1000@1 mwait.1), PSS
acpitz0 at acpi0: critical temperature is 100 degC
acpicmos0 at acpi0
acpithinkpad0 at acpi0
"PNP0C14" at acpi0 not configured
acpibat0 at acpi0: BAT0 model "45N1045" serial    99 type LION oem "SMP"
acpiac0 at acpi0: AC unit online
acpibtn0 at acpi0: PWRB
acpibtn1 at acpi0: LID0
acpivideo0 at acpi0: VID_
acpivout at acpivideo0 not configured
cpu0: Enhanced SpeedStep 2594 MHz: speeds: 2601, 2600, 2500, 2400, 2300,
2200, 2100, 2000, 1900, 1800, 1700, 1600, 1500, 1400, 1300, 1200 MHz
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel Core 3G Host" rev 0x09
inteldrm0 at pci0 dev 2 function 0 "Intel HD Graphics 4000" rev 0x09
drm0 at inteldrm0
inteldrm0: msi
inteldrm0: 1366x768, 32bpp
wsdisplay0 at inteldrm0 mux 1: console (std, vt100 emulation)
wsdisplay0: screen 1-5 added (std, vt100 emulation)
xhci0 at pci0 dev 20 function 0 "Intel 7 Series xHCI" rev 0x04: msi,
xHCI 1.0
usb0 at xhci0: USB revision 3.0
uhub0 at usb0 configuration 1 interface 0 "Intel xHCI root hub" rev
3.00/1.00 addr 1
"Intel 7 Series MEI" rev 0x04 at pci0 dev 22 function 0 not configured
ehci0 at pci0 dev 26 function 0 "Intel 7 Series USB" rev 0x04: apic 2 int 16
usb1 at ehci0: USB revision 2.0
uhub1 at usb1 configuration 1 interface 0 "Intel EHCI root hub" rev
2.00/1.00 addr 1
azalia0 at pci0 dev 27 function 0 "Intel 7 Series HD Audio" rev 0x04: msi
azalia0: codecs: Conexant CX20590, Intel/0x2806, using Conexant CX20590
audio0 at azalia0
ppb0 at pci0 dev 28 function 0 "Intel 7 Series PCIE" rev 0xc4: msi
pci1 at ppb0 bus 3
rtsx0 at pci1 dev 0 function 0 "Realtek RTS5229 Card Reader" rev 0x01: msi
sdmmc0 at rtsx0: 4-bit, dma
ppb1 at pci0 dev 28 function 1 "Intel 7 Series PCIE" rev 0xc4: msi
pci2 at ppb1 bus 4
iwn0 at pci2 dev 0 function 0 "Intel Centrino Wireless-N 2230" rev 0xc4:
msi, MIMO 2T2R, BGN, address 68:17:29:ec:ca:7f
ppb2 at pci0 dev 28 function 3 "Intel 7 Series PCIE" rev 0xc4: msi
pci3 at ppb2 bus 5
re0 at pci3 dev 0 function 0 "Realtek 8168" rev 0x07: RTL8168E/8111E-VL
(0x2c80), msi, address 28:d2:44:26:fa:e9
rgephy0 at re0 phy 7: RTL8169S/8110S/8211 PHY, rev. 5
ehci1 at pci0 dev 29 function 0 "Intel 7 Series USB" rev 0x04: apic 2 int 23
usb2 at ehci1: USB revision 2.0
uhub2 at usb2 configuration 1 interface 0 "Intel EHCI root hub" rev
2.00/1.00 addr 1
pcib0 at pci0 dev 31 function 0 "Intel HM77 LPC" rev 0x04
ahci0 at pci0 dev 31 function 2 "Intel 7 Series AHCI" rev 0x04: msi,
AHCI 1.3
ahci0: port 0: 6.0Gb/s
ahci0: port 2: 1.5Gb/s
scsibus1 at ahci0: 32 targets
sd0 at scsibus1 targ 0 lun 0: <ATA, Crucial_CT525MX3, M0C> SCSI3
0/direct fixed naa.500a075116e63f65
sd0: 500786MB, 512 bytes/sector, 1025610768 sectors, thin
cd0 at scsibus1 targ 2 lun 0: <HL-DT-ST, DVDRAM GU70N, LE20> ATAPI
5/cdrom removable
ichiic0 at pci0 dev 31 function 3 "Intel 7 Series SMBus" rev 0x04: apic
2 int 18
iic0 at ichiic0
spdmem0 at iic0 addr 0x50: 8GB DDR3 SDRAM PC3-12800 SO-DIMM
spdmem1 at iic0 addr 0x52: 8GB DDR3 SDRAM PC3-12800 SO-DIMM
isa0 at pcib0
isadma0 at isa0
pckbc0 at isa0 port 0x60/5 irq 1 irq 12
pckbd0 at pckbc0 (kbd slot)
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pms0 at pckbc0 (aux slot)
wsmouse0 at pms0 mux 0
wsmouse1 at pms0 mux 0
pms0: Synaptics clickpad, firmware 8.1, 0x1e2b1 0x940300
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
vmm0 at mainbus0: VMX/EPT
error: [drm:pid0:cpt_set_fifo_underrun_reporting] *ERROR* uncleared pch
fifo underrun on pch transcoder A
error: [drm:pid0:intel_pch_fifo_underrun_irq_handler] *ERROR* PCH
transcoder A FIFO underrun
uhub3 at uhub1 port 1 configuration 1 interface 0 "Intel Rate Matching
Hub" rev 2.00/0.00 addr 2
ugen0 at uhub3 port 3 "Intel product 0x07da" rev 2.00/78.69 addr 3
ugen1 at uhub3 port 4 "Validity Sensors VFS5011 Fingerprint Reader" rev
1.10/0.78 addr 4
uvideo0 at uhub3 port 6 configuration 1 interface 0 "Vimicro corp.
Integrated Camera" rev 2.00/39.89 addr 5
video0 at uvideo0
uhub4 at uhub2 port 1 configuration 1 interface 0 "Intel Rate Matching
Hub" rev 2.00/0.00 addr 2
vscsi0 at root
scsibus2 at vscsi0: 256 targets
softraid0 at root
scsibus3 at softraid0: 256 targets
root on sd0a (142ebbf3d95f9810.a) swap on sd0b dump on sd0b



Reply | Threaded
Open this post in threaded view
|

Re: clocksource tsc sometimes not available within debian vm on OpenBSD 6.4

Joe M
I have the same issue and have been using this driver. It sets the
correct time every 5 seconds. For this purpose, this solution is a
hack, but, I could not figure out a better solution.

https://github.com/voutilad/virtio_vmmci/issues/1

Also, I noticed that vm clock would be very slow. It loses 28 minutes
for every 30 minutes.

Another thing, watch out regarding tsc=unreliable kernel command line
option, the vm is getting unresponsive with it after an hour or so.

Reply | Threaded
Open this post in threaded view
|

Re: clocksource tsc sometimes not available within debian vm on OpenBSD 6.4

Dave Voutila-2
On Fri, Mar 8, 2019 at 1:55 PM Joe M <[hidden email]> wrote:
>
> I have the same issue and have been using this driver. It sets the
> correct time every 5 seconds. For this purpose, this solution is a
> hack, but, I could not figure out a better solution.
>
> https://github.com/voutilad/virtio_vmmci/issues/1

As the author of the driver, let me clarify that the behavior now
mimics the native vmmci(4) behavior in that it only syncs the clock
when told to by the host system, which corresponds to the virtual
mc146818 detected its own clock drift (see vmd(8) source code). In
practice, this usually happens when the host system comes out of
suspension/hibernation.

The driver is not meant to resolve major issues like chronic clock
drift that even ntp can't fix.

>
> Also, I noticed that vm clock would be very slow. It loses 28 minutes
> for every 30 minutes.
>
> Another thing, watch out regarding tsc=unreliable kernel command line
> option, the vm is getting unresponsive with it after an hour or so.
>

-Dave

Reply | Threaded
Open this post in threaded view
|

Re: clocksource tsc sometimes not available within debian vm on OpenBSD 6.4

Robert Paschedag
Am 8. März 2019 20:29:36 MEZ schrieb Dave Voutila <[hidden email]>:

>On Fri, Mar 8, 2019 at 1:55 PM Joe M <[hidden email]> wrote:
>>
>> I have the same issue and have been using this driver. It sets the
>> correct time every 5 seconds. For this purpose, this solution is a
>> hack, but, I could not figure out a better solution.
>>
>> https://github.com/voutilad/virtio_vmmci/issues/1
>
>As the author of the driver, let me clarify that the behavior now
>mimics the native vmmci(4) behavior in that it only syncs the clock
>when told to by the host system, which corresponds to the virtual
>mc146818 detected its own clock drift (see vmd(8) source code). In
>practice, this usually happens when the host system comes out of
>suspension/hibernation.
>
>The driver is not meant to resolve major issues like chronic clock
>drift that even ntp can't fix.

The drift is not my primary problem. Yes, the drift is still large but it really solves the problem that "containerd" does not eat 100% CPU. It was unusable before.

But I'll keep that in mind.

Thanks
Robert

>
>>
>> Also, I noticed that vm clock would be very slow. It loses 28 minutes
>> for every 30 minutes.
>>
>> Another thing, watch out regarding tsc=unreliable kernel command line
>> option, the vm is getting unresponsive with it after an hour or so.
>>
>
>-Dave


--
sent from my mobile device