ldom.conf.5: clarify vcpu strides

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

ldom.conf.5: clarify vcpu strides

Klemens Nanni-2
They way strides work is everything but intuitive and the manual doesn't
really help;  I've had multiple hackers/users ask me how to use them.

`vcpu 8' assigns eight virtual CPUs to a domain.

`vcpu 8:2' allocates eight VCPUs two times but assigns eight VCPUs
only once, leaving the other eight allocated (read: unusable) but not
assigned to any domain.

`vcpu 8:3' would allocate 24 VCPUs and assign eight to a domain.

This multiplicative property is not obvious from the manual; the way I
read the current wording is `vcpu 8:2' allocating ten VCPUs and assign
eight, i.e. the stride being an additive count.

stsp brought this up and we came up with the following diff.
Feedback? OK?


Index: ldom.conf.5
===================================================================
RCS file: /cvs/src/usr.sbin/ldomctl/ldom.conf.5,v
retrieving revision 1.13
diff -u -p -r1.13 ldom.conf.5
--- ldom.conf.5 21 Feb 2020 19:39:28 -0000 1.13
+++ ldom.conf.5 2 Sep 2020 14:26:58 -0000
@@ -38,8 +38,13 @@ If no configuration for the primary doma
 all CPU and memory resources not used by any guest domains.
 .It Ic vcpu Ar number Ns Op : Ns Ar stride
 Declare the number of virtual CPUs assigned to a domain.
-Optionally a stride can be specified to allocate additional virtual CPUs
-but not assign them to a domain.
+Optionally a stride factor can be specified to allocate
+.Ar number
+virtual CPUs
+.Ar stride
+times but not assign more than
+.Ar number
+virtual CPUs to a domain, leaving the rest unassigned.
 This can be used to distribute virtual CPUs over the available CPU cores.
 .It Ic memory Ar bytes
 Declare the amount of memory assigned to a domain, in bytes.

Reply | Threaded
Open this post in threaded view
|

Re: ldom.conf.5: clarify vcpu strides

Stefan Sperling-5
On Wed, Sep 02, 2020 at 04:41:49PM +0200, Klemens Nanni wrote:

> They way strides work is everything but intuitive and the manual doesn't
> really help;  I've had multiple hackers/users ask me how to use them.
>
> `vcpu 8' assigns eight virtual CPUs to a domain.
>
> `vcpu 8:2' allocates eight VCPUs two times but assigns eight VCPUs
> only once, leaving the other eight allocated (read: unusable) but not
> assigned to any domain.
>
> `vcpu 8:3' would allocate 24 VCPUs and assign eight to a domain.
>
> This multiplicative property is not obvious from the manual; the way I
> read the current wording is `vcpu 8:2' allocating ten VCPUs and assign
> eight, i.e. the stride being an additive count.
>
> stsp brought this up and we came up with the following diff.
> Feedback? OK?

My problem with the existing wording is that it says a "stride" exists
but semantics are not specified. I ended up assuming stride corresponded
to the amount of unused CPUs (e.g. 8:4 would mean allocate 8 CPUs, and
leave 4 of those CPUs unused).
I could only find out how it actually works by reading code.

I would like to suggest an example for the EXAMPLES section which
illustrates how a suitable stride factor can be determined (divide the
number of desired "unused" cpus by the number of desired "used" cpus):

diff f2b93fb3c1fc6097f79ea2bfd28de3a4ab8b938b /usr/src
blob - 1d72ee44a4ab333146b004907e240dfd939f11c5
file + usr.sbin/ldomctl/ldom.conf.5
--- usr.sbin/ldomctl/ldom.conf.5
+++ usr.sbin/ldomctl/ldom.conf.5
@@ -112,6 +112,12 @@ domain "salmah" {
 .Pp
 On a machine with 32 cores and 64GB physical memory, this leaves 12 cores and
 58GB memory to the primary domain.
+.Pp
+In order to use 2 CPUs out of 8, leaving 6 CPUs allocated but unused,
+a stride factor of 3 must be used (6 divided by 2):
+.Bd -literal -offset indent
+ vcpu 2:3
+.Ed
 .Sh SEE ALSO
 .Xr eeprom 8 ,
 .Xr ldomctl 8 ,

Reply | Threaded
Open this post in threaded view
|

Re: ldom.conf.5: clarify vcpu strides

Klemens Nanni-2
On Wed, Sep 02, 2020 at 04:58:39PM +0200, Stefan Sperling wrote:
> I would like to suggest an example for the EXAMPLES section which
> illustrates how a suitable stride factor can be determined (divide the
> number of desired "unused" cpus by the number of desired "used" cpus):
We can do with an example, but to me yours does not read obvious enough.

Also, `vcpu' denotes *virtual* CPUs inside domains, not CPUs on the
machine, so "CPU" (without "V") reads off in your example and conflicts
with the otherwise consistent mentions of "virtual CPUs" in this manual.

Here's my last diff incl. an example which reads a tad clearer to me and
is placed in the EXAMPLES section instead.

Feedback? OK?


Index: ldom.conf.5
===================================================================
RCS file: /cvs/src/usr.sbin/ldomctl/ldom.conf.5,v
retrieving revision 1.13
diff -u -p -r1.13 ldom.conf.5
--- ldom.conf.5 21 Feb 2020 19:39:28 -0000 1.13
+++ ldom.conf.5 14 Sep 2020 17:51:39 -0000
@@ -38,8 +38,13 @@ If no configuration for the primary doma
 all CPU and memory resources not used by any guest domains.
 .It Ic vcpu Ar number Ns Op : Ns Ar stride
 Declare the number of virtual CPUs assigned to a domain.
-Optionally a stride can be specified to allocate additional virtual CPUs
-but not assign them to a domain.
+Optionally a stride factor can be specified to allocate
+.Ar number
+virtual CPUs
+.Ar stride
+times but not assign more than
+.Ar number
+virtual CPUs to a domain, leaving the rest unassigned.
 This can be used to distribute virtual CPUs over the available CPU cores.
 .It Ic memory Ar bytes
 Declare the amount of memory assigned to a domain, in bytes.
@@ -112,6 +117,20 @@ domain "salmah" {
 .Pp
 On a machine with 32 cores and 64GB physical memory, this leaves 12 cores and
 58GB memory to the primary domain.
+.Pp
+Use
+.Ar stride
+factors to distribute virtual CPUs:
+.Bd -literal -offset indent
+domain "sun" {
+ vcpu 2:4
+ memory 4G
+ vdisk "/home/sun/vdisk0"
+}
+.Ed
+On a machine with eight threads per physical core, this allocates four strides
+of two virtual CPUs to the guest domain but only assigns one stride to it, i.e.
+make it occupy an entire physical core while running on only two threads.
 .Sh SEE ALSO
 .Xr eeprom 8 ,
 .Xr ldomctl 8 ,