"max" field in "netstat -m" is ambiguous

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

"max" field in "netstat -m" is ambiguous

Mike Belopuhov-5
Hi,

After some changes in the way mbuf cluster pool limits are set up,
we have a situation where the "max" number doesn't reflect what it
used to and is ambiguous most of the time.  Right now I have:

  36/144/64 mbuf 2048 byte clusters in use (current/peak/max)
  0/45/120 mbuf 2112 byte clusters in use (current/peak/max)
  256/312/64 mbuf 4096 byte clusters in use (current/peak/max)
  0/40/64 mbuf 8192 byte clusters in use (current/peak/max)
  0/14/112 mbuf 9216 byte clusters in use (current/peak/max)
  0/30/80 mbuf 12288 byte clusters in use (current/peak/max)
  0/40/64 mbuf 16384 byte clusters in use (current/peak/max)
  0/40/64 mbuf 65536 byte clusters in use (current/peak/max)

Several users expressed their concern regarding this and I agree
that this was one of the important metrics that we used to look at.

Now that kern.maxclusters defines how much memory (in 2k chunks) in
total can be spent on (all) clusters, there's no well defined maximum
value for each individual pool as they share this global limit.  But
we shouldn't provide values that are misinterpreted by users.

Here's my take on how to improve the situation.

One line in the "netstat -m" output talks about memory usage:

  5748 Kbytes allocated to network (21% in use)

This shows how much backing memory has been allocated by all cluster
pools from the UVM and percentage of how much of it has been taken
out by pool_get operations.

I think we can extend this by adding an additional number for the
upper boundary (kern.maxclusters), like so:

  saru:usr.bin/netstat% ./obj/netstat -m
  539 mbufs in use:
  385 mbufs allocated to data
  13 mbufs allocated to packet headers
  141 mbufs allocated to socket names and addresses
  19/144 mbuf 2048 byte clusters in use (current/peak)
  0/45 mbuf 2112 byte clusters in use (current/peak)
  256/312 mbuf 4096 byte clusters in use (current/peak)
  0/48 mbuf 8192 byte clusters in use (current/peak)
  0/28 mbuf 9216 byte clusters in use (current/peak)
  0/40 mbuf 12288 byte clusters in use (current/peak)
  0/40 mbuf 16384 byte clusters in use (current/peak)
  0/40 mbuf 65536 byte clusters in use (current/peak)
  5876 out of 524288 Kbytes allocated to network (20% in use)
  0 requests for memory denied
  0 requests for memory delayed
  0 calls to protocol drain routines

I gather this isn't very friendly towards existing scripts parsing
this output, but YMMV.

diff --git usr.bin/netstat/mbuf.c usr.bin/netstat/mbuf.c
index f7970a57c32..701385b2e6b 100644
--- usr.bin/netstat/mbuf.c
+++ usr.bin/netstat/mbuf.c
@@ -86,11 +86,11 @@ bool seen[256]; /* "have we seen this type yet?" */
  */
 void
 mbpr(void)
 {
  unsigned long totmem, totused, totmbufs;
- int totpct;
+ int maxclusters, totpct;
  int i, mib[4], npools;
  struct kinfo_pool pool;
  struct mbtypes *mp;
  size_t size;
 
@@ -99,10 +99,20 @@ mbpr(void)
     "%s: unexpected change to mbstat; check source\n",
     __progname);
  return;
  }
 
+ mib[0] = CTL_KERN;
+ mib[1] = KERN_MAXCLUSTERS;
+ size = sizeof(maxclusters);
+
+ if (sysctl(mib, 2, &maxclusters, &size, NULL, 0) < 0) {
+ printf("Can't retrieve value of maxclusters from the "
+    "kernel: %s\n",  strerror(errno));
+ return;
+ }
+
  mib[0] = CTL_KERN;
  mib[1] = KERN_MBSTAT;
  size = sizeof(mbstat);
 
  if (sysctl(mib, 2, &mbstat, &size, NULL, 0) < 0) {
@@ -176,24 +186,23 @@ mbpr(void)
     plural(mbstat.m_mtypes[i]), i);
  }
  totmem = (mbpool.pr_npages * mbpool.pr_pgsize);
  totused = mbpool.pr_nout * mbpool.pr_size;
  for (i = 0; i < mclp; i++) {
- printf("%u/%lu/%lu mbuf %d byte clusters in use"
-    " (current/peak/max)\n",
+ printf("%u/%lu mbuf %d byte clusters in use"
+    " (current/peak)\n",
     mclpools[i].pr_nout,
     (unsigned long)
  (mclpools[i].pr_hiwat * mclpools[i].pr_itemsperpage),
-    (unsigned long)
- (mclpools[i].pr_maxpages * mclpools[i].pr_itemsperpage),
     mclpools[i].pr_size);
  totmem += (mclpools[i].pr_npages * mclpools[i].pr_pgsize);
  totused += mclpools[i].pr_nout * mclpools[i].pr_size;
  }
 
  totpct = (totmem == 0) ? 0 : (totused/(totmem / 100));
- printf("%lu Kbytes allocated to network (%d%% in use)\n",
-    totmem / 1024, totpct);
+ printf("%lu out of %lu Kbytes allocated to network (%d%% in use)\n",
+    totmem / 1024, (unsigned long)(maxclusters * MCLBYTES) / 1024,
+    totpct);
  printf("%lu requests for memory denied\n", mbstat.m_drops);
  printf("%lu requests for memory delayed\n", mbstat.m_wait);
  printf("%lu calls to protocol drain routines\n", mbstat.m_drain);
 }

Reply | Threaded
Open this post in threaded view
|

Re: "max" field in "netstat -m" is ambiguous

Todd C. Miller
On Wed, 25 Oct 2017 19:46:56 +0200, Mike Belopuhov wrote:

> I think we can extend this by adding an additional number for the
> upper boundary (kern.maxclusters), like so:
>
>   saru:usr.bin/netstat% ./obj/netstat -m
>   539 mbufs in use:
>   385 mbufs allocated to data
>   13 mbufs allocated to packet headers
>   141 mbufs allocated to socket names and addresses
>   19/144 mbuf 2048 byte clusters in use (current/peak)
>   0/45 mbuf 2112 byte clusters in use (current/peak)
>   256/312 mbuf 4096 byte clusters in use (current/peak)
>   0/48 mbuf 8192 byte clusters in use (current/peak)
>   0/28 mbuf 9216 byte clusters in use (current/peak)
>   0/40 mbuf 12288 byte clusters in use (current/peak)
>   0/40 mbuf 16384 byte clusters in use (current/peak)
>   0/40 mbuf 65536 byte clusters in use (current/peak)
>   5876 out of 524288 Kbytes allocated to network (20% in use)
>   0 requests for memory denied
>   0 requests for memory delayed
>   0 calls to protocol drain routines

That's definitely an improvement.  OK millert@

 - todd

Reply | Threaded
Open this post in threaded view
|

Re: "max" field in "netstat -m" is ambiguous

Claudio Jeker
On Wed, Oct 25, 2017 at 01:39:35PM -0600, Todd C. Miller wrote:

> On Wed, 25 Oct 2017 19:46:56 +0200, Mike Belopuhov wrote:
>
> > I think we can extend this by adding an additional number for the
> > upper boundary (kern.maxclusters), like so:
> >
> >   saru:usr.bin/netstat% ./obj/netstat -m
> >   539 mbufs in use:
> >   385 mbufs allocated to data
> >   13 mbufs allocated to packet headers
> >   141 mbufs allocated to socket names and addresses
> >   19/144 mbuf 2048 byte clusters in use (current/peak)
> >   0/45 mbuf 2112 byte clusters in use (current/peak)
> >   256/312 mbuf 4096 byte clusters in use (current/peak)
> >   0/48 mbuf 8192 byte clusters in use (current/peak)
> >   0/28 mbuf 9216 byte clusters in use (current/peak)
> >   0/40 mbuf 12288 byte clusters in use (current/peak)
> >   0/40 mbuf 16384 byte clusters in use (current/peak)
> >   0/40 mbuf 65536 byte clusters in use (current/peak)
> >   5876 out of 524288 Kbytes allocated to network (20% in use)
> >   0 requests for memory denied
> >   0 requests for memory delayed
> >   0 calls to protocol drain routines
>
> That's definitely an improvement.  OK millert@
>

The math for the percentage in use is doing something different at least
20% of 524288 is not 5876. AFAIK the percentage is calculated against the
pool size and not the maximum size.
Would be great if netstat could show the current and peak memory usage.

--
:wq Claudio

Reply | Threaded
Open this post in threaded view
|

Re: "max" field in "netstat -m" is ambiguous

Stuart Henderson
In reply to this post by Mike Belopuhov-5
On 2017/10/25 19:46, Mike Belopuhov wrote:
> I gather this isn't very friendly towards existing scripts parsing
> this output, but YMMV.

I don't see anything in ports that cares. The only thing that parses
netstat -m output at all seems to be some code in gnupg/libgcrypt/mcrypt
that uses it as (one among several) sources to seed RNG.

There are some things that do look at mbuf use for monitoring etc (symon
for example) but I think usually not per-interface.

Reply | Threaded
Open this post in threaded view
|

Re: "max" field in "netstat -m" is ambiguous

Stuart Henderson
On 2017/10/25 21:03, Stuart Henderson wrote:

> On 2017/10/25 19:46, Mike Belopuhov wrote:
> > I gather this isn't very friendly towards existing scripts parsing
> > this output, but YMMV.
>
> I don't see anything in ports that cares. The only thing that parses
> netstat -m output at all seems to be some code in gnupg/libgcrypt/mcrypt
> that uses it as (one among several) sources to seed RNG.
>
> There are some things that do look at mbuf use for monitoring etc (symon
> for example) but I think usually not per-interface.
>

...don't know what I was thinking about per-interface there.

Anyway at least symon is using sysctl for this, see
symon/platform/OpenBSD/sm_mbuf.c, so netstat -m changes won't affect it,
but the underlying problem making "max" useless probably affects it too.
(I don't use it any more..)

Reply | Threaded
Open this post in threaded view
|

Re: "max" field in "netstat -m" is ambiguous

Mike Belopuhov-5
In reply to this post by Claudio Jeker
On Wed, Oct 25, 2017 at 21:56 +0200, Claudio Jeker wrote:

> On Wed, Oct 25, 2017 at 01:39:35PM -0600, Todd C. Miller wrote:
> > On Wed, 25 Oct 2017 19:46:56 +0200, Mike Belopuhov wrote:
> >
> > > I think we can extend this by adding an additional number for the
> > > upper boundary (kern.maxclusters), like so:
> > >
> > >   saru:usr.bin/netstat% ./obj/netstat -m
> > >   539 mbufs in use:
> > >   385 mbufs allocated to data
> > >   13 mbufs allocated to packet headers
> > >   141 mbufs allocated to socket names and addresses
> > >   19/144 mbuf 2048 byte clusters in use (current/peak)
> > >   0/45 mbuf 2112 byte clusters in use (current/peak)
> > >   256/312 mbuf 4096 byte clusters in use (current/peak)
> > >   0/48 mbuf 8192 byte clusters in use (current/peak)
> > >   0/28 mbuf 9216 byte clusters in use (current/peak)
> > >   0/40 mbuf 12288 byte clusters in use (current/peak)
> > >   0/40 mbuf 16384 byte clusters in use (current/peak)
> > >   0/40 mbuf 65536 byte clusters in use (current/peak)
> > >   5876 out of 524288 Kbytes allocated to network (20% in use)
> > >   0 requests for memory denied
> > >   0 requests for memory delayed
> > >   0 calls to protocol drain routines
> >
> > That's definitely an improvement.  OK millert@
> >
>
> The math for the percentage in use is doing something different at least
> 20% of 524288 is not 5876. AFAIK the percentage is calculated against the
> pool size and not the maximum size.

Correct and I didn't say otherwise. I wrote:

  This shows how much backing memory has been allocated by all cluster
  pools from the UVM and percentage of how much of it has been taken
  out by pool_get operations.

It's 20% of 5876K that is in use.

> Would be great if netstat could show the current and peak memory usage.
>

Current is 5876.  Maximum is 524288.  Do you want to display them in
the x/y/z format?

  5876/xxxx/524288 Kbytes allocated to network, 20% in use (current/peak/max)

Something like this? Any other ideas?

Reply | Threaded
Open this post in threaded view
|

Re: "max" field in "netstat -m" is ambiguous

Claudio Jeker
On Wed, Oct 25, 2017 at 11:46:05PM +0200, Mike Belopuhov wrote:

> On Wed, Oct 25, 2017 at 21:56 +0200, Claudio Jeker wrote:
> > On Wed, Oct 25, 2017 at 01:39:35PM -0600, Todd C. Miller wrote:
> > > On Wed, 25 Oct 2017 19:46:56 +0200, Mike Belopuhov wrote:
> > >
> > > > I think we can extend this by adding an additional number for the
> > > > upper boundary (kern.maxclusters), like so:
> > > >
> > > >   saru:usr.bin/netstat% ./obj/netstat -m
> > > >   539 mbufs in use:
> > > >   385 mbufs allocated to data
> > > >   13 mbufs allocated to packet headers
> > > >   141 mbufs allocated to socket names and addresses
> > > >   19/144 mbuf 2048 byte clusters in use (current/peak)
> > > >   0/45 mbuf 2112 byte clusters in use (current/peak)
> > > >   256/312 mbuf 4096 byte clusters in use (current/peak)
> > > >   0/48 mbuf 8192 byte clusters in use (current/peak)
> > > >   0/28 mbuf 9216 byte clusters in use (current/peak)
> > > >   0/40 mbuf 12288 byte clusters in use (current/peak)
> > > >   0/40 mbuf 16384 byte clusters in use (current/peak)
> > > >   0/40 mbuf 65536 byte clusters in use (current/peak)
> > > >   5876 out of 524288 Kbytes allocated to network (20% in use)
> > > >   0 requests for memory denied
> > > >   0 requests for memory delayed
> > > >   0 calls to protocol drain routines
> > >
> > > That's definitely an improvement.  OK millert@
> > >
> >
> > The math for the percentage in use is doing something different at least
> > 20% of 524288 is not 5876. AFAIK the percentage is calculated against the
> > pool size and not the maximum size.
>
> Correct and I didn't say otherwise. I wrote:
>
>   This shows how much backing memory has been allocated by all cluster
>   pools from the UVM and percentage of how much of it has been taken
>   out by pool_get operations.
>
> It's 20% of 5876K that is in use.
>
> > Would be great if netstat could show the current and peak memory usage.
> >
>
> Current is 5876.  Maximum is 524288.  Do you want to display them in
> the x/y/z format?
>
>   5876/xxxx/524288 Kbytes allocated to network, 20% in use (current/peak/max)
>
> Something like this? Any other ideas?

I think that would be an improvement. I normally look for peak values. The
current is normally not interesting when tuning systems.
Maybe we can even drop the use percentage since it more confusing than
anything.

--
:wq Claudio

Reply | Threaded
Open this post in threaded view
|

Re: "max" field in "netstat -m" is ambiguous

Mike Belopuhov-5
On Thu, Oct 26, 2017 at 08:58 +0200, Claudio Jeker wrote:

> On Wed, Oct 25, 2017 at 11:46:05PM +0200, Mike Belopuhov wrote:
> > On Wed, Oct 25, 2017 at 21:56 +0200, Claudio Jeker wrote:
> > > Would be great if netstat could show the current and peak memory usage.
> > >
> >
> > Current is 5876.  Maximum is 524288.  Do you want to display them in
> > the x/y/z format?
> >
> >   5876/xxxx/524288 Kbytes allocated to network, 20% in use (current/peak/max)
> >
> > Something like this? Any other ideas?
>
> I think that would be an improvement. I normally look for peak values. The
> current is normally not interesting when tuning systems.
> Maybe we can even drop the use percentage since it more confusing than
> anything.
>

How about this then?

  saru:usr.bin/netstat% ./obj/netstat -m
  532 mbufs in use:
  379 mbufs allocated to data
  12 mbufs allocated to packet headers
  141 mbufs allocated to socket names and addresses
  18/208 mbuf 2048 byte clusters in use (current/peak)
  0/45 mbuf 2112 byte clusters in use (current/peak)
  256/320 mbuf 4096 byte clusters in use (current/peak)
  0/48 mbuf 8192 byte clusters in use (current/peak)
  0/42 mbuf 9216 byte clusters in use (current/peak)
  0/50 mbuf 12288 byte clusters in use (current/peak)
  0/48 mbuf 16384 byte clusters in use (current/peak)
  0/48 mbuf 65536 byte clusters in use (current/peak)
  5952/7236/524288 Kbytes allocated to network (current/peak/max)
  0 requests for memory denied
  0 requests for memory delayed
  0 calls to protocol drain routines

OK?

diff --git usr.bin/netstat/mbuf.c usr.bin/netstat/mbuf.c
index f7970a57c32..79c2c16a6c3 100644
--- usr.bin/netstat/mbuf.c
+++ usr.bin/netstat/mbuf.c
@@ -85,13 +85,12 @@ bool seen[256]; /* "have we seen this type yet?" */
  * Print mbuf statistics.
  */
 void
 mbpr(void)
 {
- unsigned long totmem, totused, totmbufs;
- int totpct;
- int i, mib[4], npools;
+ unsigned long totmem, totpeak, totmbufs;
+ int i, maxclusters, mib[4], npools;
  struct kinfo_pool pool;
  struct mbtypes *mp;
  size_t size;
 
  if (nmbtypes != 256) {
@@ -99,10 +98,20 @@ mbpr(void)
     "%s: unexpected change to mbstat; check source\n",
     __progname);
  return;
  }
 
+ mib[0] = CTL_KERN;
+ mib[1] = KERN_MAXCLUSTERS;
+ size = sizeof(maxclusters);
+
+ if (sysctl(mib, 2, &maxclusters, &size, NULL, 0) < 0) {
+ printf("Can't retrieve value of maxclusters from the "
+    "kernel: %s\n",  strerror(errno));
+ return;
+ }
+
  mib[0] = CTL_KERN;
  mib[1] = KERN_MBSTAT;
  size = sizeof(mbstat);
 
  if (sysctl(mib, 2, &mbstat, &size, NULL, 0) < 0) {
@@ -174,26 +183,24 @@ mbpr(void)
  printf("\t%u mbuf%s allocated to <mbuf type %d>\n",
     mbstat.m_mtypes[i],
     plural(mbstat.m_mtypes[i]), i);
  }
  totmem = (mbpool.pr_npages * mbpool.pr_pgsize);
- totused = mbpool.pr_nout * mbpool.pr_size;
+ totpeak = mbpool.pr_hiwat * mbpool.pr_pgsize;
  for (i = 0; i < mclp; i++) {
- printf("%u/%lu/%lu mbuf %d byte clusters in use"
-    " (current/peak/max)\n",
+ printf("%u/%lu mbuf %d byte clusters in use"
+    " (current/peak)\n",
     mclpools[i].pr_nout,
     (unsigned long)
  (mclpools[i].pr_hiwat * mclpools[i].pr_itemsperpage),
-    (unsigned long)
- (mclpools[i].pr_maxpages * mclpools[i].pr_itemsperpage),
     mclpools[i].pr_size);
  totmem += (mclpools[i].pr_npages * mclpools[i].pr_pgsize);
- totused += mclpools[i].pr_nout * mclpools[i].pr_size;
+ totpeak += mclpools[i].pr_hiwat * mclpools[i].pr_pgsize;
  }
 
- totpct = (totmem == 0) ? 0 : (totused/(totmem / 100));
- printf("%lu Kbytes allocated to network (%d%% in use)\n",
-    totmem / 1024, totpct);
+ printf("%lu/%lu/%lu Kbytes allocated to network "
+    "(current/peak/max)\n", totmem / 1024, totpeak / 1024,
+    (unsigned long)(maxclusters * MCLBYTES) / 1024);
  printf("%lu requests for memory denied\n", mbstat.m_drops);
  printf("%lu requests for memory delayed\n", mbstat.m_wait);
  printf("%lu calls to protocol drain routines\n", mbstat.m_drain);
 }

Reply | Threaded
Open this post in threaded view
|

Re: "max" field in "netstat -m" is ambiguous

Claudio Jeker
On Sat, Oct 28, 2017 at 11:06:16AM +0200, Mike Belopuhov wrote:

> On Thu, Oct 26, 2017 at 08:58 +0200, Claudio Jeker wrote:
> > On Wed, Oct 25, 2017 at 11:46:05PM +0200, Mike Belopuhov wrote:
> > > On Wed, Oct 25, 2017 at 21:56 +0200, Claudio Jeker wrote:
> > > > Would be great if netstat could show the current and peak memory usage.
> > > >
> > >
> > > Current is 5876.  Maximum is 524288.  Do you want to display them in
> > > the x/y/z format?
> > >
> > >   5876/xxxx/524288 Kbytes allocated to network, 20% in use (current/peak/max)
> > >
> > > Something like this? Any other ideas?
> >
> > I think that would be an improvement. I normally look for peak values. The
> > current is normally not interesting when tuning systems.
> > Maybe we can even drop the use percentage since it more confusing than
> > anything.
> >
>
> How about this then?
>
>   saru:usr.bin/netstat% ./obj/netstat -m
>   532 mbufs in use:
>   379 mbufs allocated to data
>   12 mbufs allocated to packet headers
>   141 mbufs allocated to socket names and addresses
>   18/208 mbuf 2048 byte clusters in use (current/peak)
>   0/45 mbuf 2112 byte clusters in use (current/peak)
>   256/320 mbuf 4096 byte clusters in use (current/peak)
>   0/48 mbuf 8192 byte clusters in use (current/peak)
>   0/42 mbuf 9216 byte clusters in use (current/peak)
>   0/50 mbuf 12288 byte clusters in use (current/peak)
>   0/48 mbuf 16384 byte clusters in use (current/peak)
>   0/48 mbuf 65536 byte clusters in use (current/peak)
>   5952/7236/524288 Kbytes allocated to network (current/peak/max)
>   0 requests for memory denied
>   0 requests for memory delayed
>   0 calls to protocol drain routines
>
> OK?

I like it. Ok claudio@
 

> diff --git usr.bin/netstat/mbuf.c usr.bin/netstat/mbuf.c
> index f7970a57c32..79c2c16a6c3 100644
> --- usr.bin/netstat/mbuf.c
> +++ usr.bin/netstat/mbuf.c
> @@ -85,13 +85,12 @@ bool seen[256]; /* "have we seen this type yet?" */
>   * Print mbuf statistics.
>   */
>  void
>  mbpr(void)
>  {
> - unsigned long totmem, totused, totmbufs;
> - int totpct;
> - int i, mib[4], npools;
> + unsigned long totmem, totpeak, totmbufs;
> + int i, maxclusters, mib[4], npools;
>   struct kinfo_pool pool;
>   struct mbtypes *mp;
>   size_t size;
>  
>   if (nmbtypes != 256) {
> @@ -99,10 +98,20 @@ mbpr(void)
>      "%s: unexpected change to mbstat; check source\n",
>      __progname);
>   return;
>   }
>  
> + mib[0] = CTL_KERN;
> + mib[1] = KERN_MAXCLUSTERS;
> + size = sizeof(maxclusters);
> +
> + if (sysctl(mib, 2, &maxclusters, &size, NULL, 0) < 0) {
> + printf("Can't retrieve value of maxclusters from the "
> +    "kernel: %s\n",  strerror(errno));
> + return;
> + }
> +
>   mib[0] = CTL_KERN;
>   mib[1] = KERN_MBSTAT;
>   size = sizeof(mbstat);
>  
>   if (sysctl(mib, 2, &mbstat, &size, NULL, 0) < 0) {
> @@ -174,26 +183,24 @@ mbpr(void)
>   printf("\t%u mbuf%s allocated to <mbuf type %d>\n",
>      mbstat.m_mtypes[i],
>      plural(mbstat.m_mtypes[i]), i);
>   }
>   totmem = (mbpool.pr_npages * mbpool.pr_pgsize);
> - totused = mbpool.pr_nout * mbpool.pr_size;
> + totpeak = mbpool.pr_hiwat * mbpool.pr_pgsize;
>   for (i = 0; i < mclp; i++) {
> - printf("%u/%lu/%lu mbuf %d byte clusters in use"
> -    " (current/peak/max)\n",
> + printf("%u/%lu mbuf %d byte clusters in use"
> +    " (current/peak)\n",
>      mclpools[i].pr_nout,
>      (unsigned long)
>   (mclpools[i].pr_hiwat * mclpools[i].pr_itemsperpage),
> -    (unsigned long)
> - (mclpools[i].pr_maxpages * mclpools[i].pr_itemsperpage),
>      mclpools[i].pr_size);
>   totmem += (mclpools[i].pr_npages * mclpools[i].pr_pgsize);
> - totused += mclpools[i].pr_nout * mclpools[i].pr_size;
> + totpeak += mclpools[i].pr_hiwat * mclpools[i].pr_pgsize;
>   }
>  
> - totpct = (totmem == 0) ? 0 : (totused/(totmem / 100));
> - printf("%lu Kbytes allocated to network (%d%% in use)\n",
> -    totmem / 1024, totpct);
> + printf("%lu/%lu/%lu Kbytes allocated to network "
> +    "(current/peak/max)\n", totmem / 1024, totpeak / 1024,
> +    (unsigned long)(maxclusters * MCLBYTES) / 1024);
>   printf("%lu requests for memory denied\n", mbstat.m_drops);
>   printf("%lu requests for memory delayed\n", mbstat.m_wait);
>   printf("%lu calls to protocol drain routines\n", mbstat.m_drain);
>  }
>

--
:wq Claudio

Reply | Threaded
Open this post in threaded view
|

Re: "max" field in "netstat -m" is ambiguous

Mike Belopuhov-5
In reply to this post by Mike Belopuhov-5
On Sat, Oct 28, 2017 at 11:06 +0200, Mike Belopuhov wrote:

> On Thu, Oct 26, 2017 at 08:58 +0200, Claudio Jeker wrote:
> > On Wed, Oct 25, 2017 at 11:46:05PM +0200, Mike Belopuhov wrote:
> > > On Wed, Oct 25, 2017 at 21:56 +0200, Claudio Jeker wrote:
> > > > Would be great if netstat could show the current and peak memory usage.
> > > >
> > >
> > > Current is 5876.  Maximum is 524288.  Do you want to display them in
> > > the x/y/z format?
> > >
> > >   5876/xxxx/524288 Kbytes allocated to network, 20% in use (current/peak/max)
> > >
> > > Something like this? Any other ideas?
> >
> > I think that would be an improvement. I normally look for peak values. The
> > current is normally not interesting when tuning systems.
> > Maybe we can even drop the use percentage since it more confusing than
> > anything.
> >
>
> How about this then?
>
>   saru:usr.bin/netstat% ./obj/netstat -m
>   532 mbufs in use:
>   379 mbufs allocated to data
>   12 mbufs allocated to packet headers
>   141 mbufs allocated to socket names and addresses
>   18/208 mbuf 2048 byte clusters in use (current/peak)
>   0/45 mbuf 2112 byte clusters in use (current/peak)
>   256/320 mbuf 4096 byte clusters in use (current/peak)
>   0/48 mbuf 8192 byte clusters in use (current/peak)
>   0/42 mbuf 9216 byte clusters in use (current/peak)
>   0/50 mbuf 12288 byte clusters in use (current/peak)
>   0/48 mbuf 16384 byte clusters in use (current/peak)
>   0/48 mbuf 65536 byte clusters in use (current/peak)
>   5952/7236/524288 Kbytes allocated to network (current/peak/max)
>   0 requests for memory denied
>   0 requests for memory delayed
>   0 calls to protocol drain routines
>
> OK?
>

Ian Darwin suggested using fmt_scaled for an output like this:
6.1M/7.1M/512Mbytes allocated to network (current/peak/max)

netstat is already linked against libutil.

Any objections?

diff --git usr.bin/netstat/mbuf.c usr.bin/netstat/mbuf.c
index f7970a57c32..27412f9e217 100644
--- usr.bin/netstat/mbuf.c
+++ usr.bin/netstat/mbuf.c
@@ -42,10 +42,11 @@
 #include <errno.h>
 #include <limits.h>
 #include <stdio.h>
 #include <string.h>
 #include <unistd.h>
+#include <util.h>
 #include "netstat.h"
 
 #define YES 1
 typedef int bool;
 
@@ -85,13 +86,13 @@ bool seen[256]; /* "have we seen this type yet?" */
  * Print mbuf statistics.
  */
 void
 mbpr(void)
 {
- unsigned long totmem, totused, totmbufs;
- int totpct;
- int i, mib[4], npools;
+ unsigned long totmem, totpeak, totmbufs;
+ int i, maxclusters, mib[4], npools;
+ char fmt[FMT_SCALED_STRSIZE];
  struct kinfo_pool pool;
  struct mbtypes *mp;
  size_t size;
 
  if (nmbtypes != 256) {
@@ -99,10 +100,20 @@ mbpr(void)
     "%s: unexpected change to mbstat; check source\n",
     __progname);
  return;
  }
 
+ mib[0] = CTL_KERN;
+ mib[1] = KERN_MAXCLUSTERS;
+ size = sizeof(maxclusters);
+
+ if (sysctl(mib, 2, &maxclusters, &size, NULL, 0) < 0) {
+ printf("Can't retrieve value of maxclusters from the "
+    "kernel: %s\n",  strerror(errno));
+ return;
+ }
+
  mib[0] = CTL_KERN;
  mib[1] = KERN_MBSTAT;
  size = sizeof(mbstat);
 
  if (sysctl(mib, 2, &mbstat, &size, NULL, 0) < 0) {
@@ -174,26 +185,34 @@ mbpr(void)
  printf("\t%u mbuf%s allocated to <mbuf type %d>\n",
     mbstat.m_mtypes[i],
     plural(mbstat.m_mtypes[i]), i);
  }
  totmem = (mbpool.pr_npages * mbpool.pr_pgsize);
- totused = mbpool.pr_nout * mbpool.pr_size;
+ totpeak = mbpool.pr_hiwat * mbpool.pr_pgsize;
  for (i = 0; i < mclp; i++) {
- printf("%u/%lu/%lu mbuf %d byte clusters in use"
-    " (current/peak/max)\n",
+ printf("%u/%lu mbuf %d byte clusters in use"
+    " (current/peak)\n",
     mclpools[i].pr_nout,
     (unsigned long)
  (mclpools[i].pr_hiwat * mclpools[i].pr_itemsperpage),
-    (unsigned long)
- (mclpools[i].pr_maxpages * mclpools[i].pr_itemsperpage),
     mclpools[i].pr_size);
  totmem += (mclpools[i].pr_npages * mclpools[i].pr_pgsize);
- totused += mclpools[i].pr_nout * mclpools[i].pr_size;
+ totpeak += mclpools[i].pr_hiwat * mclpools[i].pr_pgsize;
  }
 
- totpct = (totmem == 0) ? 0 : (totused/(totmem / 100));
- printf("%lu Kbytes allocated to network (%d%% in use)\n",
-    totmem / 1024, totpct);
+ if (fmt_scaled(totmem, fmt) == 0)
+ printf("%s/", fmt);
+ else
+ printf("%luK/", totmem / 1024);
+ if (fmt_scaled(totpeak, fmt) == 0)
+ printf("%s/", fmt);
+ else
+ printf("%luK/", totpeak / 1024);
+ if (fmt_scaled(maxclusters * MCLBYTES, fmt) == 0)
+ printf("%s", fmt);
+ else
+ printf("%uK", (maxclusters * MCLBYTES) / 1024);
+ printf("bytes allocated to network (current/peak/max)\n");
  printf("%lu requests for memory denied\n", mbstat.m_drops);
  printf("%lu requests for memory delayed\n", mbstat.m_wait);
  printf("%lu calls to protocol drain routines\n", mbstat.m_drain);
 }

Reply | Threaded
Open this post in threaded view
|

Re: "max" field in "netstat -m" is ambiguous

Theo de Raadt-2
> Ian Darwin suggested using fmt_scaled for an output like this:
> 6.1M/7.1M/512Mbytes allocated to network (current/peak/max)
>
> netstat is already linked against libutil.

No thank you, it doesn't read well at a glance.

Humans can read just fine.