Theorical question on dual core vs single CPU in routing setup.

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Theorical question on dual core vs single CPU in routing setup.

Daniel Ouellet
Here a question I found interesting for my own education, and I am
trying to come to peace with as far as applications usage with dual
core, or multi-processor vs single one.

I was asking myself if I would actually benefit from a dual core
processor, or multi-processor system in a routing setup and more I think
about it, I would think not as the application is not multi-treads to
start with and there isn't must else running as well.

Am I wrong in my understanding?

Looking at the code of bgpd/ospfs, I don't see it design as using
multiple treads ( doesn't mean I understand it fully either) so it
wouldn't benefit from a dual core server then, and as the routing table
basically is process by the kernel, I would think it would be useless to
have multi core no?

In a setup where multiple applications are running, or where the
applications are design with treads in it, yes, but here am I wrong to
think that for a setup where routing with multiple Ethernet ports and
where bgpd/ospfd is running with pf that it wouldn't really be a
benefit? They all are dependent on each other and as such would need to
wait anyway if the routing table changed.

Can someone correct my understanding, or lack there of, I was curious
about that now.

Multi-processor is only useful when you can do multiple things, not
related to each other at the same time, or the application is design
with treads in mind, so here I guess the benefit would be minimal no?

Unless I miss something in the code, or something in how bsd.mp works
(as it would be required to run dual core CPU), may as well put the
money for the speed instead of dual core no?

It's not a big issue, but it got me thinking about it at the point that
I really got curious as to the outcome now, and wonder if I actually
understand it right, or if I am full of it!

Thanks for your time.

Daniel

Reply | Threaded
Open this post in threaded view
|

Re: Theorical question on dual core vs single CPU in routing setup.

Otto Moerbeek
On Sat, 26 Nov 2005, Daniel Ouellet wrote:

> Here a question I found interesting for my own education, and I am trying to
> come to peace with as far as applications usage with dual core, or
> multi-processor vs single one.
>
> I was asking myself if I would actually benefit from a dual core processor, or
> multi-processor system in a routing setup and more I think about it, I would
> think not as the application is not multi-treads to start with and there isn't
> must else running as well.
>
> Am I wrong in my understanding?

If you run a routing daemon, and are doing routing your are doing
multiple things simultaneously: an application (which in some cases
consists of multiple processes) and the kernel both do work.

>
> Looking at the code of bgpd/ospfs, I don't see it design as using multiple
> treads ( doesn't mean I understand it fully either) so it wouldn't benefit
> from a dual core server then, and as the routing table basically is process by
> the kernel, I would think it would be useless to have multi core no?

The current thread library does not take advantage of mult-core or
multi-procesoer setups. There's work being done on an alternative
threading implementation which does take advantage of having more than
one processor, but that is not finished yet.

Currently, the unit of scheduling is a process. The scheduling of
threads is done in userland by the pthreads library.

Currently a multi-threaded application will not benefit directly from
MP.

>
> In a setup where multiple applications are running, or where the applications
> are design with treads in it, yes, but here am I wrong to think that for a
> setup where routing with multiple Ethernet ports and where bgpd/ospfd is
> running with pf that it wouldn't really be a benefit? They all are dependent
> on each other and as such would need to wait anyway if the routing table
> changed.
>
> Can someone correct my understanding, or lack there of, I was curious about
> that now.

The kernel itself does not take advantage of multiple CPUs. But the
routing daemon _might_ benefit from having multiple CPUs. The answer is
also dependent on the impact GENERIC.MP has on performance, because it
does introduce overhead, havig to coordinate things between the CPUs.

So the only real way of answering this is to do measurements.

> Multi-processor is only useful when you can do multiple things, not related to
> each other at the same time, or the application is design with treads in mind,
> so here I guess the benefit would be minimal no?
>
> Unless I miss something in the code, or something in how bsd.mp works (as it
> would be required to run dual core CPU), may as well put the money for the
> speed instead of dual core no?
>
> It's not a big issue, but it got me thinking about it at the point that I
> really got curious as to the outcome now, and wonder if I actually understand
> it right, or if I am full of it!
>
> Thanks for your time.
>
> Daniel

If the kernel is busy, an MP setup might help since the other
processor(s) are available for user processes, but if these processes
often require services from the kernel, the benefits of MP will be low, and
in some cases even be negative.

        -Otto

Reply | Threaded
Open this post in threaded view
|

Re: Theorical question on dual core vs single CPU in routing setup.

Ted Unangst-2
In reply to this post by Daniel Ouellet
if the price is similar, or not an issue, i would just buy the dual
core system.  you can always run the up kernel on an mp system (while
the reverse is true, it doesn't usually help much), and later it will
be more flexible.

Reply | Threaded
Open this post in threaded view
|

Re: Theorical question on dual core vs single CPU in routing setup.

Joachim Schipper
In reply to this post by Daniel Ouellet
On Sat, Nov 26, 2005 at 02:52:11AM -0500, Daniel Ouellet wrote:

> Here a question I found interesting for my own education, and I am
> trying to come to peace with as far as applications usage with dual
> core, or multi-processor vs single one.
>
> I was asking myself if I would actually benefit from a dual core
> processor, or multi-processor system in a routing setup and more I think
> about it, I would think not as the application is not multi-treads to
> start with and there isn't must else running as well.
>
> Am I wrong in my understanding?
>
> Looking at the code of bgpd/ospfs, I don't see it design as using
> multiple treads ( doesn't mean I understand it fully either) so it
> wouldn't benefit from a dual core server then, and as the routing table
> basically is process by the kernel, I would think it would be useless to
> have multi core no?
>
> In a setup where multiple applications are running, or where the
> applications are design with treads in it, yes, but here am I wrong to
> think that for a setup where routing with multiple Ethernet ports and
> where bgpd/ospfd is running with pf that it wouldn't really be a
> benefit? They all are dependent on each other and as such would need to
> wait anyway if the routing table changed.
>
> Can someone correct my understanding, or lack there of, I was curious
> about that now.
>
> Multi-processor is only useful when you can do multiple things, not
> related to each other at the same time, or the application is design
> with treads in mind, so here I guess the benefit would be minimal no?
>
> Unless I miss something in the code, or something in how bsd.mp works
> (as it would be required to run dual core CPU), may as well put the
> money for the speed instead of dual core no?
>
> It's not a big issue, but it got me thinking about it at the point that
> I really got curious as to the outcome now, and wonder if I actually
> understand it right, or if I am full of it!

Theoretically, more processors are good. However, practically, almost
all of what gets done on a router is pushing packets. I've never used
any of the routing daemons, but I'd be *very* surprised if they caused
any load to speak of during normal operations.

So, for practical purposes, all of the work is in the kernel. And
OpenBSD's kernel does not really do multiprocessor.

All in all, I wouldn't expect much benefit from a dual-processor machine
in such a case.

Then again, in many cases, bandwidth and slow buses are more likely to
be the bottleneck than the processor if only doing straight routing.
IPsec and such is another matter, of course.

(And a userspace VPN daemon, like OpenVPN, will benefit you here - I
suppose - since it can be run on another processor than the kernel.)

                Joachim

Reply | Threaded
Open this post in threaded view
|

Re: Theorical question on dual core vs single CPU in routing setup.

Daniel Ouellet
Thank you to Otto, Ted and Joachim for your answers and time.

It confirmed most of my thinking and I was happy to see different point
of view on the subject. So, I can spend my money a bit more wisely, or
pretend to anyway! (:>

Thanks

Daniel

Reply | Threaded
Open this post in threaded view
|

Re: Theorical question on dual core vs single CPU in routing setup.

Henning Brauer
In reply to this post by Daniel Ouellet
* Daniel Ouellet <[hidden email]> [2005-11-26 08:57]:
> I was asking myself if I would actually benefit from a dual core
> processor, or multi-processor system in a routing setup and more I think
> about it, I would think not as the application is not multi-treads to
> start with and there isn't must else running as well.

routing itself does not benefit from MP, currently. for pure
forwarding performance a UP system could be faster than an MP system
(with same parameters otherwise), except that we use the ioapics on MP
systems and that gives some benefit... but that's a side-effect :)

> Looking at the code of bgpd/ospfs, I don't see it design as using
> multiple treads ( doesn't mean I understand it fully either) so it
> wouldn't benefit from a dual core server then, and as the routing table
> basically is process by the kernel, I would think it would be useless to
> have multi core no?


bgpd and ospfd are not threaded at all, on purpose.

however, they are multiple processes. in case of bgpd, the session
engine and the parent process on one CPU and the RDE on the other
should give performance benefits. and you're not only doing bgpd, you
are also forwarding packets, so MP might really improve your
total performance.

Reply | Threaded
Open this post in threaded view
|

Re: Theorical question on dual core vs single CPU in routing setup.

Henning Brauer
In reply to this post by Joachim Schipper
* Joachim Schipper <[hidden email]> [2005-11-26 11:48]:
> Theoretically, more processors are good. However, practically, almost
> all of what gets done on a router is pushing packets. I've never used
> any of the routing daemons, but I'd be *very* surprised if they caused
> any load to speak of during normal operations.

they don't really. bgpd eats CPU time for breakfast when a full-mesh
neighbor goes away tho, naturally.

> All in all, I wouldn't expect much benefit from a dual-processor machine
> in such a case.

the ioapics could make the difference, and it could help to actually
'decouple' bgpd and the actual packet forwarding. but yeah, I am not
certain wether the latter really helps all that much.

--
BS Web Services, http://www.bsws.de/
OpenBSD-based Webhosting, Mail Services, Managed Servers, ...
Unix is very simple, but it takes a genius to understand the simplicity.
(Dennis Ritchie)

Reply | Threaded
Open this post in threaded view
|

Re: Theorical question on dual core vs single CPU in routing setup.

Daniel Ouellet
In reply to this post by Henning Brauer
Henning Brauer wrote:

> * Daniel Ouellet <[hidden email]> [2005-11-26 08:57]:
>>Looking at the code of bgpd/ospfs, I don't see it design as using
>>multiple treads ( doesn't mean I understand it fully either) so it
>>wouldn't benefit from a dual core server then, and as the routing table
>>basically is process by the kernel, I would think it would be useless to
>>have multi core no?
>
>
>
> bgpd and ospfd are not threaded at all, on purpose.
>
> however, they are multiple processes. in case of bgpd, the session
> engine and the parent process on one CPU and the RDE on the other
> should give performance benefits. and you're not only doing bgpd, you
> are also forwarding packets, so MP might really improve your
> total performance.
>

I thought about this, but I am still not sure that it would benefit. I
didn't asked before on Otto answer where:

"If you run a routing daemon, and are doing routing your are doing
multiple things simultaneously: an application (which in some cases
consists of multiple processes) and the kernel both do work"

Isn't it that all the routing changes anyway, either from BGPd or OSPFd
are both ending in the PF table at the kernel level. That's what I
understand anyway and unless I miss something PF in that case would be
the party making the routing here and PF been single tread and looking
at the process list, I don't see multiple pf processes, so the routing
would be a linear process so multi core, or even multi processor
wouldn't really help for that. Yes, they could for BGP session flap,
etc. But that's about it no? In short, it the setup was perfect ( no
such thing obviously) meaning no route changes for flap, but only for
new additions of networks, etc. The routing table wouldn't change and as
such the process would be PF doing the work in a linear fashion making
the use of multi core, or multi processor useless in that case, or may
be even negatively impact it?

I agree that there might be something I don't understand that may affect
this and that's why I try to see what may change that, but other then
small process doing very little to start with and some changes to
routing table because of flap, the routing itself of moving packets
around wouldn't benefit, or may actually be impair par the scheduling of
process no?

I know this is a very remote question, but I am trying to make sure I
understand it right. That's what I thought above anyway, is it right or
wrong?

Daniel

PS: Don't get me wrong, one OpenBSD router kills many times over a good
size Cisco one, no questions there! But I am more interested in the
efficiency of OpenBSD itself compare it itself as an understanding of
how things work under the hood.

Reply | Threaded
Open this post in threaded view
|

Re: Theorical question on dual core vs single CPU in routing setup.

Otto Moerbeek
On Sun, 27 Nov 2005, Daniel Ouellet wrote:

> Henning Brauer wrote:
> > * Daniel Ouellet <[hidden email]> [2005-11-26 08:57]:
> > > Looking at the code of bgpd/ospfs, I don't see it design as using multiple
> > > treads ( doesn't mean I understand it fully either) so it wouldn't benefit
> > > from a dual core server then, and as the routing table basically is
> > > process by the kernel, I would think it would be useless to have multi
> > > core no?
> >
> >
> >
> > bgpd and ospfd are not threaded at all, on purpose.
> >
> > however, they are multiple processes. in case of bgpd, the session engine
> > and the parent process on one CPU and the RDE on the other should give
> > performance benefits. and you're not only doing bgpd, you are also
> > forwarding packets, so MP might really improve your total performance.
> >
>
> I thought about this, but I am still not sure that it would benefit. I didn't
> asked before on Otto answer where:
>
> "If you run a routing daemon, and are doing routing your are doing
> multiple things simultaneously: an application (which in some cases
> consists of multiple processes) and the kernel both do work"
>
> Isn't it that all the routing changes anyway, either from BGPd or OSPFd are
> both ending in the PF table at the kernel level. That's what I understand

That's a misunderstanding: pf does not route, it's the ip layer code
in the kernel. Hint: routing existed before anybody even though of firewalls.

> anyway and unless I miss something PF in that case would be the party making
> the routing here and PF been single tread and looking at the process list, I
> don't see multiple pf processes, so the routing would be a linear process so
> multi core, or even multi processor wouldn't really help for that. Yes, they
> could for BGP session flap, etc. But that's about it no? In short, it the
> setup was perfect ( no such thing obviously) meaning no route changes for
> flap, but only for new additions of networks, etc. The routing table wouldn't

Indeed, if the route deamon does nothing, only the kernel is doing
work. In general the work being done will be something like:

Interface receives a packet, driver puts it on queue, ip layer picks
up packet, decides on route, puts it on queue and kicks the driver of
the outgoing interface, which sends it out. If pf ius active, it will
do its thing too, of course.

> change and as such the process would be PF doing the work in a linear fashion
> making the use of multi core, or multi processor useless in that case, or may
> be even negatively impact it?

As Henning said, in some cases interrupt processing on MP machines
might be faster.

> I agree that there might be something I don't understand that may affect this
> and that's why I try to see what may change that, but other then small process
> doing very little to start with and some changes to routing table because of
> flap, the routing itself of moving packets around wouldn't benefit, or may
> actually be impair par the scheduling of process no?
>
> I know this is a very remote question, but I am trying to make sure I
> understand it right. That's what I thought above anyway, is it right or wrong?

Yes, if the routing daemon is idle you are right. But in practise the
routing daemon will be active, and the impact on performance will be
hard to predict, bcause it is very much dependant on a lot of things.

>
> Daniel
>
> PS: Don't get me wrong, one OpenBSD router kills many times over a good size
> Cisco one, no questions there! But I am more interested in the efficiency of
> OpenBSD itself compare it itself as an understanding of how things work under
> the hood.

        -Otto

Reply | Threaded
Open this post in threaded view
|

Re: Theorical question on dual core vs single CPU in routing setup.

Daniel Ouellet
Otto Moerbeek wrote:

> On Sun, 27 Nov 2005, Daniel Ouellet wrote:
>
>>Henning Brauer wrote:
>>
>>>* Daniel Ouellet <[hidden email]> [2005-11-26 08:57]:
>>Isn't it that all the routing changes anyway, either from BGPd or OSPFd are
>>both ending in the PF table at the kernel level. That's what I understand
>
> That's a misunderstanding: pf does not route, it's the ip layer code
> in the kernel. Hint: routing existed before anybody even though of firewalls.

Big lack of understanding on my part here! Thanks for correcting me!
More brain farts on my part, nothing that a few beer couldn't help I guess.

>>change and as such the process would be PF doing the work in a linear fashion
>>making the use of multi core, or multi processor useless in that case, or may
>>be even negatively impact it?
>
>
> As Henning said, in some cases interrupt processing on MP machines
> might be faster.

Well based on feedback for the Intel quad network card I think, from the
list, using the bsd.mp did reduce the interrupts level as pointed out on
the list before. I thought that may be that was more of a driver issue,
but I guess not and I am sure you guys are right. Most likely a side
effect like Henning said before.

>>I agree that there might be something I don't understand that may affect this
>>and that's why I try to see what may change that, but other then small process
>>doing very little to start with and some changes to routing table because of
>>flap, the routing itself of moving packets around wouldn't benefit, or may
>>actually be impair par the scheduling of process no?
>>
>>I know this is a very remote question, but I am trying to make sure I
>>understand it right. That's what I thought above anyway, is it right or wrong?
>
>
> Yes, if the routing daemon is idle you are right. But in practise the
> routing daemon will be active, and the impact on performance will be
> hard to predict, bcause it is very much dependant on a lot of things.

Sure thing! I am glad I understand it a bit better then yesterday anyway
and hopefully less then tomorrow!

Thanks for your time.

Daniel