State of multiprocessing and multithreading in OpenBSD

classic Classic list List threaded Threaded
37 messages Options
12
Reply | Threaded
Open this post in threaded view
|

State of multiprocessing and multithreading in OpenBSD

Juan Miscaro-2
What is the current state of multiprocessing and multithreading in
OpenBSD?  Also, what applications are multithreaded?  In particular,
someone told me that pf is "garbage" because it is not multithreaded?
What truth is there to this?  Under what kind of load would an OpenBSD
firewall's performance suffer due to it being non-multithreaded?

--
/jm

Reply | Threaded
Open this post in threaded view
|

Re: State of multiprocessing and multithreading in OpenBSD

Marco Peereboom
On Tue, May 04, 2010 at 10:15:09PM -0400, Juan Miscaro wrote:
> What is the current state of multiprocessing and multithreading in
> OpenBSD?  Also, what applications are multithreaded?  In particular,
> someone told me that pf is "garbage" because it is not multithreaded?
> What truth is there to this?  Under what kind of load would an OpenBSD
> firewall's performance suffer due to it being non-multithreaded?

Wow!  Just wow!

This is easily the dumbest thing I heard all year.  Thanks for the
laugh.

>
> --
> /jm

Reply | Threaded
Open this post in threaded view
|

Re: State of multiprocessing and multithreading in OpenBSD

Christiano F. Haesbaert
In reply to this post by Juan Miscaro-2
What a bunch of crap...

misc is better than usual this week.

Reply | Threaded
Open this post in threaded view
|

Re: State of multiprocessing and multithreading in OpenBSD

gwes-2
In reply to this post by Juan Miscaro-2
Juan Miscaro <[hidden email]> wrote on Tue, 4 May 2010 22:15:09 -0400

>What is the current state of multiprocessing and multithreading in
>OpenBSD?  Also, what applications are multithreaded?  In particular,
>someone told me that pf is "garbage" because it is not multithreaded?
>What truth is there to this?  Under what kind of load would an OpenBSD
>firewall's performance suffer due to it being non-multithreaded?

Ha. Note: bsd.mp

Search the misc archives for "threaded sshd".

PF is interrupt-driven inside the kernel and thus faster than any
threaded program.

Take whoever told you that load of garbled nonsense and push him
or her into a midden-heap. That's where it belongs.
Threads were invented as a very bad workaround for slow context
switching on ancient hardware using primitive OS versions.
The people who invented them said they were bad.
Any teacher or programmer who says otherwise is ignorant.
There's a paper from Berkeley showing how a threaded program can
never be fully debugged and should be presumed to be broken,
probably fatally broken.

geoff steckel
curmudgeon for hire, lease, or loan
been there since 1966

Reply | Threaded
Open this post in threaded view
|

Re: State of multiprocessing and multithreading in OpenBSD

Jan Stary
In reply to this post by Juan Miscaro-2
On May 04 22:15:09, Juan Miscaro wrote:
> What is the current state of multiprocessing and multithreading in
> OpenBSD?  Also, what applications are multithreaded?  In particular,
> someone told me that pf is "garbage" because it is not multithreaded?
> What truth is there to this?  Under what kind of load would an OpenBSD
> firewall's performance suffer due to it being non-multithreaded?

STFU, GTFO, and all that.

Reply | Threaded
Open this post in threaded view
|

Re: State of multiprocessing and multithreading in OpenBSD

Lars Nooden-2
In reply to this post by gwes-2
On Wed, 5 May 2010, Geoff wrote:
> There's a paper from Berkeley showing how a threaded program can
> never be fully debugged and should be presumed to be broken,
> probably fatally broken.

Geoff, can you post the URL or any details that might help finding and
retrieving that particular article or ones like it?

/Lars

Reply | Threaded
Open this post in threaded view
|

Re: State of multiprocessing and multithreading in OpenBSD

Peter Nicolai Mathias Hansteen
In reply to this post by Juan Miscaro-2
Juan Miscaro <[hidden email]> writes:

> someone told me that pf is "garbage" because it is not multithreaded?
> What truth is there to this?  Under what kind of load would an OpenBSD
> firewall's performance suffer due to it being non-multithreaded?

I would think that would be a fair question to ask the person who told
you PF is garbage because it is multithreaded:

    "Under what circumstances would threading or lack thereof have
     have identifiable impact on performance?"

If they can't come up with a coherent answer to that question, this is
just yet another iteration of the "OpenBSD is {crap,insecure,written
and used only by dickheads} because it does not have $my_favorite_toy"
nonsense.

- P
--
Peter N. M. Hansteen, member of the first RFC 1149 implementation team
http://bsdly.blogspot.com/ http://www.bsdly.net/ http://www.nuug.no/
"Remember to set the evil bit on all malicious network traffic"
delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.

Reply | Threaded
Open this post in threaded view
|

Re: State of multiprocessing and multithreading in OpenBSD

Tony Abernethy
In reply to this post by Lars Nooden-2
Lars Nooden wrote:

>
> On Wed, 5 May 2010, Geoff wrote:
> > There's a paper from Berkeley showing how a threaded program can
> > never be fully debugged and should be presumed to be broken,
> > probably fatally broken.
>
> Geoff, can you post the URL or any details that might help finding and
> retrieving that particular article or ones like it?
>
> /Lars

http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-1.pdf
first choice googling: threads berkeley

Choice quote: (quoting Sutter and Laurs)
"humans are quicly overwhelmed by concurrency and find it much more
difficult to reason about concurrent than sequential code. Even careful
people miss possible interleavings among even simple collections of
partially ordered operations."

Other than some stunts with data binding I don't think I've seen
anything that is competent to handle partial orders. And that one breaks
down horribly if storage cells take on more than one value during execution.

Reply | Threaded
Open this post in threaded view
|

Re: State of multiprocessing and multithreading in OpenBSD

Peter Nicolai Mathias Hansteen
In reply to this post by Peter Nicolai Mathias Hansteen
[hidden email] (Peter N. M. Hansteen) writes:

> I would think that would be a fair question to ask the person who told
> you PF is garbage because it is multithreaded:

eh, "because it is *not* multithreaded:"

--
Peter N. M. Hansteen, member of the first RFC 1149 implementation team
http://bsdly.blogspot.com/ http://www.bsdly.net/ http://www.nuug.no/
"Remember to set the evil bit on all malicious network traffic"
delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.

Reply | Threaded
Open this post in threaded view
|

Re: State of multiprocessing and multithreading in OpenBSD

Tony Abernethy
Peter N. M. Hansteen wrote:
> [hidden email] (Peter N. M. Hansteen) writes:
>
> > I would think that would be a fair question to ask the person who
> told
> > you PF is garbage because it is multithreaded:
>
> eh, "because it is *not* multithreaded:"
>
Now watch when application programmers use multithreaded stuff because
they think it will somehow solve all their problems.
If you ***CAN*** ***EVER*** make such a typo, do you really think
that they even stand a chance?

Couple this with wrong-way branches on equal comparisons (edges), and
you do not even need to get into error-recovery stuff to find a mess.

Reply | Threaded
Open this post in threaded view
|

Re: State of multiprocessing and multithreading in OpenBSD

Lars Nooden-2
In reply to this post by Tony Abernethy
On Wed, 5 May 2010, Tony Abernethy wrote:
> http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-1.pdf
> first choice googling: threads berkeley

Thanks.  You have better luck with Google than I did.  berkeley threading
won't find it.  Repeating once more for the archive:
  http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-1.pdf

/Lars

Reply | Threaded
Open this post in threaded view
|

Re: State of multiprocessing and multithreading in OpenBSD

Stas Miasnikou
In reply to this post by Tony Abernethy
Tony Abernethy wrote:

> Lars Nooden wrote:
>> On Wed, 5 May 2010, Geoff wrote:
>>> There's a paper from Berkeley showing how a threaded program can
>>> never be fully debugged and should be presumed to be broken,
>>> probably fatally broken.
>> Geoff, can you post the URL or any details that might help finding and
>> retrieving that particular article or ones like it?
>>
>> /Lars
>
> http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-1.pdf
> first choice googling: threads berkeley
>
> Choice quote: (quoting Sutter and Laurs)
> "humans are quicly overwhelmed by concurrency and find it much more
> difficult to reason about concurrent than sequential code. Even careful
> people miss possible interleavings among even simple collections of
> partially ordered operations."

Some people, when confronted with a problem, think "I know, I'll use
multithreading." Now they have N problems.
        --(almost) jwz

> Other than some stunts with data binding I don't think I've seen
> anything that is competent to handle partial orders. And that one breaks
> down horribly if storage cells take on more than one value during execution.

                                        Stas

Reply | Threaded
Open this post in threaded view
|

Re: State of multiprocessing and multithreading in OpenBSD

Brad Tilley-4
In reply to this post by Tony Abernethy
Tony Abernethy wrote:

> Peter N. M. Hansteen wrote:
>> [hidden email] (Peter N. M. Hansteen) writes:
>>
>>> I would think that would be a fair question to ask the person who
>> told
>>> you PF is garbage because it is multithreaded:
>> eh, "because it is *not* multithreaded:"
>>
> Now watch when application programmers use multithreaded stuff because
> they think it will somehow solve all their problems.

I only find threads useful in GUI programming when there's a need to
make the GUI seem responsive while other stuff is going on. That's about
all the use I have ever gotten from threads although I'm sure some apps
(video encoding, etc.) make heavy use of them since now everyone has
6-way cores, etc.

Brad

> If you ***CAN*** ***EVER*** make such a typo, do you really think
> that they even stand a chance?
>
> Couple this with wrong-way branches on equal comparisons (edges), and
> you do not even need to get into error-recovery stuff to find a mess.

Reply | Threaded
Open this post in threaded view
|

Re: State of multiprocessing and multithreading in OpenBSD

Benny Lofgren
In reply to this post by Jan Stary
Jan Stary wrote:
> On May 04 22:15:09, Juan Miscaro wrote:
>> What is the current state of multiprocessing and multithreading in
>> OpenBSD?  Also, what applications are multithreaded?  In particular,
>> someone told me that pf is "garbage" because it is not multithreaded?
>> What truth is there to this?  Under what kind of load would an OpenBSD
>> firewall's performance suffer due to it being non-multithreaded?
>
> STFU, GTFO, and all that.

Still, I think the question itself merits some discussion.

If we filter out the op:s PF nonsense, the state of rthreads development
is relevant to several well-used software packages, the first one that
comes to mind being MySQL.

I run several webservers using OpenBSD for my customers with PHP and
MySQL as important ingredients, and while I couldn't agree more to those
that would immediately say "dump MySQL, use PostgreSQL" it isn't always
possible to do so.

It is of course well known that MySQL doesn't run well (as in "as fast
as its potential allows") in a pthreads-environment, and yes, seven of
my gallant eight-core servers cores sits near idle waiting for the poor
MySQL process to finish up its work with the web server running rings
around it utilizing all cores very nicely. (Still, I use it with its
shortcomings, because I wouldn't trust any other os to do that kind of
heavily internet exposed work.)

I'm constantly scanning the changelogs in the vain hope that some day
there will be a message about rthreads being promoted to be the default
threading model. :-)   Keep up the good work!

/B

--
internetlabbet.se     / work:   +46 8 551 124 80      / "Words must
Benny Lvfgren        /  mobile: +46 70 718 11 90     /   be weighed,
                     /   fax:    +46 8 551 124 89    /    not counted."
                    /    email:  [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: State of multiprocessing and multithreading in OpenBSD

Christiano F. Haesbaert
In reply to this post by gwes-2
On 5 May 2010 01:07, Geoff <[hidden email]> wrote:

> Juan Miscaro <[hidden email]> wrote on Tue, 4 May 2010 22:15:09 -0400
>
>>What is the current state of multiprocessing and multithreading in
>>OpenBSD?  Also, what applications are multithreaded?  In particular,
>>someone told me that pf is "garbage" because it is not multithreaded?
>>What truth is there to this?  Under what kind of load would an OpenBSD
>>firewall's performance suffer due to it being non-multithreaded?
>
> Ha. Note: bsd.mp
>
> Search the misc archives for "threaded sshd".
>
> PF is interrupt-driven inside the kernel and thus faster than any
> threaded program.
>
> Take whoever told you that load of garbled nonsense and push him
> or her into a midden-heap. That's where it belongs.
> Threads were invented as a very bad workaround for slow context
> switching on ancient hardware using primitive OS versions.

Actually I believe it's more of a workaround for poor IPC techniques,
not slow context switching (you still have context switching among
processes).

> The people who invented them said they were bad.
> Any teacher or programmer who says otherwise is ignorant.
> There's a paper from Berkeley showing how a threaded program can
> never be fully debugged and should be presumed to be broken,
> probably fatally broken.

Reply | Threaded
Open this post in threaded view
|

Re: [Bulk] Re: State of multiprocessing and multithreading in OpenBSD

Kevin Chadwick-3
I heard that after being stuck at around 3ghz at a reasonable temp for
ages. Intel decided to go multicore and just after the time the
decision was made, a breakthrough in single core was made and ignored
as development was redirected. I imagine they would have hit another
barrier though, otherwise the development spent on processor management
could be saved.

I notice OpenBSD states one processor for applications and one for
boot. Does that increase security via priviledge/memory separation or
is it just because only one is used during boot?

Reply | Threaded
Open this post in threaded view
|

Re: [Bulk] Re: State of multiprocessing and multithreading in OpenBSD

Henning Brauer
* Kevin Chadwick <[hidden email]> [2010-05-05 18:00]:
> I notice OpenBSD states one processor for applications and one for
> boot. Does that increase security via priviledge/memory separation or
> is it just because only one is used during boot?

the term "application processor" is misleading. once booted the
processors are treated the same. one is just special up to the point
where the secondary CPUs are spun up. well, in general, that is the
story.

--
Henning Brauer, [hidden email], [hidden email]
BS Web Services, http://bsws.de
Full-Service ISP - Secure Hosting, Mail and DNS Services
Dedicated Servers, Rootservers, Application Hosting

Reply | Threaded
Open this post in threaded view
|

Re: State of multiprocessing and multithreading in OpenBSD

Marco Peereboom
In reply to this post by Benny Lofgren
On Wed, May 05, 2010 at 02:00:17PM +0200, Benny L?fgren wrote:

> Jan Stary wrote:
>> On May 04 22:15:09, Juan Miscaro wrote:
>>> What is the current state of multiprocessing and multithreading in
>>> OpenBSD?  Also, what applications are multithreaded?  In particular,
>>> someone told me that pf is "garbage" because it is not multithreaded?
>>> What truth is there to this?  Under what kind of load would an OpenBSD
>>> firewall's performance suffer due to it being non-multithreaded?
>>
>> STFU, GTFO, and all that.
>
> Still, I think the question itself merits some discussion.

Not really.  Threads are mostly stupid, humans are mostly stupid.
Combine the two and you end up with some really really stupid software.

> If we filter out the op:s PF nonsense, the state of rthreads development  
> is relevant to several well-used software packages, the first one that  
> comes to mind being MySQL.

Which is a steaming pile of german scheisse.  A prime example as to why
threads are mostly stupid.

> I run several webservers using OpenBSD for my customers with PHP and  
> MySQL as important ingredients, and while I couldn't agree more to those  
> that would immediately say "dump MySQL, use PostgreSQL" it isn't always  
> possible to do so.

Sure that is a real live problem but no amount of code and fixes will
ever fix something as complex as mysqueel running with threads.  You are
stuck with the suck so make sure you can restart the individual pieces
easily and often.

> It is of course well known that MySQL doesn't run well (as in "as fast  
> as its potential allows") in a pthreads-environment, and yes, seven of  
> my gallant eight-core servers cores sits near idle waiting for the poor  
> MySQL process to finish up its work with the web server running rings  
> around it utilizing all cores very nicely. (Still, I use it with its  
> shortcomings, because I wouldn't trust any other os to do that kind of  
> heavily internet exposed work.)

I love the threads over multiple core argument :-)

Lets defeat caching on the cpus and fight continuously over locks!

> I'm constantly scanning the changelogs in the vain hope that some day  
> there will be a message about rthreads being promoted to be the default  
> threading model. :-)   Keep up the good work!

rthreads wont fix world hunger.  It might make shit software have less
peanuts and corn niblets in it.

Wouldn't it be adorable if people learned to program FSMs instead of
java in those fancy universities?

>
> /B
>
> --
> internetlabbet.se     / work:   +46 8 551 124 80      / "Words must
> Benny Lvfgren        /  mobile: +46 70 718 11 90     /   be weighed,
>                     /   fax:    +46 8 551 124 89    /    not counted."
>                    /    email:  [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: State of multiprocessing and multithreading in OpenBSD

Juan Miscaro-2
On 5 May 2010 14:09, Marco Peereboom <[hidden email]> wrote:

> On Wed, May 05, 2010 at 02:00:17PM +0200, Benny L?fgren wrote:
>> Jan Stary wrote:
>>> On May 04 22:15:09, Juan Miscaro wrote:
>>>> What is the current state of multiprocessing and multithreading in
>>>> OpenBSD? B Also, what applications are multithreaded? B In particular,
>>>> someone told me that pf is "garbage" because it is not multithreaded?
>>>> What truth is there to this? B Under what kind of load would an OpenBSD
>>>> firewall's performance suffer due to it being non-multithreaded?
>>>
>>> STFU, GTFO, and all that.
>>
>> Still, I think the question itself merits some discussion.
>
> Not really. B Threads are mostly stupid, humans are mostly stupid.
> Combine the two and you end up with some really really stupid software.

Thanks everyone.  From all the stuff written in this thread (a
multithread?) I have extracted the following information:

"PF is interrupt-driven inside the kernel and thus faster than any
threaded program."

Thank you to the one that wrote that (Geoff).

I also learned that:

1. multithreading was introduced due to the processing limitations of
the average computer at the time
2. multithreaded applications are difficult to debug and therefore
pose a significant security risk

However, I'm not sure why there was so much talk of steaming piles of
shit; shit that contains less peanuts and corn niblets; "bunch of
crap"; and STFU/GTFO.

I have been using OpenBSD for many years and I was just trying to
learn more about these issues so as to be in a better position to
promote/defend the OS.  I'm not a troll and I don't know why there is
so much rudeness.

--
/jm

Reply | Threaded
Open this post in threaded view
|

Re: State of multiprocessing and multithreading in OpenBSD

Richard Toohey
Quoting Juan Miscaro <[hidden email]>:

> On 5 May 2010 14:09, Marco Peereboom <[hidden email]> wrote:
> > On Wed, May 05, 2010 at 02:00:17PM +0200, Benny L?fgren wrote:
> >> Jan Stary wrote:
> >>> On May 04 22:15:09, Juan Miscaro wrote:
> >>>> What is the current state of multiprocessing and multithreading in
> >>>> OpenBSD? B Also, what applications are multithreaded? B In
> particular,
> >>>> someone told me that pf is "garbage" because it is not
> multithreaded?
... cut ...
> However, I'm not sure why there was so much talk of steaming piles of
> shit; shit that contains less peanuts and corn niblets; "bunch of
> crap"; and STFU/GTFO.
>
> I have been using OpenBSD for many years and I was just trying to
> learn more about these issues so as to be in a better position to
> promote/defend the OS. I'm not a troll and I don't know why there is
> so much rudeness.
>

You've told the developers that their work has been described as "garbage" and
you wonder why you get a rude response?  You couldn't have phrased it in a less
inflammatory way?

"Someone" told me my Atari ST was "garbage" and their Amiga was better.  Ford is
better than Holden, vim is better than Emacs, MySQL is better than PostgreSQL,
FreeBSD is better than OpenBSD, Windows is better than Linux, etc., etc., etc.,
etc., etc.

Don't listen to the someones - you've got to try stuff for yourself.

> --
> /jm

12