Integrating "safe" languages into OpenBSD?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

Integrating "safe" languages into OpenBSD?

Nicolas Schmidt
Hi,

I recently watched a recording of Theo's talk on pledge at EuroBSDCon 2017, in which the question of memory-safe languages and their practical usefulness came up. Specifically, someone in the audience criticized the approach taken by OpenBSD, which (as I understand) accepts that all software is broken and mitigates the damage caused by various classes of exploits through techniques like ASLR, and suggested that instead one should stick to "memory safe languages" to avoid these exploits altogether.

As a response to this, Theo asked rhetorically "Where's ls, where's cat, where's grep, and where's sort?", implying that noone so far bothered to write implementations of even the basic unix utilities in such a language.

This brings me to the question, what if someone actually bothered? Under what conditions would you consider replacing one of the current C implementations with an implementation written in another, "safer" language? Note that with Cgrep and haskell-ls, there do in fact exist implementations/analogues of two of the mentioned utilities in a memory safe language (Haskell).

Best,
Nicolas Schmidt
Reply | Threaded
Open this post in threaded view
|

Re: Integrating "safe" languages into OpenBSD?

Theo de Raadt-2
> As a response to this, Theo asked rhetorically "Where's ls, where's cat,
> where's grep, and where's sort?", implying that noone so far bothered to
> write implementations of even the basic unix utilities in such a
> language.

I wasn't implying.  I was stating a fact.  There has been no attempt
to move the smallest parts of the ecosystem, to provide replacements
for base POSIX utilities.

As a general trend the only things being written in these new
languages are new web-facing applications, quite often proprietory or
customized to narrow roles.  Not Unix parts.

Right now, there are zero usage cases in the source tree to require
those compiler tools.  We won't put a horse into the source tree when
society lacks cart builders.

> This brings me to the question, what if someone actually bothered?

So rather than bothering to begin, you wrote an email.

Awesome.

Yes, now I am implying something: you won't bother to rewrite the
utilities.

And I understand, why would anyone bother?  It took about 10 years for
gnu grep to be replaced sufficiently well in our tree.  This stuff
doesn't happen overnight.

However there is a rampant fiction that if you supply a new safer
method everyone will use it.  For gods sake, the simplest of concepts
like the stack protector took nearly 10 years for adoption, let people
should switch languages?  DELUSION.

> Under what conditions would you consider replacing one of the
> current C implementations with an implementation written in another,
> "safer" language?

In OpenBSD there is a strict requirement that base builds base.

So we cannot replace any base utility, unless the toolchain to build
it is in the base.  Adding such a toolchain would take make build time
from 40 minutes to hours.  I don't see how that would happen.

> Note that with Cgrep and haskell-ls, there do in fact exist
> implementations/analogues of two of the mentioned utilities in a
> memory safe language (Haskell).

Are they POSIX compliant?  No.  They are completely different programs
that have borrowed the names.

By the way, this is how long it takes to compile our grep:

    0m00.62s real     0m00.63s user     0m00.53s system

Does Cgrep compile in less than 10 minutes?

Such ecosystems come with incredible costs.  For instance, rust cannot
even compile itself on i386 at present time because it exhausts the
address space.

Consider me a skeptic -- I think these compiler ecosystems face a grim
bloaty future.

Reply | Threaded
Open this post in threaded view
|

Re: Integrating "safe" languages into OpenBSD?

Daniel Wilkins
In reply to this post by Nicolas Schmidt
And on top of what Theo said: rewriting stuff in "safe" languages doesn't reduce
the need for mitigations *anyway*. Nobody's rewriting all of the ports tree in
memory safe languages.

Reply | Threaded
Open this post in threaded view
|

Re: Integrating "safe" languages into OpenBSD?

bytevolcano
I've always subscribed to the idea that too much safety results in too
may idiots, and the same is true for all these "safe" programming
languages. "Oh I don't have to write any form of bounds-checking,
because the language will do it for me."

To add further insult to injury, if the language's bounds checking kicks
in first your program may do something worse than just corrupting its
own memory. In my experience, apps written in these "safe" languages
(usually web apps or bloatware) actually have been the most bug-ridden
and bloated.

On Sun, 3 Dec 2017 15:54:43 -0500
Daniel Wilkins <[hidden email]> wrote:

> And on top of what Theo said: rewriting stuff in "safe" languages doesn't reduce
> the need for mitigations *anyway*. Nobody's rewriting all of the ports tree in
> memory safe languages.
>

Reply | Threaded
Open this post in threaded view
|

Re: Integrating "safe" languages into OpenBSD?

Nick Holland
On 12/03/17 20:19, [hidden email] wrote:

> I've always subscribed to the idea that too much safety results in too
> may idiots, and the same is true for all these "safe" programming
> languages. "Oh I don't have to write any form of bounds-checking,
> because the language will do it for me."
>
> To add further insult to injury, if the language's bounds checking kicks
> in first your program may do something worse than just corrupting its
> own memory. In my experience, apps written in these "safe" languages
> (usually web apps or bloatware) actually have been the most bug-ridden
> and bloated.

Oh yeah.
I recently discovered a very major business operations application where
rather than using the OS's FTP and SFTP functions, they wrote their own
in "safe" Java.  I don't know why.

It's used as part of a file transfer system -- you "print" a file to a
print queue, and it is magically transported to another machine.

If the other machine is being serviced?  Network broke?  receiving
machine unable to recieve?  Oh well.  Magic doesn't work, the file is
lost, without alerting the "sending" program.

Error reporting?  Well, for a long time, I thought it was non-existent,
but I recently found they just dumped all the java runtime output to a
file.  Nothing is actually done with this info in the application, but
if 100+ lines of J-crap is your favorite way to see "server timeout",
this is your tool.

Idiots who shouldn't be coding, coding.
"safe" languages being trusted to be safe when in the hands of idiots.
Like you said.

The more I see of "safe" languages, the more I love assembly.  Most
people who call themselves programmers...shouldn't.

Nick.

Reply | Threaded
Open this post in threaded view
|

Re: Integrating "safe" languages into OpenBSD?

Nicolas Schmidt
> Am 04.12.2017 um 14:45 schrieb Nick Holland <[hidden email]>:
...
>
> Oh yeah.
> I recently discovered a very major business operations application where
> rather than using the OS's FTP and SFTP functions, they wrote their own
> in "safe" Java.  I don't know why.
...
> If the other machine is being serviced?  Network broke?  receiving
> machine unable to recieve?  Oh well.  Magic doesn't work, the file is
> lost, without alerting the "sending" program.
>
> Error reporting?  Well, for a long time, I thought it was non-existent,
> but I recently found they just dumped all the java runtime output to a
> file.  Nothing is actually done with this info in the application, but
> if 100+ lines of J-crap is your favorite way to see "server timeout",
> this is your tool.
...
> Nick.

So they wrote a program that was a) shitty and b) memory-safe? Those are two orthogonal dimensions. Also, the anecdotal evidence that safe languages attract bad programmers does not imply that using safe languages is bad: a good programmer won't suddenly commit such atrocities as you mentioned, just because they use a safe language.

Finally, your example probably speaks more about business practices than about safe programming languages. If you want to compare Java to a non-memory-safe language, you should compare it to one that is also designed *for* (instead of *by*) programmers, like Cobol.
Reply | Threaded
Open this post in threaded view
|

Re: Integrating "safe" languages into OpenBSD?

bytevolcano
On Mon, 4 Dec 2017 16:24:52 +0100
Nicolas Schmidt <[hidden email]> wrote:

> So they wrote a program that was a) shitty and b) memory-safe? Those are two orthogonal dimensions. Also, the anecdotal evidence that safe languages attract bad programmers does not imply that using safe languages is bad: a good programmer won't suddenly commit such atrocities as you mentioned, just because they use a safe language.

A good programmer won't even need these languages in the first
place. Case in point, the entire OpenBSD dev team. :)

> Finally, your example probably speaks more about business practices than about safe programming languages. If you want to compare Java to a non-memory-safe language, you should compare it to one that is also designed *for* (instead of *by*) programmers, like Cobol.

It goes back to the point I make though, that these languages encourage
this kind of behavior by promoting a false sense of security, hence complacency.

Reply | Threaded
Open this post in threaded view
|

Re: Integrating "safe" languages into OpenBSD?

Gareth Nelson
Just throwing my 2 cents in here:
I don't think it'd be appropriate in OpenBSD base, but i'd love to get
involved in writing a *nix environment in a "safe" language and at one
point was thinking of building a linux distro where all the core tools are
in Python - more for the fun of it than anything serious, but if anyone
wants to get together to do this kind of project with a focus on security
and safety i'd love to get back to it.

I've also experimented (as all serious coders have) with my own lisp
variant, and would love to make that the basis of a small set of unix tools
with quick compilation. Reply off-list if interested.

On Tue, Dec 5, 2017 at 9:47 AM, <[hidden email]> wrote:

> On Mon, 4 Dec 2017 16:24:52 +0100
> Nicolas Schmidt <[hidden email]> wrote:
>
> > So they wrote a program that was a) shitty and b) memory-safe? Those are
> two orthogonal dimensions. Also, the anecdotal evidence that safe languages
> attract bad programmers does not imply that using safe languages is bad: a
> good programmer won't suddenly commit such atrocities as you mentioned,
> just because they use a safe language.
>
> A good programmer won't even need these languages in the first
> place. Case in point, the entire OpenBSD dev team. :)
>
> > Finally, your example probably speaks more about business practices than
> about safe programming languages. If you want to compare Java to a
> non-memory-safe language, you should compare it to one that is also
> designed *for* (instead of *by*) programmers, like Cobol.
>
> It goes back to the point I make though, that these languages encourage
> this kind of behavior by promoting a false sense of security, hence
> complacency.
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Integrating "safe" languages into OpenBSD?

Karel Gardas
On Tue, Dec 5, 2017 at 11:29 AM, Gareth Nelson <[hidden email]> wrote:
> Just throwing my 2 cents in here:
> I don't think it'd be appropriate in OpenBSD base, but i'd love to get
> involved in writing a *nix environment in a "safe" language and at one

If you are not opposed to switching your tooling, then have a look at
https://www.redox-os.org/ -- OS done in Rust. Currently probably the
most viable not-in-C OS besides a tons of academic exercises.

Reply | Threaded
Open this post in threaded view
|

Re: Integrating "safe" languages into OpenBSD?

Karel Gardas
In reply to this post by Nicolas Schmidt
On Sun, Dec 3, 2017 at 7:59 PM, Nicolas Schmidt
<[hidden email]> wrote:
> Hi,
>
> I recently watched a recording of Theo's talk on pledge at EuroBSDCon 2017, in which the question of memory-safe languages and their practical usefulness came up. Specifically, someone in the audience criticized the approach taken by OpenBSD, which (as I understand) accepts that all software is broken and mitigates the damage caused by various classes of exploits through techniques like ASLR, and suggested that instead one should stick to "memory safe languages" to avoid these exploits altogether.
>
> As a response to this, Theo asked rhetorically "Where's ls, where's cat, where's grep, and where's sort?", implying that noone so far bothered to write implementations of even the basic unix utilities in such a language.
>
> This brings me to the question, what if someone actually bothered? Under what conditions would you consider replacing one of the current C implementations with an implementation written in another, "safer" language? Note that with Cgrep and haskell-ls, there do in fact exist implementations/analogues of two of the mentioned utilities in a memory safe language (Haskell).

IMHO it does not make any sense since (1) it was just rhetorical
question probably and (2) even if you do have set of base utils
written in Haskell, you would still need Haskell compiler/interpreter
and due to size it's probably a no go into the base anyway. And you
know the mantra: base should build base.

On the other hand, it'd also be a waste of energy probably since even
Haskell is high-productive language, you still compete with let's say
99% of utility code written in C. So what do you do if you need to do
a bug fix in this? Rewrite 100% in Haskell or just rewrite/write 1% of
bugfix in C? My bet is that if the question is not about a complete
rewrite than fixing in C is more productive.

Another point: look into the history. Systems usually stays written in
the language where they started. Unix -> C. Cobol -> Cobol. Windows ->
mix C/C++/ now also .Net. But you layer on top of those additional
functionality. Look into Unix(-like) and how it drives current world
of WWW and or mobile/phone/device industry. The core is still the
same. Unix- philosophy kernel written in C and on top of this some
additional functionality and/or web services.

So to me it looks like: use language of the target system if there is
a need to hack system functionality/bugfix, but let's use modern
language if building functionality on top of that. IMHO quite
evolutionary approach. The revolutionary approach or system written in
modern language like Haskell, I'm afraid this is not going to fly also
due to limited number of interested people not be able to go over
certain critical people-threshhold/mass. For Haskell specifically see
House, Kinetic or Hos.
https://wiki.haskell.org/Applications_and_libraries/Operating_system

The question for consideration is if microservices/unikernels approach
is not the best combination of both worlds, e.g. having something like
Mirage or HalVM based application/service running on top of OpenBSD in
its VMM, that may be interesting. Unfortunately so far both supports
IIRC just Xen.

Have fun!

Reply | Threaded
Open this post in threaded view
|

Re: Integrating "safe" languages into OpenBSD?

Adam Steen
> The question for consideration is if microservices/unikernels approach
> is not the best combination of both worlds, e.g. having something like
> Mirage or HalVM based application/service running on top of OpenBSD in
> its VMM, that may be interesting. Unfortunately so far both supports
> IIRC just Xen.
>

Just a note on the above statements, i currently have solo5/vmm (with
lots of help from Mike L and Mike B) [1] running on OpenBSD Current
and am working towards [2] getting MirageOS [3] working too, if you
would like more information please let me know.

Cheers
Adam

[1] https://github.com/adamsteen/solo5
[2] https://github.com/Solo5/solo5/issues/206
[3] https://mirage.io/

Reply | Threaded
Open this post in threaded view
|

Re: Integrating "safe" languages into OpenBSD?

webmaster
In reply to this post by Nicolas Schmidt


> -------- Original Message --------
> Subject: Re: Integrating "safe" languages into OpenBSD?
> From: Nick Holland <[hidden email]>
> Date: Mon, December 04, 2017 7:45 am
> To: [hidden email]
>
>
> On 12/03/17 20:19, [hidden email] wrote:
> > I've always subscribed to the idea that too much safety results in too
> > may idiots, and the same is true for all these "safe" programming
> > languages. "Oh I don't have to write any form of bounds-checking,
> > because the language will do it for me."
> >
> > To add further insult to injury, if the language's bounds checking kicks
> > in first your program may do something worse than just corrupting its
> > own memory. In my experience, apps written in these "safe" languages
> > (usually web apps or bloatware) actually have been the most bug-ridden
> > and bloated.
>

> Idiots who shouldn't be coding, coding.
> "safe" languages being trusted to be safe when in the hands of idiots.
> Like you said.
>
> The more I see of "safe" languages, the more I love assembly.  Most
> people who call themselves programmers...shouldn't.
>
> Nick.


The issue of being in base has been raised. Pretty important. Who would
really
want to spend such enormous amounts of time to add in yet another
language?

C is in base. Assembly is in base.
But Perl is also in base. I don't think anyone would want to change all
of the
fantastic pkg_* tools into either C or assembly.
None of these three are "safe" languages.

A while ago I was told that moving to a newer version of Perl was being
held up
by needing to deal with mod_perl. That's obviously been dealt with.

I like Perl's way of flowing and doing things.

I tried learning some assembly on my own, really to just get a better
idea of
what was going on with different C commands and variables.
But the developers kept adding little changes, for good reasons, that
made
compiling with NASM changing. Plus i386 vs. amd vs. hardware I don't
have
a bit too much to deal with.


Personally, I would like to learn to properly program in C for OpenBSD.
Yet with so many changes, it's a bit of a constantly moving target.
(Hurrah!)

But as my attempts previously didn't work out too well, I see a problem
for me
and others in my position.
I don't want to hold up any active developers from doing their work.
I really don't think I can get good enough without some hand holding to
get to
a good enough understanding of OpenBSD's usage of C, since there are so
many
points that one needs to be able to tie together to "get it".
I do not have enough money to go back to school to learn C in a class. C
itself
seems pretty simple to use, but hard to put into useful contributions.

Perl has a nice collection of modules that do really useful stuff on
CPAN.
I would guess that OpenBSD has basically the same thing going on in C in
the
src tree. Every time I've tried to follow the chain along I just find
myself
lost and overwhelmed by too much to follow down the rabbit hole.

Is there anyone(s) who, preferably both not busy with active development
who
would genuinely be both willing and capable of helping follow down the
rabbit
hole? I would not be capable of doing that myself if the position were
reversed.
I just don't have the patience and personality to keep up with some
idiot like
me.

I realize that if a hundred people jump up and ask for the same thing,
maybe two
will really mean it and perhaps one will actually follow through.

If someone(s) would like to help, please let me know on or off list.
But let's not waste each others time. I don't want to exchange 20 emails
and then
get ignored. Or vice versa.


As far as the topic being discussed, I think that nothing needs to be
changed.
Lowest level to high level we have is just fine.
Assembly -> C -> Perl
I don't see any need to add to base. It's a good, strong foundation.

Chris Bennett



Reply | Threaded
Open this post in threaded view
|

Re: Integrating "safe" languages into OpenBSD?

webmaster
In reply to this post by Nicolas Schmidt

> -------- Original Message --------
> Subject: Re: Integrating "safe" languages into OpenBSD?
> From: Nick Holland <[hidden email]>
> Date: Mon, December 04, 2017 7:45 am
> To: [hidden email]
>
>
> On 12/03/17 20:19, [hidden email] wrote:
> > I've always subscribed to the idea that too much safety results in too
> > may idiots, and the same is true for all these "safe" programming
> > languages. "Oh I don't have to write any form of bounds-checking,
> > because the language will do it for me."
> >
> > To add further insult to injury, if the language's bounds checking kicks
> > in first your program may do something worse than just corrupting its
> > own memory. In my experience, apps written in these "safe" languages
> > (usually web apps or bloatware) actually have been the most bug-ridden
> > and bloated.
>

> Idiots who shouldn't be coding, coding.
> "safe" languages being trusted to be safe when in the hands of idiots.
> Like you said.
>
> The more I see of "safe" languages, the more I love assembly.  Most
> people who call themselves programmers...shouldn't.
>
> Nick.


The issue of being in base has been raised. Pretty important. Who would
really
want to spend such enormous amounts of time to add in yet another
language?

C is in base. Assembly is in base.
But Perl is also in base. I don't think anyone would want to change all
of the
fantastic pkg_* tools into either C or assembly.
None of these three are "safe" languages.

A while ago I was told that moving to a newer version of Perl was being
held up
by needing to deal with mod_perl. That's obviously been dealt with.

I like Perl's way of flowing and doing things.

I tried learning some assembly on my own, really to just get a better
idea of
what was going on with different C commands and variables.
But the developers kept adding little changes, for good reasons, that
made
compiling with NASM changing. Plus i386 vs. amd vs. hardware I don't
have
a bit too much to deal with.


Personally, I would like to learn to properly program in C for OpenBSD.
Yet with so many changes, it's a bit of a constantly moving target.
(Hurrah!)

But as my attempts previously didn't work out too well, I see a problem
for me
and others in my position.
I don't want to hold up any active developers from doing their work.
I really don't think I can get good enough without some hand holding to
get to
a good enough understanding of OpenBSD's usage of C, since there are so
many
points that one needs to be able to tie together to "get it".
I do not have enough money to go back to school to learn C in a class. C
itself
seems pretty simple to use, but hard to put into useful contributions.

Perl has a nice collection of modules that do really useful stuff on
CPAN.
I would guess that OpenBSD has basically the same thing going on in C in
the
src tree. Every time I've tried to follow the chain along I just find
myself
lost and overwhelmed by too much to follow down the rabbit hole.

Is there anyone(s) who, preferably both not busy with active development
who
would genuinely be both willing and capable of helping follow down the
rabbit
hole? I would not be capable of doing that myself if the position were
reversed.
I just don't have the patience and personality to keep up with some
idiot like
me.

I realize that if a hundred people jump up and ask for the same thing,
maybe two
will really mean it and perhaps one will actually follow through.

If someone(s) would like to help, please let me know on or off list.
But let's not waste each others time. I don't want to exchange 20 emails
and then
get ignored. Or vice versa.


As far as the topic being discussed, I think that nothing needs to be
changed.
Lowest level to high level we have is just fine.
Assembly -> C -> Perl
I don't see any need to add to base. It's a good, strong foundation.

Chris Bennett

Reply | Threaded
Open this post in threaded view
|

Re: Integrating "safe" languages into OpenBSD?

Kevin Chadwick-4
Ada 2012? increased the use of pointers but still limits their usage.

Aside from a couple of mentions in style(9) is there any info on
OpenBSD's rules around pointers or is it simply avoid unless necessary
and following general good practice?

Thanks

Reply | Threaded
Open this post in threaded view
|

Re: Integrating "safe" languages into OpenBSD?

Theo de Raadt-2
> Ada 2012? increased the use of pointers but still limits their usage.
>
> Aside from a couple of mentions in style(9) is there any info on
> OpenBSD's rules around pointers or is it simply avoid unless necessary
> and following general good practice?

Wow what a broad useless question.