Why isn't src included with OpenBSD? (documentation)

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

Why isn't src included with OpenBSD? (documentation)

Andras Farkas
Not sure whether to post this on misc@ or tech@, so trying misc@ first:

Why isn't src included on OpenBSD, perhaps as an install fileset?
Lots of documentation is unavailable outside of the /usr/src tree.

For example, today I had a server mishap which had me using fsck_ffs
after.  I needed to figure out what
PARTIALLY TRUNCATED INODE I=
meant.
I saw in fsck_ffs.8
https://man.openbsd.org/fsck_ffs.8
that the answers could be found in
Fsck_ffs - The UNIX File System Check Program
This is perfectly fine.  Not every piece of information belongs in a
man page.  Man pages are the right format for some sorts of info, and
absolutely the wrong format for some other sorts.
BUT: I looked and couldn't find it, and ended up using
https://docs.freebsd.org/44doc/smm/03.fsck/paper.pdf
which is where I found my answer.
Only after I already solved the problem did I find that the mentioned
file exists here:
https://cvsweb.openbsd.org/cgi-bin/cvsweb/src/sbin/fsck_ffs/SMM.doc/
This is a situation I occasionally run into, where useful
documentation isn't included with OpenBSD, nor is available on
OpenBSD's website (FAQ, etc).  It's occasional, but it's frustrating
every time.

Not only are the USD, PSD, and SMM missing, but other bits of info
often are, too.  For example, I first learnt vi a few years ago, back
when I was first learning Unix, with these files:
https://cvsweb.openbsd.org/src/usr.bin/vi/docs/tutorial/
Without them, or if I didn't find them, I'd have had a much more
difficult time learning vi.

People too young to have grown up with Unix need this sort of
documentation.  We can't live on man pages alone.

Reply | Threaded
Open this post in threaded view
|

Re: Why isn't src included with OpenBSD? (documentation)

Theo de Raadt-2
Andras Farkas <[hidden email]> wrote:

> Not sure whether to post this on misc@ or tech@, so trying misc@ first:
>
> Why isn't src included on OpenBSD, perhaps as an install fileset?

Because then we'd need to adjust the disk-layout expectations on every
architecture, and consider and match a variety of build patterns.
We do not want to do that.

> Lots of documentation is unavailable outside of the /usr/src tree.

Lots of documentation isn't in your brain at birth either, and you
learn where to find it.

> People too young to have grown up with Unix need this sort of
> documentation.  We can't live on man pages alone.

YES WE CAN.

Reply | Threaded
Open this post in threaded view
|

Re: Why isn't src included with OpenBSD? (documentation)

Frank Beuth
On Mon, May 18, 2020 at 11:10:59AM -0600, Theo de Raadt wrote:
>> People too young to have grown up with Unix need this sort of
>> documentation.  We can't live on man pages alone.
>
>YES WE CAN.

Proposed release poster design:

Puffy with puffed out cheeks & paper sticking out of his mouth.

Headline: "Man pages are all you need to live!"

Alternate headlines:
"We *can* live on man pages alone!"
"Man pages: a complete breakfast!"
"Man pages: they're delicious and nutritious!"

Reply | Threaded
Open this post in threaded view
|

Re: Why isn't src included with OpenBSD? (documentation)

Ingo Schwarze
In reply to this post by Andras Farkas
Hi Andras,

Andras Farkas wrote on Mon, May 18, 2020 at 01:07:36PM -0400:

> Lots of documentation is unavailable outside of the /usr/src tree.

It isn't "lots", it's only a tiny number of documents.

> that the answers could be found in
> Fsck_ffs - The UNIX File System Check Program
> This is perfectly fine.

Not really.  If it's reference documentation, it would be better
to have it in the manual page.

Of course, in some cases, whether some piece of information should
be part of the reference documentation or whether it is theoretical
background that would exceed the scope of documentation and be more
adequately explained in a research paper may be a matter of debate.

Explaining the meaning of messages that are displayed to the user
is, IMHO, usually in scope for documentation, unless the meaning
of these messages is totally obvious in the first place.

> https://cvsweb.openbsd.org/cgi-bin/cvsweb/src/sbin/fsck_ffs/SMM.doc/

The ancient non-manual-page roff(7) documents were sacrificed a
decade ago because:

 (1) We want the base system self-contained, and keeping groff -
     which is piece of GNU "C with classes" software - in base
     just to format a handful of historical documents felt
     excessive.  Similarly, writing an -ms input mode for
     mandoc - which would cause a few weeks of work, and add
     probably a few thousand lines of code to the tree -
     also felt excessive for such a small set of documents:
     nobody has been writing new documentation in -ms or -me
     or even -mm for decades.  [kettenis@ did say that he'd
     appreciate a BSD-licensed roff in base, but that would
     cause even more work, so nobody tackled that, either.
     The licensing of the main modern roff implementations (and
     even those unmaintained historical ones that i'm aware of) is
     non-free: groff is GPL (viral), Heirloom and Solaris 10 troff
     are both CDDL (viral), Plan9 is Lucent Public License (polluted
     by verbiage about patents and explicitly asserting U.S. law),
     DWB is Eclipse Public License (viral).]

 (2) While no one denies that some parts of these ancient documents
     can occasionally still be useful, many parts are probably
     outdated since they haven't been maintained for deacdes.
     No one volunteered so far to check their content and
     properly add those parts that still apply to the respective
     manual pages.

 (3) Since they are unmaintained anyway, pointing to old copies
     on the web is not that much worse than having them installed,
     in this special case.  Of course, having the content installed
     would be even better, but it's not trivial to achieve.

> This is a situation I occasionally run into, where useful
> documentation isn't included with OpenBSD, nor is available on
> OpenBSD's website (FAQ, etc).  It's occasional, but it's frustrating
> every time.
>
> Not only are the USD, PSD, and SMM missing, but other bits of info
> often are, too.

In such cases, as an immediate measure to improve the situation,
please submit a patch to the SEE ALSO section of the respective
manual page.  I'd consider it a documnetation bug if a useful
supplemental document exists, no matter whether in the tree or
elsewehere, but isn't mentioned.

> For example, I first learnt vi a few years ago, back
> when I was first learning Unix, with these files:
> https://cvsweb.openbsd.org/src/usr.bin/vi/docs/tutorial/

Hum, looks like a reference to those files is indeed missing
from the manual page.

Also, i don't see what would be wrong with installing them to
  /usr/share/doc/vi/tutorial/
Yes, the tutorial is painfully slow and extremely wordy, and some
parts are hilariously antiquated, like this:
  "Learning a new computer system implies learning a new text editor."
That wasn't even accurate at the time it was presumably written,
probably around 4.4BSD (i.e. almost 30 years ago).  It may have
been more or less true 40 years ago, though.

Can somebody work through the tutorial and confirm that everything
still works as described with our -current vi(1)?  It is too
wordy for my personal taste, so i would hate having to read it all.

Yours,
  Ingo

Reply | Threaded
Open this post in threaded view
|

Re: Why isn't src included with OpenBSD? (documentation)

Marc Espie-2
In reply to this post by Andras Farkas
On Mon, May 18, 2020 at 01:07:36PM -0400, Andras Farkas wrote:

> I saw in fsck_ffs.8
> https://man.openbsd.org/fsck_ffs.8
> that the answers could be found in
> Fsck_ffs - The UNIX File System Check Program
> This is perfectly fine.  Not every piece of information belongs in a
> man page.  Man pages are the right format for some sorts of info, and
> absolutely the wrong format for some other sorts.
> BUT: I looked and couldn't find it, and ended up using
> https://docs.freebsd.org/44doc/smm/03.fsck/paper.pdf
> which is where I found my answer.
> Only after I already solved the problem did I find that the mentioned
> file exists here:
> https://cvsweb.openbsd.org/cgi-bin/cvsweb/src/sbin/fsck_ffs/SMM.doc/

Most of these files haven't been updated in ages.

The few remaining in the source tree are there as reference for people
when writing documentation and updating the source.

Each time we see them, we cringe, and keep tidying the source code and
the man page.

Speaking for personal experience working on make.

If you really want the source code for documentation purposes, what prevents
you from getting it off cvs  or a github mirror ?

Seriously, *none* of those files are necessary *for beginners*. Once you
reach the stage where you might benefit from them (say, because you actually
want to become a developer, and could learn from worse sources), you should
be able to figure out how to get them.

Having an extra set here means... more options... more code that can go
wrong, and thus a SLOWER turn-around for snapshots, which is definitely
a bad thing.

Reply | Threaded
Open this post in threaded view
|

Re: Why isn't src included with OpenBSD? (documentation)

Ottavio Caruso
In reply to this post by Andras Farkas
On Mon, 18 May 2020 at 18:07, Andras Farkas <[hidden email]> wrote:

>
> Not sure whether to post this on misc@ or tech@, so trying misc@ first:
>
> Why isn't src included on OpenBSD, perhaps as an install fileset?
> Lots of documentation is unavailable outside of the /usr/src tree.
>
> For example, today I had a server mishap which had me using fsck_ffs
> after.  I needed to figure out what
> PARTIALLY TRUNCATED INODE I=
> meant.
> I saw in fsck_ffs.8
> https://man.openbsd.org/fsck_ffs.8
> that the answers could be found in
> Fsck_ffs - The UNIX File System Check Program
> This is perfectly fine.  Not every piece of information belongs in a
> man page.  Man pages are the right format for some sorts of info, and
> absolutely the wrong format for some other sorts.
> BUT: I looked and couldn't find it, and ended up using
> https://docs.freebsd.org/44doc/smm/03.fsck/paper.pdf
> which is where I found my answer.
> Only after I already solved the problem did I find that the mentioned
> file exists here:
> https://cvsweb.openbsd.org/cgi-bin/cvsweb/src/sbin/fsck_ffs/SMM.doc/
> This is a situation I occasionally run into, where useful
> documentation isn't included with OpenBSD, nor is available on
> OpenBSD's website (FAQ, etc).  It's occasional, but it's frustrating
> every time.
>
> Not only are the USD, PSD, and SMM missing, but other bits of info
> often are, too.  For example, I first learnt vi a few years ago, back
> when I was first learning Unix, with these files:
> https://cvsweb.openbsd.org/src/usr.bin/vi/docs/tutorial/
> Without them, or if I didn't find them, I'd have had a much more
> difficult time learning vi.
>
> People too young to have grown up with Unix need this sort of
> documentation.  We can't live on man pages alone.
>

I believe you are referring to the historic 4.4 BSD lite documents.
These are interesting for their historical value, and in some cases
(for example vi and the lpd tutorials) are somewhat still relevant.

Some of these documents have a proprietary licence attached to it and
I believe it's due to the 1994 AT&T settlement. There are third party
collections (like this: https://github.com/sergev/4.4BSD-Lite2) but
I'm not sure if one could import them all in the source tree or in the
ports tree without violating some copyright here and there.

--
Ottavio Caruso

Reply | Threaded
Open this post in threaded view
|

Re: Why isn't src included with OpenBSD? (documentation)

Christopher Turkel
In reply to this post by Frank Beuth
On Monday, May 18, 2020, Frank Beuth <[hidden email]> wrote:

> On Mon, May 18, 2020 at 11:10:59AM -0600, Theo de Raadt wrote:
>
>> People too young to have grown up with Unix need this sort of
>>> documentation.  We can't live on man pages alone.
>>>
>>
>> YES WE CAN.
>>
>
> Proposed release poster design:
>
> Puffy with puffed out cheeks & paper sticking out of his mouth.
>
> Headline: "Man pages are all you need to live!"
>
> Alternate headlines:
> "We *can* live on man pages alone!"
> "Man pages: a complete breakfast!"
> "Man pages: they're delicious and nutritious!"



The man pages are excellent! Even a total noob like me found them well
written with plenty of examples. There is also the mailing lists and you
know, $SEARCHENGINE of your choice.
Reply | Threaded
Open this post in threaded view
|

Re: Why isn't src included with OpenBSD? (documentation)

Marc Espie-2
In reply to this post by Ottavio Caruso
On Mon, May 18, 2020 at 08:43:19PM +0100, Ottavio Caruso wrote:
> Some of these documents have a proprietary licence attached to it and
> I believe it's due to the 1994 AT&T settlement. There are third party
> collections (like this: https://github.com/sergev/4.4BSD-Lite2) but
> I'm not sure if one could import them all in the source tree or in the
> ports tree without violating some copyright here and there.

If the README.md is accurate, that could be imported in the ports tree,
but there would be a lot of work to extract useful stuff from there.

Looks like a standard 4-clauses old-style BSD licence.

I've had a look, and it is very strange, some of the PSD documents have
been converted to pdf, BUT not all of them.

There are also no actual releases, so you'd have to pull a specific tag
from github, always hasardous...

Reply | Threaded
Open this post in threaded view
|

Re: Why isn't src included with OpenBSD? (documentation)

Ottavio Caruso
On Tue, 19 May 2020 at 13:31, Marc Espie <[hidden email]> wrote:

>
> On Mon, May 18, 2020 at 08:43:19PM +0100, Ottavio Caruso wrote:
> > Some of these documents have a proprietary licence attached to it and
> > I believe it's due to the 1994 AT&T settlement. There are third party
> > collections (like this: https://github.com/sergev/4.4BSD-Lite2) but
> > I'm not sure if one could import them all in the source tree or in the
> > ports tree without violating some copyright here and there.
>
> If the README.md is accurate, that could be imported in the ports tree,
> but there would be a lot of work to extract useful stuff from there.
>
> Looks like a standard 4-clauses old-style BSD licence.
>
> I've had a look, and it is very strange, some of the PSD documents have
> been converted to pdf, BUT not all of them.

That's because of the legal proviso, as below.

>
> There are also no actual releases, so you'd have to pull a specific tag
> from github, always hasardous...

Not sure if it's safe import from that project, as it lacks all the
legal proviso, like this for example:
https://svnweb.freebsd.org/base/head/share/doc/psd/title/Title?revision=307807&view=co&pathrev=307807

As long as I know, the FreeBSD SVN has all the PSD, SMM and USD source
tree that can be used legally. How one makes an OpenBSD ports from
that, I don't know:
https://svnweb.freebsd.org/base/head/share/doc/

--
Ottavio Caruso

Reply | Threaded
Open this post in threaded view
|

Re: Why isn't src included with OpenBSD? (documentation)

Mihai Popescu-3
In reply to this post by Andras Farkas
> Proposed release poster design:
> Puffy with puffed out cheeks & paper sticking out of his mouth.
> Headline: "Man pages are all you need to live!"
> Alternate headlines:
> "We *can* live on man pages alone!"

I think it's better to clarify who this "we" is/are.
Not _all_ needed info for install and usage is in the man, but I think it
was not the intent to crowd _all_ the info there.
Reply | Threaded
Open this post in threaded view
|

Re: Why isn't src included with OpenBSD? (documentation)

Andras Farkas
In reply to this post by Andras Farkas
First of all, I'd like to thank everyone who replied to me!
Thank you for explaining things.

On Mon, May 18, 2020 at 2:59 PM Ingo Schwarze <[hidden email]> wrote:

> Andras Farkas wrote on Mon, May 18, 2020 at 01:07:36PM -0400:
> > For example, I first learnt vi a few years ago, back
> > when I was first learning Unix, with these files:
> > https://cvsweb.openbsd.org/src/usr.bin/vi/docs/tutorial/
>
> Hum, looks like a reference to those files is indeed missing
> from the manual page.
>
> Also, i don't see what would be wrong with installing them to
>   /usr/share/doc/vi/tutorial/
> Yes, the tutorial is painfully slow and extremely wordy, and some
> parts are hilariously antiquated, like this:
>   "Learning a new computer system implies learning a new text editor."
> That wasn't even accurate at the time it was presumably written,
> probably around 4.4BSD (i.e. almost 30 years ago).  It may have
> been more or less true 40 years ago, though.
I'd say that sentence still contains some truth: when someone's trying
out OpenBSD as a desktop OS when they've only used Windows as a
desktop OS before...

> Can somebody work through the tutorial and confirm that everything
> still works as described with our -current vi(1)?  It is too
> wordy for my personal taste, so i would hate having to read it all.
I can do this.  I'll begin sometime this week, and send a report on
it, and a diff too if necessary.  I'd love to see it included.
:D

On Mon, May 18, 2020 at 3:01 PM Marc Espie <[hidden email]> wrote:

> On Mon, May 18, 2020 at 01:07:36PM -0400, Andras Farkas wrote:
> > I saw in fsck_ffs.8
> > https://man.openbsd.org/fsck_ffs.8
> > that the answers could be found in
> > Fsck_ffs - The UNIX File System Check Program
> > This is perfectly fine.  Not every piece of information belongs in a
> > man page.  Man pages are the right format for some sorts of info, and
> > absolutely the wrong format for some other sorts.
> > BUT: I looked and couldn't find it, and ended up using
> > https://docs.freebsd.org/44doc/smm/03.fsck/paper.pdf
> > which is where I found my answer.
> > Only after I already solved the problem did I find that the mentioned
> > file exists here:
> > https://cvsweb.openbsd.org/cgi-bin/cvsweb/src/sbin/fsck_ffs/SMM.doc/
> Most of these files haven't been updated in ages.
That is true, and important.

> Seriously, *none* of those files are necessary *for beginners*. Once you
> reach the stage where you might benefit from them (say, because you actually
> want to become a developer, and could learn from worse sources), you should
> be able to figure out how to get them.
I disagree.  In general: man pages are reference material (meant for
people who are already familiar with unix or with the man page's
topic, often not meant to be read cover to cover, definitely doesn't
hold your hand) and in contrast, a considerable amount of the
documentation (though certainly not all!) in /usr/src is tutorial
material (meant for learning something for the first time, easily read
cover to cover, holds your hand) usable by newbs.
Examples supporting this comparison:
man 1 ed versus the Brian W. Kernighan "A Tutorial Introduction to the
UNIX Text Editor"
man 1 yacc versus the Stephen C. Johnson Yacc "Yacc: Yet Another
Compiler-Compiler"
man 1 vi versus vi.beginner (though indeed, vi.1 has the Fast Startup
section, vi.1 is an exception rather than the norm)
Just looking at a list of BSD USD (UNIX User's Supplementary
Documents) documents or Unix v7 supplementary documents (The UNIX
Programmer's Manual volume 2) can provide more examples.  Some of them
don't particularly have a man page they match up to, by being broader
and more newb-oriented in theme.
The only point against this, though a BIG and important one, is that
the datedness of the tutorial materials does make them less usable for
newbs.  I agree much of the stuff is outdated and less usable than it
used to be.  I also think having it with a disclaimer is better than
not having it at all.  Ideally... I myself should put in effort into
checking some of them and fixing and updating them.

Also: I do like OpenBSD's man pages and FAQ!  Don't let anything I've
said above convince you otherwise.  I use them a lot and appreciate
them.
Thanks so much, everybody.

Reply | Threaded
Open this post in threaded view
|

Re: Why isn't src included with OpenBSD? (documentation)

Sebastian Benoit
In reply to this post by Andras Farkas
Andras Farkas([hidden email]) on 2020.05.18 13:07:36 -0400:
> Not sure whether to post this on misc@ or tech@, so trying misc@ first:
>
> Why isn't src included on OpenBSD, perhaps as an install fileset?
> Lots of documentation is unavailable outside of the /usr/src tree.
[...]
> This is a situation I occasionally run into, where useful
> documentation isn't included with OpenBSD, nor is available on
> OpenBSD's website (FAQ, etc).  It's occasional, but it's frustrating
> every time.

There you are:

  https://ftp.openbsd.org/pub/OpenBSD/6.7/src.tar.gz

ready to be unpacked at a place of your convenience.

Reply | Threaded
Open this post in threaded view
|

Re: Why isn't src included with OpenBSD? (documentation)

Ingo Schwarze
In reply to this post by Andras Farkas
Hi Andras,

Andras Farkas wrote on Tue, May 19, 2020 at 05:26:24PM -0400:
> On Mon, May 18, 2020 at 2:59 PM Ingo Schwarze <[hidden email]> wrote:

>> Can somebody work through the tutorial and confirm that everything
>> still works as described with our -current vi(1)?  It is too
>> wordy for my personal taste, so i would hate having to read it all.

> I can do this.  I'll begin sometime this week, and send a report on
> it, and a diff too if necessary.  I'd love to see it included.
> :D

Good, i'll just wait for your results.

> In general: man pages are reference material

Yes.

> (meant for people who are already familiar with unix or with the man
> page's topic,

I disagree with your definition of "reference manual".  A reference
manual is simply documentation of a tool that is complete, precise,
concise, and organized according to the inner logic of the tool.
Whether the reader is already familiar with the topic has nothing
to do with it.  Which preliminary knowledge is already needed to
even start contemplating a topic has nothing to do with it either.
You can't use the vi(1) tutorial either unless you already know what
these terms refer to: a directory, a file, a text file, a shell, ...

When i want to learn about a completely new tool, i usually strongly
prefer a good reference manual over a good user manual (i.e. an
incomplete manual trying to use simplified terms and to provide
simplified but longer explanations, organized according to didactial
considerations) over a good tutotial (and often i even prefer a
mediocre reference manual over good documentation of any other
kind).  In user manuals and tutorials, i rarely read more than the
section titles.  Reading those titles is occasionally useful to
understand what the software authors consider most relevant for
beginners.  Reading substantial amounts of text in a user manual
or tutorial usually feels like a waste of time to me.  If i need
the tool just once, there is just too much text in non-reference
documentation, and the relevant bits are harder to find.  If i
potentially want to use the tool often, sooner or later, i have to
read the reference manual anyway, so again no point to additionally
read user manuals or tutorials, which are often longer.  (I realise
this in part involves personal learning style, people who like
tutorials and user manuals do exist, but i guess they are less
common among OpenBSD users than elsewhere.)

> often not meant to be read cover to cover,

Most of our reference manuals = manual pages are well suited to
being read from cover to cover, and using them that way is definitely
encouraged.  There are some rare exceptions, though, for example
perlfunc(1) and openssl(1).  (To avoid misunderstanding, of course
they are also well-suited to looking up particular details you
aren't sure you remember correctly.)

> definitely doesn't hold your hand)

When needed, they most definitely do hand-holding.
For example, look at

  https://man.openbsd.org/httpd.conf#EXAMPLES

Often, hand-holding just wouldn't be useful:

  https://man.openbsd.org/true

> a considerable amount of the documentation (though certainly not all!)
> in /usr/src is tutorial material (meant for learning something for the
> first time, easily read cover to cover, holds your hand) usable by newbs.

If they are correct and work with the current code, tutorials are
appropriate for installation in /usr/share/doc/ - like the mg(1)
tutorial which is already installed there.  Completeness is irrelevant
for tutorials.

If there are few tutorials there, that's likely because OpenBSD
developers were less eager to spend time on them than they were
about spending time elsewhere, and nobody else jumped in either.

We don't want user manuals in OpenBSD because we can't afford the
extra maintenance effort.  Even for a bloody beginner, an outdated
or inaccurate user manual is much worse than a good reference manual.

Yours,
  Ingo

Reply | Threaded
Open this post in threaded view
|

Re: Why isn't src included with OpenBSD? (documentation)

Chris Bennett-4
I keep seeing people not getting the idea that OpenBSD has more of a
philosophy of users needing to put out their own special efforts at
learning, vs. other OS's.

Do you think that mentioning this on the homepage/FAQ would be useful?

It took me quite a while to understand that myself. Realizing that
brought me a great relief and no longer feeling frustrated.
I found it a bit inspiring and more enthusiastic about the whole
project.

Read the code because you MUST versus because you ought to.
I find that as a path to follow, pridefully.

Chris Bennett


Reply | Threaded
Open this post in threaded view
|

Re: Why isn't src included with OpenBSD? (documentation)

Theo de Raadt-2
No.

Because what you say isn't true at all.  That's limiting spin.



Chris Bennett <[hidden email]> wrote:

> I keep seeing people not getting the idea that OpenBSD has more of a
> philosophy of users needing to put out their own special efforts at
> learning, vs. other OS's.
>
> Do you think that mentioning this on the homepage/FAQ would be useful?
>
> It took me quite a while to understand that myself. Realizing that
> brought me a great relief and no longer feeling frustrated.
> I found it a bit inspiring and more enthusiastic about the whole
> project.
>
> Read the code because you MUST versus because you ought to.
> I find that as a path to follow, pridefully.
>
> Chris Bennett
>
>