outside base directory warnings when install package "grace"

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|

outside base directory warnings when install package "grace"

Norm Finlay
Hello:

Installing the graphing package "grace" created the following warnings.
This occurred in the v6.3 release.  The folks on irc suggested I email this
to [hidden email].

Sincerely,
Norm Finlay
Dept of Physics and Astronomy
Brandon University
Brandon MB Canada



# uname -a
OpenBSD mirage.Brandonu.ca 6.3 GENERIC.MP#107 amd64


# pkg_add -v -i grace
Update candidates: quirks-2.414 -> quirks-2.414
quirks-2.414 signed on 2018-03-28T14:24:37Z
grace-5.1.15p7:t1lib-5.1.2p0: ok
grace-5.1.15p7:pdflib-4.0.3p5: ok
grace-5.1.15p7: ok
/usr/local/man/: /usr/local/grace/doc/grace.1: outside base directory
/usr/local/man/: /usr/local/grace/doc/grace.1: outside base directory
/usr/local/man/: /usr/local/grace/doc/grace.1: outside base directory
/usr/local/man/: /usr/local/grace/doc/grconvert.1: outside base directory
system(/usr/sbin/makewhatis, -d, /usr/local/man, --,
/usr/local/man/man1/xmgrace.1, /usr/local/man/man1/grace.1,
/usr/local/man/man1/gracebat.1, /usr/local/man/man1/grconvert.1) failed:
exit(5)
Extracted 9356432 from 9358312

mirage# ls /usr/local
bin     grace   include info    lib     libdata libexec man     sbin
share
mirage# ls
/usr/local/grace/
auxiliary    doc          fonts        gracerc.user lib
bin          examples     gracerc      include      templates
Reply | Threaded
Open this post in threaded view
|

Re: outside base directory warnings when install package "grace"

Ingo Schwarze
Hi,

Norm Finlay wrote on Wed, Jun 20, 2018 at 04:55:04PM -0500:

> Installing the graphing package "grace" created the following warnings.
> This occurred in the v6.3 release.  The folks on irc suggested I email this
> to [hidden email].

Right.  The post-install target is plainly wrong.  Never install
manual pages outside the intended manpath.  Mandoc doesn't allow
that.

Here is a fix.

Also add the missing @bin markers while here.

OK?
  Ingo


Index: Makefile
===================================================================
RCS file: /cvs/ports/math/grace/Makefile,v
retrieving revision 1.43
diff -u -p -r1.43 Makefile
--- Makefile 5 May 2016 19:52:28 -0000 1.43
+++ Makefile 21 Jun 2018 16:04:40 -0000
@@ -3,7 +3,7 @@
 COMMENT= GRaphing, Advanced Computation and Exploration of data
 
 DISTNAME= grace-5.1.15
-REVISION= 7
+REVISION= 8
 CATEGORIES= math
 
 HOMEPAGE= http://plasma-gate.weizmann.ac.il/Grace/
@@ -71,12 +71,12 @@ post-install:
  ln -sf ${GRACE_DIR}/bin/xmgrace ${PREFIX}/bin/grace
  ln -sf ${GRACE_DIR}/bin/xmgrace ${PREFIX}/bin/gracebat
  ln -sf ${GRACE_DIR}/bin/xmgrace ${PREFIX}/bin/xmgrace
- ln -sf ../../grace/doc/xmgrace.1 ${PREFIX}/man/man1/xmgrace.1
+ mv ${PREFIX}/grace/doc/xmgrace.1 ${PREFIX}/man/man1/xmgrace.1
 .endif
  ln -sf ${GRACE_DIR}/include/grace_np.h ${PREFIX}/include/grace_np.h
  ln -sf ${GRACE_DIR}/lib/libgrace_np.a ${PREFIX}/lib/libgrace_np.a
- ln -sf ../../grace/doc/grace.1 ${PREFIX}/man/man1/grace.1
- ln -sf ../../grace/doc/gracebat.1 ${PREFIX}/man/man1/gracebat.1
- ln -sf ../../grace/doc/grconvert.1 ${PREFIX}/man/man1/grconvert.1
+ mv ${PREFIX}/grace/doc/grace.1 ${PREFIX}/man/man1/grace.1
+ mv ${PREFIX}/grace/doc/gracebat.1 ${PREFIX}/man/man1/gracebat.1
+ mv ${PREFIX}/grace/doc/grconvert.1 ${PREFIX}/man/man1/grconvert.1
 
 .include <bsd.port.mk>
Index: pkg/PFRAG.no-no_x11
===================================================================
RCS file: /cvs/ports/math/grace/pkg/PFRAG.no-no_x11,v
retrieving revision 1.2
diff -u -p -r1.2 PFRAG.no-no_x11
--- pkg/PFRAG.no-no_x11 8 Oct 2004 10:37:24 -0000 1.2
+++ pkg/PFRAG.no-no_x11 21 Jun 2018 16:04:40 -0000
@@ -1,5 +1,4 @@
 @comment $OpenBSD: PFRAG.no-no_x11,v 1.2 2004/10/08 10:37:24 espie Exp $
 bin/xmgrace
-grace/bin/xmgrace
-grace/doc/xmgrace.1
+@bin grace/bin/xmgrace
 @man man/man1/xmgrace.1
Index: pkg/PLIST
===================================================================
RCS file: /cvs/ports/math/grace/pkg/PLIST,v
retrieving revision 1.6
diff -u -p -r1.6 PLIST
--- pkg/PLIST 8 Oct 2004 10:37:24 -0000 1.6
+++ pkg/PLIST 21 Jun 2018 16:04:40 -0000
@@ -7,10 +7,10 @@ grace/auxiliary/README
 grace/auxiliary/convcal.c
 grace/auxiliary/magic.grace
 grace/bin/
-grace/bin/convcal
+@bin grace/bin/convcal
 grace/bin/fdf2fit
 grace/bin/gracebat
-grace/bin/grconvert
+@bin grace/bin/grconvert
 grace/doc/
 grace/doc/10.1.dat
 grace/doc/10a.dat
@@ -30,9 +30,6 @@ grace/doc/Tutorial.html
 grace/doc/UsersGuide.dvi
 grace/doc/UsersGuide.html
 grace/doc/convcal.1
-grace/doc/grace.1
-grace/doc/gracebat.1
-grace/doc/grconvert.1
 grace/doc/mygraph.png
 grace/doc/nohelp.html
 grace/doc/philosophical-gnu-sm.jpg

Reply | Threaded
Open this post in threaded view
|

Re: outside base directory warnings when install package "grace"

Landry Breuil-5
On Thu, Jun 21, 2018 at 06:09:31PM +0200, Ingo Schwarze wrote:

> Hi,
>
> Norm Finlay wrote on Wed, Jun 20, 2018 at 04:55:04PM -0500:
>
> > Installing the graphing package "grace" created the following warnings.
> > This occurred in the v6.3 release.  The folks on irc suggested I email this
> > to [hidden email].
>
> Right.  The post-install target is plainly wrong.  Never install
> manual pages outside the intended manpath.  Mandoc doesn't allow
> that.

Hmmm. Isnt it what tcl/tk and a bunch of other ports are doing, for
perfectly valid reasons ? what if you have a package installing
manpages with 'generic' names ?

Reply | Threaded
Open this post in threaded view
|

Re: outside base directory warnings when install package "grace"

Stuart Henderson
On 2018/06/22 09:39, Landry Breuil wrote:

> On Thu, Jun 21, 2018 at 06:09:31PM +0200, Ingo Schwarze wrote:
> > Hi,
> >
> > Norm Finlay wrote on Wed, Jun 20, 2018 at 04:55:04PM -0500:
> >
> > > Installing the graphing package "grace" created the following warnings.
> > > This occurred in the v6.3 release.  The folks on irc suggested I email this
> > > to [hidden email].
> >
> > Right.  The post-install target is plainly wrong.  Never install
> > manual pages outside the intended manpath.  Mandoc doesn't allow
> > that.
>
> Hmmm. Isnt it what tcl/tk and a bunch of other ports are doing, for
> perfectly valid reasons ? what if you have a package installing
> manpages with 'generic' names ?
>

That's a different case, there the manuals are just installed to a
different directory and you use -m/MANPATH/man.conf to locate them,
here they're installed to a different directory and symlinked.

Reply | Threaded
Open this post in threaded view
|

Re: outside base directory warnings when install package "grace"

Ingo Schwarze
Hi,

Stuart Henderson wrote on Fri, Jun 22, 2018 at 10:05:07AM +0100:
> On 2018/06/22 09:39, Landry Breuil wrote:
>> On Thu, Jun 21, 2018 at 06:09:31PM +0200, Ingo Schwarze wrote:
>>> Norm Finlay wrote on Wed, Jun 20, 2018 at 04:55:04PM -0500:

>>>> Installing the graphing package "grace" created the following warnings.
>>>> This occurred in the v6.3 release.  The folks on irc suggested I email this
>>>> to [hidden email].

>>> Right.  The post-install target is plainly wrong.  Never install
>>> manual pages outside the intended manpath.  Mandoc doesn't allow
>>> that.

>> Hmmm. Isnt it what tcl/tk and a bunch of other ports are doing, for
>> perfectly valid reasons ? what if you have a package installing
>> manpages with 'generic' names ?

> That's a different case, there the manuals are just installed to a
> different directory and you use -m/MANPATH/man.conf to locate them,
> here they're installed to a different directory and symlinked.

Stuat is of course spot on.  I already had the following more detailed
explanation written when his answer arrived and just failed to send it;
sending it anyway for the possible benefit of list members less
familiar with how these things work.

I'm appending the proposed patch again at the end for convenience
in case anybody wants to OK it.

Yours,
  Ingo


 ----- 8< ----- schnipp ----- >8 ----- 8< ----- schnapp ----- >8 -----

> Hmmm. Isnt it what tcl/tk and a bunch of other ports are doing, for
> perfectly valid reasons ?

Sorry for being unclear by being terse.  For tcl and some other
ports, the intended manpath is e.g. /usr/local/lib/tcl/tcl8.6/man/.
That manpath is intentionally outside the manpaths searched by
default, such that it can only be accessed by explicit -M/-m, by
setting $MANPATH, or by setting "manpath" in man.conf(5).  That is OK.

What this port is trying to do is installing the manual pages in
/usr/local/grace/doc/, which is outside any manpath, and then trying
to symlink them into /usr/local/man/.  That does not fully work
with makewhatis(8).

> what if you have a package installing
> manpages with 'generic' names ?

You mean like /usr/local/lib/tcl/tcl8.6/man/mann/close.n ?
In a private manpath, that is OK, it won't disturb "man close".

Or you mean like "ls" from GNU coreutils wanting to install ls(1)?
That is not OK, and that's why we rename it to "gls" - both the
program and the manual.  Note that you still get

   $ man -w ls
  /usr/share/man/man1/ls.1
  /usr/local/man/man1/gls.1

   $ man -a ls | col -b | grep -A 1 ^NAME
  NAME
     ls  list directory contents
  NAME
       ls - list directory contents

But again, that's OK, in plain man(1), base system pages are
preferred, and that you also get the ports variants with -w and -a
is actually helpful.

Or you mean like python-2.7.14 and python-3.6.5 both wanting to
install python(1)?  That's usually solved by having file names like

   $ man -w python
  /usr/local/man/man1/python2.1
  /usr/local/man/man1/python3.1

with the NAME sections in both just saying "python" without a
version number, such that "man python" can find both.

 ----- 8< ----- schnipp ----- >8 ----- 8< ----- schnapp ----- >8 -----

Index: Makefile
===================================================================
RCS file: /cvs/ports/math/grace/Makefile,v
retrieving revision 1.43
diff -u -p -r1.43 Makefile
--- Makefile 5 May 2016 19:52:28 -0000 1.43
+++ Makefile 21 Jun 2018 16:04:40 -0000
@@ -3,7 +3,7 @@
 COMMENT= GRaphing, Advanced Computation and Exploration of data
 
 DISTNAME= grace-5.1.15
-REVISION= 7
+REVISION= 8
 CATEGORIES= math
 
 HOMEPAGE= http://plasma-gate.weizmann.ac.il/Grace/
@@ -71,12 +71,12 @@ post-install:
  ln -sf ${GRACE_DIR}/bin/xmgrace ${PREFIX}/bin/grace
  ln -sf ${GRACE_DIR}/bin/xmgrace ${PREFIX}/bin/gracebat
  ln -sf ${GRACE_DIR}/bin/xmgrace ${PREFIX}/bin/xmgrace
- ln -sf ../../grace/doc/xmgrace.1 ${PREFIX}/man/man1/xmgrace.1
+ mv ${PREFIX}/grace/doc/xmgrace.1 ${PREFIX}/man/man1/xmgrace.1
 .endif
  ln -sf ${GRACE_DIR}/include/grace_np.h ${PREFIX}/include/grace_np.h
  ln -sf ${GRACE_DIR}/lib/libgrace_np.a ${PREFIX}/lib/libgrace_np.a
- ln -sf ../../grace/doc/grace.1 ${PREFIX}/man/man1/grace.1
- ln -sf ../../grace/doc/gracebat.1 ${PREFIX}/man/man1/gracebat.1
- ln -sf ../../grace/doc/grconvert.1 ${PREFIX}/man/man1/grconvert.1
+ mv ${PREFIX}/grace/doc/grace.1 ${PREFIX}/man/man1/grace.1
+ mv ${PREFIX}/grace/doc/gracebat.1 ${PREFIX}/man/man1/gracebat.1
+ mv ${PREFIX}/grace/doc/grconvert.1 ${PREFIX}/man/man1/grconvert.1
 
 .include <bsd.port.mk>
Index: pkg/PFRAG.no-no_x11
===================================================================
RCS file: /cvs/ports/math/grace/pkg/PFRAG.no-no_x11,v
retrieving revision 1.2
diff -u -p -r1.2 PFRAG.no-no_x11
--- pkg/PFRAG.no-no_x11 8 Oct 2004 10:37:24 -0000 1.2
+++ pkg/PFRAG.no-no_x11 21 Jun 2018 16:04:40 -0000
@@ -1,5 +1,4 @@
 @comment $OpenBSD: PFRAG.no-no_x11,v 1.2 2004/10/08 10:37:24 espie Exp $
 bin/xmgrace
-grace/bin/xmgrace
-grace/doc/xmgrace.1
+@bin grace/bin/xmgrace
 @man man/man1/xmgrace.1
Index: pkg/PLIST
===================================================================
RCS file: /cvs/ports/math/grace/pkg/PLIST,v
retrieving revision 1.6
diff -u -p -r1.6 PLIST
--- pkg/PLIST 8 Oct 2004 10:37:24 -0000 1.6
+++ pkg/PLIST 21 Jun 2018 16:04:40 -0000
@@ -7,10 +7,10 @@ grace/auxiliary/README
 grace/auxiliary/convcal.c
 grace/auxiliary/magic.grace
 grace/bin/
-grace/bin/convcal
+@bin grace/bin/convcal
 grace/bin/fdf2fit
 grace/bin/gracebat
-grace/bin/grconvert
+@bin grace/bin/grconvert
 grace/doc/
 grace/doc/10.1.dat
 grace/doc/10a.dat
@@ -30,9 +30,6 @@ grace/doc/Tutorial.html
 grace/doc/UsersGuide.dvi
 grace/doc/UsersGuide.html
 grace/doc/convcal.1
-grace/doc/grace.1
-grace/doc/gracebat.1
-grace/doc/grconvert.1
 grace/doc/mygraph.png
 grace/doc/nohelp.html
 grace/doc/philosophical-gnu-sm.jpg

Reply | Threaded
Open this post in threaded view
|

Re: outside base directory warnings when install package "grace"

Stuart Cassoff-4
I sometimes wonder if /usr/local/share/doc/tcl8.6/man/ would be a better place.

Stu

Reply | Threaded
Open this post in threaded view
|

Re: outside base directory warnings when install package "grace"

Ingo Schwarze
Hi Stuart,

Stuart Cassoff wrote on Sat, Jun 23, 2018 at 08:30:36AM -0400:

> I sometimes wonder if /usr/local/share/doc/tcl8.6/man/
> would be a better place.

That question is part of a bigger picture: logical organization of
manual pages and representating that organization in the file system,
on the command line, and in URIs.

The corpus of all manual pages can be organized on the following
levels:

 * Operating system, e.g. OpenBSD, FreeBSD, NetBSD, ...
   File system representation: Not needed because on a local
     machine, you have only one system installed.
     (Uh oh, but we do have the books/man-pages-posix port
     using /usr/local/share/doc/posix/man/
     but lacking the @mandir annotation.)
   Command line representation: Not needed for the same reason.
     But note that Eaton man (ca. 1991-1993) and hence the ancient
     man-1.6 still used by some Linux systems and the more modern
     man-db used by some other Linux systems do have a command
     line option to display pages for a foreign operating system
     and abuse the -m option for it, even though -m was already
     used earlier in 4.3BSD-Reno (1990) in the modern sense now
     known in OpenBSD and NetBSD.
     I think we should *not* introduce a command line option
     for displaying foreign manual pages.
   URI representation: /*system*-version/arch/name.section -
     that is, using a dedicated family of manpaths.

 * Operating system version, e.g. 5.2, 5.3, 5.4, ...
   File system representation: Not needed for the same reason.
   Command line representation: Not needed for the same reason.
   URI representation: /system-*version*/arch/name.section -
     that is, using a dedicated manpath.

 * Subsystem, e.g. tcl, tk, lowfat, openssl, ...
   File system representation:
     - in a top-level portdir (seems rather flat):
       /usr/local/heirloom-doctools/man/
       /usr/local/jdk-1.8.0/man/
       /usr/local/jre-1.8.0/man/
       /usr/local/mingw32/man/
     - below share/portdir (also a bit flat):
       /usr/local/share/lowfat/man/
     - below lib/portdir (seems somewhat wrong, manuals aren't libs):
       /usr/local/lib/eopenssl/man/
       /usr/local/lib/erlang16/man/
       /usr/local/lib/qt3/man/
       /usr/local/lib/tcl/{tcl,tk}8.{5,6}/man/
       /usr/local/lib/stk/4.0.11/man/
     - totally broken:
       /usr/local/man/man3am/   # lang/gawk
     - The most logical /usr/local/share/doc/portdir/man/
       does not seem to occur except for man-pages-posix;
       then again, it is awkwardly long.
    This is all seriously inconsistent, part of which can be explained
    by the usual upstream inconsistentcy of some ports using a
    top-level portdir and some distributing their files logically
    across common dirs like lib and share.
    Note that man-db represents subsystem names as suffixes to
    numerical section numbers, for example 3tcl, 3posix, ...
    That is slightly confusing because logically, subsystems form
    a higher level of organization than numerical sections; for
    example, OpenSSL has pages in sections 1, 3, 5, and 7.
    Solaris and illumos also use a wide variety of such suffixes
    for similar purposes, basically for both functional and genetic
    subsystems.
    Then again, it's merely a matter of the order of the two syntax
    elements "subsystem name" and "section number", which probably
    doesn't matter all that much.  So, in order to not invent our
    own bikeshed that differs only in colour, maybe we *should* use
    paths like
      /usr/local/man/man3tcl8.6
      /usr/local/man/man3posix
    in the future.  That does need adjustments in mandoc though to
    not search such directories by default - or maybe there even
    isn't even anything wrong with searching them by default as
    long as the non-suffixed directories are preferred?
  Command line representation:  Traditionally, subsystems were
    supposed to be accessed with man(1) -s just like numerical
    sections, but that functionality was never fully implemented
    as far as i'm aware, in the sense that the old man.conf(5) only
    supported "-s X11", "-s X11R6", and "-s local" by default, which
    was almost useless, so support was removed with the new man.conf(5)
    format.
    If we go for the above subsystem suffix scheme, we get the side
    benefit of restoring part of that traditional functionality with
    a more useful practical application, namely that "man -s 3tcl8.6"
    will just work, and if we want, we can let "man -s tcl8.6" search
    all the sections [1-9]tcl8.6, it's quite easy to implement.
  URI representation: None so far, and not needed unless we want to
    publish ports manual pages on man.openbsd.org.

 * Language, e.g. es, ja, pt_BR, zh_CN, ...
   File system representation: /usr/local/man/*language*/
   Command line representation: Use -m or -M.
     Note that man-db inspects $LC_MESSAGES for this purpose, but
     that is logically wrong, because even if users want to see
     translated messages, that doesn't imply they also want translated
     manual pages, which are almost always outdated, buggy, and
     incomplete.  We should not follow that bad practice - and
     besides, we do not support $LC_MESSAGES in the first place.
   URI representation: Not needed.

 * (Numerical) section.
   File system representation: /usr/local/man/man*section*/
   Command line representation: -s
   URI representation: /system-version/arch/name.*section*


So, what do you think about consistently moving stuff to

  /usr/local/man/man*section**suffix*/

and supporting

  man -s *section**suffix*

and

  man -s *suffix*

in addition to

  man -s *section* ?

If so, should the following work out of the box when tcl is installed,
or should they require explicitly specifying -s tcl - i tend to think
they should just work?

  man [-k] tclsh
  man Tcl_WaitForEvent
  man -k WaitForEvent

Yours,
  Ingo

Reply | Threaded
Open this post in threaded view
|

Re: outside base directory warnings when install package "grace"

Marc Espie-2
On Sat, Jun 23, 2018 at 07:01:36PM +0200, Ingo Schwarze wrote:

> Hi Stuart,
>
> Stuart Cassoff wrote on Sat, Jun 23, 2018 at 08:30:36AM -0400:
>
> > I sometimes wonder if /usr/local/share/doc/tcl8.6/man/
> > would be a better place.
>
> That question is part of a bigger picture: logical organization of
> manual pages and representating that organization in the file system,
> on the command line, and in URIs.
>
> The corpus of all manual pages can be organized on the following
> levels:
>
>  * Operating system, e.g. OpenBSD, FreeBSD, NetBSD, ...
>    File system representation: Not needed because on a local
>      machine, you have only one system installed.
>      (Uh oh, but we do have the books/man-pages-posix port
>      using /usr/local/share/doc/posix/man/
>      but lacking the @mandir annotation.)
>    Command line representation: Not needed for the same reason.
>      But note that Eaton man (ca. 1991-1993) and hence the ancient
>      man-1.6 still used by some Linux systems and the more modern
>      man-db used by some other Linux systems do have a command
>      line option to display pages for a foreign operating system
>      and abuse the -m option for it, even though -m was already
>      used earlier in 4.3BSD-Reno (1990) in the modern sense now
>      known in OpenBSD and NetBSD.
>      I think we should *not* introduce a command line option
>      for displaying foreign manual pages.
>    URI representation: /*system*-version/arch/name.section -
>      that is, using a dedicated family of manpaths.
>
It would still be *very cool* if we could have optional paths in
man confs, just so that man would find /usr/ports/infrastructure/man
if it's around, and not give errors if it's not...

Just having to add that on each ports build machine is a pain in the butt

Reply | Threaded
Open this post in threaded view
|

Re: outside base directory warnings when install package "grace"

Ingo Schwarze
Hi Marc,

Marc Espie wrote on Sat, Jun 23, 2018 at 07:37:25PM +0200:

> It would still be *very cool* if we could have optional paths in
> man confs, just so that man would find /usr/ports/infrastructure/man
> if it's around, and not give errors if it's not...
>
> Just having to add that on each ports build machine is a pain in the butt

That is relatively unrelated to what Stuart Cassoff and myself were
discussing, which was (1) ports manual pages as opposed to
infrastructure manual pages and (2) where to install them as opposed
to whether to search them by default.

That said, the question whether to search /usr/ports/infrastructure/man/
by default was discussed before.  I was mildly in favour of it, but
there was no consensus that people wanted it, so in the end, we stopped
doing it:


RCS file: /cvs/src/etc/Attic/man.conf,v
[...]
revision 1.22
date: 2015/01/20 19:17:28;  author: schwarze;  state: Exp;  lines: +2 -3
Do not use /usr/ports/infrastructure/man/ by default because this
directory is not contained in OpenBSD base, and because even people
having the directory often don't understand that they need to run
makewhatis(8) - and instead complain about the resulting warnings.
This commit reverts revisions 1.17 and 1.21.

Requested by deraadt@ millert@ kettenis@ who argue that people
using /usr/ports/infrastructure/bin/ already need to set PATH,
so editing man.conf (or, though more fragile, setting MANPATH)
should not be a big deal for them.
----------------------------
revision 1.21
date: 2014/12/09 05:55:42;  author: schwarze;  state: Exp;  lines: +3 -2
If it exists, use /usr/ports/infrastructure/man/ by default for apropos(1);
we already used it by default for man(1) before.
While here, fix the order of Xenocara and ports.
ok millert@ sthen@
----------------------------
[...]
revision 1.17
date: 2011/04/11 14:45:41;  author: sthen;  state: Exp;  lines: +2 -2;
allow man(1) to find the ports/infrastructure manpages by default.
ok dcoppa@ phessler@ jasper@ landry@ rpointel@ giovanni@ ajacoutot@ henning@
no problem jmc@


Nowadays, including it by default would be done by MANPATH_DEFAULT
in manpath.c, not via /usr/src/etc/man.conf, which no longer exists.
But unless deraadt@ millert@ kettenis@ speak up, i assume that
there still isn't consensus that we want that directory searched
by default.

Yours,
  Ingo

P.S.
If we wanted the directory searched by default, the warning that
/usr/ports/infrastructure/man/ does not contain mandoc.db(5) before
the first weekly(8) run could certainly be suppressed in some way.

Reply | Threaded
Open this post in threaded view
|

Re: outside base directory warnings when install package "grace"

Stuart Cassoff-4
Perhaps a mention in ports(7) about adding the ports man dir to /etc/man.conf?
The thought of an /etc/rc.d/ports or something in rc.* crossed my mind as well.

Stu

> ---------- Original Message ----------
> From: Ingo Schwarze <[hidden email]>
> Date: June 23, 2018 at 3:13 PM
>
>
> Hi Marc,
>
> Marc Espie wrote on Sat, Jun 23, 2018 at 07:37:25PM +0200:
>
> > It would still be *very cool* if we could have optional paths in
> > man confs, just so that man would find /usr/ports/infrastructure/man
> > if it's around, and not give errors if it's not...
> >
> > Just having to add that on each ports build machine is a pain in the butt
>
> That is relatively unrelated to what Stuart Cassoff and myself were
> discussing, which was (1) ports manual pages as opposed to
> infrastructure manual pages and (2) where to install them as opposed
> to whether to search them by default.
>
> That said, the question whether to search /usr/ports/infrastructure/man/
> by default was discussed before.  I was mildly in favour of it, but
> there was no consensus that people wanted it, so in the end, we stopped
> doing it:
>
>
> RCS file: /cvs/src/etc/Attic/man.conf,v
> [...]
> revision 1.22
> date: 2015/01/20 19:17:28;  author: schwarze;  state: Exp;  lines: +2 -3
> Do not use /usr/ports/infrastructure/man/ by default because this
> directory is not contained in OpenBSD base, and because even people
> having the directory often don't understand that they need to run
> makewhatis(8) - and instead complain about the resulting warnings.
> This commit reverts revisions 1.17 and 1.21.
>
> Requested by deraadt@ millert@ kettenis@ who argue that people
> using /usr/ports/infrastructure/bin/ already need to set PATH,
> so editing man.conf (or, though more fragile, setting MANPATH)
> should not be a big deal for them.
> ----------------------------
> revision 1.21
> date: 2014/12/09 05:55:42;  author: schwarze;  state: Exp;  lines: +3 -2
> If it exists, use /usr/ports/infrastructure/man/ by default for apropos(1);
> we already used it by default for man(1) before.
> While here, fix the order of Xenocara and ports.
> ok millert@ sthen@
> ----------------------------
> [...]
> revision 1.17
> date: 2011/04/11 14:45:41;  author: sthen;  state: Exp;  lines: +2 -2;
> allow man(1) to find the ports/infrastructure manpages by default.
> ok dcoppa@ phessler@ jasper@ landry@ rpointel@ giovanni@ ajacoutot@ henning@
> no problem jmc@
>
>
> Nowadays, including it by default would be done by MANPATH_DEFAULT
> in manpath.c, not via /usr/src/etc/man.conf, which no longer exists.
> But unless deraadt@ millert@ kettenis@ speak up, i assume that
> there still isn't consensus that we want that directory searched
> by default.
>
> Yours,
>   Ingo
>
> P.S.
> If we wanted the directory searched by default, the warning that
> /usr/ports/infrastructure/man/ does not contain mandoc.db(5) before
> the first weekly(8) run could certainly be suppressed in some way.
>

Reply | Threaded
Open this post in threaded view
|

ports/infrastructure/man by default

Landry Breuil-5
In reply to this post by Ingo Schwarze
On Sat, Jun 23, 2018 at 09:13:50PM +0200, Ingo Schwarze wrote:

> Hi Marc,
>
> Marc Espie wrote on Sat, Jun 23, 2018 at 07:37:25PM +0200:
>
> > It would still be *very cool* if we could have optional paths in
> > man confs, just so that man would find /usr/ports/infrastructure/man
> > if it's around, and not give errors if it's not...
> >
> > Just having to add that on each ports build machine is a pain in the butt
>
> That is relatively unrelated to what Stuart Cassoff and myself were
> discussing, which was (1) ports manual pages as opposed to
> infrastructure manual pages and (2) where to install them as opposed
> to whether to search them by default.
>
> That said, the question whether to search /usr/ports/infrastructure/man/
> by default was discussed before.  I was mildly in favour of it, but
> there was no consensus that people wanted it, so in the end, we stopped
> doing it:

I've always been in favour of it. Strongly.

> Requested by deraadt@ millert@ kettenis@ who argue that people
> using /usr/ports/infrastructure/bin/ already need to set PATH,
> so editing man.conf (or, though more fragile, setting MANPATH)
> should not be a big deal for them.

This doesnt make sense: To work with ports, i never had to set PATH,
i dont need to manually call tools there, as they're often called by the
portstree itself.
And i always curse when i run 'man dpb' on my machines and never get
what i expect.

Wasnt OpenBSD supposed to be about safe defaults, and for developers ?

Reply | Threaded
Open this post in threaded view
|

Re: ports/infrastructure/man by default

Theo de Raadt-2
> This doesnt make sense: To work with ports, i never had to set PATH,
> i dont need to manually call tools there, as they're often called by the
> portstree itself.
> And i always curse when i run 'man dpb' on my machines and never get
> what i expect.

Well then the ports infrastructure should install the manual pages.

The man(1) command has never before automatically looked in "source
directories", which this obviously is.

Install the ports infrastructure tools to /usr/local/man something or
other, and call it a day.

A proposal would be that the ports infrastructure have a "Let's begin
working on ports" command sequence, which gets run when one starts, and
voila, the documentation is now installed within man's default paths.

In /usr/local, probably.  Which is port's playground.

> Wasnt OpenBSD supposed to be about safe defaults, and for developers ?

Non sequitur.  The manual tools look in specified directories of the base.

It looks in "build result" directories, not "src" directories.

If I make a change to /usr/src/lib/libc/sys/open.2, man does not find
that file.  It looks in INSTALLED directories, not some strange source
directory.

Why should this directory be special?  To me, everything in /usr/ports
smells like source, including the tooling.

Reply | Threaded
Open this post in threaded view
|

Re: ports/infrastructure/man by default

Antoine Jacoutot-7
On Sat, Jun 23, 2018 at 03:24:55PM -0600, Theo de Raadt wrote:

> > This doesnt make sense: To work with ports, i never had to set PATH,
> > i dont need to manually call tools there, as they're often called by the
> > portstree itself.
> > And i always curse when i run 'man dpb' on my machines and never get
> > what i expect.
>
> Well then the ports infrastructure should install the manual pages.
>
> The man(1) command has never before automatically looked in "source
> directories", which this obviously is.
>
> Install the ports infrastructure tools to /usr/local/man something or
> other, and call it a day.
>
> A proposal would be that the ports infrastructure have a "Let's begin
> working on ports" command sequence, which gets run when one starts, and
> voila, the documentation is now installed within man's default paths.
>
> In /usr/local, probably.  Which is port's playground.
>
> > Wasnt OpenBSD supposed to be about safe defaults, and for developers ?
>
> Non sequitur.  The manual tools look in specified directories of the base.
>
> It looks in "build result" directories, not "src" directories.
>
> If I make a change to /usr/src/lib/libc/sys/open.2, man does not find
> that file.  It looks in INSTALLED directories, not some strange source
> directory.
>
> Why should this directory be special?  To me, everything in /usr/ports
> smells like source, including the tooling.

We already have some ports related man pages in base (e.g. port-modules(5)).
I agree the ports tooling should stay under ports/infrastructure. However I
think the documentation should be available in base.
It costs nothing really. Let's just move the shit over to src/, install it
as part of the base system and be done with it.

--
Antoine

Reply | Threaded
Open this post in threaded view
|

Re: ports/infrastructure/man by default

Ingo Schwarze
Hi Antoine,

Antoine Jacoutot wrote on Sat, Jun 23, 2018 at 11:44:53PM +0200:
> On Sat, Jun 23, 2018 at 03:24:55PM -0600, Theo de Raadt wrote:

>> Well then the ports infrastructure should install the manual pages.
>>
>> The man(1) command has never before automatically looked in "source
>> directories", which this obviously is.
>>
>> Install the ports infrastructure tools to /usr/local/man something or
>> other, and call it a day.
>>
>> A proposal would be that the ports infrastructure have a "Let's begin
>> working on ports" command sequence, which gets run when one starts, and
>> voila, the documentation is now installed within man's default paths.
>>
>> In /usr/local, probably.  Which is port's playground.
>>
>>> Wasnt OpenBSD supposed to be about safe defaults, and for developers ?
>>
>> Non sequitur.  The manual tools look in specified directories of the base.
>>
>> It looks in "build result" directories, not "src" directories.
>>
>> If I make a change to /usr/src/lib/libc/sys/open.2, man does not find
>> that file.  It looks in INSTALLED directories, not some strange source
>> directory.
>>
>> Why should this directory be special?  To me, everything in /usr/ports
>> smells like source, including the tooling.

> We already have some ports related man pages in base (e.g. port-modules(5)).
> I agree the ports tooling should stay under ports/infrastructure. However I
> think the documentation should be available in base.
> It costs nothing really. Let's just move the shit over to src/, install it
> as part of the base system and be done with it.

That sounds very reasonable to me.

No special code, no special commands, and no special configuration are
needed that way, and everything becomes simpler.

The fact that "man dpb" will work even before people install the
ports tree to /usr/ports/ is an asset rather than a downside.  People
can more easily see (including with apropos(1) on a virgin system)
what is available and supported, so they will probably also more
easily find out how to install that stuff.  The SEE ALSO section
of these pages could even contain a short sentence like

  Using this utility requires the ports tree to be installed; see
  .Lk https://www.openbsd.org/faq/ports/ports.html
  for details.

I don't see a sync problem either.  We are not FreeBSD.  We already
require that people keep base and ports in sync.

I think you should indeed move those manual pages over.
Scattering documentation to different places provides no benefit.
KISS.

Yours,
  Ingo

Reply | Threaded
Open this post in threaded view
|

Re: ports/infrastructure/man by default

Marc Espie-2
So this is what moving those manpages to base looks like.

Okay ?


Index: man1/Makefile
===================================================================
RCS file: /cvs/src/share/man/man1/Makefile,v
retrieving revision 1.7
diff -u -p -r1.7 Makefile
--- man1/Makefile 12 Feb 2017 09:57:47 -0000 1.7
+++ man1/Makefile 25 Jun 2018 14:03:04 -0000
@@ -3,4 +3,13 @@
 
 MAN= clang-local.1 help.1 intro.1 gcc-local.1
 
+# ports tools doc
+MAN+= check-lib-depends.1 check-manpages.1 check-problems.1 \
+ clean-old-distfiles.1 dolock.1 dpb-replay.1 dpb.1 \
+ getpkgpath.1 out-of-date.1 outdated-perl-ports.1 \
+ pkg_subst.1 portbump.1 portcheck.1 portgen.1 \
+ portimport.1 portslogger.1 proot.1 register-plist.1 \
+ resolve-lib.1 retrieve-index.1 update-patches.1 \
+ update-plist.1
+
 .include <bsd.prog.mk>
Index: man1/check-lib-depends.1
===================================================================
RCS file: man1/check-lib-depends.1
diff -N man1/check-lib-depends.1
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ man1/check-lib-depends.1 25 Jun 2018 14:03:04 -0000
@@ -0,0 +1,145 @@
+.\" $OpenBSD: check-lib-depends.1,v 1.8 2017/07/19 14:16:13 espie Exp $
+.\"
+.\" Copyright (c) 2010 Marc Espie <[hidden email]>
+.\"
+.\" Permission to use, copy, modify, and distribute this software for any
+.\" purpose with or without fee is hereby granted, provided that the above
+.\" copyright notice and this permission notice appear in all copies.
+.\"
+.\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+.\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+.\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+.\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+.\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+.\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+.\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+.\"
+.Dd $Mdocdate: July 19 2017 $
+.Dt CHECK-LIB-DEPENDS 1
+.Os
+.Sh NAME
+.Nm check-lib-depends
+.Nd scan port for shared libraries
+.Sh SYNOPSIS
+.Nm check-lib-depends
+.Op Fl fimoqx
+.Op Fl B Ar destdir
+.Op Fl d Ar pkgrepo
+.Op Fl D Ar directory
+.Op Fl F Ar missing
+.Op Fl O Ar dest
+.Op Fl S Ar VAR Ns = Ns Ar value
+.Op Fl s Ar source
+.Op Ar package ...
+.Sh DESCRIPTION
+.Nm
+scans a port or package for shared libraries, and verify that all of them
+are properly recorded.
+.Pp
+.Nm
+can either scan files from a built port after the
+.Ar fake stage
+from the
+.Ev WRKINST
+directory, or look directly inside a
+.Ar package .
+.Pp
+Likewise, it can also verify dependencies off installed packages,
+or figure them out directly from the port directory.
+.Pp
+.Nm
+actually consists of two distinct stages:
+.Bl -bullet
+.It
+scan all binaries inside a package to extract libraries and rpaths,
+using
+.Xr objdump 1
+or
+.Xr ldd 1 .
+.It
+compare those libraries against the port's registered dependencies
+and report problems.
+.El
+.Pp
+The options are as follows:
+.Bl -tag -width keyword
+.It Fl B Ar destdir
+Scan files under
+.Ar destdir .
+.It Fl d Ar pkgrepo
+Look for all needed packages under directory
+.Ar pkgrepo .
+.It Fl D Ar directory
+Create debug log files under
+.Ar directory
+that show in more details the run of
+.Xr objdump 1 .
+.It Fl f
+Give full reports of every file that requires a missing library.
+.It Fl F Ar missing
+Works in tandem with
+.Fl S .
+Allow backsubstituting even if the
+.Ar missing
+library
+is actually not part of the actual
+.Ev WANTLIB
+of the package.
+Mostly used to waive the presence of
+.Ar pthread ,
+a mandatory dependency of C++ libraries on some architectures.
+.It Fl i
+Read packing-list from standard input.
+.It Fl m
+Always show progress-meter, even if not run from terminal.
+.It Fl o
+Scan binaries using
+.Xr ldd 1 ,
+which also works for old a.out binaries, instead of
+.Xr objdump 1 ,
+which only works for
+.Xr elf 5
+binaries.
+.It Fl O Ar dest
+Save result of first stage in file
+.Ar dest .
+.It Fl q
+Be quiet, do not emit
+.Sq Extra:
+wantlibs unless there's also an actual problem.
+.It Fl S Ar var Ns = Ns Ar value
+recognize list of libraries listed in a variable, and replace said list
+with variable value.
+For instance,
+.Fl S Ar COMPILER_LIBCXX Ns = Ns Ar stdc++
+will replace
+.Sq stdc++
+with
+.Sq ${COMPILER_LIBCXX}
+in
+.Ev WANTLIB
+lists for compatibility with both clang and gcc.
+Order matters.
+The first
+.Fl S
+option will be handled first.
+.It Fl s Ar src
+Don't scan, directly read result of first stage from file
+.Ar src .
+.It Fl x
+Don't show progress-meter, even if run from terminal.
+.El
+.Pp
+If no
+.Ar package
+is given, and the
+.Fl i
+option is used,
+.Nm
+will retrieve the packing-list from stdin, and scan the package according
+to other options.
+.Sh SEE ALSO
+.Xr ldd 1 ,
+.Xr objdump 1 ,
+.Xr bsd.port.mk 5 ,
+.Xr elf 5
Index: man1/check-manpages.1
===================================================================
RCS file: man1/check-manpages.1
diff -N man1/check-manpages.1
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ man1/check-manpages.1 25 Jun 2018 14:03:04 -0000
@@ -0,0 +1,60 @@
+.\" $OpenBSD: check-manpages.1,v 1.2 2013/01/11 21:08:44 rpe Exp $
+.\"
+.\" Copyright (c) 2010 Marc Espie <[hidden email]>
+.\"
+.\" Permission to use, copy, modify, and distribute this software for any
+.\" purpose with or without fee is hereby granted, provided that the above
+.\" copyright notice and this permission notice appear in all copies.
+.\"
+.\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+.\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+.\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+.\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+.\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+.\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+.\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+.\"
+.Dd $Mdocdate: January 11 2013 $
+.Dt CHECK-MANPAGES 1
+.Os
+.Sh NAME
+.Nm check-manpages
+.Nd verify manpage subjects in packages
+.Sh SYNOPSIS
+.Nm check-manpages
+.Op Fl p
+.Op Ar pkgname ...
+.Sh DESCRIPTION
+.Nm
+is used to check manpages in a collection of packages.
+It will extract all manpages from the
+.Ar pkgname
+arguments into a directory named
+.Pa manpages.XXXXXXXXXXXX
+.Po
+see
+.Xr mktemp 1
+.Pc
+and run them through
+.Xr makewhatis 8 .
+.Pp
+Note that this directory is not cleared on exit, so that the problematic
+manpages can be examined more precisely.
+.Pp
+If no
+.Ar pkgname
+argument is passed, it will process all
+.Pa *.tgz
+files in the current directory.
+.Pp
+Options are as follows:
+.Bl -tag -width packing-list
+.It Fl p
+Run
+.Xr makewhatis 8
+in picky mode.
+.El
+.Pp
+Errors reported in picky mode should usually be fixed, as it means the
+subject line stored in the whatis database will be useless to find out
+the corresponding manpage.
Index: man1/check-problems.1
===================================================================
RCS file: man1/check-problems.1
diff -N man1/check-problems.1
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ man1/check-problems.1 25 Jun 2018 14:03:04 -0000
@@ -0,0 +1,191 @@
+.\" $OpenBSD: check-problems.1,v 1.4 2016/06/10 12:52:57 espie Exp $
+.\"
+.\" Copyright (c) 2010-2015 Marc Espie <[hidden email]>
+.\"
+.\" Permission to use, copy, modify, and distribute this software for any
+.\" purpose with or without fee is hereby granted, provided that the above
+.\" copyright notice and this permission notice appear in all copies.
+.\"
+.\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+.\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+.\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+.\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+.\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+.\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+.\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+.\"
+.Dd $Mdocdate: June 10 2016 $
+.Dt CHECK-PROBLEMS 1
+.Os
+.Sh NAME
+.Nm check-problems
+.Nd verify problems in package collections
+.Sh SYNOPSIS
+.Nm check-problems
+.Op Fl CDesv
+.Op Fl d Ar plist_dir
+.Op Fl o Ar output
+.Op Fl p Ar ports_dir
+.Op Ar pkgname ...
+.Sh DESCRIPTION
+.Nm
+is used to check a collection of packages for problems that cannot be
+noticed in individual packages, such as conflict and directory registration
+issues.
+.Pp
+A conflict is a file that belongs to two distinct packages.
+.Pp
+By default, packages with the same stem
+.Po
+see
+.Xr packages-specs 7
+.Pc
+are registered to conflict.
+This can be changed to add or remove extra conflict registration with
+.Cm @option no-default-conflict
+or
+.Cm @conflict
+.Po
+see
+.Xr pkg_create 1
+.Pc .
+.Pp
+Conflicts between packages mean that a user can't have both packages installed
+at the same time.
+In some cases (same software, different version), this is expected, but
+the packages should be marked as conflicting, so that
+.Xr pkg_add 1
+knows how to deal with it.
+In other cases, this is an oversight from the ports maintainer, and action
+should be taken to make sure both packages can be installed simultaneously.
+Conflicts also interact with dependencies, and sometimes a missing dependency
+will show up as an unregistered conflict.
+.Pp
+Directories can belong to distinct packages, as long as they are registered
+somewhere.
+Every directory that is not part of the standard
+.Xr mtree 8
+set should appear in a packing-list somewhere.
+.Pp
+Contrary to files, directories can be shared between several packages.
+.Pp
+.Nm
+knows enough about packages to trace directories back to a common dependency,
+or to not flag two packages as conflicting when they depend on already
+conflicting packages.
+.Pp
+.Nm
+requires a set of packing-lists to operate, which it can obtain from
+several sources.
+The preferred source is the packing-list databases
+.Po
+see
+.Ar PACKAGE_REPOSITORY
+in
+.Xr bsd.port.mk 5
+and
+.Xr register-plist 1
+.Pc ,
+as it persists over time and can be used to fix update information as well
+as current version information.
+.Pp
+It should be supplemented with the ports tree itself in order to figure
+out current versions.
+.Pp
+Options are as follows:
+.Bl -tag -width packing-list
+.It Fl C
+Do not compute conflict information.
+.It Fl D
+Do not compute common dir information.
+.It Fl e
+Check for extra logical conflicts, by adding supplementary
+formatted/unformatted manpages, as same name manpages under the
+same base directory will shadow one another.
+.It Fl o Ar output
+Save results to output file as well as writing it on the screen.
+.It Fl s
+Be silent.
+.It Fl v
+Be verbose.
+.It Fl d Ar plist-dir
+Scan directory
+.Ar plist-dir
+for packing-lists.
+.It Fl p Ar portsdir
+Traverse the ports directory for packing-lists, using
+.Ar print-plist-all .
+More than one directory could be specified by delimiting them by colon,
+e.g.:
+.Pa /usr/ports/x11/kde:/usr/ports/x11/kde4 .
+.Pp
+If
+.Fl d Ar plist-dir
+is also used, traverse only the directories corresponding to the packages
+under
+.Ar plist-dir
+to figure out current pkgnames, and annotate those with a
+.Sq \&!
+in the report.
+.El
+.Sh SAMPLE OUTPUT
+.Bd -literal
+Dependency not found zarafa-webaccess-7.1.11p2
+Dependency not found kwebkitpart-1.2p3
+[...]
+Common dirs:
+!claws-mail-gdata-3.11.1p1-ldap(mail/claws-mail,-gdata,ldap): /usr/local/lib/claws-mail
+!claws-mail-pdfviewer-3.11.1p1-ldap(mail/claws-mail,-pdfviewer,ldap): /usr/local/lib/claws-mail
+[...]
+Conflicts:
+!arm-none-eabi-gdb-7.9.1(devel/arm-none-eabi/gdb),!gdb-7.9.1(devel/gdb)
+ /usr/local/include/gdb/jit-reader.h
+ /usr/local/share/gdb/system-gdbinit/elinos.py
+ /usr/local/share/gdb/system-gdbinit/wrs-linux.py
+!libnet-1.0.2ap0v0(net/libnet/1.0),libnet-1.0.2ap2(net/libnet/1.0),libnet-1.0.2av0(net/libnet/1.0)
+ /usr/local/bin/libnet-config-1.0
+ /usr/local/include/libnet-1.0/libnet.h
+ /usr/local/include/libnet-1.0/libnet/libnet-asn1.h
+ /usr/local/include/libnet-1.0/libnet/libnet-functions.h
+ /usr/local/include/libnet-1.0/libnet/libnet-headers.h
+ /usr/local/include/libnet-1.0/libnet/libnet-macros.h
+ /usr/local/include/libnet-1.0/libnet/libnet-ospf.h
+ /usr/local/include/libnet-1.0/libnet/libnet-structures.h
+ /usr/local/lib/libnet-1.0/libnet.a
+ /usr/local/lib/libnet-1.0/libnet.so.0.0
+ /usr/local/lib/libnet-1.0/libpwrite.a
+ /usr/local/lib/libnet.so.0.0
+ /usr/local/man/cat3/libnet.0
+[...]
+zathura-pdf-mupdf-0.2.7(textproc/zathura/plugins/mupdf),zathura-pdf-poppler-0.2.5(textproc/zathura/plugins/poppler)
+ /usr/local/lib/zathura/pdf.so
+.Ed
+.Bl -bullet
+.It
+Dependency not found: means some package names could not be found.
+Most often happens because of panic issues, and fsck not recovering some
+dependency files.
+.It
+claws-mail directory: missing directory registration.
+.Nm
+could not find a common ancestor to claws-mail-gdata and claws-mail-pdfviewer,
+so it should be added to both.
+.It
+Conflict between arm-non-eabi-gdb and gdb:
+note that both packages are current.
+Two choices: either remove the offending files from one package (might break
+the other), or mark an explicit conflict between the packages (will prevent
+people from installing both at the same time).
+.It
+Conflict between libnet-...v0 and libnet:
+Note that only the v0 version is current.
+People forgot to augment the
+.Cm @option no-default-conflict
+with conflicts against the old version when bumping the
+.Ar EPOCH .
+.El
+.Sh SEE ALSO
+.Xr pkg_add 1 ,
+.Xr pkg_create 1 ,
+.Xr bsd.port.mk 5 ,
+.Xr packages-specs 7
Index: man1/clean-old-distfiles.1
===================================================================
RCS file: man1/clean-old-distfiles.1
diff -N man1/clean-old-distfiles.1
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ man1/clean-old-distfiles.1 25 Jun 2018 14:03:04 -0000
@@ -0,0 +1,95 @@
+.\" $OpenBSD: clean-old-distfiles.1,v 1.5 2015/07/23 08:05:18 bentley Exp $
+.\"
+.\" Copyright (c) 2012 Marc Espie <[hidden email]>
+.\"
+.\" Permission to use, copy, modify, and distribute this software for any
+.\" purpose with or without fee is hereby granted, provided that the above
+.\" copyright notice and this permission notice appear in all copies.
+.\"
+.\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+.\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+.\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+.\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+.\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+.\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+.\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+.\"
+.Dd $Mdocdate: July 23 2015 $
+.Dt CLEAN-OLD-DISTFILES 1
+.Os
+.Sh NAME
+.Nm clean-old-distfiles
+.Nd remove old distfiles
+.Sh SYNOPSIS
+.Nm clean-old-distfiles
+.Op Fl nv
+.Op Fl e Ar except
+.Op Fl h Ar history
+.Op Ar timestamp
+.Sh DESCRIPTION
+.Nm
+uses information provided by
+.Xr dpb 1
+to remove obsolete distfiles:
+each time
+.Xr dpb 1
+is run for a full scan of the ports tree, it appends to
+an history log that records obsolete distfiles.
+.Pp
+.Nm
+can be used to prune that log prior to a given
+.Ar timestamp
+and remove the corresponding distfiles.
+.Pp
+For normal files,
+.Nm
+verifies
+.Xr sha256 1
+checksums prior to removal.
+.Nm
+also removes links under
+.Pa by_cipher/sha256 ,
+but it trusts that the checksums for these are accurate.
+.Pp
+The
+.Ar timestamp
+optional argument is given in seconds since the epoch,
+see
+.Xr date 1 .
+.Pp
+Options are as follows:
+.Bl -tag -width nnnn
+.It Fl e Ar except
+Read exception list from file
+.Ar except .
+Each line is the name of a file that shouldn't be removed.
+.It Fl h Ar history
+Read log from history file
+.Ar history
+instead of the default
+.Pa /usr/ports/distfiles/history .
+Turns on
+.Fl nv ,
+as this is a testing option.
+.It Fl n
+Don't really remove files.
+.It Fl v
+Display what will be removed, as
+.Li rm file
+lines.
+.El
+.Sh FILES
+.Bl -tag -width /usr/ports
+.It Pa /usr/ports
+base of the ports tree, can be overridden with env variable
+.Ev PORTSDIR .
+.It Pa /usr/ports/distfiles
+location of the distfiles repository, can be overridden with env variable
+.Ev DISTDIR .
+.It Pa /usr/ports/distfiles/history
+log of obsolete distfiles.
+.El
+.Sh SEE ALSO
+.Xr dpb 1 ,
+.Xr sha256 1 ,
+.Xr mirroring-ports 7
Index: man1/dolock.1
===================================================================
RCS file: man1/dolock.1
diff -N man1/dolock.1
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ man1/dolock.1 25 Jun 2018 14:03:04 -0000
@@ -0,0 +1,51 @@
+.\" $OpenBSD: dolock.1,v 1.5 2015/11/22 23:34:34 tim Exp $
+.\"
+.\" Copyright (c) 2010 Marc Espie <[hidden email]>
+.\"
+.\" Permission to use, copy, modify, and distribute this software for any
+.\" purpose with or without fee is hereby granted, provided that the above
+.\" copyright notice and this permission notice appear in all copies.
+.\"
+.\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+.\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+.\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+.\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+.\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+.\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+.\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+.\"
+.Dd $Mdocdate: November 22 2015 $
+.Dt DOLOCK 1
+.Os
+.Sh NAME
+.Nm dolock
+.Nd lock port for concurrent builds
+.Sh SYNOPSIS
+.Nm dolock
+.Op Fl g Ar group
+.Op Fl m Ar mode
+.Ar lockname Op comment ...
+.Sh DESCRIPTION
+.Nm
+creates a lock named
+.Ar lockname
+using
+.Xr open 2 .
+If the lock is already held, it will display one single wait message on
+stderr, sleep and retry until satisfaction.
+.Pp
+If necessary, creates the lock directory, using
+.Fl m Ar mode
+as a mode argument or the default if that option is not specified,
+and
+.Fl g Ar group
+as the group owner if that option is specified.
+.Pp
+This implements a locking mechanism for ports building, see
+.Sq LOCK INFRASTRUCTURE
+in
+.Xr ports 7
+for details.
+See also
+.Xr dpb 1
+for a complementary lock infrastructure when using dpb.
Index: man1/dpb-replay.1
===================================================================
RCS file: man1/dpb-replay.1
diff -N man1/dpb-replay.1
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ man1/dpb-replay.1 25 Jun 2018 14:03:04 -0000
@@ -0,0 +1,48 @@
+.\" $OpenBSD: dpb-replay.1,v 1.3 2014/03/23 12:24:14 nigel Exp $
+.\"
+.\" Copyright (c) 2013 Marc Espie <[hidden email]>
+.\"
+.\" Permission to use, copy, modify, and distribute this software for any
+.\" purpose with or without fee is hereby granted, provided that the above
+.\" copyright notice and this permission notice appear in all copies.
+.\"
+.\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+.\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+.\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+.\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+.\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+.\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+.\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+.\"
+.Dd $Mdocdate: March 23 2014 $
+.Dt DPB-REPLAY 1
+.Os
+.Sh NAME
+.Nm dpb-replay
+.Nd replays dpb terminal output
+.Sh SYNOPSIS
+.Nm
+.Op Fl c
+.Op Fl s Ar speedup
+.Ar file
+.Sh DESCRIPTION
+.Nm
+is used to replay
+.Xr dpb 1
+terminal logs
+.Po
+.Pa %L/term-report.log
+.Pc
+later, usually at a much higher speed than the first run,
+for demonstration and optimization purposes.
+The options are as follows
+.Bl -tag -width keywordaaaa
+.It Fl c
+Color display.
+.It Fl s Ar speedup
+Sets the time acceleration to
+.Ar speedup
+(defaults to 10).
+.El
+.Sh SEE ALSO
+.Xr dpb 1
Index: man1/dpb.1
===================================================================
RCS file: man1/dpb.1
diff -N man1/dpb.1
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ man1/dpb.1 25 Jun 2018 14:03:04 -0000
@@ -0,0 +1,1673 @@
+.\" $OpenBSD: dpb.1,v 1.129 2018/01/29 18:20:12 espie Exp $
+.\"
+.\" Copyright (c) 2010-2013 Marc Espie <[hidden email]>
+.\"
+.\" Permission to use, copy, modify, and distribute this software for any
+.\" purpose with or without fee is hereby granted, provided that the above
+.\" copyright notice and this permission notice appear in all copies.
+.\"
+.\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+.\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+.\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+.\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+.\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+.\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+.\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+.\"
+.Dd $Mdocdate: January 29 2018 $
+.Dt DPB 1
+.Os
+.Sh NAME
+.Nm dpb
+.Nd distributed ports builder
+.Sh SYNOPSIS
+.Nm dpb
+.Bk -words
+.Op Fl acemqrRsuUvx
+.Op Fl A Ar arch
+.Op Fl b Ar logfile
+.Op Fl B Ar chroot
+.Op Fl C Ar pathlist
+.Op Fl D Ar PARAM Ns = Ns Ar value
+.Op Fl f Ar m
+.Op Fl F Ar m
+.Op Fl h Ar hosts
+.Op Fl I Ar pathlist
+.Op Fl J Ar p
+.Op Fl j Ar n
+.Op Fl L Ar logdir
+.Op Fl l Ar lockdir
+.Op Fl M Ar threshold
+.Op Fl P Ar pathlist
+.Op Fl p Ar parallel
+.Op Fl S Ar logfile
+.Op Fl X Ar pathlist
+.Op Ar pathlist ...
+.Ek
+.Sh DESCRIPTION
+.Nm
+is used to build ports on a cluster of machines, or on a single machine
+with several cores.
+.Nm
+walks the ports tree to figure out dependencies, and starts building ports
+as soon as it can.
+.Pp
+.Nm
+will run with sensible defaults if used without options.
+Note, however, that it will produce logs, lock files, packages, and package
+installations.
+.Pp
+If run as non-root,
+.Nm
+will warn.
+The preferred way is to run it as root (and preferably under a chroot).
+.Nm
+will then change its identity to different users as needed.
+See
+.Sq THE SECURITY MODEL OF DPB
+for details.
+.Pp
+.Nm
+can be restricted to a subset of the tree by giving it
+.Ar pathlist ...
+to build as parameters.
+.Pp
+A
+.Ar pathlist
+is either a
+.Xr pkgpath 7
+to build, or a filename that contains pkgpaths (one per line).
+.Ar pathlist
+parameters can also take the form
+.Li filename*scale
+in order to multiply the weights of all
+.Xr pkgpath 7
+in a file by a given
+.Ar scale ,
+or
+.Li pkgpath=value ,
+in order to set the weight of a given
+.Xr pkgpath 7
+to a specific value.
+.Pp
+.Nm
+supports
+.Sq hot-fixes :
+if a particular port errors out, it is possible to fix the problem, remove
+the corresponding lockfile, and
+.Nm
+will pick it up without needing to be stopped and restarted.
+.Pp
+In order to build on a cluster, the ports tree itself should be identical
+on each machine (shared through NFS or copied at start).
+.Pp
+Some directories must be shared:
+.Ev PACKAGE_REPOSITORY ,
+.Ev DISTDIR ,
+and
+.Ev PLIST_REPOSITORY .
+The
+.Ev WRKOBJDIR
+and
+.Ev LOCKDIR
+should be local to each machine, and on a high-speed partition.
+.Pp
+Also note that
+.Nm Ns 's
+logs and locks are managed by the main
+.Nm
+process, which runs locally, and hence those directories do not need to
+be shared on the cluster.
+.Pp
+Some log files ("rolling logs") are kept from one run to the run and
+stored under
+.Pa ${DISTDIR}/build-stats .
+.Pp
+Option
+.Fl h Ar file
+is used to specify hosts to use, where
+.Ar file
+may contain lots of information,
+but can be as simple as a list of hosts to use, one host per line
+(however, it is recommended to also include a
+.Ar STARTUP
+script).
+.Pp
+Most filenames will go through some control sequence expansions.
+For instance, the default logdir location can be specified as
+.Pa %p/logs/%a .
+The following sequences are recognized:
+.Bl -tag -offset aaaa -width %aa
+.It Cm %a
+architecture being used.
+.It Cm %d
+date at start of
+.Nm ,
+GMtime, formatted as yyyy-mm-dd@hh:mm:ss.
+.It Cm %f
+fetch distfiles location (DISTDIR).
+.It Cm %h
+hostname running
+.Nm .
+.It Cm %L
+logdir location.
+.It Cm %p
+portsdir location.
+.It Cm %t
+timestamp (number of seconds since January 1 1970) at start of
+.Nm .
+.El
+.Pp
+Options are as follows:
+.Bl -tag -width pkgpathlong
+.It Fl A Ar arch
+Build packages for given architecture, selecting relevant hosts from the
+cluster.
+By default, the current host's architecture will be used.
+.It Fl a
+Walk the whole tree and builds all packages (default if no
+.Ar pathlist
+is given).
+.It Fl b Ar logfile
+Explicitly prime the heuristics module with a previous build log,
+so that packages that take a long time to build will happen earlier.
+The rolling log under
+.Pa %f/build-stats/%a
+is automatically used.
+.It Fl B Ar chroot
+chroot to
+.Ar chroot
+before building.
+See
+.Xr proot 1
+for preparing such an environment.
+.It Fl c
+Clean port working directory and log before each build.
+.It Fl C Ar pathlist
+Don't clean port working directories after build.
+Only use simple
+.Xr pkgpath 7
+in the list,
+as this does not take subpackages and flavors into account.
+.It Fl D Ar PARAM Ns = Ns Ar value
+Set defined parameter to value.
+Known parameters are as follows:
+.Bl -tag -width DISP
+.It Ar ALWAYS_CLEAN
+Set to 1 if
+.Nm
+should clean work directories even if the port errored out.
+.It Ar BUILD_USER
+Default value for
+.Ar build_user
+if you want to specify it on the command line, and want to ensure even
+the small "discover PORTSDIR" activity at the beginning of
+.Nm
+is not run as root.
+.It Ar CDROM_ONLY
+Don't fetch distfiles that are not allowed for cdrom.
+.It Ar COLOR
+Set to 1 to have the normal display in color.
+.It Ar CONTROL
+Let
+.Nm
+create a unix socket of the given name for external control.
+.It Ar CONNECTION_TIMEOUT
+Connection timeout for ssh.
+Defaults to 60 seconds.
+.It Ar DISPLAY_TIMEOUT
+Display timeout (in seconds) while waiting for jobs to finish, so that the
+display is updated even if jobs didn't finish.
+Defaults to 10 seconds.
+.It Ar DONT_BUILD_ONCE
+By default,
+.Nm
+will use the
+.Ev BUILD_ONCE
+optimization
+.Po
+see
+.Xr bsd.port.mk 5
+.Pc
+if run with
+.Fl a :
+pseudo-flavors that disable subpackages and are not necessary for bootstrap
+will be disabled, so that the same port is built once, as far as possible.
+This flag disables that optimization, which might be desireable if you want
+to build a small subset of packages which would pull in the kitchen sink
+otherwise.
+.It Ar DONT_CLEAN_LOCKS
+By default,
+.Nm
+will clean old locks from dpb running on the same host that no longer exist,
+provided they didn't end in error.
+This is usually the right thing to do after a crash, or after killing dpb
+abruptly.
+Sometimes, one may want manual control over which locks to remove.
+.It Ar FETCH_JOBS
+Alternate way to specify the number of fetch jobs.
+.It Ar FETCH_USER
+User for all fetch activities if possible
+.Po defaults to
+.Ar _pfetch
+.Pc .
+.It Ar FETCH_TIMEOUT
+Timeout (in seconds) after which fetches that don't show
+any progress will be killed.
+.It Ar FTP_ONLY
+Don't fetch distfiles that are not allowed for ftp.
+.It Ar HISTORY_ONLY
+Don't fetch or build anything.
+Only run
+.Nm
+to figure out old distfiles and update
+.Pa %f/history .
+.It Ar LOGDIR
+Alternate way to specify the logging directory.
+.It Ar LOG_USER
+User
+for all log files if possible
+.Po defaults to
+.Ar build_user
+.Pc .
+.It Ar LOCKDIR
+Alternate way to specify the locking directory.
+.It Ar MIRROR
+Applicable to fetch modes.
+If 0, will only fetch normal
+.Ev DISTFILES
+.Po
+default for
+.Nm Fl f
+.Pc .
+If 1, will also fetch extra
+.Ev SUPDISTFILES
+.Po
+default for
+.Nm Fl F
+.Pc .
+.It Ar NO_CHECKSUM
+Do not run
+.Ar checksum
+again for files already fetched.
+.It Ar NO_CURSOR
+Make the terminal cursor invisible if possible.
+Avoids flickering on slow graphics cards.
+.It Ar NO_BUILD_STATS
+Disable reading/saving of default build stats under
+.Pa ${DISTDIR}/build-stats/${ARCH} .
+.It Ar NO_HISTORY
+Do not update the distfiles history.
+For instance, if
+.Nm
+is run a second time after a problem during the first run.
+.It Ar PORT_USER
+User that can write to the ports tree.
+Not really used for anything yet.
+.It Ar RECORD
+Define a file which will save all terminal output.
+Mostly useful for presentations, as a way to save
+.Nm dpb
+output and replay it later at a faster rate.
+Defaults to
+.Pa %L/term-report.log ,
+can be set to nothing to disable.
+.It Ar STARTUP
+Define a start-up script on the command-line, override any host file contents.
+.It Ar STUCK_TIMEOUT
+Timeout (in seconds * speed factor) after which tasks that don't show
+any progress will be killed.
+This can be instead set on a per-core basis as the
+.Sq stuck
+property.
+Note that this will always be divided by the core's speed factor.
+.It Ar SYSLOG
+Make
+.Nm
+call
+.Xr syslog 3
+on every task start/end while creating packages.
+This does produce lots of messages, it is intended to route the logging
+on another machine, while tracking down panics and other hangs.
+.It Ar WANTSIZE
+Alternate way to specify
+.Fl s .
+.El
+.It Fl e
+The listing job is extra and won't be given back to the pool when it's
+finished.
+.It Fl f Ar m
+Create
+.Ar m
+jobs for fetching files.
+Those are separate from the build jobs, since they don't consume cpu, and they
+run on the localhost.
+Defaults to 2.
+Can be set to 0 to bypass fetching jobs entirely,
+and reduce
+.Nm
+memory footprint by a lot.
+.It Fl F Ar m
+Fetch-only mode, for mirroring hosts.
+Do not build any package but fetch everything, disregarding
+.Ev BROKEN
+and
+.Ev ONLY_FOR_ARCHS
+information.
+Create
+.Ar m
+localhost jobs for fetching files.
+.It Fl h Ar hosts
+File with hosts to use for building.
+One host per line, plus properties, such as:
+.Bd -literal -offset indent
+espie@aeryn jobs=4 arch=i386
+.Ed
+.Pp
+Lines starting with a known variable name such as
+.Bd -literal -offset indent
+STARTUP=path
+.Ed
+or
+.Bd -literal -offset indent
+FETCH_JOBS=5
+.Ed
+can also be set inside a configuration file, to reduce the number of
+options you must pass on the command line.
+.Pp
+The special hostname
+.Ar DEFAULT
+can be used to preset defaults.
+It should be used at the start of the file.
+.Pp
+Use
+.Ar localhost
+to specify the local machine.
+.Nm
+will special-case it and not use
+.Xr ssh 1
+to connect.
+.Pp
+Properties are as follows:
+.Bl -tag -width memory=150
+.It arch=value
+Architecture of the concerned host.
+(there should be a startup task to check consistency, but
+currently this has to be set manually on heterogeneous networks.)
+.It always_clean=n
+Set to 0 or 1 on per-host basis.
+See
+.Ar ALWAYS_CLEAN
+parameter.
+.It build_user=user
+Use
+.Ar user
+for non root jobs if possible (defaults to
+.Xr whoami 1
+value).
+.It chroot=dir
+Chroot to
+.Ar dir
+before building.
+.It jobs=n
+Number of jobs to run on that host, defaults to hw.ncpu.
+.It junk=n
+Junk unused packages each n steps.
+See
+.Fl J
+option.
+.It memory=thr
+Build everything below that wrkdir threshold with
+.Ev USE_MFS Ns = Ns Sq Yes ,
+assuming the ports tree has been configured so that
+.Ev WRKOBJDIR_MFS
+points to a memory filesystem.
+.Ar thr
+is the sum, in KBytes, of ports that will be allowed to build in memory.
+.Nm
+understands suffixes, such as
+.Fl M Ar 2G
+or
+.Fl M Ar 500M .
+.Pp
+Note that you should always allow for some margin, as
+.Nm
+makes its decision based on the size information collected during previous
+builds, so in cases of significant updates, the work directory size will
+usually grow.
+.It nochecksum=0/1
+Defaults to 1.
+During the junk stage, run
+.Xr pkg_delete 1
+with the
+.Fl q
+(no checksum) option.
+.It parallel=p
+Run big ports on several cores.
+See
+.Fl p
+option.
+.It repair=0/1
+Defaults to 1.
+Run
+.Xr pkg_add 1
+with the repair option.
+This is useful on some bulk machines which tend to crash a lot, leaving
+.Pa /var/db/pkg
+in a weird state.
+.It sf=n
+Speed factor.
+An estimate of that machine's speed with that number of jobs
+compared to other machines in the same network.
+Works better with small values, in the range of 1..50.
+The machine (or machines) with the highest speed factor will
+get access to all jobs, whereas other machines will be clamped
+to stuff which does not take too long.
+Requires previous build information to be effective.
+Defaults to 1.
+.It small=s
+Small threshold (in seconds * sf):
+ports known to build under that duration are deemed to be small, so
+.Nm
+won't bother calling fine-grained steps for patch/configure/fake.
+It will go straight to build and package instead.
+Defaults to 120 seconds.
+.It squiggles=n
+Number of squiggles on this host (see
+.Sq the squiggle heuristics
+below).
+Defaults to 1 squiggle for hosts with 4 jobs or more, 0.7 for hosts with more than 1 job,
+0 for single job hosts.
+.It stuck=s
+Stuck timeout (in seconds * sf) after which tasks which show no progress
+will get killed.
+.It timeout=s
+Defines a specific connection timeout for ssh to that host.
+.El
+.Pp
+There are no fine-grained options to control
+.Xr ssh 1
+options, as those can be specified through virtual host declarations in
+.Xr ssh_config 5 .
+.It Fl I Ar pathlist
+List of
+.Xr pkgpath 7
+to install, on the local box.
+This will also add them to the list of things to build.
+.It Fl J Ar p
+Override value for the
+.Dq junk
+property.
+Delete unneeded installed packages during the build.
+Each
+.Ar prepare
+stage is followed by a
+.Ar show-prepare-results
+stage.
+After every
+.Ar p
+new dependencies, it will be followed by a
+.Ar junk
+stage which uses
+.Xr pkg_delete 1
+with the
+.Fl aXI
+options to delete automatically installed packages that are currently
+not needed.
+.Pp
+.Nm
+keeps track of list of dependencies on a given host, by storing each
+dependency list in the lockfile corresponding to the package being built.
+.Pp
+To avoid a race condition between the
+.Ar depends
+and
+.Ar junk
+stages,
+.Nm
+allows only one job on a given host to be in the
+.Ar depends
+\&...
+.Ar junk
+stages at one time, by using a per-host lock.
+.Pp
+Defaults to
+.Ar 150 .
+Can be disabled by setting to
+.Ar 0 .
+.Pp
+Some ports, most notably cmake-based, have an annoying dependency handling
+bug: they compute their makefile dependencies based on all include files
+present, not just the ones that are actually enabled.
+Those ports' build may be broken by a
+.Ar junk
+phase that removes some unused includes that were added as makefile
+prerequisites.
+Those ports should be annotated with
+DPB_PROPERTIES = nojunk
+until that bug is fixed:
+while a port with the
+.Sq nojunk
+property is building,
+.Ar junk
+will be postponed.
+.Pp
+Those ports will be marked with a
+.Sq \&!
+in the display, to make it more obvious why junk seems to be ineffective.
+.Pp
+Note that the
+.Sq nojunk
+property is still active for ports in error, in the belief that trivial fixes
+can be made that will allow the port build to finish.
+.It Fl j Ar n
+Number of jobs to run on a single host (defaults to hw.ncpu).
+.It Fl L Ar logdir
+Choose a log directory.
+.Po
+Defaults to
+.Pa %p/logs/%a
+.Pc .
+.It Fl l Ar lockdir
+Choose a lock directory.
+.Po
+Defaults to
+.Pa %L/locks
+.Pc .
+Override to keep local, as locks don't really like NFS.
+.It Fl M Ar threshold
+Build ports below the memory threshold under a memory
+filesystem, as configured through
+.Ev WRKOBJDIR_MFS
+.Po
+see
+.Xr bsd.port.mk 5
+.Pc .
+.Ar threshold
+is the sum, in KBytes, of ports allowed to build there.
+.It Fl m
+Force tty-style reporting.
+.It Fl P Ar pathlist
+Read list of
+.Xr pkgpath 7
+from file.
+.It Fl p Ar parallel
+Override value for the
+.Dq parallel
+property.
+.Pp
+Run big jobs on several cores on the same host, by using
+MAKE_JOBS=k .
+.Pp
+Once such a job has started,
+.Nm
+will not start new jobs on the same host until the big job has
+stolen enough cores from other finishing jobs.
+.Pp
+Only big ports which are safe for parallel building (annotated with
+DPB_PROPERTIES = parallel in their Makefile) will be affected.
+.Pp
+It is advisable to set k to an integral fraction of the
+number of cores available on a given host.
+.Ar parameter
+can be an integer, or of the form
+.Sq /n ,
+in which case,
+.Nm
+will set k to a fraction of the total number of jobs
+on the machine, but never below 2.
+.Pp
+Defaults to
+.Sq /2 .
+.It Fl q
+Don't quit while errors/locks are around.
+.It Fl r
+Random build order.
+Disregard any kind of smart heuristics.
+Useful to try to find missing build dependencies.
+.It Fl R
+Rebuild existing packages based on discrepancies between the package
+signature and what the port says it should be.
+Concretely, use to run a partial bulk build after some library change.
+.Pp
+Note that
+.Fl R
+won't always work, as rebuilding a package when another version is already
+installed is not supported.
+Building in a chroot is strongly recommended.
+.It Fl S Ar logfile
+Read
+.Ar logfile
+as an initial workdir size log.
+.It Fl s
+Compute workdir sizes before cleaning up, and stash them in log file
+.Pa %L/size.log .
+Also maintain a rolling log of build sizes under
+.Pa %f/build-stats/%a-size .
+In order to save time,
+.Nm
+will actually not always compute new sizes for known directories, but mostly
+for new ones, or when the package name changes.
+.It Fl u
+Update existing packages during dependency solving.
+Can be used to run a bulk-build on a machine with installed packages,
+but might break a bit, since some packages only build on a clean machine
+right now.
+.It Fl U
+Insist on updating existing packages during dependency solving,
+even if the new package apparently didn't change.
+.It Fl x
+No tty report, only report really important things, like hosts going down
+and coming back up, build errors, or builds not progressing.
+.It Fl X Ar pathlist
+Read a list of
+.Xr pkgpath 7
+from file, and pass them along in the junk phase:
+those are packages that should stay on the machine if they've been
+installed by a dependency.
+Can be used to avoid endlessly removing/reinstalling the most common
+packages, e.g.,
+.Pa devel/gmake .
+.El
+.Pp
+.Nm
+figures out in which order to build things on the fly, and constantly
+displays information relative to what's currently building.
+There's a list of what is currently running, one line per job.
+Those jobs are ordered in strict chronological order, which means that
+long running builds will tend to percolate to the top of the list.
+Normal jobs look like this:
+.Bd -literal -offset indent
+www/mozilla-firefox(build) [9452] 41% unchanged for 92 seconds
+.Ed
+.Pp
+This contains:
+.Bl -dash
+.It
+an optional
+.Sq ~
+squiggle marker (see below),
+.It
+the pkgpath being built,
+.It
+the step currently being run,
+.It
+an optional
+.Sq \&!
+for ports with the
+.Sq nojunk
+property.
+.It
+an optional
+.Sq +
+for ports built in memory.
+.It
+the pid running that task (note that this is always a pid on the host
+running dpb: for distributed builds, it will be an
+.Xr ssh 1
+to another machine),
+.It
+the current size of the log file (displayed as a percentage if previous
+build statistics are available).
+.It
+and a possible notice that things might be stuck when
+the log file doesn't change for long periods.
+.El
+.Pp
+And fetch jobs look like this:
+.Bd -literal -offset indent
+<dist-3.0.tgz(#1) [4321] 25%
+.Ed
+.Pp
+This contains:
+.Bl -dash
+.It
+the file being fetched
+.It
+the number of the
+.Ev MASTER_SITE
+being tried
+.It
+the pid of the
+.Xr ftp 1
+process (note that fetch jobs are always local).
+.It
+a progress percentage.
+.El
+.Pp
+This is followed by a host line, containing the name
+of each host used by dpb.
+Host names may be tagged with kde3 or kde4.
+They are followed by a
+.Sq `-'
+for unresponsive hosts, and the pid of the ssh master
+for distant hosts.
+.Pp
+This ends with a summary display:
+.Bl -tag -width BB=
+.It I=
+number of built packages that can be installed.
+.It B=
+number of built packages, not yet known to be installable,
+because of run depends that still need to be built.
+.It Q=
+number of packages in the queue, e.g., stuff that can be built now, assuming
+we have a free slot.
+.It T=
+number of packages to build, where dependencies are not yet resolved.
+.It F=
+number of distfiles to fetch, when
+.Fl f
+is used.
+.It !=
+number of ignored packages.
+Details in
+.Pa engine.log .
+.It L=
+list of packages that cannot currently be built because of locks.
+.It E=
+list of packages in error, that cannot currently be built.
+.It H=
+list of packages that haven't shown up yet, usually due to nfs, but
+watch out for revision bumps.
+.El
+.Pp
+If those three lists are empty, they won't even show up.
+Packages in errors may be followed by a
+.Sq \&!
+if they prevent junk from happening.
+.Pp
+Note that those numbers refer to pkgpaths known to
+.Nm .
+In general, those numbers will be slightly higher than the actual number
+of packages being built, since several paths may lead to the same package.
+.Pp
+.Nm
+uses some heuristics to try to maximise the queue as soon as possible.
+There are also provisions for a feedback-directed build, where information from
+previous builds can be used to try to build long-running jobs first.
+.Pp
+Similarly, fetches will use the continue option of
+.Xr ftp 1 ,
+since distfiles are checksummed after the fetch anyways.
+.Ss THE SQUIGGLE HEURISTICS
+However, on machines with lots of cores, the basic scheduling heuristics
+yields a tail of very small jobs, where
+.Nm
+will mostly wait on
+.Xr pkg_add 1
+to solve dependencies.
+Starting with
+.Ox 5.5 ,
+a new mechanism (squiggles) was introduced to counter-balance this effect:
+big machines devote some of their cores to
+.Sq squiggles ,
+jobs that walk the queue in reverse, thus building smallest ports first.
+As a result, small ports are built as a trickle alongside the largest ports,
+thus offsetting the negative effect of the exponential queue for a large part.
+.Pp
+Note that
+.Sq squiggles
+can be a non-integral value, usually lower than 1, in which case they
+represent the fraction of cores that should be affected to squiggles,
+as decided randomly at the start of each build.
+0.7 or 0.8 might be a good choice for dual core machines.
+.Ss DPB PROPERTIES
+The
+.Xr bsd.port.mk 5
+variable
+.Ev DPB_PROPERTIES
+may hold several annotations that only
+.Nm
+will look at.
+These properties are as follows:
+.Bl -tag -width pkgpathlong
+.It Ar lonesome
+Large port that stresses the memory limits of the machine, should be built
+alone.
+Prevents
+.Nm
+from scheduling anything else on the same host after it starts building.
+\".It Ar memoryhog
+.It Ar nojunk
+Port that hardcodes includes in its Makefile mechanisms.
+Prevents
+.Ar junk
+from running while port is building.
+.It Ar parallel
+Port that can be built in parallel, uses
+.Ev MAKE_JOBS
+and several build slots.
+.It Ar tag:kde3
+kde3 port that conflicts with kde4 ports.
+Prevent scheduling ports with
+.Ar tag:kde4
+on the same host.
+.It Ar tag:kde4
+kde4 port that conflicts with kde3 ports.
+Prevent scheduling ports with
+.Ar tag:kde3
+on the same host.
+.El
+.Sh THE SECURITY MODEL OF DPB
+When
+.Nm
+is run as root, it uses a privilege drop model instead of the
+dangerous privilege elevation model of
+.Xr doas 1 .
+When run as root, by default,
+.Ar _pbuild
+is used as the build and log user, and
+.Ar _pfetch
+is used as the fetch user.
+.Bl -bullet
+.It
+Start
+.Nm
+as root.
+.It
+.Nm
+will drop privileges for every operation except
+.Xr pkg_add 1 ,
+.Xr pkg_delete 1
+and the
+.Ar STARTUP
+script.
+.It
+For cluster builds,
+provide an
+.Xr ssh 1
+connection to distant hosts from root as root.
+.It
+.Ar build_user
+is used to build stuff locally or distantly (can be per-host), using:
+.Li chroot -u build_user /build_root
+(with
+.Pa /build_root
+=
+.Pa /
+if there is no actual chroot needed).
+It must have read access to ${DISTDIR} and ${PORTSDIR}, and write
+access to ${WRKOBJDIR}, ${PACKAGE_REPOSITORY}, and ${PLIST_REPOSITORY}.
+It does not require network access.
+.It
+.Ar LOG_USER
+is used to open all log files.
+.Ar LOG_USER
+only needs to exist locally.
+It needs write access to the log directories, including
+${DISTDIR}/build-stats.
+It does not need network access.
+.It
+.Ar FETCH_USER
+is used to fetch distfiles and handle corresponding log info.
+It needs write access to ${DISTDIR}, and network access.
+Thus,
+.Xr ftp 1
+does not happen as root.
+.It
+.Ar _dpb
+is used as a fail-safe for any other activities that do not require any rights.
+.It
+.Nm
+creates local directories as root, then gives them to the appropriate user.
+.El
+.Sh LOCKS AND ERRORS
+.Nm
+still uses the normal ports tree mechanism while building, which includes
+.Ev LOCKDIR .
+When starting up
+.Nm
+will normally detect stale locks from old dpb runs, and remove them.
+If this does not happen, builds will stay stuck in their initial stage,
+that is:
+.Ar show-prepare-results , patch , build
+depending on the port.
+A telltale message
+.Sq Awaiting lock ...
+can be found in the corresponding logfile
+.Pa paths/pkgpath.log
+.Pp
+In addition, when building a package,
+.Nm
+produces a lockfile in the locks directory, whose name is deduced from
+the basic pkgpath with slashes replaced by dots.
+This lockfile is filled with such info as the build start time or the host,
+or the needed dependencies for this pkgpath.
+.Pp
+The lockfile will also contain the name of a parent pkgpath, for paths that
+were discovered as dependencies.
+This is particularly useful for bogus paths, where it would be hard to
+know where the path came from otherwise.
+.Pp
+At the end of a successful build, these lockfiles are removed.
+The lock will stay around in case of errors.
+.Po
+raw
+value from
+.Xr wait 2
+.Pc ,
+and the name of the next task in the build pipeline (with todo=<nothing>
+in case of failure during clean-up).
+Normal list of tasks is:
+.Ar depends prepare fetch patch configure build fake package clean .
+.Pp
+At the end of each job,
+.Nm
+rechecks the locks directory for existing lockfiles.
+If some locks have vanished,
+it will put the corresponding paths back in the queue and attempt
+another build.
+.Pp
+This eases manual repairs: if a package does not build, the user can look
+at the log, go to the port directory, fix the problem, and then remove the lock.
+.Nm
+will pick up the ball and keep building without interruption.
+.Pp
+It is perfectly safe to run several
+.Nm
+in parallel on the same machine.
+This is not optimal, since each
+.Nm
+ignores the others, and only uses the lock info to avoid the other's
+current work, but it can be handy: in an emergency, one can start a second
+.Nm
+to obtain a specific package right now, in parallel with the original
+.Nm .
+.Pp
+Note that
+.Nm
+is very careful not to run two builds from the same pkgpath at the
+same time, even on different machines:
+in some cases, MULTI_PACKAGES and FLAVOR combinations may lead to the
+same package being built simultaneously, and since the package repository
+is shared, this can easily lead to trouble.
+.Pp
+Handling of shared log files and history is also done very carefully by
+systematically appending to files or using atomic mv operations.
+.Pp
+For obvious reasons, this won't work as well with masters running on distinct
+machines sharing their logs through NFS.
+.Ss BUILD CYCLES
+There are some various interdependencies in package builds that can be hard
+to trace in case something goes wrong.
+Refer to
+.Pa summary.log
+to fix those specific issues.
+.Sh AFFINITY
+.Nm
+now maintains a list of pkgpath-per-host that are currently building in the
+.Pa affinity
+directory of its log directory, along with building-in-memory status.
+.Pp
+That information is only wiped out when a given build finishes successfully.
+.Pp
+Otherwise
+.Nm
+will try to restart that build on the same host, which can be handy if you
+interrupt
+.Nm
+while it is building a large port, or if you remove a lock after fixing a
+problem.
+.Sh TAGS FOR BUILDING KDE
+Currently, kde3 and kde4 can't be built simultaneously.
+Conflicting ports have been annotated with
+DPB_PROPERTIES=tag:kde3 ,
+DPB_PROPERTIES=tag:kde4
+respectively.
+.Pp
+.Nm
+now keeps track of those tags, and will postpone ports with the wrong
+tag while a given host is used by the other tag.
+.Pp
+This heavily relies on the
+.Ar junk
+stage to clean-up hosts periodically,
+and it can even forcibly provoke a
+.Ar junk
+stage even if junk=0.
+.Pp
+This
+.Sq force-junk
+stage is actually implemented as a pseudo path called
+.Ar junk-proxy ,
+which only does junk.
+.Pp
+In order for builds to proceed gracefully, machines should start
+in a clean slate, without kde3 or kde4 installed.
+.Pp
+As a special-case, failing ports with a kde3 or kde4 tag will not
+interfere with clean-up, so that hosts do not get locked down to
+a specific tag.
+This also means that their dependencies
+may vanish before human intervention addresses the problem.
+.Pp
+This is supposed to be a temporary hack, as kde4 is large and
+having official packages helps a great deal in debugging it.
+.Sh EXTERNAL CONTROL
+If
+.Fl D Ar CONTROL Ns = Ns Ar path
+if used,
+.Nm
+will create a Unix socket at the given
+.Pa path ,
+only accessible by
+.Ar LOG_USER ,
+that can accept a few commands, .e.g.,
+usable as
+.Li nc -U path
+.Pp
+Currents commands are as follows:
+.Bl -tag -offset aaaa -with addhost
+.It Cm addhost Ar hostline
+Add a new host
+.It Cm addpath Ar fullpkgpath ...
+Add new fullpkgpath to scan
+.It Cm bye
+close the socket connection.
+.It Cm dontclean Ar pkgpath ...
+Add new pkgpath to list of paths that should not be cleaned after build
+.It Cm help
+Self explanatory
+.It Cm stats
+Show the current stats line
+.El
+.Pp
+.Sh SHUTTING DOWN GRACEFULLY
+.Nm
+periodically checks for a file named
+.Pa stop
+in its log directory.
+If this file exists, then it won't start new jobs, and shutdown when
+the current jobs are finished unless
+.Fl q .
+.Pp
+.Nm
+also checks for files named
+.Pa stop-<hostname>
+in its log directory.
+If such a file exists, then it won't start new jobs on
+the corresponding machine.
+.Sh FILES
+Apart from producing packages,
+.Nm
+may create temporary files as
+.Pa ${FULLDISTDIR}/${DISTFILE}.part .
+.Pp
+In fetch mode
+.Po
+.Fl f
+and
+.Fl F
+.Pc ,
+.Nm
+populates
+.Pa ${DISTDIR}/by_cipher/sha256
+with links.
+It also uses
+.Pa ${DISTDIR}/distinfo
+and
+.Pa ${DISTDIR}/history
+as a
+.Sq permanent log :
+.Bl -tag -width distinfo
+.It distinfo
+cache of distfiles checksum.
+Contains all
+.Xr sha256 1
+checksums of known files under
+.Pa ${DISTDIR} .
+Fetching uses this to avoid re-checksumming known files.
+.It history
+Log of old files under distinfo.
+After successfully scanning a full ports tree
+.Po
+.Nm Fl a
+.Pc ,
+the fetch engine knows precisely which files are needed by the build
+(and their checksums).
+Anything that is
+.Bl -bullet
+.It
+recorded in distinfo but unneeded
+.It
+recorded in distinfo but with the wrong checksum
+.It
+not recorded in distinfo, but not needed
+.El
+will be entered at the end of history as a line:
+.Pp
+.Li ts SHA256 (file) = value
+.Pp
+with
+.Ar ts
+a timestamp from Unix epoch.
+.Pp
+When cleaning up old files, with a tool such as
+.Xr clean-old-distfiles 1 ,
+it is vital to check both the checksum and
+the file name: since mirroring stores permanent links under
+.Pa by_cipher ,
+files which are still needed will appear in history under their old
+checksums, as an indication the link should be removed, but possibly not
+the file itself.
+.El
+.Pp
+If
+.Pa ${DISTDIR}
+ever becomes corrupted,
+removing
+.Pa ${DISTDIR}/distinfo
+will force
+.Nm
+into checking all files again.
+.Pp
+All those files belong to the
+.Ar FETCH_USER
+if it is defined.
+They should be readable for the
+.Ar build_user .
+.Pp
+.Nm
+also records rolling build statistics under
+.Pa ${DISTDIR}/build-stats/${ARCH} ,
+and uses them automatically in the absence of
+.Fl b Ar logfile .
+That file belongs to the
+.Ar LOG_USER
+if it is defined.
+.Pp
+If
+.Fl s
+is used, size information for successful builds will be recorded under
+.Pa ${DISTDIR}/build-stats/${ARCH}-size
+.Po
+by default, location adjustable with
+.Fl S Ar sizelog
+.Pc .
+This is then reused for the mfs threshold option.
+That file also belongs to the
+.Ar LOG_USER
+if it is defined.
+.Pp
+.Nm
+also maintains a list of pkgpath frequencies
+.Pa ${DISTDIR}/build-stats/${ARCH}-dependencies ,
+filled at end of LISTING if
+.Fl a .
+This list will be automatically reused when restarting a build:
+a quick LISTING of the most important dependencies will happen
+before the general LISTING,
+in order to prime further LISTING steps with most common ports first.
+.Pp
+.Nm
+will also create a large number of log files under
+.Pa ${PORTSDIR}/logs/${ARCH} ,
+which will belong
+to
+.Ar LOG_USER
+if it is defined:
+.Bl -tag -width engine.log
+.It Pa affinity/
+Affinity information.
+One file per full pkgpath, with slash replaced by dots
+like so:
+.Pa affinity/lang.ghc,-main .
+.It Pa affinity.log
+On startup
+.Nm
+reads existing affinity information, and records it in that log,
+together with its pid.
+This log just exists to verify, along with
+.Pa engine.log ,
+whether correct affinity was heeded.
+.It Pa awaiting-locks.log
+This is purely for gathering performance statistics, about how much
+lock contention happened around
+.Xr pkg_add 1
+and
+.Xr pkg_delete 1
+usage.
+Plotting cumulated time may help in fine-tuning squiggles parameters.
+.It Pa build.log
+Actual build log.
+Each line summarizes build of a single pkgpath, as:
+.Sq pkgpath host time logsize (detailed timing)[!]
+where time is the actual build time in seconds, host is the machine name
+where this occurred, logsize is the corresponding log file size,
+and a ! is appended in case the build didn't succeed.
+.Pp
+The detailed timing info gives a run-down of the build, with clean, fetch,
+prepare, patch (actually extract+patch), configure, build, fake, package, clean
+detailed timing info.
+Note that the actual build time starts at
+.Sq extract
+and finishes at
+.Sq package .
+.It Pa concurrent.log
+Shows the actual concurrency achieved as a result of job starvation /
+parallel handling.
+Only gets a new line when the value changes: pid timestamp jobs
+.It Pa debug.log
+contains various information related to the main engine spinning (RTFS, haven't
+figured that one yet) along with the more useful warning and die traces that
+happen when something wrong occurs.
+Especially useful for the warning messages that tend to be overwritten by
+subsequent displays.
+.It Pa dist/<distfile>.log
+Log of the
+.Xr ftp 1
+process(es) that attempted to fetch the distfile.
+.It Pa dump.log
+A long log file generated at the end of build that yields any information
+pertinent to ports still in the
+.Sq to build
+and the
+.Sq built
+queues.
+See also
+.Pa summary.log
+for an expurged version of same.
+.It Pa engine.log
+Build engine log.
+Each line corresponds to a state change for a pkgpath and starts with the pid
+of
+.Nm ,
+plus a timestamp of the log entry.
+.Bl -tag -width BB:
+.It ^
+pkgpath temporarily put aside, because a job is running in the same directory.
+.It !
+pkgpath ignored, either directly, or indirectly because a dependency was
+ignored.
+End of the line states reason why ignored.
+.It A
+affinity mismatch: path considered for build, but not the right host,
+followed by the affinity information.
+.It B
+pkgpath built / distfile found.
+.It C
+forcible clean-up before building a port with a kde tag.
+.It E
+error in build or fetch.
+.It F
+distfile queued for download.
+.It H
+package still not found due to nfs on this run.
+.It I
+pkgpath can be installed.
+.It J
+job to build pkgpath started.
+Also records the host used for the build.
+.It K
+kde mismatch, no build until host has been cleaned up.
+.It L
+job did not start, existing lock detected.
+.It N
+job did not finish.
+The host may have gone down.
+.It P
+built package is no longer required for anything.
+.It Q
+pkgpath queued as buildable whenever a slot is free.
+.It T
+pkgpath to build / distfile to download.
+.It V
+pkgpath put back in the buildable queue, after job that was running in
+the same directory returned.
+.It X
+only happens when rescanning after an error.
+The engine temporarily locks paths that are incomplete (detained).
+These will be kept in a separate list for later examination until the
+end of the new scan.
+.It x
+only happens when rescanning after an error.
+Releases a path for building after the new scan is finished.
+.It Y
+affinity mismatch, but job will start on the wrong host anyways, as the queue
+contains no other buildable path.
+.El
+.Pp
+Please note that the engine is no longer run after each package build event
+because of performance considerations, so the
+.Sq Q
+and
+.Sq I
+changes may be delayed by a few
+.Sq B .
+.It Pa equiv.log
+Lists of equivalent pkgpaths for the build, when default flavors and default subpackages have been resolved.
+.It Pa fetch/bad.log
+List of URLs that did not lead to a correct distfile, either because
+they were not responding, or because of incorrect checksums.
+.It Pa fetch/good.log
+List of URLs that fetched correctly, along with timing statistics.
+.It Pa fetch/manually.log
+List of pkgpaths that require manual intervention, in human-readable form.
+.It Pa <hostname>.sig.log
+Complete library signature of the host.
+.It Pa init.<hostname>.log
+Captured output of the initialization job for each host.
+.It Pa junk.log
+Option
+.Fl J
+counts the number of dependencies directly added to decide when to run
+.Nm pkg_delete Fl a .
+This file sums up how many ports were built, and how many ports had
+dependencies each time
+.Nm
+decides to junk.
+.It Pa locks/
+Directory where locks are created.
+There are three types of locks:
+.Bl -bullet
+.It
+pkgpath locks for building, where the slash in a pkgpath is replaced
+with a dot like so:
+.Pa locks/devel.make
+to flatten the structure.
+.It
+distfile locks for fetching, using the distfile name without the path like so:
+.Pa locks/distfile.dist .
+.It
+host locks for dependency handling and junking, like so:
+.Pa locks/host:hostname .
+.El
+.It Pa packages/pkgname.log
+one file or symlink per pkgname.
+.It Pa paths/some/path.log
+one file or symlink per pkgpath.
+.It Pa performance.log
+Some parts of
+.Nm
+are computationally intensive, such as the engine runs to determine
+new stuff that can be built, and the actual display reports.
+.Pp
+Both those activities are rate-limited, so that
+.Nm
+doesn't run its engine at each new package build,
+and doesn't update its display every time there is a phase change.
+.Pp
+Lines tagged with
+.Sq ENG
+correspond to the engine;
+lines tagged with
+.Sq REP
+correspond to the display reports.
+.Pp
+Lines ending with a dash
+.Sq -
+correspond to new activity that didn't trigger
+a computation.
+.Pp
+Other lines will feature a plus
+.Sq +
+for normal runs, or an exclamation point
+.Sq !
+for forced runs, followed by two numbers:
+the next timestamp at which we'll be allowed to run, and
+a measure of how much time it took to run this pass.
+.Pp
+That information is mostly relevant while
+.Nm
+is building lots of small packages very quickly.
+.It Pa signature.log
+Discrepancies between hosts that prevent them from starting up.
+.It Pa size.log
+Size of work directory at the end of each build, built only with
+.Fl s .
+.It Pa stats.log
+Simple log of the B=... line summaries.
+Mostly useful for making plots and tweaking performance.
+.It Pa stop
+Not a logfile at all, but a file created by the user to stop
+.Nm
+creating new jobs.
+.It Pa stop-<hostname>
+Not a logfile at all, but created by the user to stop hostname creating
+new jobs.
+.It Pa summary.log
+A summary file generated at end of build that lists packages not built
+or not installable, along with a reason for it.
+This summarizes packages not built because of existing locks, because of
+errors, but also because they depend on something that was not built.
+.Pp
+In that last case,
+.Pa summary.log
+contains a chain of dependencies leading to the problematic package, or
+in case of build cycles, stopping at the first loop.
+.It Pa term-report.log
+Saves all terminal output, so that it can be replayed at hi speed with
+.Xr dpb-replay 1 .
+.It Pa vars.log
+Logs the directories that were walked in the ports tree for dependency
+information, including the path to a dependency that triggered this
+particular step.
+.El
+.Sh DIAGNOSTICS
+.Bl -tag -offset aaaa -width truc
+.It Waiting for hosts to finish STARTUP...
+Displayed on the console while
+.Nm
+is setting up hosts, getting essential data from the ports tree,
+running a
+.Ar STARTUP
+script, collecting base library signatures.
+.It stuck on <lockfilename>
+Display on the console when
+.Nm
+detects a "frozen" port has happened outside of
+.Nm Ns 's
+purview, namely because the ports tree itself has that specific
+port locked without
+.Nm Ns 's
+knowledge.
+See
+.Xr bsd.port.mk 5 ,
+.Xr dolock 1 .
+.It (Junk lock obtained for <host> at <time>)
+.It (Junk lock released for <host> at <time>)
+Printed in a
+.Pa paths/pkgpath.log
+file when attempting to get a
+.Sq junk lock .
+On a given host, all dependency operations are serialized.
+The dependency computation itself is handled by the main
+.Nm
+process, which needs to know exactly which dependencies are used
+at a given point, so that
+.Ar junk
+can clean up the host correctly.
+In particular,
+.Ar junk
+will not clean up dependencies already scheduled for installation.
+Ports that do no obtain the lock on first try are put to sleep.
+.It Received IO
+Printed in a
+.Pa paths/pkgpath.log
+file when woken up before trying attempting to obtain a
+.Ar junk
+lock again...
+.It Woken up <fullpkgpath>
+Printed in a
+.Pa paths/pkgpath.log
+when waking another task by sending it SIGIO,
+so that it may attempt to obtain the junk lock again.
+.It (Junk lock failure for <host> at <time>)
+All ports sleeping for a
+.Ar junk
+lock are woken at the same time, so only one of them will obtain the lock,
+and the others will fail and be put to sleep again.
+.It Short-cut: depends already handled by <fullpkgpath>
+Printed in a
+.Pa paths/pkgpath.log
+when a port wakes up after others that ran
+.Xr pkg_add 1 .
+As
+.Nm
+maintains dependencies for a given host globally, it coalesces depends lists
+together.
+.It Don't run junk because nojunk in <fullpkgpath>
+Printed in a
+.Pa paths/pkgpath.log
+while evaluating whether to run
+.Ar junk .
+Normally,
+.Ar junk
+happens at regular intervals, but ports marked
+.Sq nojunk
+will delay that.
+.Nm
+still keeps track of attempted junks.
+.It Still tainted: <bool>
+A host may have a tag (kde3/kde4) that prevents building differently tagged
+ports.
+This will be cleansed by
+.Ar junk
+eventually.
+This prints in
+.Ar path/pkgpath.log
+to indicate whether this particular
+.Ar junk
+will keep the host tainted with a tag or not.
+.It Forced junk, retainting: <tag>
+Printed at end of
+.Ar prepare-results ,
+when an eventual junk was run even though some ports still hold a tag.
+.It Can't run junk because of lock on <fullpkgpath>
+.Ar junk
+can't happen because
+.Ar fullpkgpath
+is locked and is marked
+.Sq nojunk .
+.It Avoided depends for <dependencies>
+As dependencies are handled globally per-host, some ports can avoid
+.Xr pkg_add 1
+altogether because another port already installed the correct dependencies.
+.It SPINNING ON MAIN
+Printed in
+.Ar debug.log ,
+this is an actual bug: the engine said it can build, there are cores available,
+but
+.Nm
+can't start a new build job.
+.It SPINNING ON FETCH
+Printed in
+.Ar debug.log ,
+this is an actual bug: the engine said it can fetch, there are fetching
+cores available, but
+.Nm
+can't start a new fetch job.
+.It KILLED: <job> stuck at <somewhere>
+Printed in
+.Ar path/pkgpath.log
+when a port exceeds its timeout.
+.El
+.Sh BUGS AND LIMITATIONS
+.Nm
+performs best with lots of paths to build.
+When just used to build a few ports, there's a high risk of starvation
+as there are bottlenecks in parts of the tree.
+.Pp
+Fetch jobs don't deal with checksum changes yet:
+if a fetch fails because of a wrong checksum, if you update the distinfo
+file and remove the lock,
+.Nm
+won't pick it up.
+.Pp
+Note that
+.Nm
+does not manage installed packages in any intelligent way, it will just
+call
+.Xr pkg_add 1
+during its depend stage to install its dependencies.
+With
+.Fl u ,
+it will call pkg_add -r.
+With
+.Fl U ,
+it will call pkg_add -r -D installed,
+but there is nothing else going on.
+This is especially true when using
+.Fl R ,
+ensure the machine is clean of possibly older packages first, or run
+.Nm
+with
+.Fl U .
+.Pp
+In particular
+.Fl R
+and
+.Fl J
+together may lead to strange issues.
+.Pp
+On heterogeneous networks, calibration of build info and choice of speed
+factors is not perfect, and somewhat a dark art.
+Using distinct speed factors on a build log that comes from a single
+machine works fine, but using the build info coming from several machines
+does not work all that well.
+.Pp
+.Nm
+should check
+.Pa /usr/include
+and
+.Pa /usr/X11R6/include
+for consistency, but it doesn't.
+.Pp
+When a host fails consistency check, there is not yet a way to re-add it
+after fixing the problem.
+You have to stop
+.Nm ,
+cleanup and restart.
+.Pp
+The default limits in
+.Pa login.conf
+are too small for bulk builds on any kind of parallel machines.
+Bump number of processes, file descriptors, and memory.
+.Pp
+Even though
+.Nm
+tries really hard to check heterogeneous networks for sanity (checking
+shared libraries and .la files), it is still dependent on the user to
+make sure all the hosts build ports the same way.
+.Pp
+Make sure your NFS setup is consistent.
+The ports dir itself should be exported or synchronized.
+Distfiles, the package repository,  and the plist repository should be exported,
+but WRKOBJDIR should not be on NFS unless you have absolutely no choice,
+or if you exhibit deep masochistic tendencies.
+Pay particular attention to discrepancies in
+.Pa /etc/mk.conf .
+.Pp
+Also,
+.Nm
+connects to external hosts through
+.Xr ssh 1 ,
+relying on
+.Xr ssh_config 5
+for any special cases.
+.Pp
+When fetching distfiles,
+.Nm
+may freeze and spin in a tight loop while the last distfiles are being fetched.
+This is definitely a bug, which has been around for quite some time, which
+is a bit difficult to reproduce, and hasn't been fixed yet.
+So if
+.Nm
+stops updating its display right around the end of fetch, you've hit the bug.
+Just kill
+.Nm
+and restart it.
+.Sh SEE ALSO
+.Xr clean-old-distfiles 1 ,
+.Xr dpb-replay 1 ,
+.Xr proot 1 ,
+.Xr pkgpath 7
+.Sh HISTORY
+The original
+.Nm dpb
+command was written by Nikolay Sturm.
+This version is a complete rewrite from scratch using all the stuff
+we learnt over the years to make it better.
+.Sh AUTHORS
+.An Marc Espie
Index: man1/getpkgpath.1
===================================================================
RCS file: man1/getpkgpath.1
diff -N man1/getpkgpath.1
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ man1/getpkgpath.1 25 Jun 2018 14:03:04 -0000
@@ -0,0 +1,36 @@
+.\" $OpenBSD: getpkgpath.1,v 1.2 2015/09/09 21:34:04 schwarze Exp $
+.\"
+.\" Copyright (c) 2010 Marc Espie <[hidden email]>
+.\"
+.\" Permission to use, copy, modify, and distribute this software for any
+.\" purpose with or without fee is hereby granted, provided that the above
+.\" copyright notice and this permission notice appear in all copies.
+.\"
+.\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+.\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+.\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+.\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+.\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+.\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+.\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+.\"
+.Dd $Mdocdate: September 9 2015 $
+.Dt GETPKGPATH 1
+.Os
+.Sh NAME
+.Nm getpkgpath
+.Nd deduces PKGPATH from current directory
+.Sh SYNOPSIS
+.Nm getpkgpath
+.Ar dir
+.Sh DESCRIPTION
+.Nm
+deduces the
+.Ev PKGPATH
+corresponding to a given directory
+.Ar dir ,
+by trying to strip directories from the environment variable
+.Ev PORTSDIR_PATH .
+.Pp
+This is mostly a helper script for
+.Pa bsd.port.subdir.mk .
Index: man1/out-of-date.1
===================================================================
RCS file: man1/out-of-date.1
diff -N man1/out-of-date.1
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ man1/out-of-date.1 25 Jun 2018 14:03:04 -0000
@@ -0,0 +1,50 @@
+.\" $OpenBSD: out-of-date.1,v 1.5 2015/07/23 08:05:18 bentley Exp $
+.\"
+.\" Copyright (c) 2011 Marc Espie <[hidden email]>
+.\"
+.\" Permission to use, copy, modify, and distribute this software for any
+.\" purpose with or without fee is hereby granted, provided that the above
+.\" copyright notice and this permission notice appear in all copies.
+.\"
+.\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+.\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+.\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+.\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+.\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+.\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+.\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+.\"
+.Dd $Mdocdate: July 23 2015 $
+.Dt OUT-OF-DATE 1
+.Os
+.Sh NAME
+.Nm out-of-date
+.Nd compare installed packages and the ports tree
+.Sh SYNOPSIS
+.Nm out-of-date
+.Bk -words
+.Op Fl mqxv
+.Ek
+.Sh DESCRIPTION
+.Nm
+checks installed packages and compares them to the ports tree.
+It displays packages that may need updating.
+Typical output is a subdirlist with comments, suitable for bulk building.
+.Pp
+Options are as follows:
+.Bl -tag -width pkgpathlonger
+.It Fl m
+Causes
+.Nm
+to always display the progress meter in cases it would not do so by default.
+.It Fl q
+Quick check, only verifies the names and not full update signatures.
+Use is discouraged.
+.It Fl v
+Verbose mode.
+Does not do anything yet.
+.It Fl x
+Disable progress meter.
+.El
+.Sh AUTHORS
+.An Bernd Ahlers
Index: man1/outdated-perl-ports.1
===================================================================
RCS file: man1/outdated-perl-ports.1
diff -N man1/outdated-perl-ports.1
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ man1/outdated-perl-ports.1 25 Jun 2018 14:03:04 -0000
@@ -0,0 +1,46 @@
+.\" $OpenBSD: outdated-perl-ports.1,v 1.1 2011/11/27 13:52:10 jasper Exp $
+.\"
+.\" Copyright (c) 2011 Jasper Lievisse Adriaanse <[hidden email]>
+.\"
+.\" Permission to use, copy, modify, and distribute this software for any
+.\" purpose with or without fee is hereby granted, provided that the above
+.\" copyright notice and this permission notice appear in all copies.
+.\"
+.\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+.\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+.\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+.\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+.\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+.\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+.\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+.\"
+.Dd $Mdocdate: November 27 2011 $
+.Dt OUTDATED-PERL-PORTS 1
+.Os
+.Sh NAME
+.Nm outdated-perl-ports
+.Nd compare CPAN releases and the ports tree
+.Sh SYNOPSIS
+.Nm outdated-perl-ports
+.Bk -words
+.Op Fl d
+.Op Fl m Ar maintainer
+.Ek
+.Sh DESCRIPTION
+.Nm
+retrieves and compares the packages list provided by CPAN with the ports
+recorded in
+.Pa ${PORTSDIR}/INDEX
+and reports which ports have a newer version available upstream.
+.Pp
+Options are as follows:
+.Bl -tag -width maintainer
+.It Fl d
+Causes
+.Nm
+to show the path (relative to ${PORTSDIR}) for the outdated port.
+.It Fl m Ar maintainer
+Only display ports that are being maintained by
+.Ar maintainer .
+This can be specified as name or e-mail address.
+.El
Index: man1/pkg_subst.1
===================================================================
RCS file: man1/pkg_subst.1
diff -N man1/pkg_subst.1
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ man1/pkg_subst.1 25 Jun 2018 14:03:04 -0000
@@ -0,0 +1,101 @@
+.\" $OpenBSD: pkg_subst.1,v 1.11 2015/11/02 11:57:40 espie Exp $
+.\"
+.\" Copyright (c) 2010 Marc Espie <[hidden email]>
+.\"
+.\" Permission to use, copy, modify, and distribute this software for any
+.\" purpose with or without fee is hereby granted, provided that the above
+.\" copyright notice and this permission notice appear in all copies.
+.\"
+.\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+.\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+.\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+.\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+.\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+.\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+.\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+.\"
+.Dd $Mdocdate: November 2 2015 $
+.Dt PKG_SUBST 1
+.Os
+.Sh NAME
+.Nm pkg_subst
+.Nd substitute variables' values in files
+.Sh SYNOPSIS
+.Nm pkg_subst
+.Op Fl ci
+.Op Fl B Ar basedir
+.Op Fl D Ar name Ns Op = Ns Ar value
+.Op Fl g Ar group
+.Op Fl m Ar mode
+.Op Fl o Ar owner
+.Op Ar file ...
+.Sh DESCRIPTION
+.Nm
+is used within the ports tree to substitute variable values, using
+the exact same conventions used by
+.Xr pkg_create 1 .
+Constructs like
+.Li ${VAR}
+will be replaced with the variable value, according to
+.Fl D Ar name Ns Op = Ns Ar value
+options.
+.Pp
+The options are as follows
+.Bl -tag -width Dname[=value]xxx
+.It Fl B Ar basedir
+Specify a basedir in which we know file properties.
+It is forbidden to copy files from elsewhere without
+.Fl m Ar mode .
+.It Fl c
+Copy and substitute.
+Number of
+.Ar file
+arguments must be even.
+They are paired as
+.Ar src1
+.Ar dest1 ,
+.Ar src2
+.Ar dest2 ...
+and each
+.Ar src
+file is substituted and copied into the corresponding
+.Ar dest
+file.
+.It Xo
+.Fl D
+.Ar name Ns Op = Ns Ar value
+.Xc
+Define
+.Ar name
+to
+.Ar value ,
+or to 1 if no value is given.
+.It Fl g Ar group
+Change group of destination file to
+.Ar group .
+.It Fl i
+Ignore changes to group or owner that failed.
+.It Fl m Ar mode
+Change mode of destination file to
+.Ar mode .
+.It Fl o Ar owner
+Change owner of destination file to
+.Ar owner .
+.El
+.Pp
+If no
+.Ar file
+arguments is given,
+.Nm
+acts as a filter from stdin to stdout.
+.Pp
+Without
+.Fl c ,
+each
+.Ar file
+is backed up as
+.Ar file.beforesubst
+before the substitution occurs.
+.Sh SEE ALSO
+.Xr install 1 ,
+.Xr pkg_create 1
Index: man1/portbump.1
===================================================================
RCS file: man1/portbump.1
diff -N man1/portbump.1
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ man1/portbump.1 25 Jun 2018 14:03:04 -0000
@@ -0,0 +1,155 @@
+.\"     $OpenBSD: portbump.1,v 1.6 2016/11/02 21:31:37 sthen Exp $
+.\"
+.\" Copyright (c) 2014 Vadim Zhukov
+.\"
+.\" Permission to use, copy, modify, and distribute this software for any
+.\" purpose with or without fee is hereby granted, provided that the above
+.\" copyright notice and this permission notice appear in all copies.
+.\"
+.\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+.\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+.\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+.\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+.\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+.\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+.\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+.\"
+.Dd $Mdocdate: November 2 2016 $
+.Dt PORTBUMP 1
+.Os
+.Sh NAME
+.Nm portbump
+.Nd tweaks port revisions and library versions
+.Sh SYNOPSIS
+.Nm
+.Op Fl lMmnrv
+.Op Fl W Ar wlib
+.Op Fl w Ar wlib
+.Op Ar portref ...
+.Pp
+.Nm
+.Fl o Ar file
+.Op Fl lMmnrv
+.Op Fl W Ar addlib
+.Op Fl w Ar rmlib
+.Op Ar portref ...
+.Sh DESCRIPTION
+.Nm
+is used to increase ('bump')
+.Ev REVISION
+and
+.Ev SHARED_LIBS
+values in
+.Ox
+ports.
+It also can remove
+.Ev REVISION
+marks and add, update or remove
+.Ev WANTLIB
+items.
+.Pp
+.Ar portref
+has the same syntax as
+.Ev FULLPKGPATH
+port variable, see
+.Xr bsd.port.mk 5 .
+Actually, you can feed a list of pkgpaths as parameters; flavor
+information will be ignored.
+.Nm
+doesn't care about actual subdirectory portion of
+.Ar portref ,
+so you can freely pass
+.Pa \&. ,
+.Pa ../foo
+or anything else like this.
+.Pp
+If subpackage is not specified in port reference, then all subpackages
+of a port will be processed.
+If no port references are given, the port in current directory will
+be processed.
+.Pp
+Default mode of operation is increasing
+.Ev REVISION
+mark values (
+.Fl r ) .
+Operations on
+.Ev REVISION
+marks and on
+.Ev SHARED_LIBS
+could be combined by explicit specifying corresponding flags, see below.
+.Pp
+The following options are available:
+.Bl -tag -width Ds
+.It Fl d
+Delete all
+.Ev REVISION
+marks in
+.Pa Makefile .
+Mutually exclusive with
+.Fl r .
+.It Fl l
+Recreate all
+.Ev WANTLIB
+items based on output of
+.Sq make port-lib-depends-check .
+If used twice or more, the
+.Sq make lib-depends-check
+will be used instead.
+The
+.Fl l
+mode is incompatible with
+.Fl W
+and
+.Fl w .
+.It Fl M
+Increment by one major component of all
+.Ev SHARED_LIBS ,
+resetting minor one to zero if needed.
+Mutually exclusive with
+.Fl m .
+.It Fl m
+Increment by one minor component of all
+.Ev SHARED_LIBS .
+Mutually exclusive with
+.Fl M .
+.It Fl n
+Do not replace
+.Pa Makefile
+but save modified version in the
+.Pa Makefile.bump
+instead.
+.It Fl o Ar file
+Send modified Makefile contents to a given file instead of creating
+.Pa Makefile.bump
+in port's directory.
+This could be only used if not more than one port is specified;
+multiple subpackages of a single port could be specified, though.
+.It Fl r
+Increment
+.Ev REVISION
+values for all subpackages, or only for given ones.
+Mutually exclusive with
+.Fl d .
+.It Fl W Ar wlib
+.It Fl w Ar wlib
+Adds or updates (the
+.Fl W
+flag) or removes (the
+.Fl w
+flag)
+.Ev WANTLIB
+items in whole port, or in particular subpackages, if given.
+Both options could be specified multiple times, accumulating
+corresponding lists.
+The
+.Ev WANTLIB
+entries will be fully re-generated based on previous contents.
+Those modes are incompatible with
+.Fl l .
+.It Fl v
+Enable printing of diagnostic messages to standard error output.
+.El
+.Sh HISTORY
+.Nm
+first appeared in
+.Ox 5.6 .
Index: man1/portcheck.1
===================================================================
RCS file: man1/portcheck.1
diff -N man1/portcheck.1
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ man1/portcheck.1 25 Jun 2018 14:03:04 -0000
@@ -0,0 +1,152 @@
+.\"     $OpenBSD: portcheck.1,v 1.4 2015/07/23 08:05:18 bentley Exp $
+.\"
+.\" Copyright (c) 2013 Vadim Zhukov
+.\"
+.\" Permission to use, copy, modify, and distribute this software for any
+.\" purpose with or without fee is hereby granted, provided that the above
+.\" copyright notice and this permission notice appear in all copies.
+.\"
+.\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+.\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+.\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+.\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+.\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+.\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+.\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+.\"
+.Dd $Mdocdate: July 23 2015 $
+.Dt PORTCHECK 1
+.Os
+.Sh NAME
+.Nm portcheck
+.Nd validate a port before submitting
+.Sh SYNOPSIS
+.Nm
+.Op Fl dNP
+.Op Fl p Ar portsdir
+.Op Fl x Ar pattern
+.Pp
+.Nm
+.Fl A
+.Op Fl dP
+.Op Fl p Ar portsdir
+.Op Fl x Ar pattern
+.Op Ar subdir ...
+.Sh DESCRIPTION
+.Nm
+is used to validate the
+.Ox
+port or port hierarchy in current directory.
+It should be used before submitting ports for review to avoid making
+common mistakes.
+.Nm
+verifies that directory and file structure for a port is in place and
+that no bogus files exist.
+.Pp
+When it's done,
+.Nm
+will print detected value of port's
+.Ev PKGPATH
+to standard output, unless it fails in detection.
+In the latter case, the
+.Fl p
+option should be provided.
+All other (error) messages from
+.Nm
+end up on standard error output.
+.Pp
+By default,
+.Nm
+automatically picks up nearest parent directory named
+.Dq ports ,
+with an optional
+.Dq mystuff
+or
+.Dq openbsd-wip
+subdirectory component, as the ports root directory.
+For example: if the port being imported is located in
+.Pa /home/joe/cvs/ports/openbsd-wip/devel/p5-Foo ,
+then the root ports directory will be detected as being
+.Pa /home/joe/cvs/ports/openbsd-wip .
+To override this behaviour, see the
+.Fl p
+option.
+.Pp
+The following options are available:
+.Bl -tag -width Ds
+.It Fl A
+Intended for running
+.Nm
+on the whole ports tree, i.e., the one lying in
+.Ev PORTSDIR .
+This option adds several ignore patterns (see
+.Fl x
+option description) and disables some other checks (e.g., for missing
+distinfo).
+.Ev PKGPATH
+determining and printing won't be done.
+Implicit change of working directory to the ports tree root is done
+before starting any checks.
+Also, in this mode one or more
+.Ar subdir
+arguments could be specified, to narrow the check only for given
+subdirectories of ports tree root.
+.It Fl d
+Show debugging information such as calling of check routines.
+.It Fl P
+Disable expensive checks that use
+.Dq print-plist-with-depends
+target, e.g., proper usage of
+.Xr gtk-update-icon-cache 1 ,
+.Xr update-desktop-database 1
+and
+.Xr update-mime-database 1 .
+.It Fl p Ar portsdir
+Forces the given directory to be treated as ports root directory.
+Cancels autodetection of the root ports directory made by default.
+This option is useful, e.g., when you have a temporary ports tree in
+a non-standard location.
+.It Fl N
+Intended to be used when working on new ports.
+Enables the checks like the presence of REVISION markers and non-0.0
+.Ev SHARED_LIBS .
+It also enables checks for the presence of CVS directories that could
+be left by mistake when creating a new port based on another one.
+.It Fl x
+Excludes files and subdirectories matching given shell globbing pattern
+from any checks.
+Note that matching is done against relative path, and not against
+absoulte path or base name either.
+I.e., to exclude the
+.Dq x11/kde4/libs/logs
+from checks, you must pass the whole line as argument, not just
+.Dq logs .
+Multiple -x options may be specified.
+.El
+.Sh EXAMPLES
+To validate a new port you've just prepared, go to port's directory and
+run:
+.Bd -literal -offset indent
+$ portcheck -N
+.Ed
+.Pp
+If you were working on updating of an existing port in CVS tree:
+.Bd -literal -offset indent
+$ portcheck
+.Ed
+.Pp
+To run a global check of the whole
+.Dq devel
+category in ports tree, use the
+.Fl A
+option instead:
+.Bd -literal -offset indent
+$ portcheck -Ap /usr/ports devel
+.Ed
+.Sh SEE ALSO
+.Xr portimport 1
+.Sh HISTORY
+This utility was split from
+.Xr portimport 1
+in 2013 and first appeared in
+.Ox 5.5 .
Index: man1/portgen.1
===================================================================
RCS file: man1/portgen.1
diff -N man1/portgen.1
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ man1/portgen.1 25 Jun 2018 14:03:04 -0000
@@ -0,0 +1,91 @@
+.\" $OpenBSD: portgen.1,v 1.1 2016/01/18 19:01:02 tsg Exp $
+.\"
+.\" Copyright (c) 2015 Giannis Tsaraias <[hidden email]>
+.\" Copyright (c) 2015 Vadim Zhukov <[hidden email]>
+.\"
+.\" Permission to use, copy, modify, and distribute this software for any
+.\" purpose with or without fee is hereby granted, provided that the above
+.\" copyright notice and this permission notice appear in all copies.
+.\"
+.\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+.\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+.\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+.\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+.\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+.\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+.\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+.\"
+.Dd $Mdocdate: January 18 2016 $
+.Dt PORTGEN 1
+.Os
+.Sh NAME
+.Nm portgen
+.Nd port modules on the fly
+.Sh SYNOPSIS
+.Nm portgen
+.Ar type
+.Ar module-name
+.Sh DESCRIPTION
+The
+.Nm
+utility creates ports for the given
+.Ar module-name
+from
+.Ar type
+module framework.
+The module is downloaded, analyzed and translated into a port.
+Any missing dependencies are handled recursively.
+New ports will land under ${PORTSDIR}/mystuff.
+The
+.Ev PORTSDIR
+value will be taken from current user environment, and defaults to
+.Pa /usr/ports .
+.Pp
+The
+.Nm
+utility currently supports the following
+.Ar type
+values:
+.Pp
+.Bl -inset -offset indent -compact
+.It Cm p5
+for Perl modules on CPAN.
+.It Cm ruby
+for Ruby gems.
+.El
+.Pp
+The following packages must be installed prior to the invocation of
+.Nm :
+.Pp
+.Bl -item -offset indent -compact
+.It
+.Cm sqlports-compact
+or
+.Cm sqlports
+.It
+.Cm p5-DBI
+.It
+.Cm p5-DBD-SQLite
+.El
+.Sh EXIT STATUS
+.Ex -std
+.Sh EXAMPLES
+The following command will download the latest version of Config::AutoConf
+from CPAN, parse it and create a port for it:
+.Bd -ragged -offset indent
+.Nm
+p5 Config::AutoConf
+.Ed
+.Sh SEE ALSO
+.Xr port-modules 5 ,
+.Xr ports 7
+.Sh HISTORY
+The
+.Nm
+utility first appeared in
+.Ox 5.9
+as a result of Google Summer of Code 2015.
+.Sh AUTHORS
+.An Giannis Tsaraias Aq Mt [hidden email]
+.Sh CAVEATS
+Not every module could be converted to a port.
Index: man1/portimport.1
===================================================================
RCS file: man1/portimport.1
diff -N man1/portimport.1
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ man1/portimport.1 25 Jun 2018 14:03:04 -0000
@@ -0,0 +1,79 @@
+.\"     $OpenBSD: portimport.1,v 1.5 2015/07/23 08:05:18 bentley Exp $
+.\"
+.\" Copyright (c) 2013 Robert Peichaer
+.\"
+.\" Permission to use, copy, modify, and distribute this software for any
+.\" purpose with or without fee is hereby granted, provided that the above
+.\" copyright notice and this permission notice appear in all copies.
+.\"
+.\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+.\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+.\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+.\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+.\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+.\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+.\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+.\"
+.Dd $Mdocdate: July 23 2015 $
+.Dt PORTIMPORT 1
+.Os
+.Sh NAME
+.Nm portimport
+.Nd import a new port to the ports cvs repository
+.Sh SYNOPSIS
+.Nm
+.Op Fl p Ar portsdir
+.Op Fl u Ar username
+.Sh DESCRIPTION
+.Nm
+is used to import the directories and files of a new port to the
+.Ox
+ports
+.Xr cvs 1
+repository, avoiding common mistakes.
+It has to be executed from within the new port's directory.
+.Pp
+.Nm
+calls the
+.Xr portcheck 1
+internally to verify that directory and file structure for a port is in
+place and that no bogus files exist, and also to detect the
+.Ev PKGPATH
+for a port.
+.Pp
+The import is done in two steps.
+In the first step, the
+.Fl n
+option of the cvs command is used to only check for possible problems
+without changing anything and shows the cvs path where the new port will
+be imported.
+In the second step, the current ports directory is imported to the
+.Ox
+ports cvs repository.
+After the import, the new port is checked out in the respective directory
+of the local ports tree.
+.Pp
+The following options are available:
+.Bl -tag -width Ds
+.It Fl p Ar portsdir
+Forces the given directory to be treated as ports root directory.
+Passed through to
+.Xr portcheck 1 .
+.It Fl u Ar username
+Set the username used for
+.Xr ssh 1 ,
+and to compose vendor and release tags.
+Defaults to the local username.
+.El
+.Sh SEE ALSO
+.Xr cvs 1 ,
+.Xr portcheck 1
+.Sh HISTORY
+This command is based on the portimport script of Marc Espie, lightly
+modified by Stuart Henderson and rewritten by Robert Peichaer.
+Later, actual checks were moved to a separate utility,
+.Xr portcheck 1 .
+.Sh CAVEATS
+The
+.Ev CVSROOT
+environment variable is not used.
Index: man1/portslogger.1
===================================================================
RCS file: man1/portslogger.1
diff -N man1/portslogger.1
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ man1/portslogger.1 25 Jun 2018 14:03:04 -0000
@@ -0,0 +1,61 @@
+.\" $OpenBSD: portslogger.1,v 1.3 2015/07/23 08:05:18 bentley Exp $
+.\"
+.\" Copyright (c) 2010 Marc Espie <[hidden email]>
+.\"
+.\" Permission to use, copy, modify, and distribute this software for any
+.\" purpose with or without fee is hereby granted, provided that the above
+.\" copyright notice and this permission notice appear in all copies.
+.\"
+.\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+.\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+.\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+.\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+.\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+.\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+.\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+.\"
+.Dd $Mdocdate: July 23 2015 $
+.Dt PORTSLOGGER 1
+.Os
+.Sh NAME
+.Nm portslogger
+.Nd logs ports builds selectively
+.Sh SYNOPSIS
+.Nm portslogger
+.Op Fl s
+.Ar directory
+.Sh DESCRIPTION
+.Nm
+saves logs from ports build in separate files.
+.Pp
+.Nm
+handles directory change messages from the ports tree infrastructure and
+switches to a corresponding log file situated in
+.Ar directory
+accordingly, e.g., it will log the build from screen in
+.Pa directory/screen-4.0.3p1.log .
+.Nm
+will filter progress-bar noise from
+.Xr ftp 1 ,
+.Xr pkg_add 1 ,
+and
+.Xr pkg_create 1 .
+.Pp
+By default,
+.Nm
+acts as a filter and still passes its input to stdout, unless
+run with
+.Fl s .
+.Pp
+It is most often used as
+.Pp
+.Dl $ make package 2>&1 | portslogger logdir
+.Pp
+or in conjunction with
+.Xr tmux 1 ,
+such as
+.Pp
+.Dl $ tmux pipe-pane 'portslogger -s logdir'
+.Sh SEE ALSO
+.Xr dpb 1 ,
+.Xr tmux 1
Index: man1/proot.1
===================================================================
RCS file: man1/proot.1
diff -N man1/proot.1
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ man1/proot.1 25 Jun 2018 14:03:04 -0000
@@ -0,0 +1,287 @@
+.\" $OpenBSD: proot.1,v 1.17 2017/04/30 18:49:14 espie Exp $
+.\"
+.\" Copyright (c) 2016 Marc Espie <[hidden email]>
+.\"
+.\" Permission to use, copy, modify, and distribute this software for any
+.\" purpose with or without fee is hereby granted, provided that the above
+.\" copyright notice and this permission notice appear in all copies.
+.\"
+.\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+.\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+.\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+.\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+.\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+.\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+.\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+.\"
+.Dd $Mdocdate: April 30 2017 $
+.Dt PROOT 1
+.Os
+.Sh NAME
+.Nm proot
+.Nd ports chroot builder
+.Sh SYNOPSIS
+.Nm proot
+.Op Fl B Ar chroot
+.Op Fl c Ar configfile
+.Op Fl S Ar srcroot
+.Op Ar attribute Ns = Ns Ar value ...
+.Sh DESCRIPTION
+.Nm
+can fill up a chroot directory for ports building usage.
+It will perform a set of
+.Cm actions
+that should fill up a destination
+.Ar chroot
+directory from the base system (or an optional
+.Ar srcroot ) .
+.Pp
+As far as possible,
+.Nm
+will create hardlinks instead of copying files, so that cloning an existing
+chroot will often only consume i-nodes.
+.Pp
+Some attributes can take multiple values.
+As a shorthand, several values can be specified in a row without having
+to repeat the
+.Ar attribute
+name, for instance:
+.Pp
+.Dl proot -B /build actions=unpopulate_light check_symlinks
+.Pp
+A
+.Ar configfile
+mostly contains one
+.Ar attribute Ns = Ns Ar value
+assignment per line, with comments starting with a
+.Sq #
+character.
+.Pp
+Possible
+.Cm actions
+(in the order of execution) are as follows:
+.Pp
+.Bl -tag -width unpopulate_light -compact -offset indent
+.It Cm check_mount
+Verify the state of mount points in the system, specifically whether
+the chroot area is not nodev.
+.It Cm unpopulate
+Remove everything from the chroot apart from selected data.
+See
+.Sx Preserved areas and unpopulate
+for details.
+.It Cm snapshot
+Copy things from a snapshot,
+to be retrieved from an url or from the local filesystem.
+.It Cm locate
+Copy things from the base system, perusing the system
+.Xr locate 1
+databases.
+.Bf Sy
+Note that this might erase data, in case it doesn't match the base system
+and is in the way.
+.Ef
+.It Cm resolv
+Copy the system
+.Xr resolv.conf 5
+and
+.Xr hosts 5
+files.
+.It Cm copy_ports
+Copy the ports tree.
+.It Cm copy_sys
+Copy system include files (deprecated).
+.It Cm unpopulate_light
+Remove everything from the chroot apart from selected data.
+See
+.Sx Preserved areas and unpopulate
+for details.
+.It Cm ldconfig
+Rerun
+.Xr ldconfig 8 .
+.It Cm devs
+Regenerate devices using
+.Xr MAKEDEV 8
+and
+.Xr dev_mkdb 8 .
+.It Cm checkout_ports
+Check out a ports tree from cvs using a provided
+.Cm portscvs
+location.
+.It Cm ports_subdirs
+Create ports infrastructure subdirs, according to users required for
+.Xr dpb 1 .
+If
+.Cm chown_all
+is set to 1,
+.Xr chown 2
+any content within to the appropriate users.
+.It Cm stragglers
+Double-check filled up chroot for files we don't know about.
+.It Cm write_mk
+If some directory values are different from the default,
+write a skeleton
+.Xr mk.conf 5
+file.
+.It Cm check_symlinks
+Report absolute symlinks that exist under the
+.Ar chroot ,
+white-listing known base system links.
+.El
+.Pp
+By default,
+.Nm
+will run
+.Cm check_mount , unpopulate_light , devs , ldconfig , ports_subdirs ,
+.Cm resolv , write_mk .
+If
+.Cm unpopulate
+is used,
+.Cm unpopulate_light
+won't be run.
+It will also run
+.Cm snapshot
+if a snapshot location is provided,
+or
+.Cm locate
+otherwise.
+.Pp
+Add actions with
+.Cm actions Ns = Ns Ar value ,
+remove them with
+.Cm actions Ns =- Ns Ar value .
+.Pp
+Other attributes are:
+.Pp
+.Bl -tag -width mkconf_lines -offset indent -compact
+.It Cm chroot
+same as
+.Fl B Ar chroot
+.It Cm srcroot
+same as
+.Fl S Ar srcroot
+.It Cm PORT_USER
+Who the ports tree should belong to
+.It Cm BUILD_USER
+Who to build as (defaults to _pbuild)
+.It Cm FETCH_USER
+Who to fetch as (defaults to _pfetch)
+.It Cm LOG_USER
+Who to write log as (defaults to
+.Cm BUILD_USER )
+.It Cm PORTSDIR
+.It Cm DISTDIR
+.It Cm PACKAGE_REPOSITORY
+.It Cm PLIST_REPOSITORY
+.It Cm LOCKDIR
+.It Cm WRKOBJDIR
+same as in
+.Xr bsd.port.mk 5
+.Pp
+.It Cm chown_all
+set to 1 to have the action
+.Cm ports_subdirs
+change owners of every content.
+.It Cm extra
+extra file to copy into the
+.Ar chroot .
+.It Cm mkconf_lines
+Add lines to the generated
+.Pa /etc/mk.conf .
+.It Cm mkconf_tail
+Add file to the generated
+.Pa /etc/mk.conf .
+.It Cm preserve
+Supplemental directory to preserve during
+.Cm unpopulate .
+.It Cm sets
+Add or remove sets compared to the default list required to build ports
+(defaults to base comp etc xbase xfont xshare xetc).
+.It Cm snapshot
+Location of snapshot for the
+.Cm snapshot
+action.
+.It Cm portscvs
+Where to get a ports snapshot for the
+.Cm checkout_ports
+action.
+.El
+.Ss Preserved areas and unpopulate
+The following entries won't be erased during a
+.Cm locate
+or
+.Cm unpopulate*
+action:
+.Pp
+.Bl -dash -compact -offset indent
+.It
+Items explicitly marked with the
+.Cm preserve
+attribute.
+.It
+A
+.Cm snapshot
+directory containing the base sets.
+.It
+All the ports specific sub directories if they are defined,
+namely
+.Cm DISTDIR , WRKOBJDIR, LOGDIR, PACKAGE_REPOSITORY , PLIST_REPOSITY , LOCKDIR .
+.It
+The
+.Cm PORTSDIR
+itself, unless
+.Cm copy_ports
+is used.
+.It
+Any mountpoint.
+.It
+Data explicitly copied through other actions, e.g.,
+.Cm extra
+files, key files...
+.El
+.Sh EXAMPLES
+The following
+.Ar configfile
+sets things up for an initial build on a
+.Xr dpb 1
+cluster.
+.Bd -literal -offset index
+chroot=/build
+PORT_USER=espie
+extra=/home/espie/startup
+WRKOBJDIR=/tmp/pobj
+LOCKDIR=/tmp/locks
+PLIST_REPOSITORY=/data/plist
+DISTDIR=/data/distfiles
+PACKAGE_REPOSITORY=/data/packages
+actions=unpopulate_light
+ copy_ports
+.Ed
+.Pp
+It assumes
+.Pa /build/data
+is a separate partition that won't be cleaned up by
+.Cm unpopulate_light ,
+and that can be mounted on slaves.
+Note also that
+.Pa /tmp
+should be a fast local directory on every machine of the cluster.
+.Pp
+.Pa /home/espie/startup
+is a startup script for
+.Xr dpb 1 .
+.Pp
+Initial invocation on the master could be
+.Pp
+.D1 Nm Fl c Ar configfile Cm chown_all Ns =1
+.Pp
+to ensure correct permissions for existing data under various
+ports directories.
+.Pp
+Slaves should not need the extra arguments, as the directories will
+already have correct owners.
+.Sh SEE ALSO
+.Xr dpb 1 ,
+.Xr chroot 8
+.Sh AUTHOR
+Marc Espie
Index: man1/register-plist.1
===================================================================
RCS file: man1/register-plist.1
diff -N man1/register-plist.1
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ man1/register-plist.1 25 Jun 2018 14:03:04 -0000
@@ -0,0 +1,75 @@
+.\" $OpenBSD: register-plist.1,v 1.4 2016/06/10 12:52:57 espie Exp $
+.\"
+.\" Copyright (c) 2010 Marc Espie <[hidden email]>
+.\"
+.\" Permission to use, copy, modify, and distribute this software for any
+.\" purpose with or without fee is hereby granted, provided that the above
+.\" copyright notice and this permission notice appear in all copies.
+.\"
+.\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+.\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+.\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+.\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+.\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+.\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+.\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+.\"
+.Dd $Mdocdate: June 10 2016 $
+.Dt REGISTER-PLIST 1
+.Os
+.Sh NAME
+.Nm register-plist
+.Nd smart diff and registration of packing-lists
+.Sh SYNOPSIS
+.Nm register-plist
+.Ar dir
+.Op Ar pkgname ...
+.Nm
+.Fl t
+.Ar p1
+.Ar p2
+.Sh DESCRIPTION
+.Nm
+is used to check that a packing-list for a given package name doesn't change.
+By default, it is invoked at the end of
+.Li make package,
+see
+.Ev PACKAGE_REPOSITORY
+in
+.Xr bsd.port.mk 5 .
+Package names (package stem plus version) are used to uniquely identify
+packages.
+When something in the package changes, the package name should change,
+usually by increasing
+.Ev REVISION .
+.Pp
+.Nm
+records a packing-list for each
+.Ar pkgname
+into the given directory
+.Ar dir .
+If no previous packing-list for the given pkgname exists, the packing-list
+is recorded.
+If a previous packing-list exists,
+.Nm
+compares both packing-list for discrepancies.
+Whenever something significant changes,
+.Nm
+will display a diff of the differences, and exit with an error.
+.Pp
+Some items in the packing-list may change without actual problems.
+For instance, dependencies are allowed to go forward.
+On the other hand, some items like the package DESCR can't change at all.
+Since this is a generated file, porters often overlook that items such as
+the HOMEPAGE or the MAINTAINER's email address are embedded into that file,
+and thus that changing these requires a REVISION bump.
+.Pp
+.Nm
+can also be invoked with
+.Fl t
+and two packing-lists
+.Ar p1
+.Ar p2
+to directly compare those packing-lists.
+.Sh SEE ALSO
+.Xr bsd.port.mk 5
Index: man1/resolve-lib.1
===================================================================
RCS file: man1/resolve-lib.1
diff -N man1/resolve-lib.1
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ man1/resolve-lib.1 25 Jun 2018 14:03:05 -0000
@@ -0,0 +1,68 @@
+.\" $OpenBSD: resolve-lib.1,v 1.5 2016/04/04 17:12:58 naddy Exp $
+.\"
+.\" Copyright (c) 2010 Marc Espie <[hidden email]>
+.\"
+.\" Permission to use, copy, modify, and distribute this software for any
+.\" purpose with or without fee is hereby granted, provided that the above
+.\" copyright notice and this permission notice appear in all copies.
+.\"
+.\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+.\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+.\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+.\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+.\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+.\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+.\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+.\"
+.Dd $Mdocdate: April 4 2016 $
+.Dt RESOLVE-LIB 1
+.Os
+.Sh NAME
+.Nm resolve-lib
+.Nd resolve library specs during ports build
+.Sh SYNOPSIS
+.Nm resolve-lib
+.Op Fl needed
+.Op Fl silent
+.Ar spec ...
+.Sh DESCRIPTION
+.Nm
+resolve library specs against a list of installed libraries.
+.Nm
+takes a list of file names with full path on stdin,
+and matches each library
+.Ar spec
+against those libraries,
+using
+.Xr library-specs 7
+conventions.
+.Pp
+In particular,
+.Ev X11BASE
+and
+.Ev LOCALBASE
+are used to locate X11 libraries and port libraries.
+.Pp
+.Nm
+will print the best libraries found in a format suitable for
+.Cm @wantlib
+.Po
+see
+.Xr pkg_create 1
+.Pc .
+.Pp
+Each
+.Ar spec
+not found will result in an error message and exit code.
+.Pp
+If the option
+.Fl silent
+is used, nothing will be displayed.
+.Pp
+The option
+.Fl needed
+is used to sort out
+.Ev LIB_DEPENDS :
+.Nm
+will succeed if at least one of the libraries matches against a shared
+library passed on the command line.
Index: man1/retrieve-index.1
===================================================================
RCS file: man1/retrieve-index.1
diff -N man1/retrieve-index.1
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ man1/retrieve-index.1 25 Jun 2018 14:03:05 -0000
@@ -0,0 +1,45 @@
+.\" $OpenBSD: retrieve-index.1,v 1.6 2017/02/23 14:24:14 espie Exp $
+.\"
+.\" Copyright (c) 2010 Marc Espie <[hidden email]>
+.\"
+.\" Permission to use, copy, modify, and distribute this software for any
+.\" purpose with or without fee is hereby granted, provided that the above
+.\" copyright notice and this permission notice appear in all copies.
+.\"
+.\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+.\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+.\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+.\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+.\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+.\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+.\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+.\"
+.Dd $Mdocdate: February 23 2017 $
+.Dt RETRIEVE-INDEX 1
+.Os
+.Sh NAME
+.Nm retrieve-index
+.Nd find index entry in ports index according to selected criteria
+.Sh SYNOPSIS
+.Nm retrieve-index
+.Cm index Ns = Ns Ar file
+.Op Cm category Ns = Ns Ar cat
+.Op Cm maintainer Ns = Ns Ar person
+.Op Cm key Ns = Ns Ar value
+.Sh DESCRIPTION
+.Nm
+scans index file
+.Ar file
+for index entry (or entries) according to selected criteria,
+such as category, maintainer name or actual name.
+.Pp
+Matching is done using perl regular expressions, and corresponding entries
+get printed as a list on a single line on standard output.
+.Nm
+is a helper script used for searches from the ports infrastructure.
+It relies on
+.Pa ${PORTSDIR}/INDEX
+being accurate.
+.Sh SEE ALSO
+.Xr perlre 1 ,
+.Xr ports 7
Index: man1/update-patches.1
===================================================================
RCS file: man1/update-patches.1
diff -N man1/update-patches.1
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ man1/update-patches.1 25 Jun 2018 14:03:05 -0000
@@ -0,0 +1,100 @@
+.\" $OpenBSD: update-patches.1,v 1.6 2017/05/26 10:50:28 espie Exp $
+.\"
+.\" Copyright (c) 2010 Marc Espie <[hidden email]>
+.\"
+.\" Permission to use, copy, modify, and distribute this software for any
+.\" purpose with or without fee is hereby granted, provided that the above
+.\" copyright notice and this permission notice appear in all copies.
+.\"
+.\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+.\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+.\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+.\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+.\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+.\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+.\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+.\"
+.Dd $Mdocdate: May 26 2017 $
+.Dt UPDATE-PATCHES 1
+.Os
+.Sh NAME
+.Nm update-patches
+.Nd create or update patches for a given port.
+.Sh SYNOPSIS
+.Nm update-patches
+.Sh DESCRIPTION
+.Nm
+is a helper script for the target
+.Ar update-patches
+in
+.Xr bsd.port.mk 5 .
+It takes no argument, but uses quite a few environment variables as
+implicit parameters.
+.Pp
+.Nm
+looks under
+.Ev WRKDIST
+for patched files.
+Those are files with extension
+.Ev PATCHORIG ,
+except for those matching also
+.Ev DISTORIG .
+.Pp
+Once those files are found, they are matched against the contents of
+directory
+.Ev PATCHDIR ,
+most specifically files matching wildcard
+.Ev PATCH_LIST .
+Matching files are regenerated on the basis of
+.Pp
+.Dl $ diff -u -p -a ${DIFF_ARGS} $file$PATCHORIG $file
+.Pp
+and compared.
+Modified patches are updated and reported, new patches are generated,
+and old patches that no longer seem to exist are also reported.
+.Pp
+As an exception, if a
+.Pa $file.beforesubst
+original file is found, it is used instead of $file in the diff line:
+.Pp
+.Dl $ diff -u -p -a ${DIFF_ARGS} $file$PATCHORIG $file.beforesubst
+.Pp
+The patch header is normalized to always be:
+.Bd -literal -offset indent
+Index: $file
+--- $file.orig
++++ $file
+.Ed
+.Pp
+Patches with problematic
+.Xr cvs 1
+keywords may be fixed if possible, and reported if not.
+.Pp
+.Nm
+prints the list of changed patches for further study.
+.Sh ENVIRONMENT
+.Bl -tag -width FORCE_REGEN
+.It Ev DIFF_ARGS
+Extra parameters to insert into
+.Xr diff 1 .
+.It Ev DISTORIG
+Extension to original files to ignore entirely.
+.It Ev FORCE_REGEN
+Force patches to be regenerated even if they didn't change.
+.It Ev PATCHDIR
+Path to existing and generated patches.
+.It Ev PATCHORIG
+Extension to match for finding original files.
+.It Ev PATCH_LIST
+Only compare new patches to existing patches matching that pattern.
+.It Ev PATCH_VERBOSE
+Force
+.Nm
+to tell about everything it does.
+.It Ev WRKDIST
+Directory to scan for files to compare.
+.El
+.Sh SEE ALSO
+.Xr cvs 1 ,
+.Xr diff 1 ,
+.Xr patch 1
Index: man1/update-plist.1
===================================================================
RCS file: man1/update-plist.1
diff -N man1/update-plist.1
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ man1/update-plist.1 25 Jun 2018 14:03:05 -0000
@@ -0,0 +1,251 @@
+.\" $OpenBSD: update-plist.1,v 1.15 2018/06/02 09:52:22 espie Exp $
+.\"
+.\" Copyright (c) 2018 Marc Espie <[hidden email]>
+.\"
+.\" Permission to use, copy, modify, and distribute this software for any
+.\" purpose with or without fee is hereby granted, provided that the above
+.\" copyright notice and this permission notice appear in all copies.
+.\"
+.\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+.\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+.\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+.\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+.\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+.\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+.\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+.\"
+.Dd $Mdocdate: June 2 2018 $
+.Dt UPDATE-PLIST 1
+.Os
+.Sh NAME
+.Nm update-plist
+.Nd create or update packing-list(s) for a given port.
+.Sh SYNOPSIS
+.Ev PORTSDIR Ns = Ns Ar path
+.Op Ev FAKE_TREE_OWNER Ns = Ns Ar user Ev PORTS_TREE_OWNER Ns = Ns Ar user Nm doas
+.Nm
+.Op Fl Fmnqrvx
+.Op Fl C Ar dir
+.Op Fl E Ar ext
+.Op Fl e Ar ext
+.Op Fl i Ar var
+.Op Fl j Ar jobs
+.Op Fl S Ar var
+.Op Fl s Ar var
+.Op Fl X Ar path
+.Op Fl w Ar suffix
+.Fl -
+.Ar pkg_create_args ...
+.Sh DESCRIPTION
+.Nm
+is a helper script for the target
+.Ar update-plist
+in
+.Xr bsd.port.mk 5 .
+.Pp
+Along with options and environment variables,
+.Nm
+uses the exact same options and arguments that would be passed to
+.Xr pkg_create 1 .
+Since
+.Xr pkg_create 1
+normally takes one single actual
+.Ar pkgname ,
+there is no ambiguity in
+.Ev MULTI_PACKAGES
+situations.
+.Pp
+The first set of arguments correspond to the
+.Sq default package .
+.Pp
+If
+.Ev PORTS_PRIVSEP
+is in effect,
+.Nm
+is run as root, but will switch back to
+.Ev FAKE_TREE_OWNER
+for scanning the installation directory and to
+.Ev PORTS_TREE_OWNER
+for writing packing-lists.
+.Pp
+.Nm
+may need to know the
+.Ev PORTSDIR
+location in order to scan dependencies to trim common directories.
+.Pp
+The options are as follows:
+.Bl -tag -width Ds
+.It Fl C Ar dir
+Use a cache directory
+.Ar dir
+for scanning dependencies.
+Beware that this directory should then be cleaned manually if the
+dependencies's packing-lists change.
+But this will speed up packing-list regeneration for ports with
+lots of dependencies significantly.
+.It Fl E Ar ext
+Write new files with
+.Ar ext
+extension instead of the default
+.Pa .new .
+.It Fl e Ar ext
+Preserve old files as
+.Ar ext
+extension instead of the default
+.Pa .orig .
+.It Fl F
+Do not try to run
+.Xr pkg_locate 1 .
+.It Fl i Ar var
+Ignore variable
+.Ar var
+for new substitutions.
+Only existing substitutions will be considered for the new lists.
+.It Fl j Ar jobs
+Use
+.Ar jobs
+concurrent
+.Xr pkg_locate 1
+pipes.
+Default is
+.Sq hw.ncpu .
+.It Fl n
+Do not move the final packing-lists in position. Instead, leave the
+.Pa <file>.new
+files for comparison.
+.It Fl q
+Quiet mode.
+Do not display status progress messages.
+Note that this is not the opposite of verbose mode.
+.It Fl r
+Run the
+.Pa fake
+directory scan as root.
+.It Fl S Ar var
+Consider variable for substitution only at the end of paths,
+e.g., for suffixes like
+.Pa .pyc
+or
+.Pa .pyo .
+.It Fl s Ar var
+Consider variable for substitution only at start of paths,
+e.g., for directories like
+.Pa /etc/rc.d
+or
+.Pa /usr/local .
+.It Fl v
+Verbose mode.
+Explain about directories stripped from dependencies.
+Note that this is not the opposite of quiet mode.
+.It Fl w Ar suffix
+Warn about suffixes such as
+.Ar .orig
+that (usually) should not end into packing-lists.
+.It Fl X Ar path
+Exclude file at absolute
+.Ar path
+from the information recorded in the packing-list.
+.El
+.Pp
+.Nm
+assumes all multi-packages live under the same installation directory
+.Po
+.Xr pkg_create 1 Ns 's
+.Fl B
+option
+.Pc ,
+and it will scan all paths under that directory and dispatch
+them into corresponding subpackages, using the following heuristics:
+.Bl -bullet
+.It
+files that were in existing packing-list(s) will be kept in the same
+packing-list.
+.It
+new files and directories will be put in the packing-list that owns
+the corresponding directory.
+.It
+baring that, new files will be put into the first packing-list whose PREFIX
+fits.
+.El
+.Pp
+As far as possible, everything that looks like actual files will be sorted
+in alphabetical order, after variable substitutions.
+.Pp
+.Nm
+will warn for a lot of conditions, such as pre-formatted manpages, or files
+ending in pre-patch suffixes.
+Adding a
+.Cm @comment Ar intentional<reason>
+will silence the warning.
+For instance:
+.Bd -literal -offset indent
+@man man/ja_JP.EUC/cat1/kakasi.0
+@comment intentional: mandoc does't handle this locale
+.Ed
+.Pp
+.Nm
+will also scan dependencies, in order to strip common directories.
+.Pp
+If the
+.Ar pkglocatedb
+package is installed,
+.Nm
+will make use of it and report unregistered conflicts.
+This can be disabled with
+.Fl F
+as it is time-consuming.
+.Pp
+Most
+.Ev SUBST_VARS
+variables will be back-substituted into the regenerated packing-lists,
+starting from the longest variable values.
+.Nm
+tries to keep existing substitutions.
+It does not add new substitutions on
+.Ev BASE_PKGPATH
+and
+.Ev BUILD_PKGPATH ,
+it does only substitute
+.Ev FULLPKGNAME
+in
+.Pa share/doc/pkg-readmes
+and it currently does not add new substitutions for
+.Ev ARCH
+nor
+.Ev MACHINE_ARCH .
+.Pp
+Specific items such as shared libraries or binaries will gain annotations
+and special handling, for instance
+.Ev LIBlibname_VERSION .
+.Pp
+Existing packing-lists are scanned for non-file entries, such as
+.Cm newuser ,
+.Cm mode ,
+or
+.Cm exec
+markers.
+Those will be inserted into the updated packing-lists at the most likely
+position.
+.Pp
+.Nm
+will first write files as
+.Pa <file>.new ,
+then it will display which files are new and which files have changed.
+If old packing-lists already exist, it will move old packing lists into
+.Pa <file>.orig
+unless these already exist, then move the new files into position.
+.Sh SEE ALSO
+.Xr pkg_add 1 ,
+.Xr pkg_create 1 ,
+.Xr bsd.port.mk 5
+.Sh BUGS AND LIMITATIONS
+.Nm
+might be the most complicated piece of the ports infrastructure.
+There will always be cases that require manual intervention.
+.
+Since
+.Ar fake
+is now run as non-root,
+.Nm
+can't figure out users and groups for new files, so these should be
+considered carefully.
Index: man8/Makefile
===================================================================
RCS file: /cvs/src/share/man/man8/Makefile,v
retrieving revision 1.97
diff -u -p -r1.97 Makefile
--- man8/Makefile 26 Apr 2017 14:53:11 -0000 1.97
+++ man8/Makefile 25 Jun 2018 14:03:05 -0000
@@ -2,7 +2,7 @@
 # $NetBSD: Makefile,v 1.13 1996/03/28 21:36:40 mark Exp $
 # @(#)Makefile 8.1 (Berkeley) 6/5/93
 
-MAN= afterboot.8 autoinstall.8 boot_config.8 \
+MAN= afterboot.8 autoinstall.8 boot_config.8 bulk.8 \
  crash.8 daily.8 \
  diskless.8 genassym.sh.8 intro.8 netstart.8 rc.8 \
  rc.conf.8 rc.d.8 rc.shutdown.8 rc.subr.8 release.8 \
Index: man8/bulk.8
===================================================================
RCS file: man8/bulk.8
diff -N man8/bulk.8
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ man8/bulk.8 25 Jun 2018 14:03:05 -0000
@@ -0,0 +1,202 @@
+.\" $OpenBSD: bulk.8,v 1.11 2017/12/29 17:58:42 espie Exp $
+.\"
+.\" Copyright (c) 2016 Marc Espie <[hidden email]>
+.\"
+.\"
+.\" Permission to use, copy, modify, and distribute this software for any
+.\" purpose with or without fee is hereby granted, provided that the above
+.\" copyright notice and this permission notice appear in all copies.
+.\"
+.\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+.\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+.\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+.\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+.\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+.\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+.\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+.Dd $Mdocdate: December 29 2017 $
+.Dt BULK 8
+.Os
+.Sh NAME
+.Nm bulk
+.Nd building OpenBSD packages in bulk
+.Sh DESCRIPTION
+There are quite a few steps necessary to build packages on a cluster.
+They are:
+.Pp
+.Bl -enum -compact -offset indent
+.It
+Choose master machine setup and create partitions.
+.It
+Setup chrooted builds on the master.
+.It
+Add slaves and do a full bulk.
+.It
+Clean up and do subsequent bulks.
+.It
+Perform additional maintenance.
+.El
+.Ss 1. Choose master machine setup and create partitions
+Setup a master machine with enough room for a chroot, say
+.Pa /build .
+Assuming you are using a cluster of machines,
+this chroot should contain NFS exportable partitions for distfiles,
+plists, and packages (one single partition can be used for simplicity).
+A full setup requires on the order of 50GB for distfiles and 50GB for packages.
+.Pp
+It is possible to build packages without a chroot, but the space
+requirement difference is negligible (a full
+.Ox
+install is less than 1GB), and having everything chrooted means you may
+install useful tools to help with the process outside of the chroot
+.Po
+for instance
+.Xr rsync 1
+.Pc .
+.Pp
+Reserve one "scratch" partition under the chroot for WRKOBJDIR
+(for instance, mfs, async, or SSD).
+This partition should be roughly 10GB if you want to be able to
+build all ports.
+This can often double as
+.Pa /tmp
+under the chroot.
+.Pp
+Alternately, you can setup your whole chroot as a scratch partition,
+and reserve one more permanent space under it for distfiles,
+packages, and plists.
+.Pp
+Choose a strategy for the ports tree itself.
+There must be a copy under
+.Pa /build .
+You can either copy it from outside the chroot, have it in an NFS
+partition, or manually make sure all machines on the cluster have the
+same ports tree (cvs checkout, rsync ...).
+.Pp
+Note that logs are only produced on the master, and thus do not
+need to be nfs exportable, nor even inside the chroot.
+.Pp
+.Ox
+now comes with default users for package builds, namely _pbuild and _pfetch.
+.Pp
+The default
+.Xr login.conf 5
+is appropriate for most setups, but _pbuild's datasize-cur will need
+to be bumped for a few ports, like pypy.
+Likewise, maxproc-cur is too small for machines with more than 4-6 cpus.
+.Pp
+Note that _pbuild does not need network access, and is now blocked by default
+in
+.Xr pf 4 .
+.Ed
+.Pp
+Recent
+.Ox
+systems do not need any kind of
+.Xr doas 1
+setup for bulk ports builds, as
+.Xr dpb 1
+is run as root and drops permissions appropriately.
+.Pp
+However, you may still want to setup
+.Xr doas 1
+for root, if you want to manually fix ports, as
+.Ar PORTS_PRIVSEP
+relies on it.
+.Ss 2. Setup chrooted builds on the master
+Populate the initial chroot with
+.Xr proot 1 .
+Point DISTDIR, PACKAGE_REPOSITORY, PLIST_REPOSITORY, WRKOBJDIR
+to appropriate locations.
+.Pp
+Pay attention to nodev and wxallowed warnings.
+A chroot setup that can't have devices won't work at all.
+A setup without wxallowed in /usr/local and WRKOBJDIR won't
+build a lot of things.
+.Pp
+Check that this setup can build ports by running
+.Li dpb -B /build
+as root.
+Fix any issues related to file ownership at this point.
+See
+.Xr dpb 1
+for details.
+.Pp
+If your WRKOBJDIR is a temporary partition, make sure it
+belongs to _pbuild:_pbuild after a reboot.
+.Ss 3. Add slaves and do a full bulk
+Create identical slave machines with the same release material.
+Have them import the NFS partitions from the master, they
+don't need root access to the partitions.
+Set up
+.Xr ssh 1
+so that the master can connect to the slaves, using ssh protocol 2,
+as root, preferably without a password or passphrase (however,
+.Xr dpb 1
+uses a master connection, so a password would be required just once per host).
+.Pp
+Note that code on slave machines will only run as _pbuild
+(during builds) or root (during dependency installation).
+Slave machines only require highly restricted network access.
+They just need to act as nfs clients to the master and to be reachable
+through ssh from the master.
+.Pp
+Use a similar
+.Xr proot 1 config
+to populate each slave.
+.Pp
+You should now be able to build ports on the slaves.
+A simple config will just have
+.Bd -literal -offset indent
+DEFAULT chroot=/build
+localhost
+host1
+\&...
+.Ed
+.Pp
+Check that the full config can still build ports.
+.Pp
+You're now ready for a full bulk.
+Beware that even fast configs (3 amd64 with 8 cores each) may take over 24 hours
+to finish. It's generally appropriate to run
+.Xr dpb 1
+under
+.Xr tmux 1 .
+.Ss 4. Clean up and do subsequent bulks
+Before running the next bulk, you should set up rotating logs and move the
+PACKAGE_REPOSITORY away.
+Save the PLIST_REPOSITORY and DISTDIR though.
+PLIST_REPOSITORY will catch problems in packing-lists.
+.Pa ${PLIST_REPOSITORY}/${ARCH}/history
+is also used to store
+.Xr sha256 1
+history, necessary to reorder files inside packages to speed updates up.
+.Pp
+The DISTDIR contains history information as well as DISTDIR/build-stats
+to speed further runs up.
+.Pp
+How you wipe things out depends on your setup.
+If you run
+.Xr proot 1
+again each time, most things will get cleaned up automatically
+.Po
+.Pa /build/usr/local , /build/var/db/pkg ...
+.Pc .
+Note that known directories such as WRKOBJDIR do not get cleaned up
+automatically, so you may want to set up a STARTUP_SCRIPT in
+.Xr dpb 1 .
+.Ss 5. Perform additional maintenance
+.Xr clean-old-distfiles 1
+should be run occasionally since the DISTDIR will continue growing.
+.Pp
+.Xr check-problems 1
+should be run occasionally to find out conflicts and dependency issues.
+.Sh SEE ALSO
+.Xr check-problems 1 ,
+.Xr clean-old-distfiles 1 ,
+.Xr dpb 1 ,
+.Xr proot 1 ,
+.Xr register-plist 1 ,
+.Xr tmux 1 ,
+.Xr bsd.port.mk 5 ,
+.Xr release 8

Reply | Threaded
Open this post in threaded view
|

Re: ports/infrastructure/man by default

Ingo Schwarze
Hi Marc,

Marc Espie wrote on Mon, Jun 25, 2018 at 04:03:23PM +0200:

> So this is what moving those manpages to base looks like.
> Okay ?

I like the idea, and i checked that the diff does
what it is supposed to do.

So, OK schwarze.
  Ingo

Reply | Threaded
Open this post in threaded view
|

Re: ports/infrastructure/man by default

Theo de Raadt-2
Ingo Schwarze <[hidden email]> wrote:

> Hi Marc,
>
> Marc Espie wrote on Mon, Jun 25, 2018 at 04:03:23PM +0200:
>
> > So this is what moving those manpages to base looks like.
> > Okay ?
>
> I like the idea, and i checked that the diff does
> what it is supposed to do.

I am a bit concerned about these falling into the regular namespace.

dolock.1, in particular.

Reply | Threaded
Open this post in threaded view
|

Re: ports/infrastructure/man by default

Marc Espie-2
On Mon, Jun 25, 2018 at 02:22:21PM -0600, Theo de Raadt wrote:

> Ingo Schwarze <[hidden email]> wrote:
>
> > Hi Marc,
> >
> > Marc Espie wrote on Mon, Jun 25, 2018 at 04:03:23PM +0200:
> >
> > > So this is what moving those manpages to base looks like.
> > > Okay ?
> >
> > I like the idea, and i checked that the diff does
> > what it is supposed to do.
>
> I am a bit concerned about these falling into the regular namespace.
>
> dolock.1, in particular.

The subject line says things clearly.

If it ever confuses people, we may rename stuff.

This can happen *later*, I think.

Reply | Threaded
Open this post in threaded view
|

Re: ports/infrastructure/man by default

Antoine Jacoutot-7
On Mon, Jun 25, 2018 at 10:27:51PM +0200, Marc Espie wrote:

> On Mon, Jun 25, 2018 at 02:22:21PM -0600, Theo de Raadt wrote:
> > Ingo Schwarze <[hidden email]> wrote:
> >
> > > Hi Marc,
> > >
> > > Marc Espie wrote on Mon, Jun 25, 2018 at 04:03:23PM +0200:
> > >
> > > > So this is what moving those manpages to base looks like.
> > > > Okay ?
> > >
> > > I like the idea, and i checked that the diff does
> > > what it is supposed to do.
> >
> > I am a bit concerned about these falling into the regular namespace.
> >
> > dolock.1, in particular.
>
> The subject line says things clearly.
>
> If it ever confuses people, we may rename stuff.
>
> This can happen *later*, I think.

we have port-modules(5), maybe we should rename some from foo to port-foo?

--
Antoine