Partitioning recommendations for 6.3?

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

Partitioning recommendations for 6.3?

John Long-4
Been a while and don't have my other OpenBSD boxes accessible.

What are the recommended partitions and appropriate sizes for people
who want to track stable and possibly build the whole ports tree?

Thanks,

/jl

Reply | Threaded
Open this post in threaded view
|

Re: Partitioning recommendations for 6.3?

Il Ka
Do you want to really build all ports or just fetch skeletons and build
some of them?

For skeletons, automatic layout is good enough, but I recommend to increase
/usr/src a little and decrease /home.
Make sure you have ~ 5GB for /usr/src/ and /usr/obj.





On Mon, Jun 25, 2018 at 3:17 PM, John Long <[hidden email]> wrote:

> Been a while and don't have my other OpenBSD boxes accessible.
>
> What are the recommended partitions and appropriate sizes for people
> who want to track stable and possibly build the whole ports tree?
>
> Thanks,
>
> /jl
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Partitioning recommendations for 6.3?

Vijay Sankar
In reply to this post by John Long-4

Quoting John Long <[hidden email]>:

> Been a while and don't have my other OpenBSD boxes accessible.
>
> What are the recommended partitions and appropriate sizes for people
> who want to track stable and possibly build the whole ports tree?
>
> Thanks,
>
> /jl

Hi,

Hopefully more knowledgeable people may give us better advice. The  
default installation and partition sizes worked great for me till 6.1  
and I was able to build and test changes to kernel and also test ports  
etc without making any changes. However, for the past year or so, I  
have had to increase the size of /usr to 6G and /usr/local to 20G to  
build all the packages.

HTH,

Vijay


Vijay Sankar, M.Eng., P.Eng.
ForeTell Technologies Limited
[hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Partitioning recommendations for 6.3?

John Long-4
In reply to this post by Il Ka
On Mon, 2018-06-25 at 17:16 +0300, IL Ka wrote:
> Do you want to really build all ports or just fetch skeletons and
> build some of them?

Not sure, but I don't want to rule out building them all for a couple
or reasons. I have a new box which is probably fast enough to make it
worthwhile to build packages for some slower boxes I have. Second thing
is rebuilding the system from source and then building all the ports is
a good stability test. Bottom line is probably that I would rather plan
to have the space available and not need it then to need it and not
have it. Seems like in the past this was a problem for me.

> For skeletons, automatic layout is good enough, but I recommend to
> increase /usr/src a little and decrease /home.
> Make sure you have ~ 5GB for /usr/src/ and /usr/obj.
>

Thanks, this helps. The automatic layout didn't include /usr/xenocara
There used to be a recommendation in the past to have that as a
separate filesystem. How large should it be?

Is there any reason to track -stable anymore or has syspatch done away
with the need for that?

Seems to me, after trying to install OpenBSD on a new box, a lot of the
helpful in the FAQ is totally AWOL now and I find it hard to get all
the info together.

/jl

>
>
>
>
> On Mon, Jun 25, 2018 at 3:17 PM, John Long <[hidden email]> wrote:
> > Been a while and don't have my other OpenBSD boxes accessible.
> >
> > What are the recommended partitions and appropriate sizes for
> > people
> > who want to track stable and possibly build the whole ports tree?
> >
> > Thanks,
> >
> > /jl
> >
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Partitioning recommendations for 6.3?

John Long-4
In reply to this post by Vijay Sankar
On Mon, 2018-06-25 at 09:25 -0500, Vijay Sankar wrote:

> Quoting John Long <[hidden email]>:
>
> > Been a while and don't have my other OpenBSD boxes accessible.
> >
> > What are the recommended partitions and appropriate sizes for
> > people
> > who want to track stable and possibly build the whole ports tree?
> >
> > Thanks,
> >
> > /jl
>
> However, for the past year or so, I have had to increase the size of
> /usr to 6G and /usr/local to 20G to build all the packages.

I can't remember now.. ports go under /usr/local, correct? What goes in
/usr that would require 6G?

Thanks,

/jl

Reply | Threaded
Open this post in threaded view
|

Re: Partitioning recommendations for 6.3?

Vijay Sankar

Quoting John Long <[hidden email]>:

> On Mon, 2018-06-25 at 09:25 -0500, Vijay Sankar wrote:
>> Quoting John Long <[hidden email]>:
>>
>> > Been a while and don't have my other OpenBSD boxes accessible.
>> >
>> > What are the recommended partitions and appropriate sizes for
>> > people
>> > who want to track stable and possibly build the whole ports tree?
>> >
>> > Thanks,
>> >
>> > /jl
>>
>> However, for the past year or so, I have had to increase the size of
>> /usr to 6G and /usr/local to 20G to build all the packages.
>
> I can't remember now.. ports go under /usr/local, correct? What goes in
> /usr that would require 6G?
>
> Thanks,
>
> /jl

Here is my df -h output -- Just as an FYI I was testing some  
workarounds for the samba virusfilter issue and then made some  
mistakes that screwed up KDE etc. So decided to build it from scratch  
and have about 5000 packages built right now with the following disk  
usage.

$ df -h
Filesystem     Size    Used   Avail Capacity  Mounted on
/dev/sd0a     1005M    102M    852M    11%    /
/dev/sd0l      3.9G    1.8G    2.0G    48%    /builds
/dev/sd0k      127G    1.3G    119G     1%    /home
/dev/sd0d      3.9G    7.2M    3.7G     0%    /tmp
/dev/sd0f      5.9G    1.9G    3.8G    33%    /usr
/dev/sd0g      2.0G    185M    1.7G    10%    /usr/X11R6
/dev/sd0h     19.7G    9.4G    9.3G    50%    /usr/local
/dev/sd0j      5.9G    3.3G    2.3G    59%    /usr/obj
/dev/sd0i      2.0G    990M    929M    52%    /usr/src
/dev/sd0e     31.5G   57.9M   29.9G     0%    /var
/dev/sd0m      243G   83.7G    147G    36%    /usr/ports

Reason why I had to increase /usr from the default 2G to 6G was  
because I tend to build -current or -stable in addition to packages  
and the additional files in /usr/share/relink went above the 2G size.  
As a result I increased /usr partition to 6G.

Re. /usr/local, I used to be able to just run dpb (before 6.1) and get  
almost all the packages built without having to do any manual checks.  
So no packages were added to /usr/local earlier. But I may be missing  
something because nowadays I am able to only build around 2700  
packages if I run dpb blindly. I then have to do a make package  
manually for critical items like cmake and others. I found out the  
hard way that if I clean stuff up, some packages such as window  
managers don't build for me. So I leave /usr/local as is which  
resulted in me having to increase the size of /usr/local.  Also, for  
some packages such as webkit, I end up having to do a make clean all  
and then make package.

I may be doing something wrong so none of the above is a recommendation.


Vijay Sankar, M.Eng., P.Eng.
ForeTell Technologies Limited
[hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Partitioning recommendations for 6.3?

John Long-4
On Mon, 2018-06-25 at 10:15 -0500, Vijay Sankar wrote:

> Here is my df -h output -- Just as an FYI I was testing some  
> workarounds for the samba virusfilter issue and then made some  
> mistakes that screwed up KDE etc. So decided to build it from
> scratch  
> and have about 5000 packages built right now with the following
> disk  
> usage.
>
> $ df -h
> Filesystem     Size    Used   Avail Capacity  Mounted on
> /dev/sd0a     1005M    102M    852M    11%    /
> /dev/sd0l      3.9G    1.8G    2.0G    48%    /builds
> /dev/sd0k      127G    1.3G    119G     1%    /home
> /dev/sd0d      3.9G    7.2M    3.7G     0%    /tmp
> /dev/sd0f      5.9G    1.9G    3.8G    33%    /usr
> /dev/sd0g      2.0G    185M    1.7G    10%    /usr/X11R6
> /dev/sd0h     19.7G    9.4G    9.3G    50%    /usr/local
> /dev/sd0j      5.9G    3.3G    2.3G    59%    /usr/obj
> /dev/sd0i      2.0G    990M    929M    52%    /usr/src
> /dev/sd0e     31.5G   57.9M   29.9G     0%    /var
> /dev/sd0m      243G   83.7G    147G    36%    /usr/ports

Thanks, this is good info.

I am trying to find out about /usr/xenocara if it is still needed and
also whether it's still recommended to build from source and track
-stable or whether syspatch does away with that.

What is the recommended http server these days? I remember the
transition from apache to nginx. What's the conventional wisdom?

My plan for this box is sftp, http, and minidlna server.

Thank you,
 
/jl

Reply | Threaded
Open this post in threaded view
|

Re: Partitioning recommendations for 6.3?

Bruno Flueckiger
In reply to this post by John Long-4
On 25.06.2018 14:17, John Long wrote:
> Been a while and don't have my other OpenBSD boxes accessible.
>
> What are the recommended partitions and appropriate sizes for people
> who want to track stable and possibly build the whole ports tree?
>
> Thanks,
>
> /jl

Check the detailed explanation given by Ingo Schwarze:

https://marc.info/?l=openbsd-misc&m=149890809430366&w=2

Cheers,
Bruno

Reply | Threaded
Open this post in threaded view
|

Re: Partitioning recommendations for 6.3?

Bryan Harris
In reply to this post by John Long-4
The webserver is called httpd (not the apache one). I like this book but
some people don't need the extra help of a book (I do).

https://www.michaelwlucas.com/tools/relayd

On Mon, Jun 25, 2018 at 11:49 AM John Long <[hidden email]> wrote:

> On Mon, 2018-06-25 at 10:15 -0500, Vijay Sankar wrote:
> > Here is my df -h output -- Just as an FYI I was testing some
> > workarounds for the samba virusfilter issue and then made some
> > mistakes that screwed up KDE etc. So decided to build it from
> > scratch
> > and have about 5000 packages built right now with the following
> > disk
> > usage.
> >
> > $ df -h
> > Filesystem     Size    Used   Avail Capacity  Mounted on
> > /dev/sd0a     1005M    102M    852M    11%    /
> > /dev/sd0l      3.9G    1.8G    2.0G    48%    /builds
> > /dev/sd0k      127G    1.3G    119G     1%    /home
> > /dev/sd0d      3.9G    7.2M    3.7G     0%    /tmp
> > /dev/sd0f      5.9G    1.9G    3.8G    33%    /usr
> > /dev/sd0g      2.0G    185M    1.7G    10%    /usr/X11R6
> > /dev/sd0h     19.7G    9.4G    9.3G    50%    /usr/local
> > /dev/sd0j      5.9G    3.3G    2.3G    59%    /usr/obj
> > /dev/sd0i      2.0G    990M    929M    52%    /usr/src
> > /dev/sd0e     31.5G   57.9M   29.9G     0%    /var
> > /dev/sd0m      243G   83.7G    147G    36%    /usr/ports
>
> Thanks, this is good info.
>
> I am trying to find out about /usr/xenocara if it is still needed and
> also whether it's still recommended to build from source and track
> -stable or whether syspatch does away with that.
>
> What is the recommended http server these days? I remember the
> transition from apache to nginx. What's the conventional wisdom?
>
> My plan for this box is sftp, http, and minidlna server.
>
> Thank you,
>
> /jl
>
>

--
So the HP guy comes up to me and he says, 'If you say nasty things like
that to vendors you're not going to get anything'. I said 'no, in eight
years of saying nothing, we've got nothing, and I'm going to start saying
nasty things, in the hope that some of these vendors will start giving me
money so I'll shut up'.

 -Theo De Raadt
Reply | Threaded
Open this post in threaded view
|

Re: Partitioning recommendations for 6.3?

John Long-4
In reply to this post by Bruno Flueckiger
Thanks @bryanharris and @bruno

Thanks guys, I will check out the links.

/jl

Reply | Threaded
Open this post in threaded view
|

Re: Partitioning recommendations for 6.3?

Solene Rapenne
In reply to this post by John Long-4

John Long writes:

> Been a while and don't have my other OpenBSD boxes accessible.
>
> What are the recommended partitions and appropriate sizes for people
> who want to track stable and possibly build the whole ports tree?
>
> Thanks,
>
> /jl

hello

If you want to do a bulk build (aka whole ports tree), read bulk(8) (in
ports/infrastructure/man/), it says that you need at least 100 GB of
disk.


   1. Choose master machine setup and create partitions

        Setup a master machine with enough room for a chroot, say
        /build.  Assuming you are using a cluster of machines, this
        chroot should contain NFS exportable partitions for distfiles,
        plists, and packages (one single partition can be used for
        simplicity).  A full setup requires on the order of 50GB for
        distfiles and 50GB for packages.

Reply | Threaded
Open this post in threaded view
|

Re: Partitioning recommendations for 6.3?

lea.chescotta
Personally, what i always do, (i dont know if its the best practice, but it fixes my storage space problems, as i always use -stable, and build the updated ports) is to make a symlink in /home

Initial configuration
  $ cd /home
  $ doas mkdir ports
  $ doas chown -R user:wsrc ports
  $ cd ports
  $ cd build
  $ mkdir -p wrkobjdir distdir plist bulk_cookies update_cookies pkgrepo
Make symlinks in /usr
  $ cd /usr
  $ doas ln -s /home/ports .
Edit /etc/mk.conf
  $ doas vi /etc/mk.conf
  SUDO=/usr/bin/doas
  WRKOBJDIR=/home/ports/build/wrkobjdir
  DISTDIR=/home/ports/build/distdir
  PLIST_DB=/home/ports/build/plist
  BULK_COOKIES_DIR=/home/ports/build/bulk_cookies
  UPDATE_COOKIES_DIR=/home/ports/build/update_cookies
  PACKAGE_REPOSITORY=/home/ports/build/pkgrepo
  FETCH_PACKAGES=Yes


26. Jun 2018 11:23 by [hidden email] <mailto:[hidden email]>:


>
> John Long writes:
>
>> Been a while and don't have my other OpenBSD boxes accessible.
>>
>> What are the recommended partitions and appropriate sizes for people
>> who want to track stable and possibly build the whole ports tree?
>>
>> Thanks,
>>
>> /jl
>
> hello
>
> If you want to do a bulk build (aka whole ports tree), read bulk(8) (in
> ports/infrastructure/man/), it says that you need at least 100 GB of
> disk.
>
>
>    1. Choose master machine setup and create partitions
>
>         Setup a master machine with enough room for a chroot, say
>         /build.  Assuming you are using a cluster of machines, this
>         chroot should contain NFS exportable partitions for distfiles,
>         plists, and packages (one single partition can be used for
>         simplicity).  A full setup requires on the order of 50GB for
>         distfiles and 50GB for packages.
Reply | Threaded
Open this post in threaded view
|

Re: Partitioning recommendations for 6.3?

Solene Rapenne
Le 26 juin 2018 16:49:57 GMT+02:00, [hidden email] a écrit :

>Personally, what i always do, (i dont know if its the best practice,
>but it fixes my storage space problems, as i always use -stable, and
>build the updated ports) is to make a symlink in /home
>
>Initial configuration
>  $ cd /home
>  $ doas mkdir ports
>  $ doas chown -R user:wsrc ports
>  $ cd ports
>  $ cd build
>  $ mkdir -p wrkobjdir distdir plist bulk_cookies update_cookies
>pkgrepo
>Make symlinks in /usr
>  $ cd /usr
>  $ doas ln -s /home/ports .
>Edit /etc/mk.conf
>  $ doas vi /etc/mk.conf
>  SUDO=/usr/bin/doas
>  WRKOBJDIR=/home/ports/build/wrkobjdir
>  DISTDIR=/home/ports/build/distdir
>  PLIST_DB=/home/ports/build/plist
>  BULK_COOKIES_DIR=/home/ports/build/bulk_cookies
>  UPDATE_COOKIES_DIR=/home/ports/build/update_cookies
>  PACKAGE_REPOSITORY=/home/ports/build/pkgrepo
>  FETCH_PACKAGES=Yes
>
>
>26. Jun 2018 11:23 by [hidden email] <mailto:[hidden email]>:
>
>
>>
>> John Long writes:
>>
>>> Been a while and don't have my other OpenBSD boxes accessible.
>>>
>>> What are the recommended partitions and appropriate sizes for people
>>> who want to track stable and possibly build the whole ports tree?
>>>
>>> Thanks,
>>>
>>> /jl
>>
>> hello
>>
>> If you want to do a bulk build (aka whole ports tree), read bulk(8)
>(in
>> ports/infrastructure/man/), it says that you need at least 100 GB of
>> disk.
>>
>>
>>    1. Choose master machine setup and create partitions
>>
>>         Setup a master machine with enough room for a chroot, say
>>         /build.  Assuming you are using a cluster of machines, this
>>         chroot should contain NFS exportable partitions for
>distfiles,
>>         plists, and packages (one single partition can be used for
>>         simplicity).  A full setup requires on the order of 50GB for
>>         distfiles and 50GB for packages.

It has the drawback that you have to
set wxallowed on /home

You could also avoid the symlink
by adding PORTSDIR=/home/ports

Reply | Threaded
Open this post in threaded view
|

Re: Partitioning recommendations for 6.3?

lists-2
In reply to this post by John Long-4
Mon, 25 Jun 2018 14:37:32 +0000 John Long <[hidden email]>

> On Mon, 2018-06-25 at 17:16 +0300, IL Ka wrote:
> > Do you want to really build all ports or just fetch skeletons and
> > build some of them?  
>
> Not sure, but I don't want to rule out building them all for a couple
> or reasons. I have a new box which is probably fast enough to make it
> worthwhile to build packages for some slower boxes I have. Second thing
> is rebuilding the system from source and then building all the ports is
> a good stability test. Bottom line is probably that I would rather plan
> to have the space available and not need it then to need it and not
> have it. Seems like in the past this was a problem for me.
>
> > For skeletons, automatic layout is good enough, but I recommend to
> > increase /usr/src a little and decrease /home.
> > Make sure you have ~ 5GB for /usr/src/ and /usr/obj.
> >  
>
> Thanks, this helps. The automatic layout didn't include /usr/xenocara
> There used to be a recommendation in the past to have that as a
> separate filesystem. How large should it be?
>
> Is there any reason to track -stable anymore or has syspatch done away
> with the need for that?
>
> Seems to me, after trying to install OpenBSD on a new box, a lot of the
> helpful in the FAQ is totally AWOL now and I find it hard to get all
> the info together.

Hi John,

Person came from somewhere and cut out a lot of the useful hardware info.
At least now it's maintainable and can be carefully rewritten again, duh.

The frequent questions will probably go away over time as things improve.
Sound advice should have stayed however if you ask an enthusiast opinion.

The cvsweb shows historic versions of the pages if you want to reread it.
Asking this question also helps solve cases for mailing list readers too.

I have the same intention as you and am glad you raised the points again.
Ideally, the auto partition could have templates, for the cases you have.

http://man.openbsd.org/disklabel#AUTOMATIC_DISK_ALLOCATION

Interesting what the people with bulk port build and snapshot builds say.

http://man.openbsd.org/mk.conf
http://man.openbsd.org/bulk

Kind regards,
Anton Lazarov

> /jl
>
> >
> >
> >
> >
> > On Mon, Jun 25, 2018 at 3:17 PM, John Long <[hidden email]> wrote:  
> > > Been a while and don't have my other OpenBSD boxes accessible.
> > >
> > > What are the recommended partitions and appropriate sizes for
> > > people
> > > who want to track stable and possibly build the whole ports tree?
> > >
> > > Thanks,
> > >
> > > /jl
> > >  
> >
> >  
>

Reply | Threaded
Open this post in threaded view
|

Re: Partitioning recommendations for 6.3?

John Long-4
> > Seems to me, after trying to install OpenBSD on a new box, a lot of
> > the helpful in the FAQ is totally AWOL now and I find it hard to
> > get all the info together.
>
> Hi John,
>
> Person came from somewhere and cut out a lot of the useful hardware
> info.
> At least now it's maintainable and can be carefully rewritten again,
> duh.
>
> The frequent questions will probably go away over time as things
> improve.
> Sound advice should have stayed however if you ask an enthusiast
> opinion.
>
> The cvsweb shows historic versions of the pages if you want to reread
> it.

Thanks, Anton. I understood from Stuart how to find old web versions.
It's good to know. I started with OpenBSD at 3.6 or 3.7 and installed
everything until around 5.2. I still have two boxes running 5.X, they
just work and they're not internet-facing so they'll run until they
die. But it seems like there was a lot more info in the FAQ in those
days. Now I find it more difficult to get info.

> Ideally, the auto partition could have templates, for the cases you
> have.

I think this is a good idea but I guess a lot of people will bang you
on the head for suggesting it ;)

I don't know that I have ever seen the one-size fits all approach work
in any installer I have used, so I think templates make sense. Let the
flames begin...

Thanks for the other links. I will read all the stuff you and the other
guys have pointed me at.

/jl

Reply | Threaded
Open this post in threaded view
|

Re: Partitioning recommendations for 6.3?

lists-2
Tue, 26 Jun 2018 20:27:53 +0000 John Long <[hidden email]>
> > Ideally, the auto partition could have templates, for the cases you
> > have.  
>
> I think this is a good idea but I guess a lot of people will bang you
> on the head for suggesting it ;)



> I don't know that I have ever seen the one-size fits all approach work
> in any installer I have used, so I think templates make sense. Let the
> flames begin...
>
> Thanks for the other links. I will read all the stuff you and the other
> guys have pointed me at.

Hi John,

Hard hat mode, continuous improvement, work in progress, exciting times..
I was hastily reading the thread - missed a couple of the relevant links:

http://man.openbsd.org/release
http://man.openbsd.org/ports
http://man.openbsd.org/bulk
http://man.openbsd.org/dpb
http://man.openbsd.org/mk.conf
http://man.openbsd.org/autoinstall
http://man.openbsd.org/disklabel#AUTOMATIC_DISK_ALLOCATION

Templates are important as auto installation and auto upgrade works well.
Would be glad if you follow up on the thread with your progress findings.

Kind regards,
Anton Lazarov