Re-organising partitions without re-installation

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Re-organising partitions without re-installation

Stuart Longland
Hi all,

So, a few years ago now, I deployed a router VM with OpenBSD 6.1 AMD64.
 Later that got updated to 6.2, then 6.3, 6.4…

Yesterday I updated it to 6.5, then 6.6… now I'm trying to run syspatch:

> sjl-router# syspatch                                                                                                                  
> Get/Verify syspatch66-010_libcaut... 100% |********************************************************************| 20185 KB    00:16    
> Installing patch 010_libcauth
> No space left on sd0a, aborting
> sjl-router# df -h  
> Filesystem     Size    Used   Avail Capacity  Mounted on
> /dev/sd0a      129M   98.0M   24.4M    80%    /
> /dev/sd0k      472M   28.0K    448M     0%    /home
> /dev/sd0d      198M   76.0K    188M     0%    /tmp
> /dev/sd0f      2.3G    1.3G    911M    60%    /usr
> /dev/sd0h      2.1G    551M    1.4G    27%    /usr/local
> /dev/sd0j      1.3G    2.0K    1.2G     0%    /usr/obj
> /dev/sd0i      1.0G    2.0K    974M     0%    /usr/src
> /dev/sd0e      209M    119M   79.1M    60%    /var
> sjl-router# uname -a
> OpenBSD sjl-router.redhatters.home 6.6 GENERIC#353 amd64

8GB seemed like a reasonable amount for something that would just be
routing.  And looking at that `df` output, it would appear that there's
about 2.5GB locked away, in partitions that the original automatic
layout dictated I should have, but then didn't utilise.

I'm thankful I had the foresight of overruling its decision to allocate
space to /usr/X11R6… as this machine does not have X installed. (Why
would a router need that anyway?)

> sjl-router# disklabel sd0
> # /dev/rsd0c:
> type: SCSI
> disk: SCSI disk
> label: Block Device    
> duid: d7b965d8cdeaeef2
> flags:
> bytes/sector: 512
> sectors/track: 63
> tracks/cylinder: 255
> sectors/cylinder: 16065
> cylinders: 1044
> total sectors: 16777216
> boundstart: 64
> boundend: 16771860
> drivedata: 0
>
> 16 partitions:
> #                size           offset  fstype [fsize bsize   cpg]
>   a:           268416               64  4.2BSD   2048 16384  2097 # /
>   b:           373010           268480    swap                    # none
>   c:         16777216                0  unused                    
>   d:           413056           641504  4.2BSD   2048 16384  3227 # /tmp
>   e:           435744          1054560  4.2BSD   2048 16384  3390 # /var
>   f:          5006848          1490304  4.2BSD   2048 16384 12958 # /usr
>   h:          4403456          6497152  4.2BSD   2048 16384 12958 # /usr/local
>   i:          2138976         10900608  4.2BSD   2048 16384 12958 # /usr/src
>   j:          2746048         13039584  4.2BSD   2048 16384 12958 # /usr/obj
>   k:           986208         15785632  4.2BSD   2048 16384  7674 # /home

Question is, how do I re-organise this space?  There is sufficient space
there.  /usr/obj and /usr/src are pretty much unused.  /usr/local could
be made smaller too as could /home.

OpenBSD has growfs(8).  I note it is called growfs and not resizefs nor
shrinkfs.  The steps I believe I'd need to perform are:

- shrink /home to 200MB
- re-locate /home to the end
- blow away /usr/src and /usr/obj
- shrink /usr/local to 1GB
- re-locate /usr/local to just before /home
- re-locate /usr to just before /usr/local
- re-locate /var to just before /usr
- re-locate /tmp to to just before /var
- now grow / to fill the available space

In days gone by, there was PartitionMagic for doing this.  Under Linux
today, there's gparted.

OpenBSD complicates things because it ignores the native disklabel
format of the host platform (i.e. MS-DOS disklabel / GUID partition
table) in favour of its own BSD slice system.  So such a tool has to not
only understand ffs, but it also must understand the BSD disklabel
embedded in the partition allocated to OpenBSD.

Re-installing is something I did under the following conditions:
- Before the existence of the aforementioned partition management tools
- When I *really* screwed up

I'm not after a GUI tool to do this (although some sort of visualisation
is helpful in my experience, I can also use a spreadsheet to work out
the numbers), but I really don't think "reinstall" should be the default
answer to all this as that is really a measure of last resort.

Is there such a tool for manipulating partitions in this manner?
--
Stuart Longland (aka Redhatter, VK4MSL)

I haven't lost my mind...
  ...it's backed up on a tape somewhere.

Reply | Threaded
Open this post in threaded view
|

Re: Re-organising partitions without re-installation

chohag
Stuart Longland writes:

> > 16 partitions:
> > #                size           offset  fstype [fsize bsize   cpg]
> >   a:           268416               64  4.2BSD   2048 16384  2097 # /
> >   b:           373010           268480    swap                    # none
> >   c:         16777216                0  unused                    
> >   d:           413056           641504  4.2BSD   2048 16384  3227 # /tmp
> >   e:           435744          1054560  4.2BSD   2048 16384  3390 # /var
> >   f:          5006848          1490304  4.2BSD   2048 16384 12958 # /usr
> >   h:          4403456          6497152  4.2BSD   2048 16384 12958 # /usr/local
> >   i:          2138976         10900608  4.2BSD   2048 16384 12958 # /usr/src
> >   j:          2746048         13039584  4.2BSD   2048 16384 12958 # /usr/obj
> >   k:           986208         15785632  4.2BSD   2048 16384  7674 # /home
>
> Question is, how do I re-organise this space?  There is sufficient space
> there.  /usr/obj and /usr/src are pretty much unused.  /usr/local could
> be made smaller too as could /home.

Copy contents of /home, if any, to /var; Remove partitions i, j and
k, replacing with a single large i to mount at /home; format new,
larger partition i and restore the contents of /home from the backup;
update /etc/fstab.

Alternatively back up the contents of /usr/local as well and replace
partition h with a larger one if you don't need a /home (except
that now sysupgrade does, so...).

Reboot not required although you will need to stop/start anything
holding files open.

Matthew

Reply | Threaded
Open this post in threaded view
|

Re: Re-organising partitions without re-installation

Dumitru Moldovan-2
In reply to this post by Stuart Longland
On Sun, Dec 22, 2019 at 10:56:20AM +1000, Stuart Longland wrote:
>So, a few years ago now, I deployed a router VM with OpenBSD 6.1 AMD64.
> Later that got updated to 6.2, then 6.3, 6.4…
>
>Yesterday I updated it to 6.5, then 6.6… now I'm trying to run syspatch:

I have a similar issue with my desktop.  I tried to outsmart the
automatic installer to squeeze as much space as possible for /home on a
desktop with an 80GB SSD.  Which worked out OK for a few upgrade cycles,
always from stable version to next stable version.

However, after a couple of years, I had to unbreak an update that didn't
fit any more in /usr.  To my surprise, I had lots of old libs from
previous releases left on disk.  Had to manually remove a few of the
older unused libs from /usr to be able to redo the update successfully.

My understanding is that this is by design.  In an update, some libs are
overwritten (if they keep the same file name), but others are left on
disk (theoretically unused) when lib versions are incremented.  I can
see a few ways in which this eases updates for people following
-current, such as the OpenBSD devs, so it's a small price to pay.

But yes, it would be awesome if installations that use -stable would not
have to pay this price.  After all, only libs from current version of
the base system should be needed.  If you built something linked to an
older lib from a previous OS version, it should be recompiled after an
updated.  This could also be a security issue if you're sloppy and use
binaries linked to old base libs that are no longer updated for
security issues.

If I missed anything, would love to be corrected!  And sorry, I don't
have a solution for this.  Other than the obvious one of packaging the
files in base.  Is this a heresy?

Reply | Threaded
Open this post in threaded view
|

Re: Re-organising partitions without re-installation

rgcinjp
December 24, 2019 4:42 AM, "Dumitru Moldovan" <[hidden email]> wrote:

> On Sun, Dec 22, 2019 at 10:56:20AM +1000, Stuart Longland wrote:
>
>> So, a few years ago now, I deployed a router VM with OpenBSD 6.1 AMD64.
>> Later that got updated to 6.2, then 6.3, 6.4…
>>
>> Yesterday I updated it to 6.5, then 6.6… now I'm trying to run syspatch:
>
> I have a similar issue with my desktop. I tried to outsmart the
> automatic installer to squeeze as much space as possible for /home on a
> desktop with an 80GB SSD. Which worked out OK for a few upgrade cycles,
> always from stable version to next stable version.
>
> However, after a couple of years, I had to unbreak an update that didn't
> fit any more in /usr. To my surprise, I had lots of old libs from
> previous releases left on disk. Had to manually remove a few of the
> older unused libs from /usr to be able to redo the update successfully.
>
> My understanding is that this is by design. In an update, some libs are
> overwritten (if they keep the same file name), but others are left on
> disk (theoretically unused) when lib versions are incremented. I can
> see a few ways in which this eases updates for people following
> -current, such as the OpenBSD devs, so it's a small price to pay.

one thing that is useful is sysclean(8)

my process now after a doas sysupgrade is
1) doas sysclean; and review the output
2) vise /etc/sysclean.ignore; so that sysclean ignores special files i created
3) doas sysclean | xargs doas rm -rf

yorosiku ~

Reply | Threaded
Open this post in threaded view
|

Re: Re-organising partitions without re-installation

Stuart Longland
On 24/12/19 8:42 am, [hidden email] wrote:

>> My understanding is that this is by design. In an update, some libs are
>> overwritten (if they keep the same file name), but others are left on
>> disk (theoretically unused) when lib versions are incremented. I can
>> see a few ways in which this eases updates for people following
>> -current, such as the OpenBSD devs, so it's a small price to pay.
> one thing that is useful is sysclean(8)
>
> my process now after a doas sysupgrade is
> 1) doas sysclean; and review the output
> 2) vise /etc/sysclean.ignore; so that sysclean ignores special files i created
> 3) doas sysclean | xargs doas rm -rf
>
> yorosiku ~

Where do you get `sysclean` from?  I don't seem to have it:
> sjl-router# man sysclean                                                                                                
> man: No entry for sysclean in the manual.
> sjl-router# which sysclean
> which: sysclean: Command not found.

Regards,
--
Stuart Longland (aka Redhatter, VK4MSL)

I haven't lost my mind...
  ...it's backed up on a tape somewhere.

Reply | Threaded
Open this post in threaded view
|

Re: Re-organising partitions without re-installation

Philip Guenther-2
On Mon, Dec 23, 2019 at 3:10 PM Stuart Longland <[hidden email]>
wrote:
...

> Where do you get `sysclean` from?  I don't seem to have it:
> > sjl-router# man sysclean
>
> > man: No entry for sysclean in the manual.
> > sjl-router# which sysclean
> > which: sysclean: Command not found.
>

$ pkg_info sysclean
Information for
http://mirrors.sonic.net/pub/OpenBSD/snapshots/packages/amd64/sysclean-2.8.tgz

Comment:
list obsolete files between OpenBSD upgrades

Description:
sysclean is a script designed to help remove obsolete files between OpenBSD
upgrades.

sysclean compares a reference root directory against the currently installed
files, taking files from both the base system and packages into account.

sysclean does not remove any files on the system. It only reports obsolete
filenames or packages using out-of-date libraries.

Maintainer: Sebastien Marie <[hidden email]>

WWW: https://github.com/semarie/sysclean/

$
Reply | Threaded
Open this post in threaded view
|

Re: Re-organising partitions without re-installation

Edgar Pettijohn III-2
In reply to this post by Stuart Longland

On Dec 23, 2019 4:42 PM, [hidden email] wrote:

>
> December 24, 2019 4:42 AM, "Dumitru Moldovan" <[hidden email]> wrote:
>
> > On Sun, Dec 22, 2019 at 10:56:20AM +1000, Stuart Longland wrote:
> >
> >> So, a few years ago now, I deployed a router VM with OpenBSD 6.1 AMD64.
> >> Later that got updated to 6.2, then 6.3, 6.4…
> >>
> >> Yesterday I updated it to 6.5, then 6.6… now I'm trying to run syspatch:
> >
> > I have a similar issue with my desktop. I tried to outsmart the
> > automatic installer to squeeze as much space as possible for /home on a
> > desktop with an 80GB SSD. Which worked out OK for a few upgrade cycles,
> > always from stable version to next stable version.
> >
> > However, after a couple of years, I had to unbreak an update that didn't
> > fit any more in /usr. To my surprise, I had lots of old libs from
> > previous releases left on disk. Had to manually remove a few of the
> > older unused libs from /usr to be able to redo the update successfully.
> >
> > My understanding is that this is by design. In an update, some libs are
> > overwritten (if they keep the same file name), but others are left on
> > disk (theoretically unused) when lib versions are incremented. I can
> > see a few ways in which this eases updates for people following
> > -current, such as the OpenBSD devs, so it's a small price to pay.
>
> one thing that is useful is sysclean(8)
>
> my process now after a doas sysupgrade is
> 1) doas sysclean; and review the output
> 2) vise /etc/sysclean.ignore; so that sysclean ignores special files i created

Just wanted to emphasize step 2 above or else you will delete stuff you shouldn't.


> 3) doas sysclean | xargs doas rm -rf
>
> yorosiku ~
>

+1 for sysclean

Reply | Threaded
Open this post in threaded view
|

Re: Re-organising partitions without re-installation

Stuart Longland
In reply to this post by Stuart Longland
On 24/12/19 12:51 pm, Edgar Pettijohn wrote:

>
> On Dec 23, 2019 4:42 PM, [hidden email] wrote:
>>
>> December 24, 2019 4:42 AM, "Dumitru Moldovan" <[hidden email]> wrote:
>> one thing that is useful is sysclean(8)
>>
>> my process now after a doas sysupgrade is
>> 1) doas sysclean; and review the output
>> 2) vise /etc/sysclean.ignore; so that sysclean ignores special files i created
>
> Just wanted to emphasize step 2 above or else you will delete stuff you shouldn't.

Yes, I just installed it, and ran it through `tee` so I could review the
delete list before I passed it to `rm` (and manually edit it).  There
were a few configuration directories in `/etc` for non-base stuff (e.g.
`collectd`'s password file, `vpnc`, etc) that I had to prune out and add
to /etc/sysclean.ignore.

That put a dint in the used space:

> sjl-router# df -h
> Filesystem     Size    Used   Avail Capacity  Mounted on
> /dev/sd0a      129M   77.8M   44.6M    64%    /
> /dev/sd0k      472M   28.0K    448M     0%    /home
> /dev/sd0d      198M   50.0K    188M     0%    /tmp
> /dev/sd0f      2.3G    1.2G   1022M    55%    /usr
> /dev/sd0h      2.1G    324M    1.6G    16%    /usr/local
> /dev/sd0j      1.3G    2.0K    1.2G     0%    /usr/obj
> /dev/sd0i      1.0G    2.0K    974M     0%    /usr/src
> /dev/sd0e      209M   73.2M    125M    37%    /var

I can understand the update tool being conservative about what it
deletes, who knows what is linked to those .so files without scanning
each and every ELF binary?  (hello Gentoo revdep-rebuild!)  Keeping them
there is definitely the KISS approach.

I'm just re-running `syspatch` to see if I can get the remainder of the
patches in.  If this fails, I might see if I can dig up some docs on how
this disklabel and ffs stuff works and see if I can teach `gparted`
about it.  Something tells me it's not the complicated mess that LVM2
is, and it handles that just fine.

Many thanks all.
--
Stuart Longland (aka Redhatter, VK4MSL)

I haven't lost my mind...
  ...it's backed up on a tape somewhere.

Reply | Threaded
Open this post in threaded view
|

Re: Re-organising partitions without re-installation

Dumitru Moldovan-2
In reply to this post by rgcinjp
On Mon, Dec 23, 2019 at 10:42:47PM +0000, [hidden email] wrote:

>December 24, 2019 4:42 AM, "Dumitru Moldovan" <[hidden email]> wrote:
>
>> On Sun, Dec 22, 2019 at 10:56:20AM +1000, Stuart Longland wrote:
>>
>>> So, a few years ago now, I deployed a router VM with OpenBSD 6.1 AMD64.
>>> Later that got updated to 6.2, then 6.3, 6.4…
>>>
>>> Yesterday I updated it to 6.5, then 6.6… now I'm trying to run syspatch:
>>
>> I have a similar issue with my desktop. I tried to outsmart the
>> automatic installer to squeeze as much space as possible for /home on a
>> desktop with an 80GB SSD. Which worked out OK for a few upgrade cycles,
>> always from stable version to next stable version.
>>
>> However, after a couple of years, I had to unbreak an update that didn't
>> fit any more in /usr. To my surprise, I had lots of old libs from
>> previous releases left on disk. Had to manually remove a few of the
>> older unused libs from /usr to be able to redo the update successfully.
>>
>> My understanding is that this is by design. In an update, some libs are
>> overwritten (if they keep the same file name), but others are left on
>> disk (theoretically unused) when lib versions are incremented. I can
>> see a few ways in which this eases updates for people following
>> -current, such as the OpenBSD devs, so it's a small price to pay.
>
>one thing that is useful is sysclean(8)
>
>my process now after a doas sysupgrade is
>1) doas sysclean; and review the output
>2) vise /etc/sysclean.ignore; so that sysclean ignores special files i created
>3) doas sysclean | xargs doas rm -rf

Thanks for pointing out I missed sysclean.  I used it myself, at least
after the last upgrade, as I see I have it installed and `sysclean -a`
only finds my custom x.org config in /etc.

Maybe it would be worth mentioning in the FAQ?  I could only find it
here: https://www.openbsd.org/faq/upgrade63.html, but then it was not
mentioned for newer releases.

Another remedy is to follow the `Files to remove` section in the FAQ,
e.g. for 6.6: https://www.openbsd.org/faq/upgrade66.html#RmFiles.  The
FAQ article for the 6.3 upgrade suggests sysclean does that too.  This
seems to be a byproduct of the design, meaning it doesn't specifically
remove those files, but it should remove them, as long as all installed
packages are updated and no longer need them.  But this is just my
reading of the sysclean man page.

Reply | Threaded
Open this post in threaded view
|

Re: Re-organising partitions without re-installation

Stuart Longland
On 24/12/19 9:16 pm, Dumitru Moldovan wrote:

> Maybe it would be worth mentioning in the FAQ?  I could only find it
> here: https://www.openbsd.org/faq/upgrade63.html, but then it was not
> mentioned for newer releases.
>
> Another remedy is to follow the `Files to remove` section in the FAQ,
> e.g. for 6.6: https://www.openbsd.org/faq/upgrade66.html#RmFiles.  The
> FAQ article for the 6.3 upgrade suggests sysclean does that too.  This
> seems to be a byproduct of the design, meaning it doesn't specifically
> remove those files, but it should remove them, as long as all installed
> packages are updated and no longer need them.  But this is just my
> reading of the sysclean man page.

Yeah I had done that… actually for another router VM I had to do a very
brutal equivalent of it when it ran out of disk space mid-update in the
installer… I basically blew away /usr/* (minus directories that are on
different partitions like 'local') figuring it'd re-instate the files
when it unpacked the newer file sets.

This lead to some missing files in /usr/share/relink but I was able to
re-instate those from another 6.6 VM that did update cleanly
(ironically, the very one that prompted this discussion).

So far, both have now run `syspatch`, and I've got kernel re-linking
working on both now.  We shall see.

Both VMs should probably be re-built from scratch as a matter of sanity,
but I can do that at leisure now, what I have, works.
--
Stuart Longland (aka Redhatter, VK4MSL)

I haven't lost my mind...
  ...it's backed up on a tape somewhere.

Reply | Threaded
Open this post in threaded view
|

Re: Re-organising partitions without re-installation

Sriram Narayanan
On Wed, 25 Dec 2019 at 7:41 PM, Stuart Longland <[hidden email]>
wrote:

> On 24/12/19 9:16 pm, Dumitru Moldovan wrote:
> > Maybe it would be worth mentioning in the FAQ?  I could only find it
> > here: https://www.openbsd.org/faq/upgrade63.html, but then it was not
> > mentioned for newer releases.
> >
> > Another remedy is to follow the `Files to remove` section in the FAQ,
> > e.g. for 6.6: https://www.openbsd.org/faq/upgrade66.html#RmFiles.  The
> > FAQ article for the 6.3 upgrade suggests sysclean does that too.  This
> > seems to be a byproduct of the design, meaning it doesn't specifically
> > remove those files, but it should remove them, as long as all installed
> > packages are updated and no longer need them.  But this is just my
> > reading of the sysclean man page.
>
> Yeah I had done that… actually for another router VM I had to do a very
> brutal equivalent of it when it ran out of disk space mid-update in the
> installer… I basically blew away /usr/* (minus directories that are on
> different partitions like 'local') figuring it'd re-instate the files
> when it unpacked the newer file sets.
>
> This lead to some missing files in /usr/share/relink but I was able to
> re-instate those from another 6.6 VM that did update cleanly
> (ironically, the very one that prompted this discussion).
>
> So far, both have now run `syspatch`, and I've got kernel re-linking
> working on both now.  We shall see.
>
> Both VMs should probably be re-built from scratch as a matter of sanity,
> but I can do that at leisure now, what I have, works.


What hypervisor are you using? Does it support an API to create VM from ISO
images and to launch VMs from templates?


Do you have an inventory of the configuration files and the settings in
these files?

You may want to set up automation to produce a new openbsd vm and deliver
configurations into it via configuration management and then switch to this
vm after testing.

This would help you to cut over when needed, cut back in case of issues,
and have the ability to recover thanks to your automation.



> --
> Stuart Longland (aka Redhatter, VK4MSL)
>
> I haven't lost my mind...
>   ...it's backed up on a tape somewhere.
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Re-organising partitions without re-installation

Stuart Longland
On 25/12/19 10:30 pm, Sriram Narayanan wrote:
> On Wed, 25 Dec 2019 at 7:41 PM, Stuart Longland <[hidden email]>
> wrote:
>
>> Both VMs should probably be re-built from scratch as a matter of sanity,
>> but I can do that at leisure now, what I have, works.
>
>
> What hypervisor are you using? Does it support an API to create VM from ISO
> images and to launch VMs from templates?

I'm using OpenNebula atop Linux KVM with a Ceph storage back-end.
https://hackaday.io/project/10529-solar-powered-cloud-computing

It does have templates, but only one of the VMs concerned are actually
being managed by OpenNebula (my own; sjl-router).

The other (corerouter) is actually a bare KVM virtual machine managed
outside OpenNebula as the plan was to set up corosync to auto-migrate it
and the VM that runs the OpenNebula front-end between the two compute
nodes.  Since it is the route by which the OpenNebula VM reaches the
host nodes, it can't be managed by OpenNebula.

So templates are not a solution.


> This would help you to cut over when needed, cut back in case of issues,
> and have the ability to recover thanks to your automation.

It would, and if I were managing dozens of them all with a largely
identical pattern, I'd definitely look into it.

My VMs tend to be pets, not cattle¹.  They're all highly specialised to
the task they're performing and none of them are really alike, so
templating really doesn't work in that context.

Regards,
--
Stuart Longland (aka Redhatter, VK4MSL)

I haven't lost my mind...
  ...it's backed up on a tape somewhere.

1.
http://cloudscaling.com/blog/cloud-computing/the-history-of-pets-vs-cattle/
 (In truth, my hosts are also pet-like, because I only have a small
number of them.)