Upgrading issues (i386 on PPro class) 5.4->5.5 leaving system horked

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Upgrading issues (i386 on PPro class) 5.4->5.5 leaving system horked

Damon Getsman
Well, I've never cared much for the holidays...  So I figured, while
everybody else was busy with them, what a perfect time for me to take down
my BBS and other services and upgrade the system at least from 5.4-5.5.
I've done multiple upgrades in a day before when I've gotten behind like
this, and they've never been much of a problem, though I've run into an
issue here and there before.

That was before I ran into the removal of packages and reinstallation
process that is necessary for this upgrade.

Right off the bat I got issues about 'Can't locate object method
"filter_obsolete" via package "OpenBSD::Quirks1" at
/usr/libdata/perl5/OpenBSD/AddDelete.pm line 351"'.  This was at the 'final
step', and there had been no hiccups during the sysmerging process that I
wasn't used to.

After this issue, I started having packages that the system was trying to
reinstall but totally horked on.  I'm not totally sure if this is
comprehensive at this point, but here are the packages and what I had to do
with them:

* apcupsd - had to manually remove from pkg_list_manual
* cups - ditto
* fedora-base - ditto
* ntop - ditto
* hylafax & smsmail or whatever - ditto

I then ran into a bunch that I had to install unsigned, finally just
hitting the 'a'll so as to stop having to manually intervene and say 'y'es
on each one.  These started with metaauto*, autoconf-2.69p0*, bash-4.2.45,
beave, and then I lost track of the rest with the 'a'll install specified.

My system won't run like 90% of the binaries on it.

I have upgrades of my home directory, BBS directory, and SQL dumps of most
everything except my most recent changes, but I'm really hoping to avoid
that...

Does anybody have any tips?  Would an install over the top straight to 5.6
maybe work?  I've never had the install process leave me with such a
lobotomized system before.  Usually it's just something like mediawiki not
working with latex math markup any more or wordpress breaking.  :(

This is my primary server, and it pains me significantly to have it down,
and I really love OpenBSD, despite the hiccups at times.  Does anybody have
any suggestions?  Any help or pointers in the right direction would be
greatly appreciated.

Thanks in advance.

Reply | Threaded
Open this post in threaded view
|

Re: Upgrading issues (i386 on PPro class) 5.4->5.5 leaving system horked

Nick Holland
On 12/25/14 22:04, Damon Getsman wrote:
> Well, I've never cared much for the holidays...  So I figured, while
> everybody else was busy with them, what a perfect time for me to take down
> my BBS and other services and upgrade the system at least from 5.4-5.5.
> I've done multiple upgrades in a day before when I've gotten behind like
> this, and they've never been much of a problem, though I've run into an
> issue here and there before.
>
> That was before I ran into the removal of packages and reinstallation
> process that is necessary for this upgrade.

you don't mention WHEN you did this, but I can guess..

> Right off the bat I got issues about 'Can't locate object method
> "filter_obsolete" via package "OpenBSD::Quirks1" at
> /usr/libdata/perl5/OpenBSD/AddDelete.pm line 351"'.  This was at the 'final
> step', and there had been no hiccups during the sysmerging process that I
> wasn't used to.
>
> After this issue, I started having packages that the system was trying to
> reinstall but totally horked on.
...
> My system won't run like 90% of the binaries on it.

such as...?
I'm assuming you mean "add-on packages", not base utilities like "tar"
or "more".  If "more" is screwed up, you didn't unpack in the right
directory.

> I have upgrades of my home directory, BBS directory, and SQL dumps of most
> everything except my most recent changes, but I'm really hoping to avoid
> that...

should be fixable.

> Does anybody have any tips?  Would an install over the top straight to 5.6
> maybe work?

No, you have a fundamental problem that needs to be fixed, and another
upgrade won't help (probably won't really hurt, either)

> I've never had the install process leave me with such a
> lobotomized system before.  Usually it's just something like mediawiki not
> working with latex math markup any more or wordpress breaking.  :(

This is user error.  This upgrade *IS* trickier than most, but that's
why we carefully document things -- if you follow the process, it works
fine.

> This is my primary server, and it pains me significantly to have it down,
> and I really love OpenBSD, despite the hiccups at times.  Does anybody have
> any suggestions?  Any help or pointers in the right direction would be
> greatly appreciated.
>
> Thanks in advance.

Well, you didn't provide many details of what you actually did, but it
is pretty obvious you did not uninstall all packages BEFORE you did the
upgrade.  Sucks to be you, but I bet you read the instructions more
carefully next time. :)  Good news, should be recoverable.

The fact that you have binaries that don't run proves you didn't remove
the packages before upgrade.  The other options are your kernel and
userland are out of sync (i.e., forgot to upgrade kernel or userland) or
you unpacked your userland to a different location, but either of those
would probably be far worse situations.

At this point, packages will have to be forcibly removed.  See the
options of pkg_delete -- the existing removal scripts may well fail, and
you may well have your dependencies screwed up, and a -q to skip over
verifying signatures would probably be good, too, set options accordingly.

Once you do that, you should have no old binaries left to run on your
system, and you should be back to 100% functional base OpenBSD install.
 You can now reinstall your packages, or since you have another upgrade
to go, upgrade the base system to 5.6 first.

Nick.

Raf
Reply | Threaded
Open this post in threaded view
|

Re: Upgrading issues (i386 on PPro class) 5.4->5.5 leaving system horked

Raf
In reply to this post by Damon Getsman
On Thu, Dec 25, 2014 at 10:04:48PM EST, Damon Getsman wrote:

> Well, I've never cared much for the holidays...  So I figured, while
> everybody else was busy with them, what a perfect time for me to take down
> my BBS and other services and upgrade the system at least from 5.4-5.5.
> I've done multiple upgrades in a day before when I've gotten behind like
> this, and they've never been much of a problem, though I've run into an
> issue here and there before.
>
> That was before I ran into the removal of packages and reinstallation
> process that is necessary for this upgrade.
>
> [...]
>
> Does anybody have any tips?  Would an install over the top straight to 5.6
> maybe work?  I've never had the install process leave me with such a
> lobotomized system before.  Usually it's just something like mediawiki not
> working with latex math markup any more or wordpress breaking.  :(
>
> This is my primary server, and it pains me significantly to have it down,
> and I really love OpenBSD, despite the hiccups at times.  Does anybody have
> any suggestions?  Any help or pointers in the right direction would be
> greatly appreciated.

Just checking the obvious, but you have obviously followed the 5.4 ->
5.5 upgrade guide[0] and removed all packages BEFORE upgrading and
installed them again AFTER you have done everything else?

[0] http://www.openbsd.org/faq/upgrade55.html

Regards,

Raf

Reply | Threaded
Open this post in threaded view
|

Re: Upgrading issues (i386 on PPro class) 5.4->5.5 leaving system horked

Stuart Henderson
In reply to this post by Damon Getsman
On 2014-12-26, Damon Getsman <[hidden email]> wrote:

> Right off the bat I got issues about 'Can't locate object method
> "filter_obsolete" via package "OpenBSD::Quirks1" at
> /usr/libdata/perl5/OpenBSD/AddDelete.pm line 351"'.  This was at the 'final
> step', and there had been no hiccups during the sysmerging process that I
> wasn't used to.
>
> After this issue, I started having packages that the system was trying to
> reinstall but totally horked on.  I'm not totally sure if this is
> comprehensive at this point, but here are the packages and what I had to do
> with them:
>
> * apcupsd - had to manually remove from pkg_list_manual
> * cups - ditto
> * fedora-base - ditto
> * ntop - ditto
> * hylafax & smsmail or whatever - ditto
>
> I then ran into a bunch that I had to install unsigned, finally just
> hitting the 'a'll so as to stop having to manually intervene and say 'y'es
> on each one.  These started with metaauto*, autoconf-2.69p0*, bash-4.2.45,
> beave, and then I lost track of the rest with the 'a'll install specified.

The 5.5 packages *are* signed.

I think you are trying to update packages from an incorrect source.
Check that PKG_PATH in the environment and/or /etc/pkg.conf are valid for 5.5
and do "pkg_add -D installed -u" to update everything (this will reinstall all
packages). Capture the output (e.g. with script) so if it fails you can show
the exact output.

Reply | Threaded
Open this post in threaded view
|

Re: Upgrading issues (i386 on PPro class) 5.4->5.5 leaving system horked

Damon Getsman
In reply to this post by Damon Getsman
> From Nick Holland <[hidden email]>
> Date Thu, 25 Dec 2014 22:52:15 -0500

>> My system won't run like 90% of the binaries on it.
 
> such as...?
> I'm assuming you mean "add-on packages", not base utilities like
> "tar"
> or "more".  If "more" is screwed up, you didn't unpack in the right
> directory.

  My apologies for my lack of specific information.  I was a wee bit
flustered when this first happened.  :)
  Your assumption is correct; base utilities were working correctly;
I didn't notice any problems with anything in /bin or /sbin, but when I
ventured outside of there I started finding more problems.

> This is user error.  This upgrade *IS* trickier than most, but that's
> why we carefully document things -- if you follow the process, it
> works fine.

  I'm well aware of that.  I've been using OpenBSD since 4.x, and
a few times before that for brief stretches.  After another response to
my initial post I ended up tracing the problem down to /etc/pkg.conf
and $PKG_PATH, both set for the entire system, pointing at irrelevant
versions.  One was pointing at a 5.4 repository and another was
pointing at a 5.3 repository.  That seems like a rather strange portion
not to automate in the upgrade process, but I've hardly ever rolled a
distribution of an OS before.

> Well, you didn't provide many details of what you actually did, but
> it is pretty obvious you did not uninstall all packages BEFORE you did the
> upgrade.  Sucks to be you, but I bet you read the instructions more
> carefully next time. :)  Good news, should be recoverable.
 
> The fact that you have binaries that don't run proves you didn't
> remove the packages before upgrade.  The other options are your kernel and
> userland are out of sync (i.e., forgot to upgrade kernel or userland)
> or you unpacked your userland to a different location, but either of
> those would probably be far worse situations.

  Well I got a little slightly more detailed information on recovering
from foolishness that I caused like this, and have implemented a
recovery strategy to the best of my limited mindset.  ;)
Unfortunately, things are still out of sync a bit; everything, again,
out of the primary /bin and /sbin directories, seems to pretty much be
crashing on attempts to load invalid libraries.
  I'll consolidate the actual debugging output and facts of the matter
into my reply to that message; if you're still willing to help, I
certainly do appreciate your input on the problem.  Should be posted
within a relatively short amount of time here.
  Thank you for your time & help on this matter.

  -Damon

Reply | Threaded
Open this post in threaded view
|

Re: Upgrading issues (i386 on PPro class) 5.4->5.5 leaving system horked

Damon Getsman
In reply to this post by Damon Getsman
Hello everyone.

I posted that information regarding the 5.4->5.5 upgrade breaking (and
subsequent break when following someone's advice just to take it up to 5.6
since it was already horked like that) a few days back here.  I was going
to post some google groups and/or openbsd.org links to the previous
threaded posts, but I'm not having luck digging them up on google groups,
and it appears that majordomo on the openbsd.org site won't give summaries
from previous months.

Regardless, I just wanted to find out...  I usually get people willing to
give some advice, or at least willing to laugh and tell me the lesson that
I needed to know on here.  I was really kind of surprised that I haven't
heard anything back on this for so long...

So, can anybody tell me, is my situation just so hosed that it's helpless?
I mean, should I stop waiting for potential ways to fix this dependency
hosed box and reinstall and try to find a way to re-inject all of my data
into it, or are the gurus just swamped with new years tasks?  :)  If any of
you could give me some feedback I'd really appreciate it.  Like I said with
the issue when I was first mentioning it, this system is really integral to
a lot of the work that I do, and it's my sole external facing server...
It's like a knife in my gut not having it working.

Anything is very much appreciated.  Feel free to hit me up via personal
email, too, if you'd rather avoid public posting for some reason.

I hope you all have a better year than last.  :)

-Damon

Raf
Reply | Threaded
Open this post in threaded view
|

Re: Upgrading issues (i386 on PPro class) 5.4->5.5 leaving system horked

Raf
On Fri, Jan 02, 2015 at 04:10:58PM EST, Damon Getsman wrote:

> Hello everyone.

Hi Damon,

> I posted that information regarding the 5.4->5.5 upgrade breaking (and
> subsequent break when following someone's advice just to take it up to 5.6
> since it was already horked like that) a few days back here.  I was going
> to post some google groups and/or openbsd.org links to the previous
> threaded posts, but I'm not having luck digging them up on google groups,
> and it appears that majordomo on the openbsd.org site won't give summaries
> from previous months.

marc.info?

> Regardless, I just wanted to find out...  I usually get people willing to
> give some advice, or at least willing to laugh and tell me the lesson that
> I needed to know on here.  I was really kind of surprised that I haven't
> heard anything back on this for so long...

You haven't provided answers to some of the questions either.

http://marc.info/?l=openbsd-misc&m=141958400922121&w=2

> So, can anybody tell me, is my situation just so hosed that it's helpless?
> I mean, should I stop waiting for potential ways to fix this dependency
> hosed box and reinstall and try to find a way to re-inject all of my data
> into it, or are the gurus just swamped with new years tasks?  :)  If any of
> you could give me some feedback I'd really appreciate it.  Like I said with
> the issue when I was first mentioning it, this system is really integral to
> a lot of the work that I do, and it's my sole external facing server...
> It's like a knife in my gut not having it working.
>
> Anything is very much appreciated.  Feel free to hit me up via personal
> email, too, if you'd rather avoid public posting for some reason.
>
> I hope you all have a better year than last.  :)

Regards,

Raf

Reply | Threaded
Open this post in threaded view
|

Re: Upgrading issues (i386 on PPro class) 5.4->5.5 leaving system horked

Jorge Gabriel Lopez Paramount
In reply to this post by Damon Getsman
Quoting Damon Getsman <[hidden email]>:

> Hello everyone.
>
> Regardless, I just wanted to find out...  I usually get people willing to
> give some advice, or at least willing to laugh and tell me the lesson that
> I needed to know on here.  I was really kind of surprised that I haven't
> heard anything back on this for so long...

As I see it would be very difficult to diagnose your problem remotely,  
but I'm new here. Yesterday I did my first (test) upgrade of 5.5 to  
5.6 and was fine, the system was smart enough to not screw up things  
even when I set the wrong architecture and version on pkg.conf. I did  
four updates in total and the last one was very straightforward and  
quick.

The process was simple: booted systems with the installation CD,  
selected upgrade and almost everything was the default selection,  
after that booted into the upgraded server, ran the sysmerge command,  
deleted the old files, rebooted, updated packages with pkg_add -u,  
rebooted, ran again pkg_add -u just in case and everything went fine.  
I even applied patches since I'm using stable.

> So, can anybody tell me, is my situation just so hosed that it's helpless?
> I mean, should I stop waiting for potential ways to fix this dependency
> hosed box and reinstall and try to find a way to re-inject all of my data
> into it, or are the gurus just swamped with new years tasks?  :)  If any of
> you could give me some feedback I'd really appreciate it.  Like I said with
> the issue when I was first mentioning it, this system is really integral to
> a lot of the work that I do, and it's my sole external facing server...
> It's like a knife in my gut not having it working.

If I was you I would install a clean system, check differences between  
the stock and your configuration files, restore your data and check if  
everything is working again. Anyway, if you have to do more than one  
upgrade operation on the same system it might take less effort just  
reinstalling than going through all the upgrades.

Upgrading critical systems is an excellent case for virtual machines,  
if something goes wrong you just have to restore the backed up image  
that can be as easy as copying a file. I know that virtual machines  
are heresy here and viewed as a waste of resources, but in situations  
like this are priceless.

--
Best regards,
Jorge Lopez.



----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.

Reply | Threaded
Open this post in threaded view
|

Re: Upgrading issues (i386 on PPro class) 5.4->5.5 leaving system horked

trondd
In reply to this post by Damon Getsman
Damon Getsman <[hidden email]> wrote:

> So, can anybody tell me, is my situation just so hosed that it's helpless?
> I mean, should I stop waiting for potential ways to fix this dependency
> hosed box and reinstall and try to find a way to re-inject all of my data
> into it, or are the gurus just swamped with new years tasks?  :)  If any of
> you could give me some feedback I'd really appreciate it.  Like I said with
> the issue when I was first mentioning it, this system is really integral to
> a lot of the work that I do, and it's my sole external facing server...
> It's like a knife in my gut not having it working.
>
> -Damon

At this point, I don't know if anyone can tell what's wrong or how to fix it.
You might as well reinstall.  How spread out is your data?  Save /home, /etc,
and pkg_info -qm and off you go.

I usually make /etc a git repo so I can diff and roll back changes, etc. And
keep a seperate partition for backups so I can wipe the system drive and
reinstall if needed (haven't needed it yet).

You could also try to make a site.tgz of your data to make reinstallation
easier if it's that complex.

Tim.