memory limits for OpenBSD on TalosII ?

[Eric Grosse]

I have a TalosII with dual four-core Power9 and dual 16GB DDR4 ECC RAM.
Ubuntu installs ok but OpenBSD install is power cycling shortly after OPAL
says
"switching to big-endian OS."

A thread from September mentioned a compiled limit of 6GB of RAM. Is that
still the limit? If exceeded, what console output should I see?

[Theo de Raadt-2]

please try a snapshot.

we are running on much larger machines now.

Eric Grosse <[hidden email]> wrote:

> I have a TalosII with dual four-core Power9 and dual 16GB DDR4 ECC RAM.
> Ubuntu installs ok but OpenBSD install is power cycling shortly after OPAL
> says
> "switching to big-endian OS."
>
> A thread from September mentioned a compiled limit of 6GB of RAM. Is that
> still the limit? If exceeded, what console output should I see?

[Eric Grosse]

I should have said this was a snapshot miniroot downloaded yesterday
evening.

Likely my newbie mistake somehow.  I have installed OpenBSD on a number of
x86 machines over the years, but have little experience with Petitboot. I'm
not seeing any other TalosII messages on either the serial or VGA ports.
Anything useful I can capture by exiting Petitboot to the shell first, to
spot my mistake?

On Fri, Mar 5, 2021 at 4:15 PM Theo de Raadt <[hidden email]> wrote:

> please try a snapshot.
>
> we are running on much larger machines now.
>
> Eric Grosse <[hidden email]> wrote:
>
> > I have a TalosII with dual four-core Power9 and dual 16GB DDR4 ECC RAM.
> > Ubuntu installs ok but OpenBSD install is power cycling shortly after
> OPAL
> > says
> > "switching to big-endian OS."
> >
> > A thread from September mentioned a compiled limit of 6GB of RAM. Is that
> > still the limit? If exceeded, what console output should I see?
>

[George Koehler-2]

On Fri, 5 Mar 2021 15:49:30 -0800
Eric Grosse <[hidden email]> wrote:

> I have a TalosII with dual four-core Power9 and dual 16GB DDR4 ECC RAM.
> Ubuntu installs ok but OpenBSD install is power cycling shortly after OPAL
> says
> "switching to big-endian OS."

Which OpenBSD install image are you using?  Are you booting from USB
or some other way?

Just now, I dd'ed the snapshots/powerpc64/install69.img onto a USB stick
and rebooted my TalosII.  I see the serial console by ssh'ing to the bmc
and running obmc-console-client.  The USB stick appeared as "OpenBSD
install" in petitboot.  I didn't find a message saying, "switching to
big-endian OS."  The console output, after I selected "OpenBSD install"
in petitboot, was

 Performing kexec reboot
SIGTERM received, booting...
[   22.888740] kexec_core: Starting new kernel
Copyright (c) 1982, 1986, 1989, 1991, 1993
        The Regents of the University of California.  All rights reserved.
Copyright (c) 1995-2021 OpenBSD. All rights reserved.  https://www.OpenBSD.org

OpenBSD 6.9-beta (RAMDISK) #79: Sat Mar  6 12:43:46 MST 2021
    [hidden email]:/usr/src/sys/arch/powerpc64/compile/RAMDISK
real mem  = 8589934592 (8192MB)
avail mem = 7834324992 (7471MB)
random: boothowto does not indicate good seed
mainbus0 at root: T2P9S01 REV 1.01
cpu0 at mainbus0 pir c: IBM POWER9 2.2, 2700 MHz
...

Here, the copyright is the first message from the OpenBSD kernel.
Are you not seeing the copyright?

I wonder if your USB stick is bad.  If you mount its DOS partition,
you should see 2 files (boot, grub.cfg), and "boot" should be an exact
copy of "bsd.rd" from the snapshot.

A regular DOS-format USB stick would become bootable by adding just 2
files: "bsd.rd" copied from the snapshot, and a small "grub.cfg" like,

menuentry "bsd.rd" {
        linux /bsd.rd
        initrd /bsd.rd
}

I wonder if your Talos has old firmware, and if old firmware can
confuse OpenBSD.  If you pick "System information" in petitboot, it
should show several version numbers.    --George

[Eric Grosse]

George Koehler <[hidden email]> wrote:
> Which OpenBSD install image are you using? ...

I copied miniroot69.img from
cdn.openbsd.org/pub/OpenBSD/snapshots/powerpc64/
a few days ago onto a flash usb stick. Fdisk reports
  *0 id=0c start=8192 size=32668 FAT32L
   3 id=a6 start 40960 size=26624 OpenBSD
and mount /dev/sd1i as msdos shows the two files you predicted:
    boot         size 11554783, which seems right for bsd.rd
    grub.cfg     with contents
        menuentry "OpenBSD install" {
             linux /boot
             initrd /boot
         }
so the flash stick seems ok.

On a console directly connected to the motherboard serial port, Petitboot
says:
System type: T2P9D01 REV 1.01n
open-power-talos-v1.20-161-g76f78f4
buildroot-2017.11.2-8-g4b6188e0f2
skiboot-bc106a0
hostboot-884b60b
linux-v4.15.9-openpower1-p9e03417
petitboot-v1.7.1-p836d356
BMC Firmware version: 2.00.030070000
The firmware is certainly not current; Raptor issued a 2.0 update last month
for example. I had thought maybe better to use flash parts that had been
working reliably for a couple years. Happy to update if y'all expect this
is my problem.

[Theo de Raadt-2]

Eric Grosse <[hidden email]> wrote:

> System type: T2P9D01 REV 1.01n
> open-power-talos-v1.20-161-g76f78f4
> buildroot-2017.11.2-8-g4b6188e0f2
> skiboot-bc106a0
> hostboot-884b60b
> linux-v4.15.9-openpower1-p9e03417
> petitboot-v1.7.1-p836d356
> BMC Firmware version: 2.00.030070000
> The firmware is certainly not current; Raptor issued a 2.0 update last month
> for example. I had thought maybe better to use flash parts that had been
> working reliably for a couple years. Happy to update if y'all expect this
> is my problem.

Pretty sure that is too old.

If I recall correctly, there was an ELF loader bug in petitboot (or some
other piece of the boot puzzle) which was fixed in a later rev, over a
year ago now.  One of my initial development machines had the same
problem.  In that case raptor had to send me new eeproms, that machine
was cranky and would not update.

[Eric Grosse]

Eric wrote
> Raptor issued a 2.0 update last month
I misread at first; the v2.00 firmware is from February 2020.

Commenting on my fall 2018 system's firmware,
Theo wrote
> Pretty sure that is too old. ...
> raptor had to send me new eeproms...

I bought some blank flash parts from Digikey, got a BusPirate, installed
the v2.00 firmware, and OpenBSD now works splendidly. Sometime soon
I'll post more detailed notes at my web site, n2vi.com, in case a fellow
traveller someday needs to hike this path.

Thank you all for the quick advice and the underlying great work.