bsd.mp not installed on EdgeRouter Lite

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

bsd.mp not installed on EdgeRouter Lite

Scott Bennett
After reading INSTALL.octeon, I was able to write miniroot62.fs to a usb,
plug that into the ERL, and perform a normal installation. The problem is
that the installer was not able to detect both cores, so it only installed
bsd.sp (bsd.mp was not an option in the set selection).

Running 6.2-release.

I did follow the instructions for setting the coremask=0x3 when booting the
installer, and setting the coremask=0x3 in the bootcmd. It seems that the
installer just wasn't able to recognize that it's a dual core system.

To workaround this problem, I downloaded bsd.mp after installation and copied
that to the FAT partition. My ERL can now run SMP, but as you probably
guessed this does break KARL.

Has anyone been able to install bsd.mp on the ERL and not break KARL?

Selected snippets from the install process below.

Cheers,
Scott

[....................snip....................]

Octeon ubnt_e100# fatload usb 0 $loadaddr bsd.rd
reading bsd.rd
..........................
.................

8750939 bytes read
Octeon ubnt_e100# bootoctlinux rootdev=rd0 coremask=0x3
argv[2]: coremask=0x3
ELF file is 64 bit
Allocating memory for ELF segment: addr: 0xffffffff81000000 (adjusted to: 0x1000000), size 0x86f890
Allocated memory for ELF segment: addr: 0xffffffff81000000, size 0x86f890
Processing PHDR 0
  Loading 7ef710 bytes at ffffffff81000000
  Clearing 80180 bytes at ffffffff817ef710
## Loading Linux kernel with entry point: 0xffffffff81000000 ...
Bootloader: Done loading app on coremask: 0x3

[....................snip....................]

boot_desc->argc:3
boot_desc->flags:0x5
boot_desc->core_mask:0x3
boot_desc->dram_size:512
boot_desc->phy_mem_desc_addr:0

[....................snip....................]

OpenBSD 6.2 (RAMDISK) #0: Wed Oct  4 05:40:31 UTC 2017
    visa@octeon:/usr/src/sys/arch/octeon/compile/RAMDISK
real mem = 536870912 (512MB)
avail mem = 520896512 (496MB)
mainbus0 at root
cpu0 at mainbus0: CN50xx CPU rev 0.1 500 MHz, Software FP emulation
cpu0: cache L1-I 32KB 4 way D 8KB 64 way, L2 128KB 8 way
clock0 at mainbus0: int 5
iobus0 at mainbus0

[....................snip....................]

Select sets by entering a set name, a file name pattern or 'all'. De-select
sets by prepending a '-', e.g.: '-game*'. Selected sets are labelled '[X]'.
    [X] bsd           [X] comp62.tgz    [X] xbase62.tgz   [X] xserv62.tgz
    [X] bsd.rd        [X] man62.tgz     [X] xshare62.tgz
    [X] base62.tgz    [X] game62.tgz    [X] xfont62.tgz

Reply | Threaded
Open this post in threaded view
|

Re: bsd.mp not installed on EdgeRouter Lite

Stefan Sperling-5
On Fri, Jan 12, 2018 at 11:24:59AM -0500, Scott Bennett wrote:

> After reading INSTALL.octeon, I was able to write miniroot62.fs to a usb,
> plug that into the ERL, and perform a normal installation. The problem is
> that the installer was not able to detect both cores, so it only installed
> bsd.sp (bsd.mp was not an option in the set selection).
>
> Running 6.2-release.
>
> I did follow the instructions for setting the coremask=0x3 when booting the
> installer, and setting the coremask=0x3 in the bootcmd. It seems that the
> installer just wasn't able to recognize that it's a dual core system.

I've seen this, too. For some reason bsd.rd doesn't count CPUs correctly
even with coremask=0x3.

> To workaround this problem, I downloaded bsd.mp after installation and copied
> that to the FAT partition. My ERL can now run SMP, but as you probably
> guessed this does break KARL.

That's what I did, too.

> Has anyone been able to install bsd.mp on the ERL and not break KARL?

Not me.

Looking into this is somewhere at the far end of my todo list.
Not sure I'll ever get to it.

Reply | Threaded
Open this post in threaded view
|

Re: bsd.mp not installed on EdgeRouter Lite

Jan Kalkus
I had it working at some point (I think it was in 6.1-current).

During an upgrade/boot from bsd.rd, drop into a shell before beginning the upgrade, manually set numcpu (I think that was the variable name) to 2 (not sure why it gets set as 1).

After that, the upgrade script will “detect” two cpu cores and install the bsd.mp kernel.

The last part is much more fuzzy to me. I think after the kernel re-linking, I would copy it to the FAT partition.

That seemed to work, inasmuch as it did not throw any errors about kernel re-linking errors.

It has been quite a while since I’ve performed an upgrade on my ERLs. I will try it out when I get a chance.

> On Jan 13, 2018, at 03:01, Stefan Sperling <[hidden email]> wrote:
>
>> On Fri, Jan 12, 2018 at 11:24:59AM -0500, Scott Bennett wrote:
>> After reading INSTALL.octeon, I was able to write miniroot62.fs to a usb,
>> plug that into the ERL, and perform a normal installation. The problem is
>> that the installer was not able to detect both cores, so it only installed
>> bsd.sp (bsd.mp was not an option in the set selection).
>>
>> Running 6.2-release.
>>
>> I did follow the instructions for setting the coremask=0x3 when booting the
>> installer, and setting the coremask=0x3 in the bootcmd. It seems that the
>> installer just wasn't able to recognize that it's a dual core system.
>
> I've seen this, too. For some reason bsd.rd doesn't count CPUs correctly
> even with coremask=0x3.
>
>> To workaround this problem, I downloaded bsd.mp after installation and copied
>> that to the FAT partition. My ERL can now run SMP, but as you probably
>> guessed this does break KARL.
>
> That's what I did, too.
>
>> Has anyone been able to install bsd.mp on the ERL and not break KARL?
>
> Not me.
>
> Looking into this is somewhere at the far end of my todo list.
> Not sure I'll ever get to it.
>

Reply | Threaded
Open this post in threaded view
|

Re: bsd.mp not installed on EdgeRouter Lite

jungle Boogie
In reply to this post by Scott Bennett
On 12 January 2018 at 08:24, Scott Bennett <[hidden email]> wrote:
> After reading INSTALL.octeon, I was able to write miniroot62.fs to a usb,
> plug that into the ERL, and perform a normal installation. The problem is
> that the installer was not able to detect both cores, so it only installed
> bsd.sp (bsd.mp was not an option in the set selection).

See this post:
https://an.undulating.space/post/171020-erl-openbsd-smp/

See this reddit thread:
https://www.reddit.com/r/openbsd/comments/7agdgh/openbsd_62_on_edgerouter_lite_with_bsdmp/

Hope that helps

Reply | Threaded
Open this post in threaded view
|

Re: bsd.mp not installed on EdgeRouter Lite

Scott Bennett
In reply to this post by Stefan Sperling-5
On 1/13/2018 5:01 AM, Stefan Sperling wrote:

> On Fri, Jan 12, 2018 at 11:24:59AM -0500, Scott Bennett wrote:
>> After reading INSTALL.octeon, I was able to write miniroot62.fs to a usb,
>> plug that into the ERL, and perform a normal installation. The problem is
>> that the installer was not able to detect both cores, so it only installed
>> bsd.sp (bsd.mp was not an option in the set selection).
>>
>> Running 6.2-release.
>>
>> I did follow the instructions for setting the coremask=0x3 when booting the
>> installer, and setting the coremask=0x3 in the bootcmd. It seems that the
>> installer just wasn't able to recognize that it's a dual core system.
>
> I've seen this, too. For some reason bsd.rd doesn't count CPUs correctly
> even with coremask=0x3.
>
>> To workaround this problem, I downloaded bsd.mp after installation and copied
>> that to the FAT partition. My ERL can now run SMP, but as you probably
>> guessed this does break KARL.
>
> That's what I did, too.
>
>> Has anyone been able to install bsd.mp on the ERL and not break KARL?
>
> Not me.
>
> Looking into this is somewhere at the far end of my todo list.
> Not sure I'll ever get to it.

Thanks for the reply, Stefan! At least it's a known issue. I will
watch the tree for any updates in this area.

Reply | Threaded
Open this post in threaded view
|

Re: bsd.mp not installed on EdgeRouter Lite

Scott Bennett
In reply to this post by jungle Boogie
On 1/15/2018 5:30 PM, jungle Boogie wrote:

> On 12 January 2018 at 08:24, Scott Bennett <[hidden email]> wrote:
>> After reading INSTALL.octeon, I was able to write miniroot62.fs to a usb,
>> plug that into the ERL, and perform a normal installation. The problem is
>> that the installer was not able to detect both cores, so it only installed
>> bsd.sp (bsd.mp was not an option in the set selection).
>
> See this post:
> https://an.undulating.space/post/171020-erl-openbsd-smp/
>
> See this reddit thread:
> https://www.reddit.com/r/openbsd/comments/7agdgh/openbsd_62_on_edgerouter_lite_with_bsdmp/
>
> Hope that helps

Much appreciated! The reddit thread may just work.

Reply | Threaded
Open this post in threaded view
|

Re: bsd.mp not installed on EdgeRouter Lite

Sean Murphy
On Tue, Jan 16, 2018 at 3:27 PM, Scott Bennett <[hidden email]>
wrote:
> On 1/15/2018 5:30 PM, jungle Boogie wrote:
>> On 12 January 2018 at 08:24, Scott Bennett <[hidden email]>
wrote:
>>> After reading INSTALL.octeon, I was able to write miniroot62.fs to a
usb,
>>> plug that into the ERL, and perform a normal installation. The problem
is
>>> that the installer was not able to detect both cores, so it only
installed
>>> bsd.sp (bsd.mp was not an option in the set selection).
>>
>> See this post:
>> https://an.undulating.space/post/171020-erl-openbsd-smp/
>>
>> See this reddit thread:
>>
https://www.reddit.com/r/openbsd/comments/7agdgh/openbsd_62_on_edgerouter_lite_with_bsdmp/
>>
>> Hope that helps
>
> Much appreciated! The reddit thread may just work.
>

​Greetings all, and thanks for the info on getting the MP installed on the
ERL.
I performed the steps as indicated n the links above and now have GENERIC.MP
running on my ERL.  I did see that KARL failed on the initial install and
reboot,
which I performed via serial console.  I attempted
the manual relinking and a
second reboot via ssh, so I don't know if it failed
​ ​
on the second reboot.  If it did,
​I​
 can live without it
​ ​
in order to have a second core
​ ​
as intended on this device.
Thanks again, dmesg to follow.



​Copyright (c) 1982, 1986, 1989, 1991, 1993
        The Regents of the University of California.  All rights reserved.
Copyright (c) 1995-2017 OpenBSD. All rights reserved.
https://www.OpenBSD.org

OpenBSD 6.2 (GENERIC.MP) #0: Wed Oct  4 05:08:15 UTC 2017
    visa@octeon:/usr/src/sys/arch/octeon/compile/GENERIC.MP
real mem = 536870912 (512MB)
avail mem = 523943936 (499MB)
mainbus0 at root
cpu0 at mainbus0: CN50xx CPU rev 0.1 500 MHz, Software FP emulation
cpu0: cache L1-I 32KB 4 way D 8KB 64 way, L2 128KB 8 way
cpu1 at mainbus0: CN50xx CPU rev 0.1 500 MHz, Software FP emulation
cpu1: cache L1-I 32KB 4 way D 8KB 64 way, L2 128KB 8 way
clock0 at mainbus0: int 5
iobus0 at mainbus0
simplebus0 at iobus0: "soc"
octciu0 at simplebus0
cn30xxsmi0 at simplebus0
com0 at simplebus0: ns16550a, 64 byte fifo
com0: console
dwctwo0 at iobus0 base 0x1180068000000 irq 56
usb0 at dwctwo0: USB revision 2.0
uhub0 at usb0 configuration 1 interface 0 "Octeon DWC2 root hub" rev
2.00/1.00 addr 1
octrng0 at iobus0 base 0x1400000000000 irq 0
cn30xxgmx0 at iobus0 base 0x1180008000000
cnmac0 at cn30xxgmx0: RGMII, address 44:d9:e7:40:b5:c8
atphy0 at cnmac0 phy 7: AR8035 10/100/1000 PHY, rev. 2
cnmac1 at cn30xxgmx0: RGMII, address 44:d9:e7:40:b5:c9
atphy1 at cnmac1 phy 6: AR8035 10/100/1000 PHY, rev. 2
cnmac2 at cn30xxgmx0: RGMII, address 44:d9:e7:40:b5:ca
atphy2 at cnmac2 phy 5: AR8035 10/100/1000 PHY, rev. 2
/dev/ksyms: Symbol table not valid.
umass0 at uhub0 port 1 configuration 1 interface 0 "Lexar USB Flash Drive"
rev 2.10/11.00 addr 2
umass0: using SCSI over Bulk-Only
scsibus0 at umass0: 2 targets, initiator 0
sd0 at scsibus0 targ 1 lun 0: <Lexar, USB Flash Drive, 1100> SCSI4 0/direct
removable serial.05dca83aZB0L2W63LA4P
sd0: 30526MB, 512 bytes/sector, 62517248 sectors
vscsi0 at root
scsibus1 at vscsi0: 256 targets
softraid0 at root
scsibus2 at softraid0: 256 targets
boot device: sd0
root on sd0a (fb74a192b03ee90f.a) swap on sd0b dump on sd0b
WARNING: No TOD clock, believing file system.
WARNING: CHECK AND RESET THE DATE!
Reply | Threaded
Open this post in threaded view
|

Re: bsd.mp not installed on EdgeRouter Lite

Stefan Sperling-5
On Thu, Jan 18, 2018 at 09:06:44AM -0500, Sean Murphy wrote:
> I performed the steps as indicated n the links above and now have GENERIC.MP
> running on my ERL.  I did see that KARL failed on the initial install and
> reboot,

It looks like this issue was just fixed in -current by visa@

Reply | Threaded
Open this post in threaded view
|

Re: bsd.mp not installed on EdgeRouter Lite

Scott Bennett
On 1/18/2018 9:23 AM, Stefan Sperling wrote:
> On Thu, Jan 18, 2018 at 09:06:44AM -0500, Sean Murphy wrote:
>> I performed the steps as indicated n the links above and now have GENERIC.MP
>> running on my ERL.  I did see that KARL failed on the initial install and
>> reboot,
>
> It looks like this issue was just fixed in -current by visa@

I saw the commit messages. Very exciting! I'll give it a try once a new snapshot
gets rolled. Luckily I haven't deployed the ERL yet, so re-installing won't be
a problem.

Reply | Threaded
Open this post in threaded view
|

Re: bsd.mp not installed on EdgeRouter Lite

jungle Boogie
On 18 January 2018 at 07:00, Scott Bennett <[hidden email]> wrote:

> On 1/18/2018 9:23 AM, Stefan Sperling wrote:
>> On Thu, Jan 18, 2018 at 09:06:44AM -0500, Sean Murphy wrote:
>>> I performed the steps as indicated n the links above and now have GENERIC.MP
>>> running on my ERL.  I did see that KARL failed on the initial install and
>>> reboot,
>>
>> It looks like this issue was just fixed in -current by visa@
>
> I saw the commit messages. Very exciting! I'll give it a try once a new snapshot
> gets rolled. Luckily I haven't deployed the ERL yet, so re-installing won't be
> a problem.
>

From the build on the 18th:
Location of sets? (disk http nfs or 'done') [done] done
Making all device nodes...done.
Multiprocessor machine; using bsd.mp instead of bsd.


Very nice! Thanks Visa!

--
-------
inum: 883510009027723
sip: [hidden email]