NOT POSSIBLE: Fully encrypted system with keydisk

classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|

NOT POSSIBLE: Fully encrypted system with keydisk

Stefan Wollny-3
G'day!

I AM LOST!

I read the FAQ (section 14), read man bioctl, read stsp@'s article on
undeadly.org and a couple of other sites on the 'net: IT IS NOT POSSIBLE
TO INSTALL A FULLY ENCRYPTED SYSTEM WITH KEYDISK FOLLOWING THE
DOCUMENTATION on an amd64-current system!
Full stop! (~current = the last three weeks, every iteration)

Again and again and again I end up with
//install: //install.sub[xxxx]: cannot create /dev/null: No space left
on device
repeated endlessly with inbetween (probably, as pretty fast scrolling by)
uid "0" on /: out of inodes

What am I missing???

YES: I can install the system unencrypted on the same hardware
YES: I did switch to the console when running 'bsd.rd' from CD
YES: I did switch to /dev and did 'sh ./MAKEDEV all'
YES: I did 'fdisk -iy' to sd0 and sd1 (sd0=SSD, sd1=USB-keydisk)
YES: I did 'disklabel -E sd0' and 'disklabel -E sd1' accordingly,
setting every partition to type RAID
NO: 'dd if=/dev/zero ...' is not possible with bsd.rd (amd64-current)
YES: I did 'bioctl -C force -c C -l /dev/sd0d -k /dev/sd1d softraid0'
YES: I did again 'sh ./MAKEDEV all' to catch the newly created sd2
NO: I did not 'fdisk', 'disklabel' and 'newfs' on sd2 as this is
expected to be done by the install process
YES: I did switch to '/' before running 'install'
... but from now on I only see the message as stated above.

No dmesg possible as the install process stops right after entering
'install'!

What am I doing wrong, what am I missing??? I am lost, having screwed at
least one SSD...

Any hints appreciated. Thanks.

Best,
STEFAN

Reply | Threaded
Open this post in threaded view
|

Re: NOT POSSIBLE: Fully encrypted system with keydisk

Stuart Henderson
On 2015-12-10, Stefan Wollny <[hidden email]> wrote:

> G'day!
>
> I AM LOST!
>
> I read the FAQ (section 14), read man bioctl, read stsp@'s article on
> undeadly.org and a couple of other sites on the 'net: IT IS NOT POSSIBLE
> TO INSTALL A FULLY ENCRYPTED SYSTEM WITH KEYDISK FOLLOWING THE
> DOCUMENTATION on an amd64-current system!
> Full stop! (~current = the last three weeks, every iteration)
>
> Again and again and again I end up with
> //install: //install.sub[xxxx]: cannot create /dev/null: No space left
> on device
> repeated endlessly with inbetween (probably, as pretty fast scrolling by)
> uid "0" on /: out of inodes
>
> What am I missing???
>
> YES: I can install the system unencrypted on the same hardware
> YES: I did switch to the console when running 'bsd.rd' from CD
> YES: I did switch to /dev and did 'sh ./MAKEDEV all'
> YES: I did 'fdisk -iy' to sd0 and sd1 (sd0=SSD, sd1=USB-keydisk)
> YES: I did 'disklabel -E sd0' and 'disklabel -E sd1' accordingly,
> setting every partition to type RAID
> NO: 'dd if=/dev/zero ...' is not possible with bsd.rd (amd64-current)

Really? I think that should work...

> YES: I did 'bioctl -C force -c C -l /dev/sd0d -k /dev/sd1d softraid0'
> YES: I did again 'sh ./MAKEDEV all' to catch the newly created sd2

In the above step, you have run yourself out of space on the
ramdisk by creating a load of device nodes that you don't have
space for and don't need.

> NO: I did not 'fdisk', 'disklabel' and 'newfs' on sd2 as this is
> expected to be done by the install process
> YES: I did switch to '/' before running 'install'
> ... but from now on I only see the message as stated above.
>
> No dmesg possible as the install process stops right after entering
> 'install'!
>
> What am I doing wrong, what am I missing??? I am lost, having screwed at
> least one SSD...
>
> Any hints appreciated. Thanks.
>
> Best,
> STEFAN

Reply | Threaded
Open this post in threaded view
|

Re: NOT POSSIBLE: Fully encrypted system with keydisk

trondd-2
In reply to this post by Stefan Wollny-3
On Thu, December 10, 2015 6:35 pm, Stefan Wollny wrote:
> YES: I did 'disklabel -E sd0' and 'disklabel -E sd1' accordingly,
setting every partition to type RAID

How many partitions are you making on sd0?  For FDE, typically you make
one partition of type RAID filling the disk (or your desired OpenBSD area)
and all the other partitions are created inside of it.  How are you
partitioning the drives?

> YES: I did 'bioctl -C force -c C -l /dev/sd0d -k /dev/sd1d softraid0'

Why force?  Why partition d?  Again, how are you partitioning your drives?

Reply | Threaded
Open this post in threaded view
|

Re: NOT POSSIBLE: Fully encrypted system with keydisk

Alexander Hall
In reply to this post by Stuart Henderson
On December 11, 2015 1:27:52 AM GMT+01:00, Stuart Henderson <[hidden email]> wrote:
>On 2015-12-10, Stefan Wollny <[hidden email]> wrote:

>> YES: I did 'bioctl -C force -c C -l /dev/sd0d -k /dev/sd1d softraid0'
>> YES: I did again 'sh ./MAKEDEV all' to catch the newly created sd2
>
>In the above step, you have run yourself out of space on the
>ramdisk by creating a load of device nodes that you don't have
>space for and don't need.

Indeed. In particular, you tend to run out of inodes.

/Alexander

Reply | Threaded
Open this post in threaded view
|

Re: NOT POSSIBLE: Fully encrypted system with keydisk

Tati Chevron
In reply to this post by trondd-2
On Thu, Dec 10, 2015 at 07:33:57PM -0500, trondd wrote:

>On Thu, December 10, 2015 6:35 pm, Stefan Wollny wrote:
>> YES: I did 'disklabel -E sd0' and 'disklabel -E sd1' accordingly,
>setting every partition to type RAID
>
>How many partitions are you making on sd0?  For FDE, typically you make
>one partition of type RAID filling the disk (or your desired OpenBSD area)
>and all the other partitions are created inside of it.  How are you
>partitioning the drives?
>
>> YES: I did 'bioctl -C force -c C -l /dev/sd0d -k /dev/sd1d softraid0'
>
>Why force?  Why partition d?  Again, how are you partitioning your drives?

Why wouldn't a single partition d spanning the whole drive be the logical
choice for a disk that is going to be a used entirely as a softraid crypto
volume?

The RAID partition is not a root filesystem in itself, so if you are
implying that it should be a single partition a spanning the whole drive,
I disagree.

--
Tati Chevron
Perl and FORTRAN specialist.
SWABSIT development and migration department.
http://www.swabsit.com

Reply | Threaded
Open this post in threaded view
|

Re: NOT POSSIBLE: Fully encrypted system with keydisk

Stefan Sperling-5
In reply to this post by Alexander Hall
On Fri, Dec 11, 2015 at 09:53:48AM +0100, Alexander Hall wrote:

> On December 11, 2015 1:27:52 AM GMT+01:00, Stuart Henderson <[hidden email]> wrote:
> >On 2015-12-10, Stefan Wollny <[hidden email]> wrote:
>
> >> YES: I did 'bioctl -C force -c C -l /dev/sd0d -k /dev/sd1d softraid0'
> >> YES: I did again 'sh ./MAKEDEV all' to catch the newly created sd2
> >
> >In the above step, you have run yourself out of space on the
> >ramdisk by creating a load of device nodes that you don't have
> >space for and don't need.
>
> Indeed. In particular, you tend to run out of inodes.
>
> /Alexander

Yes. That step should be: sh ./MAKEDEV sd2

Reply | Threaded
Open this post in threaded view
|

Re: NOT POSSIBLE: Fully encrypted system with keydisk

Tati Chevron
In reply to this post by trondd-2
On Thu, Dec 10, 2015 at 07:33:57PM -0500, trondd wrote:

> On Thu, December 10, 2015 6:35 pm, Stefan Wollny wrote:
>> YES: I did 'disklabel -E sd0' and 'disklabel -E sd1' accordingly,
> setting every partition to type RAID
>
> How many partitions are you making on sd0?  For FDE, typically you make
> one partition of type RAID filling the disk (or your desired OpenBSD area)
> and all the other partitions are created inside of it.  How are you
> partitioning the drives?
>
>> YES: I did 'bioctl -C force -c C -l /dev/sd0d -k /dev/sd1d softraid0'
>
> Why force?  Why partition d?  Again, how are you partitioning your drives?

Why wouldn't a single partition d spanning the whole drive be the logical
choice for a disk that is going to be a used entirely as a softraid crypto
volume?

The RAID partition is not a root filesystem in itself, so if you are
implying that it should be a single partition a spanning the whole drive,
I disagree.

--
Tati Chevron
Perl and FORTRAN specialist.
SWABSIT development and migration department.
http://www.swabsit.com

Reply | Threaded
Open this post in threaded view
|

Re: NOT POSSIBLE: Fully encrypted system with keydisk

Stefan Wollny
In reply to this post by Stefan Sperling-5
> Gesendet: Freitag, 11. Dezember 2015 um 11:33 Uhr
> Von: "Stefan Sperling" <[hidden email]>
> An: "Alexander Hall" <[hidden email]>
> Cc: "Stuart Henderson" <[hidden email]>, [hidden email]
> Betreff: Re: NOT POSSIBLE: Fully encrypted system with keydisk
>
> On Fri, Dec 11, 2015 at 09:53:48AM +0100, Alexander Hall wrote:
> > On December 11, 2015 1:27:52 AM GMT+01:00, Stuart Henderson <[hidden email]> wrote:
> > >On 2015-12-10, Stefan Wollny <[hidden email]> wrote:
> >
> > >> YES: I did 'bioctl -C force -c C -l /dev/sd0d -k /dev/sd1d softraid0'
> > >> YES: I did again 'sh ./MAKEDEV all' to catch the newly created sd2
> > >
> > >In the above step, you have run yourself out of space on the
> > >ramdisk by creating a load of device nodes that you don't have
> > >space for and don't need.
> >
> > Indeed. In particular, you tend to run out of inodes.
> >
> > /Alexander
>
> Yes. That step should be: sh ./MAKEDEV sd2
>

@Alexander, Stefan & Stuart:

I can confirm that this was the cause for the error message. Doing it as you advised enabled me to install amd64-current, yet some step is still missing as afer the reboot the system does not come up (stops at the splash screen not entering any boot operation).

A few words on my use case: From my customers I get sensible personal data on their customers (not only name/address, but job related information, income statements, ratings, etc.). Loosing the laptop when traveling would be painful but loosing the confidentiality would really hit me.

My setup: The laptop has two SSDs - a big one for '/home' and a smaller one (mSata) for the system (plus some spare).

The system-SSD (=sd0) has one partiion 'd' which gets unlocked by the keydisk's 'd' partition (=> sd3)
The /home-SDD (=sd1) has one partition 'e' which gets unlocked by the keydisk's 'e' partition (=> sd4)
On the keydisk (=sd2) there are some more partitions for keys and storage:
   'f' to unlock a backup-disk which I use onsite.
   'g' and 'h' for future use to unlock other devices (like e.g. USB devices).
   'i' an additional RAID partition for other sensible stuff (e.g. passwords for clients' systems which should be accessible in case of emergency from an unencrypted OpenBSD-box as well.

@stuart: dd fails with "file system ist full \ dd: /dev/rsd3c: No space left on device"

@trondd: Not having an 'a' partition on one of the three devices seemed to be helpful to memorize that this is not a 'normal' partition. No real technical reason otherwise.

OK - follow up problem: After the installation on /dev/sd3 (plus setting up /dev/sd4 for /home) I did not reboot but run installboot(8) like so:
# /usr/sbin/installboot sd3

This last produced an error message about /usr/mdec/biosboot missing.

Mind giving me an other hint on what I might have missed? I searched the man pages but nothing obvious came to me. Has there been some recent changes?

TIA.

Best,
STEFAN


BTW: A dmesg from an unencrypted install can be found here:
http://marc.info/?l=openbsd-misc&m=144956819405937&w=2

Reply | Threaded
Open this post in threaded view
|

Re: NOT POSSIBLE: Fully encrypted system with keydisk

Stefan Sperling-5
On Fri, Dec 11, 2015 at 01:18:55PM +0100, Stefan Wollny wrote:
> OK - follow up problem: After the installation on /dev/sd3 (plus setting up /dev/sd4 for /home) I did not reboot but run installboot(8) like so:
> # /usr/sbin/installboot sd3
>
> This last produced an error message about /usr/mdec/biosboot missing.
>
> Mind giving me an other hint on what I might have missed? I searched the man pages but nothing obvious came to me. Has there been some recent changes?

In the bsd.rd ramdisk, the installed system's root disk (sd3a) is mounted
at /mnt, and the installed system's user partition is mounted at /mnt/usr.

So this should work: /mnt/usr/sbin/installboot -r /mnt sd3

Reply | Threaded
Open this post in threaded view
|

Re: NOT POSSIBLE: Fully encrypted system with keydisk

Stefan Wollny
> Gesendet: Freitag, 11. Dezember 2015 um 14:52 Uhr
> Von: "Stefan Sperling" <[hidden email]>
> An: "Stefan Wollny" <[hidden email]>
> Cc: [hidden email]
> Betreff: Re: NOT POSSIBLE: Fully encrypted system with keydisk
>
> On Fri, Dec 11, 2015 at 01:18:55PM +0100, Stefan Wollny wrote:
> > OK - follow up problem: After the installation on /dev/sd3 (plus setting up /dev/sd4 for /home) I did not reboot but run installboot(8) like so:
> > # /usr/sbin/installboot sd3
> >
> > This last produced an error message about /usr/mdec/biosboot missing.
> >
> > Mind giving me an other hint on what I might have missed? I searched the man pages but nothing obvious came to me. Has there been some recent changes?
>
> In the bsd.rd ramdisk, the installed system's root disk (sd3a) is mounted
> at /mnt, and the installed system's user partition is mounted at /mnt/usr.
>
> So this should work: /mnt/usr/sbin/installboot -r /mnt sd3
>
>

Hi Stefan,

THX a lot for (again) helping me and the explanation.

I run the command like you adviced and no error message showed up.

So far, so good - unfortunatelly the system still does not boot after the 'reboot'. Still stops at the manufacturers splash screen not recognizing any storage device to boot from.

Any other idea? Could it be that this machine's particular BIOS needs an 'a' partition on sd0 to find the device? As the system is unusable at present and I need to disassemble the SSDs anyway to get them wiped in order to proceed with the next install I will give this a try.

Best,
STEFAN

Reply | Threaded
Open this post in threaded view
|

Re: NOT POSSIBLE: Fully encrypted system with keydisk

Stefan Sperling-5
On Fri, Dec 11, 2015 at 03:30:04PM +0100, Stefan Wollny wrote:
> I run the command like you adviced and no error message showed up.
>
> So far, so good - unfortunatelly the system still does not boot after the 'reboot'. Still stops at the manufacturers splash screen not recognizing any storage device to boot from.
>
> Any other idea? Could it be that this machine's particular BIOS needs an 'a' partition on sd0 to find the device? As the system is unusable at present and I need to disassemble the SSDs anyway to get them wiped in order to proceed with the next install I will give this a try.
>
> Best,
> STEFAN

Hmm, that's odd. Could you show us the output of the following:

At the boot> prompt (if you can get there):

  machine diskinfo

In bsd.rd:

  fdisk sd0
  disklabel sd0

  fdisk sd3
  disklabel sd3

  bioctl softraid0

And also instalboot's verbose messages:

  /mnt/usr/sbin/installboot -v -r /mnt sd3

Another thing you could try to narrow down the problem is using
a passphrase instead of a key disk. Does that work?

Reply | Threaded
Open this post in threaded view
|

Re: NOT POSSIBLE: Fully encrypted system with keydisk

Tati Chevron
In reply to this post by Stefan Wollny
On Fri, Dec 11, 2015 at 03:30:04PM +0100, Stefan Wollny wrote:

>> Gesendet: Freitag, 11. Dezember 2015 um 14:52 Uhr
>> Von: "Stefan Sperling" <[hidden email]>
>> An: "Stefan Wollny" <[hidden email]>
>> Cc: [hidden email]
>> Betreff: Re: NOT POSSIBLE: Fully encrypted system with keydisk
>>
>> On Fri, Dec 11, 2015 at 01:18:55PM +0100, Stefan Wollny wrote:
>> > OK - follow up problem: After the installation on /dev/sd3 (plus setting up /dev/sd4 for /home) I did not reboot but run installboot(8) like so:
>> > # /usr/sbin/installboot sd3
>> >
>> > This last produced an error message about /usr/mdec/biosboot missing.
>> >
>> > Mind giving me an other hint on what I might have missed? I searched the man pages but nothing obvious came to me. Has there been some recent changes?
>>
>> In the bsd.rd ramdisk, the installed system's root disk (sd3a) is mounted
>> at /mnt, and the installed system's user partition is mounted at /mnt/usr.
>>
>> So this should work: /mnt/usr/sbin/installboot -r /mnt sd3
>>
>>
>
>Hi Stefan,
>
>THX a lot for (again) helping me and the explanation.
>
>I run the command like you adviced and no error message showed up.
>
>So far, so good - unfortunatelly the system still does not boot after the 'reboot'. Still stops at the manufacturers splash screen not recognizing any storage device to boot from.
>
>Any other idea? Could it be that this machine's particular BIOS needs an 'a' partition on sd0 to find the device? As the system is unusable at present and I need to disassemble the SSDs anyway to get them wiped in order to proceed with the next install I will give this a try.

The layout of the paritions within the disklabel shouldn't be of any
concern to the BIOS.

There has been discusson on the list previously about buggy BIOSes that
lock up if a disk is connected that doesn't contain a standard MBR and
partition table.  However, since you indicate that you can install and
boot OpenBSD on a non-encrypted volume on the same disks, that doesn't
seem like the problem.

--
Tati Chevron
Perl and FORTRAN specialist.
SWABSIT development and migration department.
http://www.swabsit.com

Reply | Threaded
Open this post in threaded view
|

Re: NOT POSSIBLE: Fully encrypted system with keydisk

Stuart Henderson
In reply to this post by Stefan Wollny
On 2015-12-11, Stefan Wollny <[hidden email]> wrote:
> @stuart: dd fails with "file system ist full \ dd: /dev/rsd3c: No space left on device"

Guessing that you didn't create the sd3 device nodes before doing the dd.
At this point you probably have a file (not device node) named /dev/rsd3c.

Reply | Threaded
Open this post in threaded view
|

Re: NOT POSSIBLE: Fully encrypted system with keydisk

Stefan Sperling-5
In reply to this post by Stefan Sperling-5
On Fri, Dec 11, 2015 at 05:44:36PM +0100, Stefan Wollny wrote:
> fdisk(25692): syscall 54 "ioctl"
> Abort trap
> >   disklabel sd3
> disklabel(3120): syscall 54 "ioctl"
> Abort trap

This is obviously not quite right.
It looks like you're using a snapshot with a pledge(2) bug.

What snapshot are you booting? Please ensure that you're either
booting 5.8 or the latest snapshot and send a complete dmesg
if it is still failing.

Reply | Threaded
Open this post in threaded view
|

Re: NOT POSSIBLE: Fully encrypted system with keydisk

Stefan Wollny-3
Am 12/11/15 um 18:34 schrieb Stefan Sperling:

> On Fri, Dec 11, 2015 at 05:44:36PM +0100, Stefan Wollny wrote:
>> fdisk(25692): syscall 54 "ioctl"
>> Abort trap
>>>    disklabel sd3
>> disklabel(3120): syscall 54 "ioctl"
>> Abort trap
> This is obviously not quite right.
> It looks like you're using a snapshot with a pledge(2) bug.
>
> What snapshot are you booting? Please ensure that you're either
> booting 5.8 or the latest snapshot and send a complete dmesg
> if it is still failing.
A couple of test iterations later ...

[TLDR: Still no reboot into an unencrypted system]


These are the steps (annotated) I went through:



+++++++++++++++++++++++++++++++++++++++++++++++++++
s

Prior to running bsr.rd check the chain of boot devices,
has to be CD => sd0 => PXE
's' to choose "shell"
fdisk sd0 => OK
fdisk sd1 => not OK
fdisk sd2 => not OK
cd /dev
sh ./MAKEDEV sd1
sh ./MAKEDEV sd2
cd /
fdisk -iy sd0
fdisk -iy sd1
fdisk -iy sd2
disklabel -E sd0
     entire HD: FS type RAID, partition 'd'
disklabel -E sd1
     entire HD: FS type RAID, partition 'e'
disklabel -E sd2
     partition 'd', size 1M, FS type RAID
     partition 'e', size 1M, FS type RAID
     partition 'f', size 1M, FS type RAID
     partition 'g', size 1M, FS type RAID
     partition 'h', size 1M, FS type RAID
     partition 'i', size <entire remaining area>, FS type 4.2BSD
bioctl -c C -l /dev/sd0d -k /dev/sd2d softraid0
bioctl -c C -l /dev/sd1e -k /dev/sd2e softraid0
cd /dev
sh ./MAKEDEV sd3
sh ./MAKEDEV sd4
cd /
dd if=/dev/zero of=/dev/rsd3c bs=1m count=1
dd if=/dev/zero of=/dev/rsd4c bs=1m count=1
fdisk -iy sd3
fdisk -iy sd4
install
[ ... usual install process ... ]
/mnt/usr/sbin/installboot -v -r /mnt sd3
newfs sd2i
mount /dev/sd2i /mnt2
dmesg > /mnt2/dmesg.txt
fdisk sd0 > /mnt2/fdisk-sd0.txt
fdisk sd1 > /mnt2/fdisk-sd1.txt
fdisk sd2 > /mnt2/fdisk-sd2.txt
fdisk sd3 > /mnt2/fdisk-sd3.txt
fdisk sd4 > /mnt2/fdisk-sd4.txt

fdisk sd0 > /mnt2/fdisk-sd0.txt
fdisk sd1 > /mnt2/fdisk-sd1.txt
fdisk sd2 > /mnt2/fdisk-sd2.txt
fdisk sd3 > /mnt2/fdisk-sd3.txt
fdisk sd4 > /mnt2/fdisk-sd4.txt

bioctl sd3 > /mnt2/bioctl-sd3.txt

reboot

+++++++++++++++++++++++++++++++++++++++++++++++++++


RESULT: No system start, the process stops immediately at the splash
screen (=no boot device found?)

OK - here are the protocols as given above:

SD0
     Disk: sd0    geometry: 14593/255/63 [234441648 Sectors]
     Offset: 0    Signature: 0xAA55
                   Starting         Ending         LBA Info:
      #: id      C   H   S -      C   H   S [ start:        size ]
-------------------------------------------------------------------------------
      0: 00      0   0   0 -      0   0   0 [ 0:           0 ] unused
      1: 00      0   0   0 -      0   0   0 [ 0:           0 ] unused
      2: 00      0   0   0 -      0   0   0 [ 0:           0 ] unused
     *3: A6      0   1   2 -  14592 254  63 [          64: 234436481 ]
OpenBSD

     # /dev/rsd0c:
     type: SCSI
     disk: SCSI disk
     label: Samsung SSD 850
     duid: 9a2e9576361b4dd5
     flags:
     bytes/sector: 512
     sectors/track: 63
     tracks/cylinder: 255
     sectors/cylinder: 16065
     cylinders: 14593
     total sectors: 234441648
     boundstart: 64
     boundend: 234436545
     drivedata: 0

     16 partitions:
     #                size           offset  fstype [fsize bsize  cpg]
       c:        234441648                0 unused
       d:        234436481               64 RAID

SD 1:
     Disk: sd1    geometry: 124519/255/63 [2000409264 Sectors]
     Offset: 0    Signature: 0xAA55
                   Starting         Ending         LBA Info:
      #: id      C   H   S -      C   H   S [ start:        size ]
-------------------------------------------------------------------------------
      0: 00      0   0   0 -      0   0   0 [ 0:           0 ] unused
      1: 00      0   0   0 -      0   0   0 [ 0:           0 ] unused
      2: 00      0   0   0 -      0   0   0 [ 0:           0 ] unused
     *3: A6      0   1   2 - 124518 254  63 [          64: 2000397671 ]
OpenBSD

     # /dev/rsd1c:
     type: SCSI
     disk: SCSI disk
     label: Samsung SSD 850
     duid: 5367a9a0fd3fe33c
     flags:
     bytes/sector: 512
     sectors/track: 63
     tracks/cylinder: 255
     sectors/cylinder: 16065
     cylinders: 124519
     total sectors: 2000409264
     boundstart: 64
     boundend: 2000397735
     drivedata: 0

     16 partitions:
     #                size           offset  fstype [fsize bsize  cpg]
       c:       2000409264                0 unused
       e:       2000397671               64 RAID

SD2
     Disk: sd2    geometry: 979/255/63 [15730688 Sectors]
     Offset: 0    Signature: 0xAA55
                   Starting         Ending         LBA Info:
      #: id      C   H   S -      C   H   S [ start:        size ]
-------------------------------------------------------------------------------
      0: 00      0   0   0 -      0   0   0 [ 0:           0 ] unused
      1: 00      0   0   0 -      0   0   0 [ 0:           0 ] unused
      2: 00      0   0   0 -      0   0   0 [ 0:           0 ] unused
     *3: A6      0   1   2 -    978 254  63 [ 64:    15727571 ] OpenBSD

     # /dev/rsd2c:
     type: SCSI
     disk: SCSI disk
     label: UDisk
     duid: 25af1f68379d49c8
     flags:
     bytes/sector: 512
     sectors/track: 63
     tracks/cylinder: 255
     sectors/cylinder: 16065
     cylinders: 979
     total sectors: 15730688
     boundstart: 64
     boundend: 15727635
     drivedata: 0

     16 partitions:
     #                size           offset  fstype [fsize bsize cpg]
       c:         15730688                0 unused
       d:            16001               64 RAID
       e:            16065            16065 RAID
       f:            16065            32130 RAID
       g:            16065            48195 RAID
       h:            16065            64260 RAID
       i:         15647264            80352  4.2BSD   2048 16384 1


SD3
     Disk: sd3    geometry: 14592/255/63 [234435953 Sectors]
     Offset: 0    Signature: 0xAA55
                   Starting         Ending         LBA Info:
      #: id      C   H   S -          C   H   S [ start:        size ]
-------------------------------------------------------------------------------
      0: 00      0   0   0 -          0   0   0 [ 0:           0 ] unused
      1: 00      0   0   0 -          0   0   0 [ 0:           0 ] unused
      2: 00      0   0   0 -          0   0   0 [ 0:           0 ] unused
     *3: A6      0   1   2 -  14591 254  63 [          64: 234420416 ]
OpenBSD

     # /dev/rsd3c:
     type: SCSI
     disk: SCSI disk
     label: SR CRYPTO
     duid: 4e5ebd3d927a8fb6
     flags:
     bytes/sector: 512
     sectors/track: 63
     tracks/cylinder: 255
     sectors/cylinder: 16065
     cylinders: 14592
     total sectors: 234435953
     boundstart: 64
     boundend: 234420480
     drivedata: 0

     16 partitions:
     #                size           offset  fstype [fsize bsize  cpg]
       a:         20980800               64  4.2BSD   2048 16384    1 # /mnt
       b:          2088476         20980864 swap
       c:        234435953                0 unused
       d:         20980864         23069344  4.2BSD   2048 16384    1 #
/mnt/tmp
       e:         41929664         44050208  4.2BSD   2048 16384    1 #
/mnt/var
       f:         62910528         85979872  4.2BSD   2048 16384    1 #
/mnt/usr
       g:         85530080        148890400  4.2BSD   2048 16384    1


sd4
     Disk: sd4    geometry: 124518/255/63 [2000397143 Sectors]
     Offset: 0    Signature: 0xAA55
                 Starting         Ending         LBA Info:
          #: id      C   H   S -           C   H   S [ start:        size ]
-------------------------------------------------------------------------------
          0: 00      0   0   0 -           0   0   0 [          
0:           0 ] unused
          1: 00      0   0   0 -           0   0   0 [          
0:           0 ] unused
          2: 00      0   0   0 -           0   0   0 [          
0:           0 ] unused
         *3: A6      0   1   2 - 124517 254  63 [ 64:  2000381606 ] OpenBSD

     # /dev/rsd4c:
     type: SCSI
     disk: SCSI disk
     label: SR CRYPTO
     duid: 97fd340f56beeb30
     flags:
     bytes/sector: 512
     sectors/track: 63
     tracks/cylinder: 255
     sectors/cylinder: 16065
     cylinders: 124518
     total sectors: 2000397143
     boundstart: 64
     boundend: 2000381670
     drivedata: 0

     16 partitions:
     #                size           offset  fstype [fsize bsize  cpg]
       a:       2000381504               64  4.2BSD   8192 65536    1 #
/mnt/home
       c:       2000397143                0 unused

BIOCTL SD3
     Volume      Status               Size Device
     softraid0 0 Online       120031207936 sd3     CRYPTO
               0 Online       120031207936 0:0.0   noencl <sd0d>
               1 Online           key disk 0:1.0   noencl <sd2d>


DMESG:

OpenBSD 5.8-current (RAMDISK_CD) #1574: Fri Dec 11 06:22:23 MST 2015
[hidden email]:/usr/src/sys/arch/amd64/compile/RAMDISK_CD
real mem = 17082359808 (16291MB)
avail mem = 16562933760 (15795MB)
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xeb500 (35 entries)
bios0: vendor American Megatrends Inc. version "1.05.01" date 08/05/2015
bios0: Notebook W65_67SZ
acpi0 at bios0: rev 2
acpi0: tables DSDT FACP APIC FPDT ASF! SSDT SSDT SSDT MCFG HPET SSDT
SSDT SSDT DMAR
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM) i5-4210M CPU @ 2.60GHz, 3093.30 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,SENSOR,ARAT
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4, IBE
cpu at mainbus0: not configured
cpu at mainbus0: not configured
cpu at mainbus0: not configured
ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 20, 24 pins
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 2 (RP01)
acpiprt2 at acpi0: bus 3 (RP03)
acpiprt3 at acpi0: bus 4 (RP04)
acpiprt4 at acpi0: bus 1 (P0P2)
acpiprt5 at acpi0: bus -1 (P0PA)
acpiprt6 at acpi0: bus -1 (P0PB)
acpiprt7 at acpi0: bus 1 (PEG0)
acpiec0 at acpi0
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel Core 4G Host" rev 0x06
ppb0 at pci0 dev 1 function 0 "Intel Core 4G PCIE" rev 0x06: msi
pci1 at ppb0 bus 1
vga1 at pci0 dev 2 function 0 "Intel HD Graphics 4600" rev 0x06
wsdisplay1 at vga1 mux 1: console (80x25, vt100 emulation)
"Intel Core 4G HD Audio" rev 0x06 at pci0 dev 3 function 0 not configured
xhci0 at pci0 dev 20 function 0 "Intel 8 Series xHCI" rev 0x05: msi
usb0 at xhci0: USB revision 3.0
uhub0 at usb0 "Intel xHCI root hub" rev 3.00/1.00 addr 1
"Intel 8 Series MEI" rev 0x04 at pci0 dev 22 function 0 not configured
ehci0 at pci0 dev 26 function 0 "Intel 8 Series USB" rev 0x05: apic 2 int 16
usb1 at ehci0: USB revision 2.0
uhub1 at usb1 "Intel EHCI root hub" rev 2.00/1.00 addr 1
"Intel 8 Series HD Audio" rev 0x05 at pci0 dev 27 function 0 not configured
ppb1 at pci0 dev 28 function 0 "Intel 8 Series PCIE" rev 0xd5
pci2 at ppb1 bus 2
ppb2 at pci0 dev 28 function 2 "Intel 8 Series PCIE" rev 0xd5: msi
pci3 at ppb2 bus 3
iwm0 at pci3 dev 0 function 0 "Intel Dual Band Wireless AC 7260" rev
0xbb, msi
ppb3 at pci0 dev 28 function 3 "Intel 8 Series PCIE" rev 0xd5: msi
pci4 at ppb3 bus 4
rtsx0 at pci4 dev 0 function 0 "Realtek RTL8411 Card Reader" rev 0x01: msi
sdmmc0 at rtsx0
re0 at pci4 dev 0 function 2 "Realtek 8168" rev 0x0a: RTL8411 (0x4880),
msi, address 80:fa:5b:13:a0:ad
rgephy0 at re0 phy 7: RTL8169S/8110S/8211 PHY, rev. 5
ehci1 at pci0 dev 29 function 0 "Intel 8 Series USB" rev 0x05: apic 2 int 23
usb2 at ehci1: USB revision 2.0
uhub2 at usb2 "Intel EHCI root hub" rev 2.00/1.00 addr 1
"Intel HM86 LPC" rev 0x05 at pci0 dev 31 function 0 not configured
ahci0 at pci0 dev 31 function 2 "Intel 8 Series AHCI" rev 0x05: msi,
AHCI 1.3
ahci0: port 0: 1.5Gb/s
ahci0: port 4: 6.0Gb/s
ahci0: port 5: 6.0Gb/s
scsibus0 at ahci0: 32 targets
cd0 at scsibus0 targ 0 lun 0: <TSSTcorp, CDDVDW SN-208FB, SB00> ATAPI
5/cdrom removable
sd0 at scsibus0 targ 4 lun 0: <ATA, Samsung SSD 850, EMT4> SCSI3
0/direct fixed naa.5002538d402ece0c
sd0: 114473MB, 512 bytes/sector, 234441648 sectors, thin
sd1 at scsibus0 targ 5 lun 0: <ATA, Samsung SSD 850, EXM0> SCSI3
0/direct fixed naa.500253887007d4c5
sd1: 976762MB, 512 bytes/sector, 2000409264 sectors, thin
"Intel 8 Series SMBus" rev 0x05 at pci0 dev 31 function 3 not configured
isa0 at mainbus0
pckbc0 at isa0 port 0x60/5 irq 1 irq 12
pckbd0 at pckbc0 (kbd slot)
wskbd0 at pckbd0: console keyboard, using wsdisplay1
uhidev0 at uhub0 port 2 configuration 1 interface 0 "Logitech USB
Receiver" rev 2.00/12.01 addr 2
uhidev0: iclass 3/1
ukbd0 at uhidev0
wskbd1 at ukbd0 mux 1
wskbd1: connecting to wsdisplay1
uhidev1 at uhub0 port 2 configuration 1 interface 1 "Logitech USB
Receiver" rev 2.00/12.01 addr 2
uhidev1: iclass 3/1, 8 report ids
uhid at uhidev1 reportid 2 not configured
uhid at uhidev1 reportid 3 not configured
uhid at uhidev1 reportid 4 not configured
uhid at uhidev1 reportid 8 not configured
uhidev2 at uhub0 port 2 configuration 1 interface 2 "Logitech USB
Receiver" rev 2.00/12.01 addr 2
uhidev2: iclass 3/0, 33 report ids
uhid at uhidev2 reportid 16 not configured
uhid at uhidev2 reportid 17 not configured
uhid at uhidev2 reportid 32 not configured
uhid at uhidev2 reportid 33 not configured
umass0 at uhub0 port 6 configuration 1 interface 0 "General UDisk" rev
2.00/1.00 addr 3
umass0: using SCSI over Bulk-Only
scsibus1 at umass0: 2 targets, initiator 0
sd2 at scsibus1 targ 1 lun 0: <General, UDisk, 5.00> SCSI2 0/direct
removable serial.abcd1234245144257801
sd2: 7681MB, 512 bytes/sector, 15730688 sectors
"vendor 0x8087 product 0x07dc" rev 2.00/0.01 addr 4 at uhub0 port 7 not
configured
uhub3 at uhub1 port 1 "vendor 0x8087 product 0x8008" rev 2.00/0.05 addr 2
uhub4 at uhub2 port 1 "vendor 0x8087 product 0x8000" rev 2.00/0.05 addr 2
softraid0 at root
scsibus2 at softraid0: 256 targets
root on rd0a swap on rd0b dump on rd0b
iwm0: could not read firmware iwm-7260-9 (error 2)
sd3 at scsibus2 targ 1 lun 0: <OPENBSD, SR CRYPTO, 005> SCSI2 0/direct fixed
sd3: 114470MB, 512 bytes/sector, 234435953 sectors
sd4 at scsibus2 targ 2 lun 0: <OPENBSD, SR CRYPTO, 005> SCSI2 0/direct fixed
sd4: 976756MB, 512 bytes/sector, 2000397143 sectors

Reply | Threaded
Open this post in threaded view
|

Re: NOT POSSIBLE: Fully encrypted system with keydisk

Mike Larkin
On Sat, Dec 12, 2015 at 12:27:46AM +0100, Stefan Wollny wrote:

> Am 12/11/15 um 18:34 schrieb Stefan Sperling:
> >On Fri, Dec 11, 2015 at 05:44:36PM +0100, Stefan Wollny wrote:
> >>fdisk(25692): syscall 54 "ioctl"
> >>Abort trap
> >>>   disklabel sd3
> >>disklabel(3120): syscall 54 "ioctl"
> >>Abort trap
> >This is obviously not quite right.
> >It looks like you're using a snapshot with a pledge(2) bug.
> >
> >What snapshot are you booting? Please ensure that you're either
> >booting 5.8 or the latest snapshot and send a complete dmesg
> >if it is still failing.
> A couple of test iterations later ...
>
> [TLDR: Still no reboot into an unencrypted system]
>
>
> These are the steps (annotated) I went through:
>
>
>
> +++++++++++++++++++++++++++++++++++++++++++++++++++
> s
>
> Prior to running bsr.rd check the chain of boot devices,
> has to be CD => sd0 => PXE
> 's' to choose "shell"
> fdisk sd0 => OK
> fdisk sd1 => not OK
> fdisk sd2 => not OK
> cd /dev
> sh ./MAKEDEV sd1
> sh ./MAKEDEV sd2
> cd /
> fdisk -iy sd0
> fdisk -iy sd1
> fdisk -iy sd2
> disklabel -E sd0
>     entire HD: FS type RAID, partition 'd'
> disklabel -E sd1
>     entire HD: FS type RAID, partition 'e'
> disklabel -E sd2
>     partition 'd', size 1M, FS type RAID
>     partition 'e', size 1M, FS type RAID
>     partition 'f', size 1M, FS type RAID
>     partition 'g', size 1M, FS type RAID
>     partition 'h', size 1M, FS type RAID
>     partition 'i', size <entire remaining area>, FS type 4.2BSD
> bioctl -c C -l /dev/sd0d -k /dev/sd2d softraid0
> bioctl -c C -l /dev/sd1e -k /dev/sd2e softraid0
> cd /dev
> sh ./MAKEDEV sd3
> sh ./MAKEDEV sd4
> cd /
> dd if=/dev/zero of=/dev/rsd3c bs=1m count=1
> dd if=/dev/zero of=/dev/rsd4c bs=1m count=1
> fdisk -iy sd3
> fdisk -iy sd4
> install
> [ ... usual install process ... ]
> /mnt/usr/sbin/installboot -v -r /mnt sd3

The installer will detect your install target is softraid crypto
and do that automatically. Why are you re-doing this again?

-ml

> newfs sd2i
> mount /dev/sd2i /mnt2
> dmesg > /mnt2/dmesg.txt
> fdisk sd0 > /mnt2/fdisk-sd0.txt
> fdisk sd1 > /mnt2/fdisk-sd1.txt
> fdisk sd2 > /mnt2/fdisk-sd2.txt
> fdisk sd3 > /mnt2/fdisk-sd3.txt
> fdisk sd4 > /mnt2/fdisk-sd4.txt
>
> fdisk sd0 > /mnt2/fdisk-sd0.txt
> fdisk sd1 > /mnt2/fdisk-sd1.txt
> fdisk sd2 > /mnt2/fdisk-sd2.txt
> fdisk sd3 > /mnt2/fdisk-sd3.txt
> fdisk sd4 > /mnt2/fdisk-sd4.txt
>
> bioctl sd3 > /mnt2/bioctl-sd3.txt
>
> reboot

Reply | Threaded
Open this post in threaded view
|

Re: NOT POSSIBLE: Fully encrypted system with keydisk

Stefan Wollny-3
Gesendet von meinem BlackBerry 10-Smartphone.
  Originalnachricht  
‎On Sat, Dec 12, 2015 at 12:27:46AM +0100, Stefan Wollny wrote:

> Am 12/11/15 um 18:34 schrieb Stefan Sperling:
> >On Fri, Dec 11, 2015 at 05:44:36PM +0100, Stefan Wollny wrote:
> >>fdisk(25692): syscall 54 "ioctl"
> >>Abort trap
> >>> disklabel sd3
> >>disklabel(3120): syscall 54 "ioctl"
> >>Abort trap
> >This is obviously not quite right.
> >It looks like you're using a snapshot with a pledge(2) bug.
> >
> >What snapshot are you booting? Please ensure that you're either
> >booting 5.8 or the latest snapshot and send a complete dmesg
> >if it is still failing.
> A couple of test iterations later ...
>
> [TLDR: Still no reboot into an unencrypted system]
>
>
> These are the steps (annotated) I went through:
>
>
>
> +++++++++++++++++++++++++++++++++++++++++++++++++++
> s
>
> Prior to running bsr.rd check the chain of boot devices,
> has to be CD => sd0 => PXE
> 's' to choose "shell"
> fdisk sd0 => OK
> fdisk sd1 => not OK
> fdisk sd2 => not OK
> cd /dev
> sh ./MAKEDEV sd1
> sh ./MAKEDEV sd2
> cd /
> fdisk -iy sd0
> fdisk -iy sd1
> fdisk -iy sd2
> disklabel -E sd0
> entire HD: FS type RAID, partition 'd'
> disklabel -E sd1
> entire HD: FS type RAID, partition 'e'
> disklabel -E sd2
> partition 'd', size 1M, FS type RAID
> partition 'e', size 1M, FS type RAID
> partition 'f', size 1M, FS type RAID
> partition 'g', size 1M, FS type RAID
> partition 'h', size 1M, FS type RAID
> partition 'i', size <entire remaining area>, FS type 4.2BSD
> bioctl -c C -l /dev/sd0d -k /dev/sd2d softraid0
> bioctl -c C -l /dev/sd1e -k /dev/sd2e softraid0
> cd /dev
> sh ./MAKEDEV sd3
> sh ./MAKEDEV sd4
> cd /
> dd if=/dev/zero of=/dev/rsd3c bs=1m count=1
> dd if=/dev/zero of=/dev/rsd4c bs=1m count=1
> fdisk -iy sd3
> fdisk -iy sd4
> install
> [ ... usual install process ... ]
> /mnt/usr/sbin/installboot -v -r /mnt sd3

The installer will detect your install target is softraid crypto
and do that automatically. Why are you re-doing this again?

-ml

I was adviced to do so. With or without the system doesn't reboot but stopps
at the machine's splash screen for not finding a boot device.

STEFAN ‎

Reply | Threaded
Open this post in threaded view
|

Re: NOT POSSIBLE: Fully encrypted system with keydisk

Mike Larkin
On Sat, Dec 12, 2015 at 12:51:33AM +0100, Stefan Wollny wrote:

>
>
> Gesendet??von??meinem??BlackBerry??10-Smartphone.
> ?? Originalnachricht ??
> ???On Sat, Dec 12, 2015 at 12:27:46AM +0100, Stefan Wollny wrote:
> > Am 12/11/15 um 18:34 schrieb Stefan Sperling:
> > >On Fri, Dec 11, 2015 at 05:44:36PM +0100, Stefan Wollny wrote:
> > >>fdisk(25692): syscall 54 "ioctl"
> > >>Abort trap
> > >>> disklabel sd3
> > >>disklabel(3120): syscall 54 "ioctl"
> > >>Abort trap
> > >This is obviously not quite right.
> > >It looks like you're using a snapshot with a pledge(2) bug.
> > >
> > >What snapshot are you booting? Please ensure that you're either
> > >booting 5.8 or the latest snapshot and send a complete dmesg
> > >if it is still failing.
> > A couple of test iterations later ...
> >
> > [TLDR: Still no reboot into an unencrypted system]
> >
> >
> > These are the steps (annotated) I went through:
> >
> >
> >
> > +++++++++++++++++++++++++++++++++++++++++++++++++++
> > s
> >
> > Prior to running bsr.rd check the chain of boot devices,
> > has to be CD => sd0 => PXE
> > 's' to choose "shell"
> > fdisk sd0 => OK
> > fdisk sd1 => not OK
> > fdisk sd2 => not OK
> > cd /dev
> > sh ./MAKEDEV sd1
> > sh ./MAKEDEV sd2
> > cd /
> > fdisk -iy sd0
> > fdisk -iy sd1
> > fdisk -iy sd2
> > disklabel -E sd0
> > entire HD: FS type RAID, partition 'd'
> > disklabel -E sd1
> > entire HD: FS type RAID, partition 'e'
> > disklabel -E sd2
> > partition 'd', size 1M, FS type RAID
> > partition 'e', size 1M, FS type RAID
> > partition 'f', size 1M, FS type RAID
> > partition 'g', size 1M, FS type RAID
> > partition 'h', size 1M, FS type RAID
> > partition 'i', size <entire remaining area>, FS type 4.2BSD
> > bioctl -c C -l /dev/sd0d -k /dev/sd2d softraid0
> > bioctl -c C -l /dev/sd1e -k /dev/sd2e softraid0
> > cd /dev
> > sh ./MAKEDEV sd3
> > sh ./MAKEDEV sd4
> > cd /
> > dd if=/dev/zero of=/dev/rsd3c bs=1m count=1
> > dd if=/dev/zero of=/dev/rsd4c bs=1m count=1
> > fdisk -iy sd3
> > fdisk -iy sd4
> > install
> > [ ... usual install process ... ]
> > /mnt/usr/sbin/installboot -v -r /mnt sd3
>
> The installer will detect your install target is softraid crypto
> and do that automatically. Why are you re-doing this again?
>
> -ml
>
> I was adviced to do so. With or without the system doesn't reboot but stopps at the machine's splash screen for not finding a boot device.
>
> STEFAN ???

I'd try with a simpler config first, like just making a single softraid crypto
device, without a keydisk. See if that works first before moving on to more
complex configurations.

-ml

Reply | Threaded
Open this post in threaded view
|

Re: NOT POSSIBLE: Fully encrypted system with keydisk

Stefan Sperling-5
In reply to this post by Stefan Wollny-3
On Sat, Dec 12, 2015 at 12:27:46AM +0100, Stefan Wollny wrote:

> Am 12/11/15 um 18:34 schrieb Stefan Sperling:
> >On Fri, Dec 11, 2015 at 05:44:36PM +0100, Stefan Wollny wrote:
> >>fdisk(25692): syscall 54 "ioctl"
> >>Abort trap
> >>>   disklabel sd3
> >>disklabel(3120): syscall 54 "ioctl"
> >>Abort trap
> >This is obviously not quite right.
> >It looks like you're using a snapshot with a pledge(2) bug.
> >
> >What snapshot are you booting? Please ensure that you're either
> >booting 5.8 or the latest snapshot and send a complete dmesg
> >if it is still failing.
> A couple of test iterations later ...
>
> [TLDR: Still no reboot into an unencrypted system]
>
> These are the steps (annotated) I went through:

I cannot see anything obviously wrong in there.

Perhaps the BIOS has some issue with this setup.
Could you try the same procedure on a different machine
to check if it works there?

Another thing you could try is clearing the bootable flag
in the MBR of your key disk with fdisk (run fdisk -e sd2;
type the commands 'flag 3 0' and 'exit'). Perhaps the BIOS
is looking for something to boot on the keydisk and crashes?