iSCSI mount on boot causes the machine to go to single user mode

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

iSCSI mount on boot causes the machine to go to single user mode

Ashton Fagg
>Synopsis: Mouting iSCSI disk at boot time causes machine to enter single user mode.
>Category: Bug.
>Environment:
        System      : OpenBSD 6.9
        Details     : OpenBSD 6.9-beta (GENERIC.MP) #346: Fri Feb 19 23:56:21 MST 2021
                         [hidden email]:/usr/src/sys/arch/amd64/compile/GENERIC.MP

        Architecture: OpenBSD.amd64
        Machine     : amd64
>Description:

Cross posting this from misc@, because the more I mess around with it
the more suspicious I get that a bug is present...

I have a configured iSCSI drive that I am trying to mount at boot.  If
I mount the drive manually, it works as expected.

Trying to mount at boot, however, causes the machine to go into single
user mode.

When the machine comes up, it does indeed pass mounting all of the
local partitions before trying to mount the iSCSI target.

However, it still blows up and enters single user mode. Upon entering
single user mode, the additional NIC that connects to the SAN does
appear to be configured and happy (I can ping the iSCSI host). You can
see pictures of the screen before [1] and after [2] entering and
exiting single user mode. In [3], you can see the machine does
eventually hit a login prompt.

Upon exiting single user mode, it then gives an error message that
"SIOCSIFGATTR" is an invalid argument to ifconfig, before continuing
to boot. It then says iscsid failed to start. This is visible in [2]
and [3].

Once actually logged in, it appears as though iscsid is dead in the
water. However, if I remove the /etc/fstab entry, it boots as normal
and then I can mount manually as expected. Very strange.

Checking /var/log/daemon, this is what I see:

Feb 20 18:59:28 elara iscsid[52173]: startup
Feb 20 18:59:28 elara iscsid[52173]: fatal in iscsid: vscsi_open: Device busy

iscsid won't reload, restart etc without removing the /etc/fstab entry and rebooting.

[1] https://thalassa.fagg.id.au/IMG_6318.jpg
[2] https://thalassa.fagg.id.au/IMG_6321.jpg
[3] https://thalassa.fagg.id.au/IMG_6322.jpg


>How-To-Repeat:

Adding the following line in my /etc/fstab:

   08b434e6d3d1d227.c /build ffs rw,noatime,nodev,wxallowed,net 1 2

And rebooting will result in the sequence described above.

Other relevant configs are below:
       
Here is my full and complete /etc/iscsid.conf:

  elara$ cat /etc/iscsi.conf        
  target "ports" {
          initiatoraddr 172.16.0.2
          targetaddr 172.16.0.1
          targetname "iqn.2011-01.au.id.fagg.ctl:elara-usr-ports"
  }

And the two NIC configs + an ifconfig - the NIC used for the iSCSI target is ix0.

  elara$ cat /etc/hostname.re0
  inet 192.168.1.6 0xffffff00

  elara$ cat /etc/hostname.ix0  
  inet 172.16.0.2 0xfffff000
  up

  elara$ doas ifconfig
  doas ([hidden email]) password:
  lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 32768
          index 4 priority 0 llprio 3
          groups: lo
          inet6 ::1 prefixlen 128
          inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4
          inet 127.0.0.1 netmask 0xff000000
  ix0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
          lladdr 90:e2:ba:29:76:a0
          index 1 priority 0 llprio 3
          media: Ethernet autoselect (10GSFP+Cu full-duplex,rxpause,txpause)
          status: active
          inet 172.16.0.2 netmask 0xfffff000 broadcast 172.16.15.255
  re0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
          lladdr 70:85:c2:67:19:b9
          index 2 priority 0 llprio 3
          groups: egress
          media: Ethernet autoselect (1000baseT full-duplex)
          status: active
          inet 192.168.1.6 netmask 0xffffff00 broadcast 192.168.1.255
  enc0: flags=0<>
          index 3 priority 0 llprio 3
          groups: enc
          status: active
  pflog0: flags=141<UP,RUNNING,PROMISC> mtu 33136
          index 5 priority 0 llprio 3
          groups: pflog

>Fix:

Removing the /etc/fstab entry results in a clean boot, and you can then mount manually. It is only when trying to auto-mount at boot time that the problem appears.


dmesg:
OpenBSD 6.9-beta (GENERIC.MP) #346: Fri Feb 19 23:56:21 MST 2021
    [hidden email]:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 42861899776 (40876MB)
avail mem = 41547538432 (39622MB)
random: good seed from bootblocks
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 3.0 @ 0xed2d0 (33 entries)
bios0: vendor American Megatrends Inc. version "P4.40" date 01/05/2018
bios0: ASRock AB350 Pro4
acpi0 at bios0: ACPI 6.0
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP APIC FPDT FIDT SSDT SSDT CRAT CDIT SSDT MCFG AAFT HPET SSDT UEFI IVRS SSDT SSDT WSMT
acpi0: wakeup devices GPP0(S4) GPP1(S4) GPP2(S4) PTXH(S4) GPP3(S4) GPP4(S4) GPP5(S4) GPP6(S4) GPP7(S4) GPP8(S4) GPP9(S4) GPPA(S4) GPPB(S4) GPPC(S4) GPPD(S4) GPPE(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 32 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: AMD Ryzen 7 1800X Eight-Core Processor, 3593.95 MHz, 17-01-01
cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu0: 64KB 64b/line 4-way I-cache, 32KB 64b/line 8-way D-cache, 512KB 64b/line 8-way L2 cache
cpu0: ITLB 64 4KB entries fully associative, 64 4MB entries fully associative
cpu0: DTLB 64 4KB entries fully associative, 64 4MB entries fully associative
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, IBE
cpu1 at mainbus0: apid 1 (application processor)
cpu1: AMD Ryzen 7 1800X Eight-Core Processor, 3593.25 MHz, 17-01-01
cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu1: 64KB 64b/line 4-way I-cache, 32KB 64b/line 8-way D-cache, 512KB 64b/line 8-way L2 cache
cpu1: ITLB 64 4KB entries fully associative, 64 4MB entries fully associative
cpu1: DTLB 64 4KB entries fully associative, 64 4MB entries fully associative
cpu1: smt 1, core 0, package 0
cpu2 at mainbus0: apid 2 (application processor)
cpu2: AMD Ryzen 7 1800X Eight-Core Processor, 3593.25 MHz, 17-01-01
cpu2: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu2: 64KB 64b/line 4-way I-cache, 32KB 64b/line 8-way D-cache, 512KB 64b/line 8-way L2 cache
cpu2: ITLB 64 4KB entries fully associative, 64 4MB entries fully associative
cpu2: DTLB 64 4KB entries fully associative, 64 4MB entries fully associative
cpu2: smt 0, core 1, package 0
cpu3 at mainbus0: apid 3 (application processor)
cpu3: AMD Ryzen 7 1800X Eight-Core Processor, 3593.25 MHz, 17-01-01
cpu3: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu3: 64KB 64b/line 4-way I-cache, 32KB 64b/line 8-way D-cache, 512KB 64b/line 8-way L2 cache
cpu3: ITLB 64 4KB entries fully associative, 64 4MB entries fully associative
cpu3: DTLB 64 4KB entries fully associative, 64 4MB entries fully associative
cpu3: smt 1, core 1, package 0
cpu4 at mainbus0: apid 4 (application processor)
cpu4: AMD Ryzen 7 1800X Eight-Core Processor, 3593.25 MHz, 17-01-01
cpu4: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu4: 64KB 64b/line 4-way I-cache, 32KB 64b/line 8-way D-cache, 512KB 64b/line 8-way L2 cache
cpu4: ITLB 64 4KB entries fully associative, 64 4MB entries fully associative
cpu4: DTLB 64 4KB entries fully associative, 64 4MB entries fully associative
cpu4: smt 0, core 2, package 0
cpu5 at mainbus0: apid 5 (application processor)
cpu5: AMD Ryzen 7 1800X Eight-Core Processor, 3593.25 MHz, 17-01-01
cpu5: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu5: 64KB 64b/line 4-way I-cache, 32KB 64b/line 8-way D-cache, 512KB 64b/line 8-way L2 cache
cpu5: ITLB 64 4KB entries fully associative, 64 4MB entries fully associative
cpu5: DTLB 64 4KB entries fully associative, 64 4MB entries fully associative
cpu5: smt 1, core 2, package 0
cpu6 at mainbus0: apid 6 (application processor)
cpu6: AMD Ryzen 7 1800X Eight-Core Processor, 3593.25 MHz, 17-01-01
cpu6: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu6: 64KB 64b/line 4-way I-cache, 32KB 64b/line 8-way D-cache, 512KB 64b/line 8-way L2 cache
cpu6: ITLB 64 4KB entries fully associative, 64 4MB entries fully associative
cpu6: DTLB 64 4KB entries fully associative, 64 4MB entries fully associative
cpu6: smt 0, core 3, package 0
cpu7 at mainbus0: apid 7 (application processor)
cpu7: AMD Ryzen 7 1800X Eight-Core Processor, 3593.25 MHz, 17-01-01
cpu7: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu7: 64KB 64b/line 4-way I-cache, 32KB 64b/line 8-way D-cache, 512KB 64b/line 8-way L2 cache
cpu7: ITLB 64 4KB entries fully associative, 64 4MB entries fully associative
cpu7: DTLB 64 4KB entries fully associative, 64 4MB entries fully associative
cpu7: smt 1, core 3, package 0
cpu8 at mainbus0: apid 8 (application processor)
cpu8: AMD Ryzen 7 1800X Eight-Core Processor, 3593.25 MHz, 17-01-01
cpu8: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu8: 64KB 64b/line 4-way I-cache, 32KB 64b/line 8-way D-cache, 512KB 64b/line 8-way L2 cache
cpu8: ITLB 64 4KB entries fully associative, 64 4MB entries fully associative
cpu8: DTLB 64 4KB entries fully associative, 64 4MB entries fully associative
cpu8: smt 0, core 4, package 0
cpu9 at mainbus0: apid 9 (application processor)
cpu9: AMD Ryzen 7 1800X Eight-Core Processor, 3593.25 MHz, 17-01-01
cpu9: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu9: 64KB 64b/line 4-way I-cache, 32KB 64b/line 8-way D-cache, 512KB 64b/line 8-way L2 cache
cpu9: ITLB 64 4KB entries fully associative, 64 4MB entries fully associative
cpu9: DTLB 64 4KB entries fully associative, 64 4MB entries fully associative
cpu9: smt 1, core 4, package 0
cpu10 at mainbus0: apid 10 (application processor)
cpu10: AMD Ryzen 7 1800X Eight-Core Processor, 3593.25 MHz, 17-01-01
cpu10: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu10: 64KB 64b/line 4-way I-cache, 32KB 64b/line 8-way D-cache, 512KB 64b/line 8-way L2 cache
cpu10: ITLB 64 4KB entries fully associative, 64 4MB entries fully associative
cpu10: DTLB 64 4KB entries fully associative, 64 4MB entries fully associative
cpu10: smt 0, core 5, package 0
cpu11 at mainbus0: apid 11 (application processor)
cpu11: AMD Ryzen 7 1800X Eight-Core Processor, 3593.25 MHz, 17-01-01
cpu11: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu11: 64KB 64b/line 4-way I-cache, 32KB 64b/line 8-way D-cache, 512KB 64b/line 8-way L2 cache
cpu11: ITLB 64 4KB entries fully associative, 64 4MB entries fully associative
cpu11: DTLB 64 4KB entries fully associative, 64 4MB entries fully associative
cpu11: smt 1, core 5, package 0
cpu12 at mainbus0: apid 12 (application processor)
cpu12: AMD Ryzen 7 1800X Eight-Core Processor, 3593.25 MHz, 17-01-01
cpu12: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu12: 64KB 64b/line 4-way I-cache, 32KB 64b/line 8-way D-cache, 512KB 64b/line 8-way L2 cache
cpu12: ITLB 64 4KB entries fully associative, 64 4MB entries fully associative
cpu12: DTLB 64 4KB entries fully associative, 64 4MB entries fully associative
cpu12: smt 0, core 6, package 0
cpu13 at mainbus0: apid 13 (application processor)
cpu13: AMD Ryzen 7 1800X Eight-Core Processor, 3593.25 MHz, 17-01-01
cpu13: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu13: 64KB 64b/line 4-way I-cache, 32KB 64b/line 8-way D-cache, 512KB 64b/line 8-way L2 cache
cpu13: ITLB 64 4KB entries fully associative, 64 4MB entries fully associative
cpu13: DTLB 64 4KB entries fully associative, 64 4MB entries fully associative
cpu13: smt 1, core 6, package 0
cpu14 at mainbus0: apid 14 (application processor)
cpu14: AMD Ryzen 7 1800X Eight-Core Processor, 3593.25 MHz, 17-01-01
cpu14: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu14: 64KB 64b/line 4-way I-cache, 32KB 64b/line 8-way D-cache, 512KB 64b/line 8-way L2 cache
cpu14: ITLB 64 4KB entries fully associative, 64 4MB entries fully associative
cpu14: DTLB 64 4KB entries fully associative, 64 4MB entries fully associative
cpu14: smt 0, core 7, package 0
cpu15 at mainbus0: apid 15 (application processor)
cpu15: AMD Ryzen 7 1800X Eight-Core Processor, 3593.25 MHz, 17-01-01
cpu15: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu15: 64KB 64b/line 4-way I-cache, 32KB 64b/line 8-way D-cache, 512KB 64b/line 8-way L2 cache
cpu15: ITLB 64 4KB entries fully associative, 64 4MB entries fully associative
cpu15: DTLB 64 4KB entries fully associative, 64 4MB entries fully associative
cpu15: smt 1, core 7, package 0
ioapic0 at mainbus0: apid 17 pa 0xfec00000, version 21, 24 pins, can't remap
ioapic1 at mainbus0: apid 18 pa 0xfec01000, version 21, 32 pins, can't remap
acpimcfg0 at acpi0
acpimcfg0: addr 0xf8000000, bus 0-63
acpihpet0 at acpi0: 14318180 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 1 (GPP0)
acpiprt2 at acpi0: bus -1 (GPP1)
acpiprt3 at acpi0: bus 3 (GPP2)
acpiprt4 at acpi0: bus 10 (BR3C)
acpiprt5 at acpi0: bus -1 (GPP3)
acpiprt6 at acpi0: bus -1 (GPP4)
acpiprt7 at acpi0: bus -1 (GPP5)
acpiprt8 at acpi0: bus -1 (GPP6)
acpiprt9 at acpi0: bus -1 (GPP7)
acpiprt10 at acpi0: bus 11 (GPP8)
acpiprt11 at acpi0: bus -1 (GPP9)
acpiprt12 at acpi0: bus -1 (GPPA)
acpiprt13 at acpi0: bus -1 (GPPB)
acpiprt14 at acpi0: bus -1 (GPPC)
acpiprt15 at acpi0: bus -1 (GPPD)
acpiprt16 at acpi0: bus -1 (GPPE)
acpiprt17 at acpi0: bus -1 (GPPF)
acpiprt18 at acpi0: bus 17 (GP17)
acpiprt19 at acpi0: bus 18 (GP18)
acpipci0 at acpi0 PCI0: 0x00000010 0x00000011 0x00000000
acpicmos0 at acpi0
acpibtn0 at acpi0: PWRB
amdgpio0 at acpi0 GPIO uid 0 addr 0xfed81500/0x400 irq 7, 184 pins
"AMDIF030" at acpi0 not configured
"PNP0C14" at acpi0 not configured
acpicpu0 at acpi0: C1 bad (state 1 has no substates): C2(0@400 io@0x414), C1(@1 halt!), PSS
acpicpu1 at acpi0: C1 bad (state 1 has no substates): C2(0@400 io@0x414), C1(@1 halt!), PSS
acpicpu2 at acpi0: C1 bad (state 1 has no substates): C2(0@400 io@0x414), C1(@1 halt!), PSS
acpicpu3 at acpi0: C1 bad (state 1 has no substates): C2(0@400 io@0x414), C1(@1 halt!), PSS
acpicpu4 at acpi0: C1 bad (state 1 has no substates): C2(0@400 io@0x414), C1(@1 halt!), PSS
acpicpu5 at acpi0: C1 bad (state 1 has no substates): C2(0@400 io@0x414), C1(@1 halt!), PSS
acpicpu6 at acpi0: C1 bad (state 1 has no substates): C2(0@400 io@0x414), C1(@1 halt!), PSS
acpicpu7 at acpi0: C1 bad (state 1 has no substates): C2(0@400 io@0x414), C1(@1 halt!), PSS
acpicpu8 at acpi0: C1 bad (state 1 has no substates): C2(0@400 io@0x414), C1(@1 halt!), PSS
acpicpu9 at acpi0: C1 bad (state 1 has no substates): C2(0@400 io@0x414), C1(@1 halt!), PSS
acpicpu10 at acpi0: C1 bad (state 1 has no substates): C2(0@400 io@0x414), C1(@1 halt!), PSS
acpicpu11 at acpi0: C1 bad (state 1 has no substates): C2(0@400 io@0x414), C1(@1 halt!), PSS
acpicpu12 at acpi0: C1 bad (state 1 has no substates): C2(0@400 io@0x414), C1(@1 halt!), PSS
acpicpu13 at acpi0: C1 bad (state 1 has no substates): C2(0@400 io@0x414), C1(@1 halt!), PSS
acpicpu14 at acpi0: C1 bad (state 1 has no substates): C2(0@400 io@0x414), C1(@1 halt!), PSS
acpicpu15 at acpi0: C1 bad (state 1 has no substates): C2(0@400 io@0x414), C1(@1 halt!), PSS
cpu0: 3593 MHz: speeds: 3600 3200 2200 MHz
pci0 at mainbus0 bus 0
ksmn0 at pci0 dev 0 function 0 "AMD 17h Root Complex" rev 0x00
"AMD 17h IOMMU" rev 0x00 at pci0 dev 0 function 2 not configured
pchb0 at pci0 dev 1 function 0 "AMD 17h PCIE" rev 0x00
ppb0 at pci0 dev 1 function 1 "AMD 17h PCIE" rev 0x00: msi
pci1 at ppb0 bus 1
ix0 at pci1 dev 0 function 0 "Intel 82599" rev 0x01, msix, 8 queues, address 90:e2:ba:29:76:a0
ppb1 at pci0 dev 1 function 3 "AMD 17h PCIE" rev 0x00: msi
pci2 at ppb1 bus 3
xhci0 at pci2 dev 0 function 0 "AMD 300 Series xHCI" rev 0x02: msi, xHCI 1.10
usb0 at xhci0: USB revision 3.0
uhub0 at usb0 configuration 1 interface 0 "AMD xHCI root hub" rev 3.00/1.00 addr 1
ahci0 at pci2 dev 0 function 1 "AMD 300 Series SATA" rev 0x02: msi, AHCI 1.3.1
ahci0: port busy after first PMP probe FIS
ahci0: port busy after first PMP probe FIS
ahci0: port 0: 6.0Gb/s
ahci0: port busy after first PMP probe FIS
ahci0: port busy after first PMP probe FIS
ahci0: port 1: 6.0Gb/s
scsibus1 at ahci0: 32 targets
sd0 at scsibus1 targ 0 lun 0: <ATA, CT1000BX500SSD1, M6CR> naa.500a0751e4e92f88
sd0: 953869MB, 512 bytes/sector, 1953525168 sectors, thin
sd1 at scsibus1 targ 1 lun 0: <ATA, SPCC Solid State, V3.1> naa.502b2a201d1c1b1a
sd1: 61057MB, 512 bytes/sector, 125045424 sectors, thin
ppb2 at pci2 dev 0 function 2 vendor "AMD", unknown product 0x43b2 rev 0x02
pci3 at ppb2 bus 4
ppb3 at pci3 dev 0 function 0 "AMD 300 Series PCIE" rev 0x02: msi
pci4 at ppb3 bus 5
ppb4 at pci3 dev 1 function 0 "AMD 300 Series PCIE" rev 0x02: msi
pci5 at ppb4 bus 6
ppb5 at pci3 dev 4 function 0 "AMD 300 Series PCIE" rev 0x02: msi
pci6 at ppb5 bus 7
ppb6 at pci3 dev 5 function 0 "AMD 300 Series PCIE" rev 0x02: msi
pci7 at ppb6 bus 8
ppb7 at pci3 dev 6 function 0 "AMD 300 Series PCIE" rev 0x02: msi
pci8 at ppb7 bus 9
ahci1 at pci8 dev 0 function 0 "ASMedia ASM1061 AHCI" rev 0x02: msi, AHCI 1.2
scsibus2 at ahci1: 32 targets
ppb8 at pci3 dev 7 function 0 "AMD 300 Series PCIE" rev 0x02: msi
pci9 at ppb8 bus 10
re0 at pci9 dev 0 function 0 "Realtek 8168" rev 0x11: RTL8168G/8111G (0x4c00), msi, address 70:85:c2:67:19:b9
rgephy0 at re0 phy 7: RTL8251 PHY, rev. 0
pchb1 at pci0 dev 2 function 0 "AMD 17h PCIE" rev 0x00
pchb2 at pci0 dev 3 function 0 "AMD 17h PCIE" rev 0x00
ppb9 at pci0 dev 3 function 1 "AMD 17h PCIE" rev 0x00: msi
pci10 at ppb9 bus 11
vga1 at pci10 dev 0 function 0 vendor "NVIDIA", unknown product 0x1d01 rev 0xa1
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
azalia0 at pci10 dev 0 function 1 vendor "NVIDIA", unknown product 0x0fb8 rev 0xa1: msi
azalia0: no supported codecs
pchb3 at pci0 dev 4 function 0 "AMD 17h PCIE" rev 0x00
pchb4 at pci0 dev 7 function 0 "AMD 17h PCIE" rev 0x00
ppb10 at pci0 dev 7 function 1 "AMD 17h PCIE" rev 0x00
pci11 at ppb10 bus 17
vendor "AMD", unknown product 0x145a (class instrumentation unknown subclass 0x00, rev 0x00) at pci11 dev 0 function 0 not configured
ccp0 at pci11 dev 0 function 2 "AMD 17h Crypto" rev 0x00
xhci1 at pci11 dev 0 function 3 "AMD 17h xHCI" rev 0x00: msi, xHCI 1.0
usb1 at xhci1: USB revision 3.0
uhub1 at usb1 configuration 1 interface 0 "AMD xHCI root hub" rev 3.00/1.00 addr 1
pchb5 at pci0 dev 8 function 0 "AMD 17h PCIE" rev 0x00
ppb11 at pci0 dev 8 function 1 "AMD 17h PCIE" rev 0x00
pci12 at ppb11 bus 18
vendor "AMD", unknown product 0x1455 (class instrumentation unknown subclass 0x00, rev 0x00) at pci12 dev 0 function 0 not configured
ahci2 at pci12 dev 0 function 2 "AMD FCH AHCI" rev 0x51: msi, AHCI 1.3.1
scsibus3 at ahci2: 32 targets
azalia1 at pci12 dev 0 function 3 "AMD 17h HD Audio" rev 0x00: apic 18 int 19
azalia1: codecs: Realtek ALC892
audio0 at azalia1
piixpm0 at pci0 dev 20 function 0 "AMD FCH SMBus" rev 0x59: SMI
iic0 at piixpm0
spdmem0 at iic0 addr 0x50: 16GB DDR4 SDRAM PC4-17000
spdmem1 at iic0 addr 0x51: 16GB DDR4 SDRAM PC4-17000
spdmem2 at iic0 addr 0x52: 4GB DDR4 SDRAM PC4-17000
spdmem3 at iic0 addr 0x53: 4GB DDR4 SDRAM PC4-17000
iic1 at piixpm0
pcib0 at pci0 dev 20 function 3 "AMD FCH LPC" rev 0x51
pchb6 at pci0 dev 24 function 0 "AMD 17h Data Fabric" rev 0x00
pchb7 at pci0 dev 24 function 1 "AMD 17h Data Fabric" rev 0x00
pchb8 at pci0 dev 24 function 2 "AMD 17h Data Fabric" rev 0x00
pchb9 at pci0 dev 24 function 3 "AMD 17h Data Fabric" rev 0x00
pchb10 at pci0 dev 24 function 4 "AMD 17h Data Fabric" rev 0x00
pchb11 at pci0 dev 24 function 5 "AMD 17h Data Fabric" rev 0x00
pchb12 at pci0 dev 24 function 6 "AMD 17h Data Fabric" rev 0x00
pchb13 at pci0 dev 24 function 7 "AMD 17h Data Fabric" rev 0x00
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5 irq 1 irq 12
pckbd0 at pckbc0 (kbd slot)
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
wbsio0 at isa0 port 0x2e/2: NCT6779D rev 0x62
lm1 at wbsio0 port 0x290/8: NCT6779D
uhub2 at uhub0 port 8 configuration 1 interface 0 "Genesys Logic USB2.0 Hub" rev 2.00/85.36 addr 2
uhub3 at uhub2 port 1 configuration 1 interface 0 "ASIX Electronics AX68004" rev 1.01/0.12 addr 3
uhidev0 at uhub3 port 1 configuration 1 interface 0 "Primax Kensington Eagle Trackball" rev 2.00/6.00 addr 4
uhidev0: iclass 3/1
ums0 at uhidev0: 3 buttons, Z and W dir
wsmouse0 at ums0 mux 0
uhidev1 at uhub3 port 2 configuration 1 interface 0 "Kinesis Advantage2 Keyboard" rev 2.00/1.00 addr 5
uhidev1: iclass 3/1
ums1 at uhidev1: 3 buttons, Z dir
wsmouse1 at ums1 mux 0
uhidev2 at uhub3 port 2 configuration 1 interface 1 "Kinesis Advantage2 Keyboard" rev 2.00/1.00 addr 5
uhidev2: iclass 3/1
ukbd0 at uhidev2: 8 variable keys, 6 key codes
wskbd1 at ukbd0 mux 1
wskbd1: connecting to wsdisplay0
uhidev3 at uhub3 port 2 configuration 1 interface 2 "Kinesis Advantage2 Keyboard" rev 2.00/1.00 addr 5
uhidev3: iclass 3/0, 2 report ids
uhid0 at uhidev3 reportid 1: input=1, output=0, feature=0
uhid1 at uhidev3 reportid 2: input=1, output=0, feature=0
uhidev4 at uhub3 port 6 configuration 1 interface 0 "ASIX Electronics AX68004" rev 1.10/0.02 addr 6
uhidev4: iclass 3/1
ukbd1 at uhidev4: 8 variable keys, 6 key codes
wskbd2 at ukbd1 mux 1
wskbd2: connecting to wsdisplay0
uhidev5 at uhub3 port 6 configuration 1 interface 1 "ASIX Electronics AX68004" rev 1.10/0.02 addr 6
uhidev5: iclass 3/1, 3 report ids
ums2 at uhidev5 reportid 1: 5 buttons, Z dir
wsmouse2 at ums2 mux 0
uhid2 at uhidev5 reportid 2: input=1, output=0, feature=0
uhid3 at uhidev5 reportid 3: input=2, output=0, feature=0
uhub4 at uhub2 port 2 configuration 1 interface 0 "Genesys Logic USB2.0 Hub" rev 2.00/85.36 addr 7
uvideo0 at uhub4 port 2 configuration 1 interface 0 "Logitech Webcam C270" rev 2.00/0.12 addr 8
video0 at uvideo0
uaudio0 at uhub4 port 2 configuration 1 interface 3 "Logitech Webcam C270" rev 2.00/0.12 addr 8
uaudio0: class v1, full-speed, sync, channels: 0 play, 1 rec, 2 ctls
audio1 at uaudio0
uhidev6 at uhub4 port 3 configuration 1 interface 0 "Logitech G502 HERO Gaming Mouse" rev 2.00/27.03 addr 9
uhidev6: iclass 3/1
ums3 at uhidev6: 16 buttons, Z and W dir
wsmouse3 at ums3 mux 0
uhidev7 at uhub4 port 3 configuration 1 interface 1 "Logitech G502 HERO Gaming Mouse" rev 2.00/27.03 addr 9
uhidev7: iclass 3/0, 17 report ids
ukbd2 at uhidev7 reportid 1: 8 variable keys, 6 key codes
wskbd3 at ukbd2 mux 1
wskbd3: connecting to wsdisplay0
uhid4 at uhidev7 reportid 3: input=4, output=0, feature=0
uhid5 at uhidev7 reportid 4: input=1, output=0, feature=0
uhidpp0 at uhidev7 reportid 16
vscsi0 at root
scsibus4 at vscsi0: 256 targets
softraid0 at root
scsibus5 at softraid0: 256 targets
root on sd0a (02215eb188a74570.a) swap on sd0b dump on sd0b

usbdevs:
Controller /dev/usb0:
addr 01: 1022:0000 AMD, xHCI root hub
         super speed, self powered, config 1, rev 1.00
         driver: uhub0
Controller /dev/usb1:
addr 01: 1022:0000 AMD, xHCI root hub
         super speed, self powered, config 1, rev 1.00
         driver: uhub1

Reply | Threaded
Open this post in threaded view
|

Re: iSCSI mount on boot causes the machine to go to single user mode

Claudio Jeker-3
On Mon, Feb 22, 2021 at 09:27:58PM -0500, Ashton Fagg wrote:

> >Synopsis: Mouting iSCSI disk at boot time causes machine to enter single user mode.
> >Category: Bug.
> >Environment:
> System      : OpenBSD 6.9
> Details     : OpenBSD 6.9-beta (GENERIC.MP) #346: Fri Feb 19 23:56:21 MST 2021
> [hidden email]:/usr/src/sys/arch/amd64/compile/GENERIC.MP
>
> Architecture: OpenBSD.amd64
> Machine     : amd64
> >Description:
>
> Cross posting this from misc@, because the more I mess around with it
> the more suspicious I get that a bug is present...
>
> I have a configured iSCSI drive that I am trying to mount at boot.  If
> I mount the drive manually, it works as expected.
>
> Trying to mount at boot, however, causes the machine to go into single
> user mode.
>
> When the machine comes up, it does indeed pass mounting all of the
> local partitions before trying to mount the iSCSI target.
>
> However, it still blows up and enters single user mode. Upon entering
> single user mode, the additional NIC that connects to the SAN does
> appear to be configured and happy (I can ping the iSCSI host). You can
> see pictures of the screen before [1] and after [2] entering and
> exiting single user mode. In [3], you can see the machine does
> eventually hit a login prompt.
>
> Upon exiting single user mode, it then gives an error message that
> "SIOCSIFGATTR" is an invalid argument to ifconfig, before continuing
> to boot. It then says iscsid failed to start. This is visible in [2]
> and [3].
>
> Once actually logged in, it appears as though iscsid is dead in the
> water. However, if I remove the /etc/fstab entry, it boots as normal
> and then I can mount manually as expected. Very strange.
>
> Checking /var/log/daemon, this is what I see:
>
> Feb 20 18:59:28 elara iscsid[52173]: startup
> Feb 20 18:59:28 elara iscsid[52173]: fatal in iscsid: vscsi_open: Device busy
>
> iscsid won't reload, restart etc without removing the /etc/fstab entry and rebooting.
>
> [1] https://thalassa.fagg.id.au/IMG_6318.jpg
> [2] https://thalassa.fagg.id.au/IMG_6321.jpg
> [3] https://thalassa.fagg.id.au/IMG_6322.jpg
>

Looking at the screenshot it seems there is a timing issue. The fsck -N
tries to run before the disk was actually attached. I guess the iscsid
startup script should wait to ensure the disks are ready.
You could probably add a 'sleep 5' to rc_start() in /etc/rc.d/iscsid and
see if that helps. Ideally iscsictl should actually do the wait or fail.

--
:wq Claudio

Reply | Threaded
Open this post in threaded view
|

Re: iSCSI mount on boot causes the machine to go to single user mode

Ashton Fagg
Claudio Jeker <[hidden email]> writes:

> Looking at the screenshot it seems there is a timing issue. The fsck -N
> tries to run before the disk was actually attached. I guess the iscsid
> startup script should wait to ensure the disks are ready.
> You could probably add a 'sleep 5' to rc_start() in /etc/rc.d/iscsid and
> see if that helps. Ideally iscsictl should actually do the wait or fail.

I took a bit of a look through the code, and I've worked out an
(untested, beyond building) patch. I'll be able to try it later today
probably, but wondering if this looks reasonable?

Unless I misunderstand, it looks like what's happening is that
/dev/vscsi0 isn't ready when iscsid calls vscsi_open(). My proposed fix
is to add a wait loop to this (try 3 times, waiting 5 seconds between
each attempt).

I'll try it tonight once I can reboot the machine.


diff --git a/usr.sbin/iscsid/vscsi.c b/usr.sbin/iscsid/vscsi.c
index dc8d5ed67e0..25f3feba6ff 100644
--- a/usr.sbin/iscsid/vscsi.c
+++ b/usr.sbin/iscsid/vscsi.c
@@ -29,6 +29,7 @@
 #include <fcntl.h>
 #include <stdlib.h>
 #include <string.h>
+#include <unistd.h>
 
 #include "iscsid.h"
 #include "log.h"
@@ -52,11 +53,30 @@ void vscsi_fail(void *arg);
 void vscsi_dataout(struct connection *, struct scsi_task *, u_int32_t,
     size_t, size_t);
 
+#define  VSCSI_OPEN_WAIT  5
+#define  VSCSI_OPEN_TRIES 3
+
 void
 vscsi_open(char *dev)
 {
- if ((v.fd = open(dev, O_RDWR)) == -1)
- fatal("vscsi_open");
+ /* Sometimes, it seems we try to open before this is ready.
+   This builds in a little tolerance to that. */
+ int tries;
+
+ for (tries = 1; tries <= VSCSI_OPEN_TRIES; ++tries) {
+ if ((v.fd = open(dev, O_RDWR)) == -1) {
+ /* We can't try again, so fail. */
+ if (tries == VSCSI_OPEN_TRIES)
+ fatal("vscsi_open");
+ log_debug("vscsi_open: waiting...");
+ sleep(VSCSI_OPEN_WAIT);
+ } else {
+ /* That worked, so we're good! */
+ log_debug("vscsci_open: successful");
+ break;
+                }
+ }
+
 
  event_set(&v.ev, v.fd, EV_READ|EV_PERSIST, vscsi_dispatch, NULL);
  event_add(&v.ev, NULL);
Reply | Threaded
Open this post in threaded view
|

Re: iSCSI mount on boot causes the machine to go to single user mode

Claudio Jeker-3
On Tue, Feb 23, 2021 at 07:07:34AM -0500, Ashton Fagg wrote:

> Claudio Jeker <[hidden email]> writes:
>
> > Looking at the screenshot it seems there is a timing issue. The fsck -N
> > tries to run before the disk was actually attached. I guess the iscsid
> > startup script should wait to ensure the disks are ready.
> > You could probably add a 'sleep 5' to rc_start() in /etc/rc.d/iscsid and
> > see if that helps. Ideally iscsictl should actually do the wait or fail.
>
> I took a bit of a look through the code, and I've worked out an
> (untested, beyond building) patch. I'll be able to try it later today
> probably, but wondering if this looks reasonable?
>
> Unless I misunderstand, it looks like what's happening is that
> /dev/vscsi0 isn't ready when iscsid calls vscsi_open(). My proposed fix
> is to add a wait loop to this (try 3 times, waiting 5 seconds between
> each attempt).
>
> I'll try it tonight once I can reboot the machine.
>

I'm not sure. vscsi(4) is ready the moment the kernel starts init. So it
should not result in any problem for iscsid. What made you think this is
the issue?

My assumption is that the initial exchange to get the iscsi session going
with the target just takes a bit longer than the rc script takes to do the
fsck. Because of this I think a sleep may be a quick fix.

A proper fix would be to block iscsictl until the configured session is
up. The tricky bit is that at some point a timeout needs to trigger in
case the target is unavailable.

> diff --git a/usr.sbin/iscsid/vscsi.c b/usr.sbin/iscsid/vscsi.c
> index dc8d5ed67e0..25f3feba6ff 100644
> --- a/usr.sbin/iscsid/vscsi.c
> +++ b/usr.sbin/iscsid/vscsi.c
> @@ -29,6 +29,7 @@
>  #include <fcntl.h>
>  #include <stdlib.h>
>  #include <string.h>
> +#include <unistd.h>
>  
>  #include "iscsid.h"
>  #include "log.h"
> @@ -52,11 +53,30 @@ void vscsi_fail(void *arg);
>  void vscsi_dataout(struct connection *, struct scsi_task *, u_int32_t,
>      size_t, size_t);
>  
> +#define  VSCSI_OPEN_WAIT  5
> +#define  VSCSI_OPEN_TRIES 3
> +
>  void
>  vscsi_open(char *dev)
>  {
> - if ((v.fd = open(dev, O_RDWR)) == -1)
> - fatal("vscsi_open");
> + /* Sometimes, it seems we try to open before this is ready.
> +   This builds in a little tolerance to that. */
> + int tries;
> +
> + for (tries = 1; tries <= VSCSI_OPEN_TRIES; ++tries) {
> + if ((v.fd = open(dev, O_RDWR)) == -1) {
> + /* We can't try again, so fail. */
> + if (tries == VSCSI_OPEN_TRIES)
> + fatal("vscsi_open");
> + log_debug("vscsi_open: waiting...");
> + sleep(VSCSI_OPEN_WAIT);
> + } else {
> + /* That worked, so we're good! */
> + log_debug("vscsci_open: successful");
> + break;
> +                }
> + }
> +
>  
>   event_set(&v.ev, v.fd, EV_READ|EV_PERSIST, vscsi_dispatch, NULL);
>   event_add(&v.ev, NULL);


--
:wq Claudio

Reply | Threaded
Open this post in threaded view
|

Re: iSCSI mount on boot causes the machine to go to single user mode

Ashton Fagg
Claudio Jeker <[hidden email]> writes:

> I'm not sure. vscsi(4) is ready the moment the kernel starts init. So it
> should not result in any problem for iscsid. What made you think this is
> the issue?

Yeah, so in /var/log/daemon, you see this:

Feb 20 18:59:28 elara iscsid[52173]: startup
Feb 20 18:59:28 elara iscsid[52173]: fatal in iscsid: vscsi_open: Device busy

It appears it just dies there. iscsid never actually does anything
beyond that, looking at the log.

Hence my thinking it can't talk to /dev/vscsi0 when it wants to.

> My assumption is that the initial exchange to get the iscsi session going
> with the target just takes a bit longer than the rc script takes to do the
> fsck. Because of this I think a sleep may be a quick fix.

I will try the sleep as well.

> A proper fix would be to block iscsictl until the configured session is
> up. The tricky bit is that at some point a timeout needs to trigger in
> case the target is unavailable.

Reply | Threaded
Open this post in threaded view
|

Re: iSCSI mount on boot causes the machine to go to single user mode

Claudio Jeker-3
On Tue, Feb 23, 2021 at 07:39:17AM -0500, Ashton Fagg wrote:

> Claudio Jeker <[hidden email]> writes:
>
> > I'm not sure. vscsi(4) is ready the moment the kernel starts init. So it
> > should not result in any problem for iscsid. What made you think this is
> > the issue?
>
> Yeah, so in /var/log/daemon, you see this:
>
> Feb 20 18:59:28 elara iscsid[52173]: startup
> Feb 20 18:59:28 elara iscsid[52173]: fatal in iscsid: vscsi_open: Device busy
>
> It appears it just dies there. iscsid never actually does anything
> beyond that, looking at the log.
>
> Hence my thinking it can't talk to /dev/vscsi0 when it wants to.

Isn't that the 2nd iscsid that is started? IIRC your pictures showed that
once you exited single user mode that another iscsid was started (but
maybe I just interpreted the images wrong).
 
> > My assumption is that the initial exchange to get the iscsi session going
> > with the target just takes a bit longer than the rc script takes to do the
> > fsck. Because of this I think a sleep may be a quick fix.
>
> I will try the sleep as well.
>
> > A proper fix would be to block iscsictl until the configured session is
> > up. The tricky bit is that at some point a timeout needs to trigger in
> > case the target is unavailable.

--
:wq Claudio

Reply | Threaded
Open this post in threaded view
|

Re: iSCSI mount on boot causes the machine to go to single user mode

Ashton Fagg
Claudio Jeker <[hidden email]> writes:

> Isn't that the 2nd iscsid that is started? IIRC your pictures showed that
> once you exited single user mode that another iscsid was started (but
> maybe I just interpreted the images wrong).

Ooh, yeah actually you are probably right.

I also misread your initial email - you said "iscsictl should wait and
fail", I read it as "iscsid should wait and fail". ;-)

Sorry for the confusion. I'll experiment a bit and see if I can come up
with a patch. I should not code before coffee. :-)

Reply | Threaded
Open this post in threaded view
|

Re: iSCSI mount on boot causes the machine to go to single user mode

Ashton Fagg
Ashton Fagg <[hidden email]> writes:

> Ooh, yeah actually you are probably right.
>
> I also misread your initial email - you said "iscsictl should wait and
> fail", I read it as "iscsid should wait and fail". ;-)
>
> Sorry for the confusion. I'll experiment a bit and see if I can come up
> with a patch. I should not code before coffee. :-)

Yep, so just checked this out.

Adding "sleep 5" to the rc.d script does indeed bring about a clean
boot.

I re-did my experiments and yes, it appears that the second start of
iscsid fails with the device busy error - as expected, my diff does
nothing for that except make it fail 15 seconds later than without. :-)

The "sleep 5" thing is good enough for now, at least. I'd prefer to
*not* have to remember to fix that after every snapshot upgrade, so I'll
see if I can figure out the iscsictl fix.

- Ash

Reply | Threaded
Open this post in threaded view
|

Re: iSCSI mount on boot causes the machine to go to single user mode

Ashton Fagg
Ashton Fagg <[hidden email]> writes:

> Yep, so just checked this out.
>
> Adding "sleep 5" to the rc.d script does indeed bring about a clean
> boot.
>
> I re-did my experiments and yes, it appears that the second start of
> iscsid fails with the device busy error - as expected, my diff does
> nothing for that except make it fail 15 seconds later than without. :-)
>
> The "sleep 5" thing is good enough for now, at least. I'd prefer to
> *not* have to remember to fix that after every snapshot upgrade, so I'll
> see if I can figure out the iscsictl fix.
>
> - Ash
Attached is my first attempt at a "proper" solution. I haven't tested it
beyond building as I can't take my machine down for testing right now...

But, I'd appreciate a sanity check that I'm on the right track.

Thanks,

Ash


Reply | Threaded
Open this post in threaded view
|

Re: iSCSI mount on boot causes the machine to go to single user mode

Ashton Fagg
Ashton Fagg <[hidden email]> writes:

> Attached is my first attempt at a "proper" solution. I haven't tested it
> beyond building as I can't take my machine down for testing right now...
>
> But, I'd appreciate a sanity check that I'm on the right track.

Sorry for the noise. I think my mail client mangled the attachment, so
trying again.


diff --git a/usr.sbin/iscsictl/iscsictl.c b/usr.sbin/iscsictl/iscsictl.c
index 77f9c74abde..5fc326acca6 100644
--- a/usr.sbin/iscsictl/iscsictl.c
+++ b/usr.sbin/iscsictl/iscsictl.c
@@ -40,6 +40,8 @@ struct pdu *ctl_getpdu(char *, size_t);
 int ctl_sendpdu(int, struct pdu *);
 void show_config(struct ctrlmsghdr *, struct pdu *);
 void show_vscsi_stats(struct ctrlmsghdr *, struct pdu *);
+void             poll_and_wait(void);
+void             register_poll(struct ctrlmsghdr *, struct pdu *);
 
 char cbuf[CONTROL_READ_SIZE];
 
@@ -48,6 +50,14 @@ struct control {
  int fd;
 } control;
 
+struct poll_result {
+ u_int8_t   result;
+ int        attempts;
+} pr;
+
+#define POLL_DELAY     1
+#define POLL_ATTEMPTS  10
+
 __dead void
 usage(void)
 {
@@ -68,7 +78,7 @@ main (int argc, char* argv[])
  char *sockname = ISCSID_CONTROL;
  struct session_ctlcfg *s;
  struct iscsi_config *cf;
- int ch, val = 0;
+ int ch, poll = 0, val = 0;
 
  /* check flags */
  while ((ch = getopt(argc, argv, "f:s:")) != -1) {
@@ -135,6 +145,9 @@ main (int argc, char* argv[])
     &cf->initiator, sizeof(cf->initiator)) == -1)
  err(1, "control_compose");
  }
+
+ /* Reloading, so poll afterwards. */
+ poll = 1;
  SIMPLEQ_FOREACH(s, &cf->sessions, entry) {
  struct ctrldata cdv[3];
  bzero(cdv, sizeof(cdv));
@@ -174,6 +187,12 @@ main (int argc, char* argv[])
 
  run();
 
+ /* If we've reloaded, we probably should wait in case any new connections
+   need to come up (or fail). */
+ if (poll) {
+ poll_and_wait();
+ }
+
  close(control.fd);
 
  return (0);
@@ -229,6 +248,10 @@ run_command(struct pdu *pdu)
  case CTRL_INPROGRESS:
  printf("command in progress...\n");
  break;
+ case CTRL_SESS_POLL:
+ done = 1;
+ register_poll(cmh, pdu);
+ break;
  case CTRL_INITIATOR_CONFIG:
  case CTRL_SESSION_CONFIG:
  show_config(cmh, pdu);
@@ -383,3 +406,43 @@ show_vscsi_stats(struct ctrlmsghdr *cmh, struct pdu *pdu)
     vs->cnt_t2i_status[1],
     vs->cnt_t2i_status[2]);
 }
+
+void
+poll_and_wait(void)
+{
+ pr.result = (u_int8_t) 0;
+ printf("polling...");
+
+ for (pr.attempts = 1; pr.attempts <= POLL_ATTEMPTS; ++pr.attempts) {
+ if (control_compose(NULL, CTRL_SESS_POLL, NULL, 0) == -1)
+ err(1, "control_compose");
+
+ struct pdu *pdu;
+ while ((pdu = TAILQ_FIRST(&control.channel)) != NULL) {
+ run_command(pdu);
+ }
+
+ /* Poll says we are good to go. */
+ if (pr.result != 0) {
+ printf("ok!\n");
+ return;
+ }
+
+ /* Poll says we should wait... */
+ printf("%d, ", pr.attempts);
+ sleep(POLL_DELAY);
+ }
+
+ printf("..,timer exceeded.\n");
+}
+
+void
+register_poll(struct ctrlmsghdr *cmh, struct pdu *pdu)
+{
+ u_int8_t *r = &pr.result;
+
+ if (cmh->len[0] != sizeof(u_int8_t))
+ errx(1, "poll: bad size of response");
+
+ r = pdu_getbuf(pdu, NULL, 1);
+}
diff --git a/usr.sbin/iscsid/Makefile b/usr.sbin/iscsid/Makefile
index 7a62024e68b..bac59d934f8 100644
--- a/usr.sbin/iscsid/Makefile
+++ b/usr.sbin/iscsid/Makefile
@@ -2,7 +2,7 @@
 
 PROG= iscsid
 SRCS= connection.c control.c initiator.c iscsid.c log.c logmsg.c pdu.c \
- session.c task.c util.c vscsi.c
+ poll.c session.c task.c util.c vscsi.c
 
 MAN= iscsid.8
 
diff --git a/usr.sbin/iscsid/iscsid.c b/usr.sbin/iscsid/iscsid.c
index d3526e96363..61f831c3b24 100644
--- a/usr.sbin/iscsid/iscsid.c
+++ b/usr.sbin/iscsid/iscsid.c
@@ -211,6 +211,7 @@ iscsid_ctrl_dispatch(void *ch, struct pdu *pdu)
  struct initiator_config *ic;
  struct session_config *sc;
  struct session *s;
+ struct session_poll *p;
  int *valp;
 
  cmh = pdu_getbuf(pdu, NULL, 0);
@@ -304,6 +305,20 @@ iscsid_ctrl_dispatch(void *ch, struct pdu *pdu)
 
  control_compose(ch, CTRL_SUCCESS, NULL, 0);
  break;
+ case CTRL_SESS_POLL:
+ p = poll_alloc();
+
+ TAILQ_FOREACH(s, &initiator->sessions, entry) {
+ poll_session(p, s);
+ }
+
+ poll_finalize(p);
+
+ control_compose(ch, CTRL_SESS_POLL,
+    (void *) &(p->status), sizeof(u_int8_t));
+
+ poll_dealloc(p);
+ break;
  default:
  log_warnx("unknown control message type %d", cmh->type);
  control_compose(ch, CTRL_FAILURE, NULL, 0);
diff --git a/usr.sbin/iscsid/iscsid.h b/usr.sbin/iscsid/iscsid.h
index b43fb5dcd99..08d5f251a39 100644
--- a/usr.sbin/iscsid/iscsid.h
+++ b/usr.sbin/iscsid/iscsid.h
@@ -67,7 +67,7 @@ struct ctrldata {
 #define CTRL_LOG_VERBOSE 6
 #define CTRL_VSCSI_STATS 7
 #define CTRL_SHOW_SUM 8
-
+#define CTRL_SESS_POLL          9
 
 TAILQ_HEAD(session_head, session);
 TAILQ_HEAD(connection_head, connection);
@@ -251,6 +251,13 @@ struct session {
  int action;
 };
 
+struct session_poll {
+ u_int16_t session_count; /* Total number of configured sessions*/
+ u_int16_t init_count;    /* Number of sessions in init state */
+ u_int16_t running_count; /* Number of sessions in running state */
+ u_int8_t  status;        /* Status flag */
+};
+
 struct connection {
  struct event ev;
  struct event wev;
@@ -391,6 +398,12 @@ void vscsi_status(int, int, void *, size_t);
 void vscsi_event(unsigned long, u_int, u_int);
 struct vscsi_stats *vscsi_stats(void);
 
+/* Session polling */
+struct session_poll *poll_alloc(void);
+void    poll_dealloc(struct session_poll *);
+void    poll_session(struct session_poll *, struct session *);
+void    poll_finalize(struct session_poll *);
+
 /* logmsg.c */
 void log_hexdump(void *, size_t);
 void log_pdu(struct pdu *, int);
diff --git a/usr.sbin/iscsid/poll.c b/usr.sbin/iscsid/poll.c
new file mode 100644
index 00000000000..d3c79985d71
--- /dev/null
+++ b/usr.sbin/iscsid/poll.c
@@ -0,0 +1,80 @@
+/*
+ * Copyright (c) 2021 Dr Ashton Fagg <[hidden email]>
+ *
+ * Permission to use, copy, modify, and distribute this software for any
+ * purpose with or without fee is hereby granted, provided that the above
+ * copyright notice and this permission notice appear in all copies.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+ * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+ * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+ * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+ * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+ * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+ * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+ */
+
+#include <sys/types.h>
+#include <sys/socket.h>
+
+#include <event.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+
+#include "iscsid.h"
+#include "log.h"
+
+struct session_poll
+*poll_alloc(void)
+{
+ struct session_poll *p;
+
+ if (!(p = calloc(1, sizeof(*p))))
+ fatal("poll_alloc failed.");
+
+ p->session_count = (u_int16_t) 0;
+ p->init_count    = (u_int16_t) 0;
+ p->running_count = (u_int16_t) 0;
+
+ return p;
+}
+
+
+void
+poll_dealloc(struct session_poll *p)
+{
+ free(p);
+}
+
+void
+poll_session(struct session_poll *p, struct session *s)
+{
+ if (!s)
+ fatal("poll_session failed: invalid session");
+
+ ++(p->session_count);
+
+ /* If SESS_RUNNING, this determines the session has either
+           been brought up successfully, or has failed. Either way,
+   we aren't waiting on it. */
+ if (s->state & SESS_RUNNING)
+ ++(p->running_count);
+ /* Otherwise, it is in SESS_INIT. These need to be waited on. */
+ else if (s->state & SESS_RUNNING)
+ ++(p->init_count);
+ else
+ fatal("poll_session: unknown state.");
+}
+
+void
+poll_finalize(struct session_poll *p)
+{
+ /* Perform final book keeping to determine status.
+         *
+ * status will be non-zero if the number of sessions that are
+         * in states other than SESS_INIT is equal to the total number
+ * of configured sessions. This indicates to the poller that
+ * we are not waiting for something to complete. */
+ p->status = (u_int8_t) (p->session_count == p->running_count);
+}
Reply | Threaded
Open this post in threaded view
|

Re: iSCSI mount on boot causes the machine to go to single user mode

Claudio Jeker-3
On Wed, Feb 24, 2021 at 08:03:21PM -0500, Ashton Fagg wrote:

> Ashton Fagg <[hidden email]> writes:
>
> > Attached is my first attempt at a "proper" solution. I haven't tested it
> > beyond building as I can't take my machine down for testing right now...
> >
> > But, I'd appreciate a sanity check that I'm on the right track.
>
> Sorry for the noise. I think my mail client mangled the attachment, so
> trying again.
>

> diff --git a/usr.sbin/iscsictl/iscsictl.c b/usr.sbin/iscsictl/iscsictl.c
> index 77f9c74abde..5fc326acca6 100644
> --- a/usr.sbin/iscsictl/iscsictl.c
> +++ b/usr.sbin/iscsictl/iscsictl.c
> @@ -40,6 +40,8 @@ struct pdu *ctl_getpdu(char *, size_t);
>  int ctl_sendpdu(int, struct pdu *);
>  void show_config(struct ctrlmsghdr *, struct pdu *);
>  void show_vscsi_stats(struct ctrlmsghdr *, struct pdu *);
> +void             poll_and_wait(void);
> +void             register_poll(struct ctrlmsghdr *, struct pdu *);
>  
>  char cbuf[CONTROL_READ_SIZE];
>  
> @@ -48,6 +50,14 @@ struct control {
>   int fd;
>  } control;
>  
> +struct poll_result {
> + u_int8_t   result;
> + int        attempts;
> +} pr;
> +
> +#define POLL_DELAY     1
> +#define POLL_ATTEMPTS  10
> +
>  __dead void
>  usage(void)
>  {
> @@ -68,7 +78,7 @@ main (int argc, char* argv[])
>   char *sockname = ISCSID_CONTROL;
>   struct session_ctlcfg *s;
>   struct iscsi_config *cf;
> - int ch, val = 0;
> + int ch, poll = 0, val = 0;
>  
>   /* check flags */
>   while ((ch = getopt(argc, argv, "f:s:")) != -1) {
> @@ -135,6 +145,9 @@ main (int argc, char* argv[])
>      &cf->initiator, sizeof(cf->initiator)) == -1)
>   err(1, "control_compose");
>   }
> +
> + /* Reloading, so poll afterwards. */
> + poll = 1;
>   SIMPLEQ_FOREACH(s, &cf->sessions, entry) {
>   struct ctrldata cdv[3];
>   bzero(cdv, sizeof(cdv));
> @@ -174,6 +187,12 @@ main (int argc, char* argv[])
>  
>   run();
>  
> + /* If we've reloaded, we probably should wait in case any new connections
> +   need to come up (or fail). */
> + if (poll) {
> + poll_and_wait();
> + }
> +
>   close(control.fd);
>  
>   return (0);
> @@ -229,6 +248,10 @@ run_command(struct pdu *pdu)
>   case CTRL_INPROGRESS:
>   printf("command in progress...\n");
>   break;
> + case CTRL_SESS_POLL:
> + done = 1;
> + register_poll(cmh, pdu);
> + break;
>   case CTRL_INITIATOR_CONFIG:
>   case CTRL_SESSION_CONFIG:
>   show_config(cmh, pdu);
> @@ -383,3 +406,43 @@ show_vscsi_stats(struct ctrlmsghdr *cmh, struct pdu *pdu)
>      vs->cnt_t2i_status[1],
>      vs->cnt_t2i_status[2]);
>  }
> +
> +void
> +poll_and_wait(void)
> +{
> + pr.result = (u_int8_t) 0;
> + printf("polling...");
> +
> + for (pr.attempts = 1; pr.attempts <= POLL_ATTEMPTS; ++pr.attempts) {
> + if (control_compose(NULL, CTRL_SESS_POLL, NULL, 0) == -1)
> + err(1, "control_compose");
> +
> + struct pdu *pdu;
> + while ((pdu = TAILQ_FIRST(&control.channel)) != NULL) {
> + run_command(pdu);
> + }
> +
> + /* Poll says we are good to go. */
> + if (pr.result != 0) {
> + printf("ok!\n");
> + return;
> + }
> +
> + /* Poll says we should wait... */
> + printf("%d, ", pr.attempts);
> + sleep(POLL_DELAY);
> + }
> +
> + printf("..,timer exceeded.\n");
> +}
> +
> +void
> +register_poll(struct ctrlmsghdr *cmh, struct pdu *pdu)
> +{
> + u_int8_t *r = &pr.result;
> +
> + if (cmh->len[0] != sizeof(u_int8_t))
> + errx(1, "poll: bad size of response");
> +
> + r = pdu_getbuf(pdu, NULL, 1);
> +}
> diff --git a/usr.sbin/iscsid/Makefile b/usr.sbin/iscsid/Makefile
> index 7a62024e68b..bac59d934f8 100644
> --- a/usr.sbin/iscsid/Makefile
> +++ b/usr.sbin/iscsid/Makefile
> @@ -2,7 +2,7 @@
>  
>  PROG= iscsid
>  SRCS= connection.c control.c initiator.c iscsid.c log.c logmsg.c pdu.c \
> - session.c task.c util.c vscsi.c
> + poll.c session.c task.c util.c vscsi.c
>  
>  MAN= iscsid.8
>  
> diff --git a/usr.sbin/iscsid/iscsid.c b/usr.sbin/iscsid/iscsid.c
> index d3526e96363..61f831c3b24 100644
> --- a/usr.sbin/iscsid/iscsid.c
> +++ b/usr.sbin/iscsid/iscsid.c
> @@ -211,6 +211,7 @@ iscsid_ctrl_dispatch(void *ch, struct pdu *pdu)
>   struct initiator_config *ic;
>   struct session_config *sc;
>   struct session *s;
> + struct session_poll *p;
>   int *valp;
>  
>   cmh = pdu_getbuf(pdu, NULL, 0);
> @@ -304,6 +305,20 @@ iscsid_ctrl_dispatch(void *ch, struct pdu *pdu)
>  
>   control_compose(ch, CTRL_SUCCESS, NULL, 0);
>   break;
> + case CTRL_SESS_POLL:
> + p = poll_alloc();
> +
> + TAILQ_FOREACH(s, &initiator->sessions, entry) {
> + poll_session(p, s);
> + }
> +
> + poll_finalize(p);
> +
> + control_compose(ch, CTRL_SESS_POLL,
> +    (void *) &(p->status), sizeof(u_int8_t));

No need to type-cast, no need for the () around &(p->status). I prefer to
use sizeof(p->status) since that will adapt properly.

> +
> + poll_dealloc(p);

I think you could just use a stack variable for p here instead of using
alloc and dealloc. The struct is small.

> + break;
>   default:
>   log_warnx("unknown control message type %d", cmh->type);
>   control_compose(ch, CTRL_FAILURE, NULL, 0);
> diff --git a/usr.sbin/iscsid/iscsid.h b/usr.sbin/iscsid/iscsid.h
> index b43fb5dcd99..08d5f251a39 100644
> --- a/usr.sbin/iscsid/iscsid.h
> +++ b/usr.sbin/iscsid/iscsid.h
> @@ -67,7 +67,7 @@ struct ctrldata {
>  #define CTRL_LOG_VERBOSE 6
>  #define CTRL_VSCSI_STATS 7
>  #define CTRL_SHOW_SUM 8
> -
> +#define CTRL_SESS_POLL          9
>  
>  TAILQ_HEAD(session_head, session);
>  TAILQ_HEAD(connection_head, connection);
> @@ -251,6 +251,13 @@ struct session {
>   int action;
>  };
>  
> +struct session_poll {
> + u_int16_t session_count; /* Total number of configured sessions*/
> + u_int16_t init_count;    /* Number of sessions in init state */
> + u_int16_t running_count; /* Number of sessions in running state */
> + u_int8_t  status;        /* Status flag */
> +};
> +

I would just use int for all the variables here. Just keep it simple.

>  struct connection {
>   struct event ev;
>   struct event wev;
> @@ -391,6 +398,12 @@ void vscsi_status(int, int, void *, size_t);
>  void vscsi_event(unsigned long, u_int, u_int);
>  struct vscsi_stats *vscsi_stats(void);
>  
> +/* Session polling */
> +struct session_poll *poll_alloc(void);
> +void    poll_dealloc(struct session_poll *);
> +void    poll_session(struct session_poll *, struct session *);
> +void    poll_finalize(struct session_poll *);
> +
>  /* logmsg.c */
>  void log_hexdump(void *, size_t);
>  void log_pdu(struct pdu *, int);
> diff --git a/usr.sbin/iscsid/poll.c b/usr.sbin/iscsid/poll.c
> new file mode 100644
> index 00000000000..d3c79985d71
> --- /dev/null
> +++ b/usr.sbin/iscsid/poll.c
> @@ -0,0 +1,80 @@
> +/*
> + * Copyright (c) 2021 Dr Ashton Fagg <[hidden email]>
> + *
> + * Permission to use, copy, modify, and distribute this software for any
> + * purpose with or without fee is hereby granted, provided that the above
> + * copyright notice and this permission notice appear in all copies.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
> + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
> + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
> + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
> + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
> + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
> + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
> + */
> +
> +#include <sys/types.h>
> +#include <sys/socket.h>
> +
> +#include <event.h>
> +#include <stdlib.h>
> +#include <string.h>
> +#include <unistd.h>
> +
> +#include "iscsid.h"
> +#include "log.h"
> +
> +struct session_poll
> +*poll_alloc(void)
> +{
> + struct session_poll *p;
> +
> + if (!(p = calloc(1, sizeof(*p))))
> + fatal("poll_alloc failed.");
> +

Calloc already sets all memory to 0. So the code below is not needed.

> + p->session_count = (u_int16_t) 0;
> + p->init_count    = (u_int16_t) 0;
> + p->running_count = (u_int16_t) 0;

Also there is no need to typecast here. In general try to avoid typecasts
since they can hide hard to find bugs.

> +
> + return p;
> +}
> +
> +
> +void
> +poll_dealloc(struct session_poll *p)
> +{
> + free(p);
> +}

As mentioned above I would just use a stack variable
        struct session_poll p = { 0 };
instead of dynamically allocate this.

> +void
> +poll_session(struct session_poll *p, struct session *s)
> +{
> + if (!s)
> + fatal("poll_session failed: invalid session");
> +
> + ++(p->session_count);

No need for the ()

> +
> + /* If SESS_RUNNING, this determines the session has either
> +           been brought up successfully, or has failed. Either way,
> +   we aren't waiting on it. */
> + if (s->state & SESS_RUNNING)
> + ++(p->running_count);
> + /* Otherwise, it is in SESS_INIT. These need to be waited on. */
> + else if (s->state & SESS_RUNNING)
> + ++(p->init_count);

The two if statements are the same. I guess you want SESS_INIT in the
second.

> + else
> + fatal("poll_session: unknown state.");
> +}
> +
> +void
> +poll_finalize(struct session_poll *p)
> +{
> + /* Perform final book keeping to determine status.
> +         *
> + * status will be non-zero if the number of sessions that are
> +         * in states other than SESS_INIT is equal to the total number
> + * of configured sessions. This indicates to the poller that
> + * we are not waiting for something to complete. */

Indents are strange here. Guess tab vs spaces.

> + p->status = (u_int8_t) (p->session_count == p->running_count);
> +}

I actually wonder if this code should just return the various counts to
iscsictl so it could display the status (if requested).

I think this is moving in the right direction.
--
:wq Claudio