Re: How to use proot?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: How to use proot?

Xiyue Deng
Resend to mailing list which I dropped accidentally.

Xiyue Deng <[hidden email]> writes:

> Hi Marc,
>
> Marc Espie <[hidden email]> writes:
>
>> On Fri, Dec 27, 2019 at 06:35:34PM -0800, Xiyue Deng wrote:
>>> Hi,
>>>
>>> I'm trying to set up a chroot for dpb using proot, but it looks like I'm
>>> doing something wrong and nothing has been created in the chroot
>>> directory.  According to proot man page the following command should be
>>> sufficient, but I got the following outputs and nothing happens in /build:
>>>
>>> --------8<--------
>>> $ sudo ./proot -B /build
>> Why are you still using sudo and not doas ?
>
> No particular reason, just to be able to run my other scripts from Linux.
>
>> You're not saying if you're running current.
>>
>
> Sorry forgot to mention.  I'm running 6.6-stable on mips64el/loongson.
>
>>> Password:
>>> loguser: _pbuild
>>> fetchuser: _pfetch
>>> builduser: _pbuild
>>> PORTSDIR=/usr/ports
>>> DISTDIR=/usr/ports/distfiles
>>> WRKOBJDIR=/usr/ports/pobj
>>> LOCKDIR=/usr/ports/pobj/locks
>>> LOGDIR=/usr/ports/logs
>>> PACKAGE_REPOSITORY=/usr/ports/packages
>>> PLIST_REPOSITORY=/usr/ports/plist
>>> Couldn't find mountpoint for /build ???
>>> Running locate: ok
>>> --------8<--------
>>>
>>> It looks like it treats /build as a mountpoint, but what if I just need
>>> a local chroot?  I wonder what is the correct way to use proot.  Thanks!
>>
>> No it doesn't. It's definitely not what it says.
>> The actual code will try to figure out on which mountpoint /build is located,
>> by calling dirname() until it doesn't change any more.
>>
>> How about giving us the output of mount as well ?
>
> I didn't mount /build, it's just a normal directory.

signature.asc (847 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: How to use proot?

Xiyue Deng
(Adding mailing list back to CC which I accidentally dropped in my last mail.)

Marc Espie <[hidden email]> writes:

> On Sat, Dec 28, 2019 at 03:02:59PM -0800, Xiyue Deng wrote:
>> > No it doesn't. It's definitely not what it says.
>> > The actual code will try to figure out on which mountpoint /build is located,
>> > by calling dirname() until it doesn't change any more.
>> >
>> > How about giving us the output of mount as well ?
>>
>> I didn't mount /build, it's just a normal directory.
>
> How about givins us the output of mount ?
>
> I'm not saying it's not a normal directory, I'm saying that proot runs
> "mount" to figure out where /build is located, and somehow in your
> configuration, it can't. So I'd like to see the output of mount.
Sure thing.  It's not very interesting and looks like below:

$ mount
/dev/wd0a on / type ffs (local, wxallowed, softdep)

(So basically my whole disk is mounted on /)

signature.asc (847 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: How to use proot?

Xiyue Deng
Marc Espie <[hidden email]> writes:

> On Sun, Dec 29, 2019 at 02:21:21AM -0800, Xiyue Deng wrote:
>> Sure thing.  It's not very interesting and looks like below:
>>
>> $ mount
>> /dev/wd0a on / type ffs (local, wxallowed, softdep)
>>
>> (So basically my whole disk is mounted on /)
>
> ... and there was a bug in my logic because I never got a proot straight
> mounted under /. It's now fixed.
>
>
> Index: proot
> ===================================================================
> RCS file: /cvs/ports/infrastructure/bin/proot,v
> retrieving revision 1.66
> diff -u -p -r1.66 proot
> --- proot 6 Dec 2019 13:40:46 -0000 1.66
> +++ proot 29 Dec 2019 10:34:54 -0000
> @@ -474,10 +474,15 @@ sub check_mountpoint
>   }
>   }
>   my $mnt = $root;
> - do {{
> + while (1) {
>   if (!defined $dev->{$mnt}) {
> - $mnt = dirname($mnt);
> - next;
> + my $up = dirname($mnt);
> + if ($mnt eq $up) {
> + last;
> + } else {
> + $mnt = $up;
> + next;
> + }
>   }
>   $state->errsay("#1 is under #2 which is nodev", $root, $mnt)
>      if $dev->{$mnt} == 0;
> @@ -485,8 +490,8 @@ sub check_mountpoint
>      $root, $mnt) if $nosuid->{$mnt} == 0;
>   $state->errsay("#1 is under #2 which does not have wxallowed",
>      $root, $mnt) if $wx->{$mnt} == 0;
> - return;
> - }} while ($mnt ne dirname($mnt));
> + return;
> + }
>   $state->fatal_error("Couldn't find mountpoint for #1 ???", $root);
>  }
>  
It's working now!  Thanks Marc for the quick fix!

signature.asc (847 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: How to use proot?

Marc Espie-2
In reply to this post by Xiyue Deng
On Sun, Dec 29, 2019 at 02:21:21AM -0800, Xiyue Deng wrote:

> (Adding mailing list back to CC which I accidentally dropped in my last mail.)
>
> Marc Espie <[hidden email]> writes:
>
> > On Sat, Dec 28, 2019 at 03:02:59PM -0800, Xiyue Deng wrote:
> >> > No it doesn't. It's definitely not what it says.
> >> > The actual code will try to figure out on which mountpoint /build is located,
> >> > by calling dirname() until it doesn't change any more.
> >> >
> >> > How about giving us the output of mount as well ?
> >>
> >> I didn't mount /build, it's just a normal directory.
> >
> > How about givins us the output of mount ?
> >
> > I'm not saying it's not a normal directory, I'm saying that proot runs
> > "mount" to figure out where /build is located, and somehow in your
> > configuration, it can't. So I'd like to see the output of mount.
>
> Sure thing.  It's not very interesting and looks like below:
>
> $ mount
> /dev/wd0a on / type ffs (local, wxallowed, softdep)
>
> (So basically my whole disk is mounted on /)

So it got fixed, once Xiyue sent me the output of mount.
It was a stupid bug, as expected.

(In general, when a developer asks you for something specific, just do it,
don't presume. Most bugs are stupid, and the only way to fix them is to
figure out the wrong assumption... in that specific case, the loop to figure
out where a directory was mount'd was just plain wrong)

Reply | Threaded
Open this post in threaded view
|

Re: How to use proot?

Xiyue Deng
In reply to this post by Xiyue Deng
Xiyue Deng <[hidden email]> writes:

> Marc Espie <[hidden email]> writes:
>
>> On Sun, Dec 29, 2019 at 02:21:21AM -0800, Xiyue Deng wrote:
>>> Sure thing.  It's not very interesting and looks like below:
>>>
>>> $ mount
>>> /dev/wd0a on / type ffs (local, wxallowed, softdep)
>>>
>>> (So basically my whole disk is mounted on /)
>>
>> ... and there was a bug in my logic because I never got a proot straight
>> mounted under /. It's now fixed.
>>
>>
>> Index: proot
>> ===================================================================
>> RCS file: /cvs/ports/infrastructure/bin/proot,v
>> retrieving revision 1.66
>> diff -u -p -r1.66 proot
>> --- proot 6 Dec 2019 13:40:46 -0000 1.66
>> +++ proot 29 Dec 2019 10:34:54 -0000
>> @@ -474,10 +474,15 @@ sub check_mountpoint
>>   }
>>   }
>>   my $mnt = $root;
>> - do {{
>> + while (1) {
>>   if (!defined $dev->{$mnt}) {
>> - $mnt = dirname($mnt);
>> - next;
>> + my $up = dirname($mnt);
>> + if ($mnt eq $up) {
>> + last;
>> + } else {
>> + $mnt = $up;
>> + next;
>> + }
>>   }
>>   $state->errsay("#1 is under #2 which is nodev", $root, $mnt)
>>      if $dev->{$mnt} == 0;
>> @@ -485,8 +490,8 @@ sub check_mountpoint
>>      $root, $mnt) if $nosuid->{$mnt} == 0;
>>   $state->errsay("#1 is under #2 which does not have wxallowed",
>>      $root, $mnt) if $wx->{$mnt} == 0;
>> - return;
>> - }} while ($mnt ne dirname($mnt));
>> + return;
>> + }
>>   $state->fatal_error("Couldn't find mountpoint for #1 ???", $root);
>>  }
>>  
>
> It's working now!  Thanks Marc for the quick fix!
A follow up question: It seems dpb still cannot use the created chroot
by default.  I created the chroot using the following command as
suggested in the man page:

,----
| $ proot -B /build actions=unpopulate_light check_symlinks
`----

The error messages look like below:

,----
| $ sudo /usr/ports/infrastructure/bin/dpb -B /build -P ~/localports -DCOLOR=1
| Password:
| *** Parse error in /: Could not find /usr/ports/infrastructure/mk/bsd.port.mk (<bsd.port.mk>:5)
| dpb: Can't obtain vital information from the ports tree
| Usage: dpb [-acemqrRsuUvx] [-A arch] [-B chroot] [-C plist] [-f m] [-F m]
|        [-I pathlist] [-J p] [-j n] [-p parallel] [-P pathlist] [-h hosts]
|        [-L logdir] [-l lockdir] [-b log] [-M threshold] [-X pathlist]
|        [pathlist ...]
`----

I checked the chroot under /build and it seems
/build/usr/ports/infrastructure does not exist.  Maybe the contents of
/usr/ports/infrastructure should be copied over as well? Or am I missing
some options in the dpb command?

signature.asc (847 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: How to use proot?

Marc Espie-2
On Sun, Dec 29, 2019 at 05:46:04AM -0800, Xiyue Deng wrote:

> Xiyue Deng <[hidden email]> writes:
>
> > Marc Espie <[hidden email]> writes:
> >
> >> On Sun, Dec 29, 2019 at 02:21:21AM -0800, Xiyue Deng wrote:
> >>> Sure thing.  It's not very interesting and looks like below:
> >>>
> >>> $ mount
> >>> /dev/wd0a on / type ffs (local, wxallowed, softdep)
> >>>
> >>> (So basically my whole disk is mounted on /)
> >>
> >> ... and there was a bug in my logic because I never got a proot straight
> >> mounted under /. It's now fixed.
> >>
> >>
> >> Index: proot
> >> ===================================================================
> >> RCS file: /cvs/ports/infrastructure/bin/proot,v
> >> retrieving revision 1.66
> >> diff -u -p -r1.66 proot
> >> --- proot 6 Dec 2019 13:40:46 -0000 1.66
> >> +++ proot 29 Dec 2019 10:34:54 -0000
> >> @@ -474,10 +474,15 @@ sub check_mountpoint
> >>   }
> >>   }
> >>   my $mnt = $root;
> >> - do {{
> >> + while (1) {
> >>   if (!defined $dev->{$mnt}) {
> >> - $mnt = dirname($mnt);
> >> - next;
> >> + my $up = dirname($mnt);
> >> + if ($mnt eq $up) {
> >> + last;
> >> + } else {
> >> + $mnt = $up;
> >> + next;
> >> + }
> >>   }
> >>   $state->errsay("#1 is under #2 which is nodev", $root, $mnt)
> >>      if $dev->{$mnt} == 0;
> >> @@ -485,8 +490,8 @@ sub check_mountpoint
> >>      $root, $mnt) if $nosuid->{$mnt} == 0;
> >>   $state->errsay("#1 is under #2 which does not have wxallowed",
> >>      $root, $mnt) if $wx->{$mnt} == 0;
> >> - return;
> >> - }} while ($mnt ne dirname($mnt));
> >> + return;
> >> + }
> >>   $state->fatal_error("Couldn't find mountpoint for #1 ???", $root);
> >>  }
> >>  
> >
> > It's working now!  Thanks Marc for the quick fix!
>
> A follow up question: It seems dpb still cannot use the created chroot
> by default.  I created the chroot using the following command as
> suggested in the man page:
>
> ,----
> | $ proot -B /build actions=unpopulate_light check_symlinks
> `----
>
> The error messages look like below:
>
> ,----
> | $ sudo /usr/ports/infrastructure/bin/dpb -B /build -P ~/localports -DCOLOR=1
> | Password:
> | *** Parse error in /: Could not find /usr/ports/infrastructure/mk/bsd.port.mk (<bsd.port.mk>:5)
> | dpb: Can't obtain vital information from the ports tree
> | Usage: dpb [-acemqrRsuUvx] [-A arch] [-B chroot] [-C plist] [-f m] [-F m]
> |        [-I pathlist] [-J p] [-j n] [-p parallel] [-P pathlist] [-h hosts]
> |        [-L logdir] [-l lockdir] [-b log] [-M threshold] [-X pathlist]
> |        [pathlist ...]
> `----
>
> I checked the chroot under /build and it seems
> /build/usr/ports/infrastructure does not exist.  Maybe the contents of
> /usr/ports/infrastructure should be copied over as well? Or am I missing
> some options in the dpb command?

By default, proot does not take care of checking out the ports tree.

You either do it manually through cvs, or you use the checkout_ports
or the copy_ports action

Reply | Threaded
Open this post in threaded view
|

Re: How to use proot?

Xiyue Deng
Marc Espie <[hidden email]> writes:

> On Sun, Dec 29, 2019 at 05:46:04AM -0800, Xiyue Deng wrote:
>> Xiyue Deng <[hidden email]> writes:
>>
>> > Marc Espie <[hidden email]> writes:
>> >
>> >> On Sun, Dec 29, 2019 at 02:21:21AM -0800, Xiyue Deng wrote:
>> >>> Sure thing.  It's not very interesting and looks like below:
>> >>>
>> >>> $ mount
>> >>> /dev/wd0a on / type ffs (local, wxallowed, softdep)
>> >>>
>> >>> (So basically my whole disk is mounted on /)
>> >>
>> >> ... and there was a bug in my logic because I never got a proot straight
>> >> mounted under /. It's now fixed.
>> >>
>> >>
>> >> Index: proot
>> >> ===================================================================
>> >> RCS file: /cvs/ports/infrastructure/bin/proot,v
>> >> retrieving revision 1.66
>> >> diff -u -p -r1.66 proot
>> >> --- proot 6 Dec 2019 13:40:46 -0000 1.66
>> >> +++ proot 29 Dec 2019 10:34:54 -0000
>> >> @@ -474,10 +474,15 @@ sub check_mountpoint
>> >>   }
>> >>   }
>> >>   my $mnt = $root;
>> >> - do {{
>> >> + while (1) {
>> >>   if (!defined $dev->{$mnt}) {
>> >> - $mnt = dirname($mnt);
>> >> - next;
>> >> + my $up = dirname($mnt);
>> >> + if ($mnt eq $up) {
>> >> + last;
>> >> + } else {
>> >> + $mnt = $up;
>> >> + next;
>> >> + }
>> >>   }
>> >>   $state->errsay("#1 is under #2 which is nodev", $root, $mnt)
>> >>      if $dev->{$mnt} == 0;
>> >> @@ -485,8 +490,8 @@ sub check_mountpoint
>> >>      $root, $mnt) if $nosuid->{$mnt} == 0;
>> >>   $state->errsay("#1 is under #2 which does not have wxallowed",
>> >>      $root, $mnt) if $wx->{$mnt} == 0;
>> >> - return;
>> >> - }} while ($mnt ne dirname($mnt));
>> >> + return;
>> >> + }
>> >>   $state->fatal_error("Couldn't find mountpoint for #1 ???", $root);
>> >>  }
>> >>  
>> >
>> > It's working now!  Thanks Marc for the quick fix!
>>
>> A follow up question: It seems dpb still cannot use the created chroot
>> by default.  I created the chroot using the following command as
>> suggested in the man page:
>>
>> ,----
>> | $ proot -B /build actions=unpopulate_light check_symlinks
>> `----
>>
>> The error messages look like below:
>>
>> ,----
>> | $ sudo /usr/ports/infrastructure/bin/dpb -B /build -P ~/localports -DCOLOR=1
>> | Password:
>> | *** Parse error in /: Could not find /usr/ports/infrastructure/mk/bsd.port.mk (<bsd.port.mk>:5)
>> | dpb: Can't obtain vital information from the ports tree
>> | Usage: dpb [-acemqrRsuUvx] [-A arch] [-B chroot] [-C plist] [-f m] [-F m]
>> |        [-I pathlist] [-J p] [-j n] [-p parallel] [-P pathlist] [-h hosts]
>> |        [-L logdir] [-l lockdir] [-b log] [-M threshold] [-X pathlist]
>> |        [pathlist ...]
>> `----
>>
>> I checked the chroot under /build and it seems
>> /build/usr/ports/infrastructure does not exist.  Maybe the contents of
>> /usr/ports/infrastructure should be copied over as well? Or am I missing
>> some options in the dpb command?
>
> By default, proot does not take care of checking out the ports tree.
>
> You either do it manually through cvs, or you use the checkout_ports
> or the copy_ports action
Thanks Marc!  "copy_ports" did the trick and dpb is working with the
new chroot now.

However "copy_ports" is kinda slow - the whole process now takes ~30min
compared to ~2min without copy_ports.  I wonder is there a way to bind
mount directory under the chroot (similar to bind-mount in pbuilder[1])
to make this process faster?

[1] https://manpages.debian.org/stretch-backports/pbuilder/pbuilderrc.5.en.html

signature.asc (847 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: How to use proot?

Marc Espie-2
Just have your ports tree checked out under your mount point.
Next time it will be much faster ;)

We're still shackled to our slow ffs. Copying files does take time.

Reply | Threaded
Open this post in threaded view
|

Re: How to use proot?

Xiyue Deng
(Adding [hidden email] back to CC.)

Marc Espie <[hidden email]> writes:

> Just have your ports tree checked out under your mount point.
> Next time it will be much faster ;)

Unfortunately my loongson box is extremely slow in both CPU and disk
performances and a CVS update usually takes same amount of time.  But
it should be faster on more modern systems.

>
> We're still shackled to our slow ffs. Copying files does take time.

Hope there will be some work to improve FFS performance in the future.

signature.asc (847 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: How to use proot?

Marc Espie-2
On Mon, Dec 30, 2019 at 01:11:51PM -0800, Xiyue Deng wrote:

> (Adding [hidden email] back to CC.)
>
> Marc Espie <[hidden email]> writes:
>
> > Just have your ports tree checked out under your mount point.
> > Next time it will be much faster ;)
>
> Unfortunately my loongson box is extremely slow in both CPU and disk
> performances and a CVS update usually takes same amount of time.  But
> it should be faster on more modern systems.

Do you have an amd64 box around ? using it as an NFS server might be
less painful for some things...

Reply | Threaded
Open this post in threaded view
|

Re: How to use proot?

Xiyue Deng
Marc Espie <[hidden email]> writes:

> On Mon, Dec 30, 2019 at 01:11:51PM -0800, Xiyue Deng wrote:
>> (Adding [hidden email] back to CC.)
>>
>> Marc Espie <[hidden email]> writes:
>>
>> > Just have your ports tree checked out under your mount point.
>> > Next time it will be much faster ;)
>>
>> Unfortunately my loongson box is extremely slow in both CPU and disk
>> performances and a CVS update usually takes same amount of time.  But
>> it should be faster on more modern systems.
>
> Do you have an amd64 box around ? using it as an NFS server might be
> less painful for some things...
Currently no.  But it's OK as I'm not using the Loongson box for
anything serious yet, just try to see how well OpenBSD works on it as
it's currently the only OS that officially support Loongson that's not
ancient, and so far it's much better than 6.5.

signature.asc (847 bytes) Download Attachment