pkg_add vs. ssh

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

pkg_add vs. ssh

Marc Espie-2
I think I've finally managed to get pkg_add working over ssh.
This turned out to be more complicated than I expected, so it warrants
a small explanation.

Basically, plain scp won't work correctly, because you don't control
when the fork'ed scp will die (well, you control when they die, but
the connection spawns more stuff, and that stuff tends to hang around)
=> big pkg_add operations, like -u, will tend to start grabbing lots of
files just to get to the PLIST, and abort the connection soon afterwards.

After much head scratching, I resorted to doing something rsync-like:
have just an ssh pipe, and communicate needed stuff between client and
server.

Since we're using perl, instead of needing a local copy of rsync on the
server, we can be much more devious: we just assume perl, and transfer
the server script on the pipe, first thing (using perl -x).

Then the client just asks for package listings (LIST directory), for
package contents (GET filename), or for early stop of a command (ABORT).

There was still one big problem left, which is that pkg_add doesn't
directly run the client, but runs a gunzip process which runs the client:
killing the gunzip doesn't ensure timely demise of the client process,
and I was left with a frequent deadlock, since the old client tended to
swallow the ABORTED answer from the server.

After more head scratching, I ended up using signal USR1 to communicate
between pkg_add and its client grand-child, to make sure this one was
done reading from the ssh connection before I did the ABORT and fired
the new connection.

The new code seems to work just fine so fine just far.

Reply | Threaded
Open this post in threaded view
|

Re: pkg_add vs. ssh

Marc Espie-2
On Tue, Mar 07, 2006 at 08:05:08PM +0100, Marc Espie wrote:

> There was still one big problem left, which is that pkg_add doesn't
> directly run the client, but runs a gunzip process which runs the client:
> killing the gunzip doesn't ensure timely demise of the client process,
> and I was left with a frequent deadlock, since the old client tended to
> swallow the ABORTED answer from the server.
>
> After more head scratching, I ended up using signal USR1 to communicate
> between pkg_add and its client grand-child, to make sure this one was
> done reading from the ssh connection before I did the ABORT and fired
> the new connection.
>
> The new code seems to work just fine so fine just far.

As I suspected, this part was too complicated, and susceptible to some
nasty sigpipe race condition. I ended up switching programming models,
and instead of having a classical 'shell pipe' with a child and grand-child,
having two children, doing the pipe connection manually, and micro-managing
the second child. The nice end result is that I can close($fh) on the gzip
handle and waitpid($object->{pid2}, 0) on the connection process, so that
I can be sure it died before I send the ABORT message. The only not nice
result is that I have to micro-manage those processes, e.g., make sure I
do both the close and waitpid each time (not too bad), and not forget
to close everything as expected...

There's one local area where the code is more complicated, but in general,
the processes interactions are greatly simplified, and stuff now happens
in a deterministic way...