Duplicity & /etc/daily.local

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Duplicity & /etc/daily.local

nothingness
Hi misc@,


   I'm trying to run daily backups to a sftp server for various VMs and
devices on my network, and want to use /etc/daily.local for this. I'm
calling this script from the daily.local file:

env 'GNUPG="/usr/local/bin/gpg" PASSPHRASE="mypassword"'
/root/duplicity-hostname.sh

but unfortunately duplicity can't find gnupg and errors out with this
error message:

Traceback (innermost last):
   File "/usr/local/bin/duplicity", line 1562, in <module>
     with_tempdir(main)
   File "/usr/local/bin/duplicity", line 1548, in with_tempdir
     fn()
   File "/usr/local/bin/duplicity", line 1387, in main
     action = commandline.ProcessCommandLine(sys.argv[1:])
   File "/usr/local/lib/python2.7/site-packages/duplicity/commandline.py", line 1088, in ProcessCommandLine
     globals.gpg_profile = gpg.GPGProfile()
   File "/usr/local/lib/python2.7/site-packages/duplicity/gpg.py", line 92, in __init__
     self.gpg_version = self.get_gpg_version(globals.gpg_binary)
   File "/usr/local/lib/python2.7/site-packages/duplicity/gpg.py", line 107, in get_gpg_version
     res = gnupg.run(["--version"], create_fhs=["stdout"])
   File "/usr/local/lib/python2.7/site-packages/duplicity/gpginterface.py", line 374, in run
     create_fhs, attach_fhs)
   File "/usr/local/lib/python2.7/site-packages/duplicity/gpginterface.py", line 423, in _attach_fork_exec
     self._as_child(process, gnupg_commands, args)
   File "/usr/local/lib/python2.7/site-packages/duplicity/gpginterface.py", line 462, in _as_child
     os.execvp(command[0], command)
   File "/usr/local/lib/python2.7/os.py", line 346, in execvp
     _execvpe(file, args)
   File "/usr/local/lib/python2.7/os.py", line 382, in _execvpe
     func(fullname, *argrest)
  OSError: [Errno 2] No such file or directory

GPGError: failed to determine gnupg version of None from


duplicity-hostname.sh content:

#!/bin/ksh
PASSPHRASE=mypassword
/usr/local/bin/duplicity incremental /var sftp://user@backuphost:/hostname/var
/usr/local/bin/duplicity incremental /etc sftp://user@backuphost:/hostname/etc
/usr/local/bin/duplicity incremental /root sftp://user@backuphost:/hostname/root

Can daily.local even handle this or is the environment too limited?

Cheers,

Noth

Reply | Threaded
Open this post in threaded view
|

Re: Duplicity & /etc/daily.local

trondd-2
On Mon, May 20, 2019 5:50 pm, Noth wrote:

> Hi misc@,
>
>
>  Â I'm trying to run daily backups to a sftp server for various VMs and
> devices on my network, and want to use /etc/daily.local for this. I'm
> calling this script from the daily.local file:
>
> env 'GNUPG="/usr/local/bin/gpg" PASSPHRASE="mypassword"'
> /root/duplicity-hostname.sh
>
> but unfortunately duplicity can't find gnupg
>

I don't use duplicity anymore but is the GNUPG environment variable even a
thing?  Their manpage doesn't mention it but does specifiy a --ggp-binary
commandline argument.

http://duplicity.nongnu.org/duplicity.1.html

You'll also need to be sure gpg is looking in the right place for the
keyrings.

Reply | Threaded
Open this post in threaded view
|

Re: Duplicity & /etc/daily.local

Antoine Jacoutot-7
In reply to this post by nothingness
On Mon, May 20, 2019 at 11:50:13PM +0200, Noth wrote:

> Hi misc@,
>
>
>   I'm trying to run daily backups to a sftp server for various VMs and
> devices on my network, and want to use /etc/daily.local for this. I'm
> calling this script from the daily.local file:
>
> env 'GNUPG="/usr/local/bin/gpg" PASSPHRASE="mypassword"'
> /root/duplicity-hostname.sh
>
> but unfortunately duplicity can't find gnupg and errors out with this error
> message:
>
> Traceback (innermost last):
>   File "/usr/local/bin/duplicity", line 1562, in <module>
>     with_tempdir(main)
>   File "/usr/local/bin/duplicity", line 1548, in with_tempdir
>     fn()
>   File "/usr/local/bin/duplicity", line 1387, in main
>     action = commandline.ProcessCommandLine(sys.argv[1:])
>   File "/usr/local/lib/python2.7/site-packages/duplicity/commandline.py", line 1088, in ProcessCommandLine
>     globals.gpg_profile = gpg.GPGProfile()
>   File "/usr/local/lib/python2.7/site-packages/duplicity/gpg.py", line 92, in __init__
>     self.gpg_version = self.get_gpg_version(globals.gpg_binary)
>   File "/usr/local/lib/python2.7/site-packages/duplicity/gpg.py", line 107, in get_gpg_version
>     res = gnupg.run(["--version"], create_fhs=["stdout"])
>   File "/usr/local/lib/python2.7/site-packages/duplicity/gpginterface.py", line 374, in run
>     create_fhs, attach_fhs)
>   File "/usr/local/lib/python2.7/site-packages/duplicity/gpginterface.py", line 423, in _attach_fork_exec
>     self._as_child(process, gnupg_commands, args)
>   File "/usr/local/lib/python2.7/site-packages/duplicity/gpginterface.py", line 462, in _as_child
>     os.execvp(command[0], command)
>   File "/usr/local/lib/python2.7/os.py", line 346, in execvp
>     _execvpe(file, args)
>   File "/usr/local/lib/python2.7/os.py", line 382, in _execvpe
>     func(fullname, *argrest)
>  OSError: [Errno 2] No such file or directory
>
> GPGError: failed to determine gnupg version of None from
>
>
> duplicity-hostname.sh content:
>
> #!/bin/ksh
> PASSPHRASE=mypassword
> /usr/local/bin/duplicity incremental /var sftp://user@backuphost:/hostname/var
> /usr/local/bin/duplicity incremental /etc sftp://user@backuphost:/hostname/etc
> /usr/local/bin/duplicity incremental /root sftp://user@backuphost:/hostname/root
>
> Can daily.local even handle this or is the environment too limited?

daily.local is run by cron which sets:
PATH=/bin:/sbin:/usr/bin:/usr/sbin

Try setting this in your script:
PATH=${PATH}:/usr/local/bin

--
Antoine

Reply | Threaded
Open this post in threaded view
|

Re: Duplicity & /etc/daily.local

Craig Skinner-3
In reply to this post by nothingness
Hi Noth,

On Mon, 20 May 2019 23:50:13 +0200 Noth wrote:
> /root/duplicity-hostname.sh

Does this script work properly when you're logged in as root?


>
> #!/bin/ksh
> PASSPHRASE=mypassword

try: export PASSPHRASE='mypassword'


> Can daily.local even handle this or is the environment too limited?



daily is run by root's crontab.

To get root's cron environment mailed out, add a temp root cron job:

[next 5 mins] [this hour]  *  *   *  logname; umask; pwd; printenv | sort


If your script works well when logged in as root, but not from cron,
add the missing environment elements to daily.local/script/root's crontab.


Cheers,
--
Craig Skinner | http://linkd.in/yGqkv7

Reply | Threaded
Open this post in threaded view
|

Re: Duplicity & /etc/daily.local

Patrick Wildt-3
In reply to this post by nothingness
On Mon, May 20, 2019 at 11:50:13PM +0200, Noth wrote:

> Hi misc@,
>
>
>   I'm trying to run daily backups to a sftp server for various VMs and
> devices on my network, and want to use /etc/daily.local for this. I'm
> calling this script from the daily.local file:
>
> env 'GNUPG="/usr/local/bin/gpg" PASSPHRASE="mypassword"'
> /root/duplicity-hostname.sh
>
> but unfortunately duplicity can't find gnupg and errors out with this error
> message:
>
> Traceback (innermost last):
>   File "/usr/local/bin/duplicity", line 1562, in <module>
>     with_tempdir(main)
>   File "/usr/local/bin/duplicity", line 1548, in with_tempdir
>     fn()
>   File "/usr/local/bin/duplicity", line 1387, in main
>     action = commandline.ProcessCommandLine(sys.argv[1:])
>   File "/usr/local/lib/python2.7/site-packages/duplicity/commandline.py", line 1088, in ProcessCommandLine
>     globals.gpg_profile = gpg.GPGProfile()
>   File "/usr/local/lib/python2.7/site-packages/duplicity/gpg.py", line 92, in __init__
>     self.gpg_version = self.get_gpg_version(globals.gpg_binary)
>   File "/usr/local/lib/python2.7/site-packages/duplicity/gpg.py", line 107, in get_gpg_version
>     res = gnupg.run(["--version"], create_fhs=["stdout"])
>   File "/usr/local/lib/python2.7/site-packages/duplicity/gpginterface.py", line 374, in run
>     create_fhs, attach_fhs)
>   File "/usr/local/lib/python2.7/site-packages/duplicity/gpginterface.py", line 423, in _attach_fork_exec
>     self._as_child(process, gnupg_commands, args)
>   File "/usr/local/lib/python2.7/site-packages/duplicity/gpginterface.py", line 462, in _as_child
>     os.execvp(command[0], command)
>   File "/usr/local/lib/python2.7/os.py", line 346, in execvp
>     _execvpe(file, args)
>   File "/usr/local/lib/python2.7/os.py", line 382, in _execvpe
>     func(fullname, *argrest)
>  OSError: [Errno 2] No such file or directory
>
> GPGError: failed to determine gnupg version of None from
>
>
> duplicity-hostname.sh content:
>
> #!/bin/ksh
> PASSPHRASE=mypassword
> /usr/local/bin/duplicity incremental /var sftp://user@backuphost:/hostname/var
> /usr/local/bin/duplicity incremental /etc sftp://user@backuphost:/hostname/etc
> /usr/local/bin/duplicity incremental /root sftp://user@backuphost:/hostname/root
>
> Can daily.local even handle this or is the environment too limited?
>
> Cheers,
>
> Noth
>

I have the same setup and it failed for me as well.  I somehow managed
to fix it by setting PATH and also exporting TERM:

PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/X11R6/bin:/usr/local/bin:/usr/local/sbin:/usr/games
TERM=xterm
export PATH TERM

And you should probably also do something like:

. /root/.passphrase
test -n "$PASSPHRASE" || exit 0
export PASSPHRASE

Patrick

Reply | Threaded
Open this post in threaded view
|

Re: Duplicity & /etc/daily.local

nothingness

On 22/05/2019 04:46, Patrick Wildt wrote:

> On Mon, May 20, 2019 at 11:50:13PM +0200, Noth wrote:
>> Hi misc@,
>>
>>
>>    I'm trying to run daily backups to a sftp server for various VMs and
>> devices on my network, and want to use /etc/daily.local for this. I'm
>> calling this script from the daily.local file:
>>
>> env 'GNUPG="/usr/local/bin/gpg" PASSPHRASE="mypassword"'
>> /root/duplicity-hostname.sh
>>
>> but unfortunately duplicity can't find gnupg and errors out with this error
>> message:
>>
>> Traceback (innermost last):
>>    File "/usr/local/bin/duplicity", line 1562, in <module>
>>      with_tempdir(main)
>>    File "/usr/local/bin/duplicity", line 1548, in with_tempdir
>>      fn()
>>    File "/usr/local/bin/duplicity", line 1387, in main
>>      action = commandline.ProcessCommandLine(sys.argv[1:])
>>    File "/usr/local/lib/python2.7/site-packages/duplicity/commandline.py", line 1088, in ProcessCommandLine
>>      globals.gpg_profile = gpg.GPGProfile()
>>    File "/usr/local/lib/python2.7/site-packages/duplicity/gpg.py", line 92, in __init__
>>      self.gpg_version = self.get_gpg_version(globals.gpg_binary)
>>    File "/usr/local/lib/python2.7/site-packages/duplicity/gpg.py", line 107, in get_gpg_version
>>      res = gnupg.run(["--version"], create_fhs=["stdout"])
>>    File "/usr/local/lib/python2.7/site-packages/duplicity/gpginterface.py", line 374, in run
>>      create_fhs, attach_fhs)
>>    File "/usr/local/lib/python2.7/site-packages/duplicity/gpginterface.py", line 423, in _attach_fork_exec
>>      self._as_child(process, gnupg_commands, args)
>>    File "/usr/local/lib/python2.7/site-packages/duplicity/gpginterface.py", line 462, in _as_child
>>      os.execvp(command[0], command)
>>    File "/usr/local/lib/python2.7/os.py", line 346, in execvp
>>      _execvpe(file, args)
>>    File "/usr/local/lib/python2.7/os.py", line 382, in _execvpe
>>      func(fullname, *argrest)
>>   OSError: [Errno 2] No such file or directory
>>
>> GPGError: failed to determine gnupg version of None from
>>
>>
>> duplicity-hostname.sh content:
>>
>> #!/bin/ksh
>> PASSPHRASE=mypassword
>> /usr/local/bin/duplicity incremental /var sftp://user@backuphost:/hostname/var
>> /usr/local/bin/duplicity incremental /etc sftp://user@backuphost:/hostname/etc
>> /usr/local/bin/duplicity incremental /root sftp://user@backuphost:/hostname/root
>>
>> Can daily.local even handle this or is the environment too limited?
>>
>> Cheers,
>>
>> Noth
>>
> I have the same setup and it failed for me as well.  I somehow managed
> to fix it by setting PATH and also exporting TERM:
>
> PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/X11R6/bin:/usr/local/bin:/usr/local/sbin:/usr/games
> TERM=xterm
> export PATH TERM
>
> And you should probably also do something like:
>
> . /root/.passphrase
> test -n "$PASSPHRASE" || exit 0
> export PASSPHRASE
>
> Patrick
>
Hello again,

   Sorry about the delay, I haven't been able to look at this till today.

So:

1. adding PATH does make duplicity find gpg. Thanks Antoine Jacoutot for
the suggestion on getting the env mailed via cron.

2. Next issue is that the sftp connection fails, because it can't
authenticate the host and wants confirmation on accepting the key. This
is despite having copied the user public key to
user/.ssh/.authorized_keys and tested using sftp user@hostname and sftp
[hidden email] . Current state of the script is this:


#!/bin/ksh
export PASSPHRASE="mypassword"
export PATH=${PATH}:/usr/local/bin
/usr/local/bin/duplicity incremental
--ssh-options="-oStrictHostKeyChecking=no" /var
sftp://user@backuphost:/hostname/var

As you can see I'm passing the option so it doesn't ask for
confirmation, but it still happens:


/usr/local/lib/python2.7/site-packages/paramiko/kex_ecdh_nist.py:39: CryptographyDeprecationWarning: encode_point has been deprecated on EllipticCurvePublicNumbers and will be removed in a future version. Please use EllipticCurvePublicKey.public_bytes to obtain both compressed and uncompressed point encoding.
   m.add_string(self.Q_C.public_numbers().encode_point())
/usr/local/lib/python2.7/site-packages/paramiko/kex_ecdh_nist.py:92: CryptographyDeprecationWarning: Support for unsafe construction of public numbers from encoded data will be removed in a future version. Please use EllipticCurvePublicKey.from_encoded_point
   self.curve, Q_S_bytes
/usr/local/lib/python2.7/site-packages/paramiko/kex_ecdh_nist.py:103: CryptographyDeprecationWarning: encode_point has been deprecated on EllipticCurvePublicNumbers and will be removed in a future version. Please use EllipticCurvePublicKey.public_bytes to obtain both compressed and uncompressed point encoding.
   hm.add_string(self.Q_C.public_numbers().encode_point())
BackendException: ssh connection to dupuser@tombstone:22 failed: EOF when reading a line
The authenticity of host 'tombstone' can't be established.
SSH-ED25519 key fingerprint is fd:60:79:e6:d9:06:47:41:b6:54:19:63:9c:b0:d2:89.
Are you sure you want to continue connecting (yes/no)? /usr/local/lib/python2.7/site-packages/paramiko


I tried adding StrictHostKeyChecking no to /etc/ssh/ssh_config and /root/.ssh/config but it gets ignored. Pretty much at a loss here...

Thanks in advance!