Building Unbound with Python module support

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Building Unbound with Python module support

Andy Lemin-2
Hi guys,

I’m just after some general advice as I feel like I’m doing something wrong, and having to hack around too much for what I believe should be simple.

I am developing a simple Python plugin for Unbound, and the default Unbound install on OpenBSD sadly wasn’t built with “—with-pythonmodule”.

So I grabbed the Unbound source code with a git clone from GitHub, installed dependencies, and did “./configure —with-pythonmodule”, make, make install etc..

So nothing special here. It installed to /usr/local/ rather than just /usr etc, and so fiddled around with /etc/rc.d/unbound to make the rc scripts start the custom one.

But I’m getting errors which requires some extra config settings to squash when loading the same config as with the built in Unbound. ok maybe newer unbound code..

But I am then also getting errors when trying to load the stock example python plugin as per the source built sphinx docs.

I’m not at my computer at the moment so can’t share the exact errors, but thought I’d ask as it feels like I’m missing something obvious!

Maybe I need some extra build options or static library references to make it as smooth as the built in Unbound? Or maybe I should be using a different source?

Any initial thoughts? I’ll post exact errors as soon as I can.

Thanks :)
Andy.





Sent from a teeny tiny keyboard, so please excuse typos

Reply | Threaded
Open this post in threaded view
|

Re: Building Unbound with Python module support

Stuart Henderson
On 2019-08-06, Andy Lemin <[hidden email]> wrote:

> Hi guys,
>
> I’m just after some general advice as I feel like I’m doing something wrong, and having to hack around too much for what I believe should be simple.
>
> I am developing a simple Python plugin for Unbound, and the default Unbound install on OpenBSD sadly wasn’t built with “—with-pythonmodule”.
>
> So I grabbed the Unbound source code with a git clone from GitHub, installed dependencies, and did “./configure —with-pythonmodule”, make, make install etc..
>
> So nothing special here. It installed to /usr/local/ rather than just /usr etc, and so fiddled around with /etc/rc.d/unbound to make the rc scripts start the custom one.
>
> But I’m getting errors which requires some extra config settings to squash when loading the same config as with the built in Unbound. ok maybe newer unbound code..
>
> But I am then also getting errors when trying to load the stock example python plugin as per the source built sphinx docs.
>
> I’m not at my computer at the moment so can’t share the exact errors, but thought I’d ask as it feels like I’m missing something obvious!
>
> Maybe I need some extra build options or static library references to make it as smooth as the built in Unbound? Or maybe I should be using a different source?
>
> Any initial thoughts? I’ll post exact errors as soon as I can.

Initial thoughts are "did you use the same configure flags as much as possible
as the build in base". Really need to see the errors to be able to make any
more detailed suggestions.

The default install can't include Python support, because the default install
of Unbound is in the base OS, and Python isn't.


Reply | Threaded
Open this post in threaded view
|

Re: Building Unbound with Python module support

Andy Lemin-2
Hi Stuart,

Thanks for your reply.

So I put in some leg work to set myself up so I could build a new release base system, and went digging.

And I found “/usr/src/usr.src/unbound/Makefile.bsd-wrapper” so I think I have found the correct build options to match with the base builds CONFIGURE_OPTS_UNBOUND

I will try again with these options tomorrow, and see if I have the same errors.

“The default install can't include Python support, because the default install of Unbound is in the base OS, and Python isn't.”

Facepalm.. Of course!

Is there a C plugin library? I would like to make this project native/portable so other users can use this project without having to rebuild Unbound?

Thanks Andy.


Sent from a teeny tiny keyboard, so please excuse typos

> On 6 Aug 2019, at 19:36, Stuart Henderson <[hidden email]> wrote:
>
>> On 2019-08-06, Andy Lemin <[hidden email]> wrote:
>> Hi guys,
>>
>> I’m just after some general advice as I feel like I’m doing something wrong, and having to hack around too much for what I believe should be simple.
>>
>> I am developing a simple Python plugin for Unbound, and the default Unbound install on OpenBSD sadly wasn’t built with “—with-pythonmodule”.
>>
>> So I grabbed the Unbound source code with a git clone from GitHub, installed dependencies, and did “./configure —with-pythonmodule”, make, make install etc..
>>
>> So nothing special here. It installed to /usr/local/ rather than just /usr etc, and so fiddled around with /etc/rc.d/unbound to make the rc scripts start the custom one.
>>
>> But I’m getting errors which requires some extra config settings to squash when loading the same config as with the built in Unbound. ok maybe newer unbound code..
>>
>> But I am then also getting errors when trying to load the stock example python plugin as per the source built sphinx docs.
>>
>> I’m not at my computer at the moment so can’t share the exact errors, but thought I’d ask as it feels like I’m missing something obvious!
>>
>> Maybe I need some extra build options or static library references to make it as smooth as the built in Unbound? Or maybe I should be using a different source?
>>
>> Any initial thoughts? I’ll post exact errors as soon as I can.
>
> Initial thoughts are "did you use the same configure flags as much as possible
> as the build in base". Really need to see the errors to be able to make any
> more detailed suggestions.
>
> The default install can't include Python support, because the default install
> of Unbound is in the base OS, and Python isn't.
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Building Unbound with Python module support

Andy Lemin-2
Morning Stuart,

So I’ve tested with the base build options properly, the initial errors I saw before have gone which is good. But I have a more fundamental issue with Unbound now sadly.

Swig successfully built “/usr/src/unbound/pythonmod/unboundmodule.py” and installed it to “/usr/local/lib/python2.7/site-packages/unboundmodule.py”.

However unbound is unable to find it, and the following errors are seen;
[HOME]root@bsd1:/var/unbound#/usr/local/sbin/unbound -c /var/unbound/etc/unbound.conf -dv

[1565135861] unbound[90497:0] notice: Start of unbound 1.9.3.

[1565135861] unbound[90497:0] debug: increased limit(open files) from 128 to 16478

[1565135861] unbound[90497:0] debug: creating udp4 socket 127.0.0.1 53

[1565135861] unbound[90497:0] debug: creating udp4 socket 10.10.1.5 53

[1565135861] unbound[90497:0] debug: creating unix socket /var/run/unbound.sock

[1565135861] unbound[90497:0] debug: switching log to syslog

Could not find platform independent libraries <prefix>

Could not find platform dependent libraries <exec_prefix>

Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]

Traceback (most recent call last):

  File "<string>", line 1, in <module>

ImportError: No module named distutils.sysconfig

Traceback (most recent call last):

  File "<string>", line 1, in <module>

NameError: name 'distutils' is not defined

Traceback (most recent call last):

  File "<string>", line 1, in <module>

ImportError: No module named unboundmodule



I have tried all manner of values for PYTHONHOME and I have also tried

--with-pythonmodule=/usr/local/lib/python2.7/site-packages


Searching around shows others have found the exact same issue;
https://nlnetlabs.nl/pipermail/unbound-users/2011-July/007371.html

What do you think about this in context of OpenBSD?

Thanks again for your time.

Kind regards, Andy.



Sent from a teeny tiny keyboard, so please excuse typos

> On 7 Aug 2019, at 00:03, Andy Lemin <[hidden email]> wrote:
>
> Hi Stuart,
>
> Thanks for your reply.
>
> So I put in some leg work to set myself up so I could build a new release base system, and went digging.
>
> And I found “/usr/src/usr.src/unbound/Makefile.bsd-wrapper” so I think I have found the correct build options to match with the base builds CONFIGURE_OPTS_UNBOUND
>
> I will try again with these options tomorrow, and see if I have the same errors.
>
> “The default install can't include Python support, because the default install of Unbound is in the base OS, and Python isn't.”
>
> Facepalm.. Of course!
>
> Is there a C plugin library? I would like to make this project native/portable so other users can use this project without having to rebuild Unbound?
>
> Thanks Andy.
>
>
> Sent from a teeny tiny keyboard, so please excuse typos
>
>>> On 6 Aug 2019, at 19:36, Stuart Henderson <[hidden email]> wrote:
>>>
>>> On 2019-08-06, Andy Lemin <[hidden email]> wrote:
>>> Hi guys,
>>>
>>> I’m just after some general advice as I feel like I’m doing something wrong, and having to hack around too much for what I believe should be simple.
>>>
>>> I am developing a simple Python plugin for Unbound, and the default Unbound install on OpenBSD sadly wasn’t built with “—with-pythonmodule”.
>>>
>>> So I grabbed the Unbound source code with a git clone from GitHub, installed dependencies, and did “./configure —with-pythonmodule”, make, make install etc..
>>>
>>> So nothing special here. It installed to /usr/local/ rather than just /usr etc, and so fiddled around with /etc/rc.d/unbound to make the rc scripts start the custom one.
>>>
>>> But I’m getting errors which requires some extra config settings to squash when loading the same config as with the built in Unbound. ok maybe newer unbound code..
>>>
>>> But I am then also getting errors when trying to load the stock example python plugin as per the source built sphinx docs.
>>>
>>> I’m not at my computer at the moment so can’t share the exact errors, but thought I’d ask as it feels like I’m missing something obvious!
>>>
>>> Maybe I need some extra build options or static library references to make it as smooth as the built in Unbound? Or maybe I should be using a different source?
>>>
>>> Any initial thoughts? I’ll post exact errors as soon as I can.
>>
>> Initial thoughts are "did you use the same configure flags as much as possible
>> as the build in base". Really need to see the errors to be able to make any
>> more detailed suggestions.
>>
>> The default install can't include Python support, because the default install
>> of Unbound is in the base OS, and Python isn't.
>>
>>
Reply | Threaded
Open this post in threaded view
|

Re: Building Unbound with Python module support

Claudio Jeker
On Wed, Aug 07, 2019 at 08:44:07AM +0100, Andy Lemin wrote:

> Morning Stuart,
>
> So I’ve tested with the base build options properly, the initial errors I saw before have gone which is good. But I have a more fundamental issue with Unbound now sadly.
>
> Swig successfully built “/usr/src/unbound/pythonmod/unboundmodule.py” and installed it to “/usr/local/lib/python2.7/site-packages/unboundmodule.py”.
>
> However unbound is unable to find it, and the following errors are seen;
> [HOME]root@bsd1:/var/unbound#/usr/local/sbin/unbound -c /var/unbound/etc/unbound.conf -dv
>
> [1565135861] unbound[90497:0] notice: Start of unbound 1.9.3.
>
> [1565135861] unbound[90497:0] debug: increased limit(open files) from 128 to 16478
>
> [1565135861] unbound[90497:0] debug: creating udp4 socket 127.0.0.1 53
>
> [1565135861] unbound[90497:0] debug: creating udp4 socket 10.10.1.5 53
>
> [1565135861] unbound[90497:0] debug: creating unix socket /var/run/unbound.sock
>
> [1565135861] unbound[90497:0] debug: switching log to syslog
>
> Could not find platform independent libraries <prefix>
>
> Could not find platform dependent libraries <exec_prefix>
>
> Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
>
> Traceback (most recent call last):
>
>   File "<string>", line 1, in <module>
>
> ImportError: No module named distutils.sysconfig
>
> Traceback (most recent call last):
>
>   File "<string>", line 1, in <module>
>
> NameError: name 'distutils' is not defined
>
> Traceback (most recent call last):
>
>   File "<string>", line 1, in <module>
>
> ImportError: No module named unboundmodule
>
>
>
> I have tried all manner of values for PYTHONHOME and I have also tried
>
> --with-pythonmodule=/usr/local/lib/python2.7/site-packages
>
>
> Searching around shows others have found the exact same issue;
> https://nlnetlabs.nl/pipermail/unbound-users/2011-July/007371.html
>
> What do you think about this in context of OpenBSD?
>

unbound does a chroot(2) by default ot /var/unbound and so anything in
/usr/local is unreachable. Either install the python code into the chroot
or try running unbound with chroot: "" (which disables chroot). See also
unbound.conf(5) for more info about chroot.


> Sent from a teeny tiny keyboard, so please excuse typos
>
> > On 7 Aug 2019, at 00:03, Andy Lemin <[hidden email]> wrote:
> >
> > Hi Stuart,
> >
> > Thanks for your reply.
> >
> > So I put in some leg work to set myself up so I could build a new release base system, and went digging.
> >
> > And I found “/usr/src/usr.src/unbound/Makefile.bsd-wrapper” so I think I have found the correct build options to match with the base builds CONFIGURE_OPTS_UNBOUND
> >
> > I will try again with these options tomorrow, and see if I have the same errors.
> >
> > “The default install can't include Python support, because the default install of Unbound is in the base OS, and Python isn't.”
> >
> > Facepalm.. Of course!
> >
> > Is there a C plugin library? I would like to make this project native/portable so other users can use this project without having to rebuild Unbound?
> >
> > Thanks Andy.
> >
> >
> > Sent from a teeny tiny keyboard, so please excuse typos
> >
> >>> On 6 Aug 2019, at 19:36, Stuart Henderson <[hidden email]> wrote:
> >>>
> >>> On 2019-08-06, Andy Lemin <[hidden email]> wrote:
> >>> Hi guys,
> >>>
> >>> I’m just after some general advice as I feel like I’m doing something wrong, and having to hack around too much for what I believe should be simple.
> >>>
> >>> I am developing a simple Python plugin for Unbound, and the default Unbound install on OpenBSD sadly wasn’t built with “—with-pythonmodule”.
> >>>
> >>> So I grabbed the Unbound source code with a git clone from GitHub, installed dependencies, and did “./configure —with-pythonmodule”, make, make install etc..
> >>>
> >>> So nothing special here. It installed to /usr/local/ rather than just /usr etc, and so fiddled around with /etc/rc.d/unbound to make the rc scripts start the custom one.
> >>>
> >>> But I’m getting errors which requires some extra config settings to squash when loading the same config as with the built in Unbound. ok maybe newer unbound code..
> >>>
> >>> But I am then also getting errors when trying to load the stock example python plugin as per the source built sphinx docs.
> >>>
> >>> I’m not at my computer at the moment so can’t share the exact errors, but thought I’d ask as it feels like I’m missing something obvious!
> >>>
> >>> Maybe I need some extra build options or static library references to make it as smooth as the built in Unbound? Or maybe I should be using a different source?
> >>>
> >>> Any initial thoughts? I’ll post exact errors as soon as I can.
> >>
> >> Initial thoughts are "did you use the same configure flags as much as possible
> >> as the build in base". Really need to see the errors to be able to make any
> >> more detailed suggestions.
> >>
> >> The default install can't include Python support, because the default install
> >> of Unbound is in the base OS, and Python isn't.
> >>
> >>

--
:wq Claudio

Reply | Threaded
Open this post in threaded view
|

Re: Building Unbound with Python module support

Andy Lemin-2
Hi Claudio,

I did actually try putting my python plugin and unboundmodule.py into the chroot, but I completely forgot that I would also need to install the rest of python into the chroot! Haha

Serves me right for working on it until 1am when I should should have been asleep and trying with fresh eyes the next day ;)

All working now. You guys are heros.

Thank you for the gentle nudges in the right direction.

Kindest regards.
Andy Lemin


Sent from a teeny tiny keyboard, so please excuse typos

> On 7 Aug 2019, at 09:01, Claudio Jeker <[hidden email]> wrote:
>
>> On Wed, Aug 07, 2019 at 08:44:07AM +0100, Andy Lemin wrote:
>> Morning Stuart,
>>
>> So I’ve tested with the base build options properly, the initial errors I saw before have gone which is good. But I have a more fundamental issue with Unbound now sadly.
>>
>> Swig successfully built “/usr/src/unbound/pythonmod/unboundmodule.py” and installed it to “/usr/local/lib/python2.7/site-packages/unboundmodule.py”.
>>
>> However unbound is unable to find it, and the following errors are seen;
>> [HOME]root@bsd1:/var/unbound#/usr/local/sbin/unbound -c /var/unbound/etc/unbound.conf -dv
>>
>> [1565135861] unbound[90497:0] notice: Start of unbound 1.9.3.
>>
>> [1565135861] unbound[90497:0] debug: increased limit(open files) from 128 to 16478
>>
>> [1565135861] unbound[90497:0] debug: creating udp4 socket 127.0.0.1 53
>>
>> [1565135861] unbound[90497:0] debug: creating udp4 socket 10.10.1.5 53
>>
>> [1565135861] unbound[90497:0] debug: creating unix socket /var/run/unbound.sock
>>
>> [1565135861] unbound[90497:0] debug: switching log to syslog
>>
>> Could not find platform independent libraries <prefix>
>>
>> Could not find platform dependent libraries <exec_prefix>
>>
>> Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
>>
>> Traceback (most recent call last):
>>
>>  File "<string>", line 1, in <module>
>>
>> ImportError: No module named distutils.sysconfig
>>
>> Traceback (most recent call last):
>>
>>  File "<string>", line 1, in <module>
>>
>> NameError: name 'distutils' is not defined
>>
>> Traceback (most recent call last):
>>
>>  File "<string>", line 1, in <module>
>>
>> ImportError: No module named unboundmodule
>>
>>
>>
>> I have tried all manner of values for PYTHONHOME and I have also tried
>>
>> --with-pythonmodule=/usr/local/lib/python2.7/site-packages
>>
>>
>> Searching around shows others have found the exact same issue;
>> https://nlnetlabs.nl/pipermail/unbound-users/2011-July/007371.html
>>
>> What do you think about this in context of OpenBSD?
>>
>
> unbound does a chroot(2) by default ot /var/unbound and so anything in
> /usr/local is unreachable. Either install the python code into the chroot
> or try running unbound with chroot: "" (which disables chroot). See also
> unbound.conf(5) for more info about chroot.
>
>
>> Sent from a teeny tiny keyboard, so please excuse typos
>>
>>> On 7 Aug 2019, at 00:03, Andy Lemin <[hidden email]> wrote:
>>>
>>> Hi Stuart,
>>>
>>> Thanks for your reply.
>>>
>>> So I put in some leg work to set myself up so I could build a new release base system, and went digging.
>>>
>>> And I found “/usr/src/usr.src/unbound/Makefile.bsd-wrapper” so I think I have found the correct build options to match with the base builds CONFIGURE_OPTS_UNBOUND
>>>
>>> I will try again with these options tomorrow, and see if I have the same errors.
>>>
>>> “The default install can't include Python support, because the default install of Unbound is in the base OS, and Python isn't.”
>>>
>>> Facepalm.. Of course!
>>>
>>> Is there a C plugin library? I would like to make this project native/portable so other users can use this project without having to rebuild Unbound?
>>>
>>> Thanks Andy.
>>>
>>>
>>> Sent from a teeny tiny keyboard, so please excuse typos
>>>
>>>>> On 6 Aug 2019, at 19:36, Stuart Henderson <[hidden email]> wrote:
>>>>>
>>>>> On 2019-08-06, Andy Lemin <[hidden email]> wrote:
>>>>> Hi guys,
>>>>>
>>>>> I’m just after some general advice as I feel like I’m doing something wrong, and having to hack around too much for what I believe should be simple.
>>>>>
>>>>> I am developing a simple Python plugin for Unbound, and the default Unbound install on OpenBSD sadly wasn’t built with “—with-pythonmodule”.
>>>>>
>>>>> So I grabbed the Unbound source code with a git clone from GitHub, installed dependencies, and did “./configure —with-pythonmodule”, make, make install etc..
>>>>>
>>>>> So nothing special here. It installed to /usr/local/ rather than just /usr etc, and so fiddled around with /etc/rc.d/unbound to make the rc scripts start the custom one.
>>>>>
>>>>> But I’m getting errors which requires some extra config settings to squash when loading the same config as with the built in Unbound. ok maybe newer unbound code..
>>>>>
>>>>> But I am then also getting errors when trying to load the stock example python plugin as per the source built sphinx docs.
>>>>>
>>>>> I’m not at my computer at the moment so can’t share the exact errors, but thought I’d ask as it feels like I’m missing something obvious!
>>>>>
>>>>> Maybe I need some extra build options or static library references to make it as smooth as the built in Unbound? Or maybe I should be using a different source?
>>>>>
>>>>> Any initial thoughts? I’ll post exact errors as soon as I can.
>>>>
>>>> Initial thoughts are "did you use the same configure flags as much as possible
>>>> as the build in base". Really need to see the errors to be able to make any
>>>> more detailed suggestions.
>>>>
>>>> The default install can't include Python support, because the default install
>>>> of Unbound is in the base OS, and Python isn't.
>>>>
>>>>
>
> --
> :wq Claudio