VPS default gateway in a different subnet than host

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

VPS default gateway in a different subnet than host

Jyri Hovila [iki]
Dear everyone,

I'm working with an interesting little pilot that involves running an
OpenBSD virtual host in KVM environment. The VPS provider in question
has no (serious) prior experience with OpenBSD and so there are some
rather basic level issues that need solving.

I've been able to work out most of the mysteries, but there's one
routing related issue I haven't been able to figure out myself.

Basically, the problem is that the default gateway (5.166.16.254) is in
a different subnet than the host IP address (193.34.119.15), making it
unreachable.

Here's the routing table as it exists after reboot. The address comes
from DHCP server, and as you can see, there's an entry that should make
the default gateway reachable (but it's not):

Destination      Gateway           Flags   Prio ... Iface
224/4            127.0.0.1         URS     8        lo0
5.166.16.254/32  193.34.119.15     UCS     8        em0
127/8            128.0.0.1         UGRS    8        lo0
127.0.0.1        127.0.0.1         UHhl    1        lo0
193.34.119.15    02:00:00:2c:80:a9 UHLl    1        em0
193.34.119.15/32 193.34.119.15     UCn     4        em0

When I try to ping the gateway:

ping: sendmsg: Invalid argument

I'm rather sure this is basic stuff that I should actually already
know, but I humbly admit I'm at loss here. Been doing extensive
googling and still haven't been able to figure this one out.

All assistance is much appreciated!

Yours,

Jyri

Reply | Threaded
Open this post in threaded view
|

Re: VPS default gateway in a different subnet than host

Jyri Hovila [iki]
Hi,

a brief follow-up.

With Linux, default gateway that resides in different subnet than the
host, all that has to be done is 1) adding a static route to the
default gateway and then 2) adding the default gateway to routing table.

With my OpenBSD test case, I already have a static route to the default
gateway (thanks to correctly configured DHCP server) but when I try to
add default gateway:

# route add default 5.166.16.254
add net default: gateway 5.166.16.254: Too many levels of symbolic links

I'm still googling but haven't found a solution yet.

Any tips, anyone?

- Jyri

Reply | Threaded
Open this post in threaded view
|

Re: VPS default gateway in a different subnet than host

Matthew Dempsky-3
In reply to this post by Jyri Hovila [iki]
[+mpi, jsing]

This is a known issue. Local /32 routes don't work with the new
ART-based routing table.

mpi@ is working on a fix. In the mean time, you may be able to
recompile your kernel without "option ART".

On Tue, Dec 13, 2016 at 9:09 AM, Jyri Hovila [iki.fi]
<[hidden email]> wrote:

> Dear everyone,
>
> I'm working with an interesting little pilot that involves running an
> OpenBSD virtual host in KVM environment. The VPS provider in question
> has no (serious) prior experience with OpenBSD and so there are some
> rather basic level issues that need solving.
>
> I've been able to work out most of the mysteries, but there's one
> routing related issue I haven't been able to figure out myself.
>
> Basically, the problem is that the default gateway (5.166.16.254) is in
> a different subnet than the host IP address (193.34.119.15), making it
> unreachable.
>
> Here's the routing table as it exists after reboot. The address comes
> from DHCP server, and as you can see, there's an entry that should make
> the default gateway reachable (but it's not):
>
> Destination      Gateway           Flags   Prio ... Iface
> 224/4            127.0.0.1         URS     8        lo0
> 5.166.16.254/32  193.34.119.15     UCS     8        em0
> 127/8            128.0.0.1         UGRS    8        lo0
> 127.0.0.1        127.0.0.1         UHhl    1        lo0
> 193.34.119.15    02:00:00:2c:80:a9 UHLl    1        em0
> 193.34.119.15/32 193.34.119.15     UCn     4        em0
>
> When I try to ping the gateway:
>
> ping: sendmsg: Invalid argument
>
> I'm rather sure this is basic stuff that I should actually already
> know, but I humbly admit I'm at loss here. Been doing extensive
> googling and still haven't been able to figure this one out.
>
> All assistance is much appreciated!
>
> Yours,
>
> Jyri

Reply | Threaded
Open this post in threaded view
|

Re: VPS default gateway in a different subnet than host

Matthew Dempsky-3
Here is mpi's proposed fix:
http://marc.info/?l=openbsd-tech&m=148162020419474&w=2

On Tue, Dec 13, 2016 at 1:36 PM, Matthew Dempsky <[hidden email]> wrote:

> [+mpi, jsing]
>
> This is a known issue. Local /32 routes don't work with the new
> ART-based routing table.
>
> mpi@ is working on a fix. In the mean time, you may be able to
> recompile your kernel without "option ART".
>
> On Tue, Dec 13, 2016 at 9:09 AM, Jyri Hovila [iki.fi]
> <[hidden email]> wrote:
>> Dear everyone,
>>
>> I'm working with an interesting little pilot that involves running an
>> OpenBSD virtual host in KVM environment. The VPS provider in question
>> has no (serious) prior experience with OpenBSD and so there are some
>> rather basic level issues that need solving.
>>
>> I've been able to work out most of the mysteries, but there's one
>> routing related issue I haven't been able to figure out myself.
>>
>> Basically, the problem is that the default gateway (5.166.16.254) is in
>> a different subnet than the host IP address (193.34.119.15), making it
>> unreachable.
>>
>> Here's the routing table as it exists after reboot. The address comes
>> from DHCP server, and as you can see, there's an entry that should make
>> the default gateway reachable (but it's not):
>>
>> Destination      Gateway           Flags   Prio ... Iface
>> 224/4            127.0.0.1         URS     8        lo0
>> 5.166.16.254/32  193.34.119.15     UCS     8        em0
>> 127/8            128.0.0.1         UGRS    8        lo0
>> 127.0.0.1        127.0.0.1         UHhl    1        lo0
>> 193.34.119.15    02:00:00:2c:80:a9 UHLl    1        em0
>> 193.34.119.15/32 193.34.119.15     UCn     4        em0
>>
>> When I try to ping the gateway:
>>
>> ping: sendmsg: Invalid argument
>>
>> I'm rather sure this is basic stuff that I should actually already
>> know, but I humbly admit I'm at loss here. Been doing extensive
>> googling and still haven't been able to figure this one out.
>>
>> All assistance is much appreciated!
>>
>> Yours,
>>
>> Jyri

Reply | Threaded
Open this post in threaded view
|

Re: VPS default gateway in a different subnet than host

Jyri Hovila [iki]
In reply to this post by Matthew Dempsky-3
Hi!

> This is a known issue. Local /32 routes don't work with the new
> ART-based routing table.

Thank you so much for this info! You just saved me from a *lot* of
headaches!

> mpi@ is working on a fix. In the mean time, you may be able to
> recompile your kernel without "option ART".

Yeah, I'm quite familiar with compiling the kernel. Will do this and
see if it works out!

Thanks again!

Yours,

Jyri

Reply | Threaded
Open this post in threaded view
|

Re: VPS default gateway in a different subnet than host

Dag Richards
In reply to this post by Jyri Hovila [iki]
With tcp, the default is pretty much always in the same subnet as at
least one interface of any given host. One can do things with VPN, and
gif's and gre's etc which can work around some oddball situations.

however.

if there is a local router that you use to get to your 'default'
gateway, I would characterize that local router as your default gateway.

how about showing us an ifconfig and a netsat -rn

Jyri Hovila [iki.fi] wrote:

> Hi,
>
> a brief follow-up.
>
> With Linux, default gateway that resides in different subnet than the
> host, all that has to be done is 1) adding a static route to the
> default gateway and then 2) adding the default gateway to routing table.
>
> With my OpenBSD test case, I already have a static route to the default
> gateway (thanks to correctly configured DHCP server) but when I try to
> add default gateway:
>
> # route add default 5.166.16.254
> add net default: gateway 5.166.16.254: Too many levels of symbolic links
>
> I'm still googling but haven't found a solution yet.
>
> Any tips, anyone?
>
> - Jyri
>

--
Dag H. Richards  ( no title / no lettres )

The first rule of tautology club is the first rule of tautology club.

This message may or may not contain proprietary information.
Since it is being relayed by SMTP across an unknown number of
relays to its destination, using a protocol that is traditionally
plain ASCII, it's silly to pretend it is still confidential.
If you are not the intended recipient of this message,
there is simply nothing I can do about that. Attempting to bind you
to some destruction protocol through this windbag sig paragraph is
Quixotic at best..

Reply | Threaded
Open this post in threaded view
|

Re: VPS default gateway in a different subnet than host

Jyri Hovila [iki]
In reply to this post by Matthew Dempsky-3
Hi!

The thing is, I can't recompile the kernel on the VPS I'm working on
since I have no means to transfer the sources to the host. Chicken /
egg...

> Here is mpi's proposed fix:
> http://marc.info/?l=openbsd-tech&m=148162020419474&w=2

I'll be happy to test this if it makes it's way to CURRENT. In fact,
I'd love to try it out ASAP. I just need an installation ISO with the
patch applied.

- Jyri

Reply | Threaded
Open this post in threaded view
|

Re: VPS default gateway in a different subnet than host

Matthew Dempsky-3
Do you have console access to the VM? You may be able to use:

    route delete 5.166.16.254/32
    route add 5.166.16.252/30 -gateway -iface 193.34.119.15

as a workaround so you can download the kernel sources.

On Tue, Dec 13, 2016 at 3:10 PM, Jyri Hovila [iki.fi]
<[hidden email]> wrote:

> Hi!
>
> The thing is, I can't recompile the kernel on the VPS I'm working on
> since I have no means to transfer the sources to the host. Chicken /
> egg...
>
>> Here is mpi's proposed fix:
>> http://marc.info/?l=openbsd-tech&m=148162020419474&w=2
>
> I'll be happy to test this if it makes it's way to CURRENT. In fact,
> I'd love to try it out ASAP. I just need an installation ISO with the
> patch applied.
>
> - Jyri

Reply | Threaded
Open this post in threaded view
|

Re: VPS default gateway in a different subnet than host

Jyri Hovila [iki]
In reply to this post by Dag Richards
Dag,

I'm well familiar with the typical configuration, in which the gateway
is in the same sub net.

However, this particular setup is actually quite common in the VPS
world. At OVH Hosting this configuration is de facto standard. It works
by adding a static route to the gateway over the host's IP. With Linux,
there's no problem, and actually earlier versions of OpenBSD would
probably have worked in this setup just as well. The issue has to do
with the ART-based routing which was introduced quite recently if my
memory serves me right.

Yours,

Jyri

Reply | Threaded
Open this post in threaded view
|

Re: VPS default gateway in a different subnet than host

Ax0n
In reply to this post by Jyri Hovila [iki]
I'm guessing the default route a.k.a. gateway already exists, and you're
trying to add another, duplicate route. What's the output of the following
command before and after you do the route add?

netstat -rn -finet

On Tue, Dec 13, 2016 at 2:51 PM, Jyri Hovila [iki.fi] <[hidden email]>
wrote:

> Hi,
>
> a brief follow-up.
>
> With Linux, default gateway that resides in different subnet than the
> host, all that has to be done is 1) adding a static route to the
> default gateway and then 2) adding the default gateway to routing table.
>
> With my OpenBSD test case, I already have a static route to the default
> gateway (thanks to correctly configured DHCP server) but when I try to
> add default gateway:
>
> # route add default 5.166.16.254
> add net default: gateway 5.166.16.254: Too many levels of symbolic links
>
> I'm still googling but haven't found a solution yet.
>
> Any tips, anyone?
>
> - Jyri

Reply | Threaded
Open this post in threaded view
|

Re: VPS default gateway in a different subnet than host

Jyri Hovila [iki]
In reply to this post by Matthew Dempsky-3
Hi!

A brief, positive update.

> Here is mpi's proposed fix:
> http://marc.info/?l=openbsd-tech&m=148162020419474&w=2

The fixed was included in CURRENT on December the 17th, and I can
report that OpenBSD now happily runs under the "default gateway in a
different subnet" scenario used at OVH Hosting.

Great work, Martin and everyone!

I'd also like to give credit to Zion VPS whose superb customer care
helped in locating the reason for the "temporary incompatibility"
between OpenBSD and OVH Hosting's platform. I've been advocating
OpenBSD to them, volunteering in creating the necessary automatic VPS
configuration scripts they need in order to provide the same "purchase
and it's already up" experience, this far available only to their Linux
customers. It may be that Zion VPS soon has OpenBSD as one of the
default choices for OS - this test/fix operation was supported by some
additional manual work by the Zion VPS staff. The service itself is
available already even though the automation is still missing; you'll
just have to ask the Zion VPS customer care to "add a custom ISO" to
your VPS host, and provide them with a link to the latest CURRENT
installation ISO. The installation process itself (as long as it has to
be handled manually) is handled over a browser based VNC session.

Yours,

Jyri

Reply | Threaded
Open this post in threaded view
|

Re: VPS default gateway in a different subnet than host

Jyri Hovila [iki]
In reply to this post by Jyri Hovila [iki]
A brief follow-up.

From Zion VPS:

"We have already started giving access to OpenBSD CDROM image to all
clients in VPS control panel. We will prepare a template (click and
ready) in the upcoming weeks."

Thanks, Matthew, Martin and everyone!

Yours,

Jyri