routes get assigned to a wrong interface, openbsd 5.9

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

routes get assigned to a wrong interface, openbsd 5.9

Mart Tõnso
Hello.

I am hitting a strange behaviour with openbsd 5.9.

# uname -a
OpenBSD router_dev01.lan 5.9 GENERIC.MP#1888 amd64

There's pppd running on the box (for a 3g connection) and OpenVPN
connection on top of that.

The bug is that any routes pushed from openvpn server get assigned to
ppp0 interface (instead of tun0, as I would naively expect).

It's possible reproduce this behaviour by running "route add" command
manually, for example:

# route add 1.2.3.4/32 10.88.0.1
add host 1.2.3.4/32: gateway 10.88.0.1

# netstat -rn -f inet
Routing tables

Internet:
Destination        Gateway            Flags   Refs      Use   Mtu  Prio Iface
default            10.64.64.64        UGS        1       19     -     8 ppp0
1.2.3.4            10.88.0.1          UGHS       0        0     -     8 ppp0
10.64.64.64        10.145.0.40        UH         1        1     -     8 ppp0
10.88.0/24         10.88.0.124        UGS        0      161     -     8 tun0
10.88.0.124        10.88.0.124        UHl        1        1     -     1 tun0
10.88.0.124        10.88.0.124        UH         0        0     -     8 tun0
10.90.0/24         10.88.0.1          UGS        0        0     -     8 ppp0
10.99.0/24         10.88.0.1          UGS        0        0     -     8 ppp0
10.145.0.40        10.145.0.40        UHl        0        4     -     1 ppp0
...

Note that 10.88.0/24 network is associated with interface tun0.
The new route (with gw in that network, 10.88.0.1) however get's
assigned to interface ppp0.

What's happening here?

---
Regards,

Mart

Reply | Threaded
Open this post in threaded view
|

Re: routes get assigned to a wrong interface, openbsd 5.9

Martin Pieuchot
On 12/04/16(Tue) 16:20, Mart Tõnso wrote:

> Hello.
>
> I am hitting a strange behaviour with openbsd 5.9.
>
> # uname -a
> OpenBSD router_dev01.lan 5.9 GENERIC.MP#1888 amd64
>
> There's pppd running on the box (for a 3g connection) and OpenVPN
> connection on top of that.
>
> The bug is that any routes pushed from openvpn server get assigned to
> ppp0 interface (instead of tun0, as I would naively expect).
>
> It's possible reproduce this behaviour by running "route add" command
> manually, for example:
>
> # route add 1.2.3.4/32 10.88.0.1
> add host 1.2.3.4/32: gateway 10.88.0.1
>
> # netstat -rn -f inet
> Routing tables
>
> Internet:
> Destination        Gateway            Flags   Refs      Use   Mtu  Prio Iface
> default            10.64.64.64        UGS        1       19     -     8 ppp0
> 1.2.3.4            10.88.0.1          UGHS       0        0     -     8 ppp0
> 10.64.64.64        10.145.0.40        UH         1        1     -     8 ppp0
> 10.88.0/24         10.88.0.124        UGS        0      161     -     8 tun0
> 10.88.0.124        10.88.0.124        UHl        1        1     -     1 tun0
> 10.88.0.124        10.88.0.124        UH         0        0     -     8 tun0
> 10.90.0/24         10.88.0.1          UGS        0        0     -     8 ppp0
> 10.99.0/24         10.88.0.1          UGS        0        0     -     8 ppp0
> 10.145.0.40        10.145.0.40        UHl        0        4     -     1 ppp0
> ...
>
> Note that 10.88.0/24 network is associated with interface tun0.
> The new route (with gw in that network, 10.88.0.1) however get's
> assigned to interface ppp0.
>
> What's happening here?

Hard to say since you did not include the complete routing table output.

Don't you have 10.88.0.1 configured on ppp0?  What is your ifconfig
output?  What does "$ route -n get 10.88.0.1" returns you?

Reply | Threaded
Open this post in threaded view
|

Re: routes get assigned to a wrong interface, openbsd 5.9

Mart Tõnso
Ah, yes, sorry about that. Here's the full routing info with ifconfig output:

# ifconfig
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 32768
        priority: 0
        groups: lo
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x6
        inet 127.0.0.1 netmask 0xff000000
re0: flags=18843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,MPSAFE> mtu 1500
        lladdr 00:0d:b9:40:bc:54
        priority: 0
        media: Ethernet autoselect (none)
        status: no carrier
        inet 172.16.1.0 netmask 0xffffff00 broadcast 172.16.1.255
re1: flags=18843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,MPSAFE> mtu 1500
        lladdr 00:0d:b9:40:bc:55
        priority: 0
        media: Ethernet autoselect (none)
        status: no carrier
re2: flags=18802<BROADCAST,SIMPLEX,MULTICAST,MPSAFE> mtu 1500
        lladdr 00:0d:b9:40:bc:56
        priority: 0
        media: Ethernet autoselect (10baseT half-duplex)
        status: no carrier
athn0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> mtu 1500
        lladdr 04:f0:21:17:40:15
        priority: 4
        groups: wlan
        media: IEEE802.11 autoselect
        status: no network
        ieee80211: nwid ""
enc0: flags=0<>
        priority: 0
        groups: enc
        status: active
ppp0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1500
        priority: 0
        groups: ppp egress
        inet 10.128.195.179 --> 10.64.64.64 netmask 0xff000000
tun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1500
        priority: 0
        groups: tun
        status: active
        inet 10.88.0.124 --> 10.88.0.124 netmask 0xffffff00


# netstat -rn
Routing tables

Internet:
Destination        Gateway            Flags   Refs      Use   Mtu  Prio Iface
default            10.64.64.64        UGS        2       12     -     8 ppp0
10.64.64.64        10.128.195.179     UH         1        1     -     8 ppp0
10.88.0/24         10.88.0.124        UGS        0        0     -     8 tun0
10.88.0.124        10.88.0.124        UHl        0        0     -     1 tun0
10.88.0.124        10.88.0.124        UH         0        0     -     8 tun0
10.90.0/24         10.88.0.1          UGS        0        0     -     8 ppp0
10.99.0/24         10.88.0.1          UGS        0        0     -     8 ppp0
10.128.195.179     10.128.195.179     UHl        0        3     -     1 ppp0
127/8              127.0.0.1          UGRS       0        0 32768     8 lo0
127.0.0.1          127.0.0.1          UHl        0      310 32768     1 lo0
172.16.1.0         00:0d:b9:40:bc:54  UHLl       0        0     -     1 re0
172.16.1/24        172.16.1.0         C          0        0     -     4 re0
172.16.1.255       172.16.1.0         Hb         0        0     -     1 re0
224/4              127.0.0.1          URS        0        0 32768     8 lo0

Internet6:
Destination                        Gateway
Flags   Refs      Use   Mtu  Prio Iface
::/104                             ::1                            UGRS
      0        0 32768     8 lo0
::/96                              ::1                            UGRS
      0        0 32768     8 lo0
::1                                ::1                            UHl
      0        0 32768     1 lo0
::127.0.0.0/104                    ::1                            UGRS
      0        0 32768     8 lo0
::224.0.0.0/100                    ::1                            UGRS
      0        0 32768     8 lo0
::255.0.0.0/104                    ::1                            UGRS
      0        0 32768     8 lo0
::ffff:0.0.0.0/96                  ::1                            UGRS
      0        0 32768     8 lo0
2002::/24                          ::1                            UGRS
      0        0 32768     8 lo0
2002:7f00::/24                     ::1                            UGRS
      0        0 32768     8 lo0
2002:e000::/20                     ::1                            UGRS
      0        0 32768     8 lo0
2002:ff00::/24                     ::1                            UGRS
      0        0 32768     8 lo0
fe80::/10                          ::1                            UGRS
      0        0 32768     8 lo0
fe80::1%lo0                        fe80::1%lo0                    UHl
      0        0 32768     1 lo0
fec0::/10                          ::1                            UGRS
      0        0 32768     8 lo0
ff01::/16                          ::1                            UGRS
      0        0 32768     8 lo0
ff01::%lo0/32                      ::1                            UC
      0        1 32768     4 lo0
ff02::/16                          ::1                            UGRS
      0        0 32768     8 lo0
ff02::%lo0/32                      ::1                            UC
      0        1 32768     4 lo0

Note the excerpt from openvpn log:

Apr 13 09:12:29 ruuter_dev01 openvpn[26943]: /sbin/ifconfig tun0
10.88.0.124 10.88.0.124 mtu 1500 netmask 255.255.255.0 up
Apr 13 09:12:29 ruuter_dev01 openvpn[26943]: /sbin/route add -net
10.88.0.0 10.88.0.124 -netmask 255.255.255.0
Apr 13 09:12:29 ruuter_dev01 openvpn[26943]: /sbin/route add -net
10.99.0.0 10.88.0.1 -netmask 255.255.255.0
Apr 13 09:12:29 ruuter_dev01 openvpn[26943]: /sbin/route add -net
10.90.0.0 10.88.0.1 -netmask 255.255.255.0


# route -n get 10.88.0.1
   route to: 10.88.0.1
destination: 10.88.0.0
       mask: 255.255.255.0
    gateway: 10.88.0.124
  interface: tun0
 if address: 10.88.0.124
   priority: 8 (static)
      flags: <UP,GATEWAY,DONE,STATIC>
     use       mtu    expire
       0         0         0

This seems ok, so why do the routes end up on ppp0?

---
Regards,

Mart

On Tue, Apr 12, 2016 at 4:55 PM, Martin Pieuchot <[hidden email]> wrote:

> On 12/04/16(Tue) 16:20, Mart Tõnso wrote:
>> Hello.
>>
>> I am hitting a strange behaviour with openbsd 5.9.
>>
>> # uname -a
>> OpenBSD router_dev01.lan 5.9 GENERIC.MP#1888 amd64
>>
>> There's pppd running on the box (for a 3g connection) and OpenVPN
>> connection on top of that.
>>
>> The bug is that any routes pushed from openvpn server get assigned to
>> ppp0 interface (instead of tun0, as I would naively expect).
>>
>> It's possible reproduce this behaviour by running "route add" command
>> manually, for example:
>>
>> # route add 1.2.3.4/32 10.88.0.1
>> add host 1.2.3.4/32: gateway 10.88.0.1
>>
>> # netstat -rn -f inet
>> Routing tables
>>
>> Internet:
>> Destination        Gateway            Flags   Refs      Use   Mtu  Prio
Iface
>> default            10.64.64.64        UGS        1       19     -     8
ppp0
>> 1.2.3.4            10.88.0.1          UGHS       0        0     -     8
ppp0
>> 10.64.64.64        10.145.0.40        UH         1        1     -     8
ppp0
>> 10.88.0/24         10.88.0.124        UGS        0      161     -     8
tun0
>> 10.88.0.124        10.88.0.124        UHl        1        1     -     1
tun0
>> 10.88.0.124        10.88.0.124        UH         0        0     -     8
tun0
>> 10.90.0/24         10.88.0.1          UGS        0        0     -     8
ppp0
>> 10.99.0/24         10.88.0.1          UGS        0        0     -     8
ppp0
>> 10.145.0.40        10.145.0.40        UHl        0        4     -     1
ppp0

>> ...
>>
>> Note that 10.88.0/24 network is associated with interface tun0.
>> The new route (with gw in that network, 10.88.0.1) however get's
>> assigned to interface ppp0.
>>
>> What's happening here?
>
> Hard to say since you did not include the complete routing table output.
>
> Don't you have 10.88.0.1 configured on ppp0?  What is your ifconfig
> output?  What does "$ route -n get 10.88.0.1" returns you?

Reply | Threaded
Open this post in threaded view
|

Re: routes get assigned to a wrong interface, openbsd 5.9

Martin Pieuchot
Hello Mart,

On 13/04/16(Wed) 09:22, Mart Tõnso wrote:
> Ah, yes, sorry about that. Here's the full routing info with ifconfig output:
>
> # ifconfig
> [...]
> ppp0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1500
>         priority: 0
>         groups: ppp egress
>         inet 10.128.195.179 --> 10.64.64.64 netmask 0xff000000
                                                      ^^^^^^^^^^
Here is the problem.  For historical reasons the code that finds
a matching interface to attach your route matches your gateway
with ppp0's address/netmask.

A workaround would be to change your ppp0 setup to use a /32 mask.

A correct fix is included below, I'll be interested to hear if it
works for you.

> tun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1500
>         priority: 0
>         groups: tun
>         status: active
>         inet 10.88.0.124 --> 10.88.0.124 netmask 0xffffff00
                ^^^^^^^^^^^^^^^^^^^^^^^^^^
By the way why do you use the same src and dst address?

Index: net/route.c
===================================================================
RCS file: /cvs/src/sys/net/route.c,v
retrieving revision 1.298
diff -u -p -r1.298 route.c
--- net/route.c 26 Mar 2016 21:56:04 -0000 1.298
+++ net/route.c 13 Apr 2016 07:38:11 -0000
@@ -740,20 +740,16 @@ ifa_ifwithroute(int flags, struct sockad
  ifa = ifaof_ifpforaddr(dst, ifp);
  if_put(ifp);
  } else {
- ifa = ifa_ifwithnet(gateway, rtableid);
- }
- }
- if (ifa == NULL) {
- struct rtentry *rt = rtalloc(gateway, 0, rtableid);
- /* The gateway must be local if the same address family. */
- if (!rtisvalid(rt) || ((rt->rt_flags & RTF_GATEWAY) &&
-    rt_key(rt)->sa_family == dst->sa_family)) {
+ struct rtentry *rt;
+
+ rt = rtalloc(gateway, RT_RESOLVE, rtableid);
+ if (rt != NULL)
+ ifa = rt->rt_ifa;
  rtfree(rt);
- return (NULL);
  }
- ifa = rt->rt_ifa;
- rtfree(rt);
  }
+ if (ifa == NULL)
+ return (NULL);
  if (ifa->ifa_addr->sa_family != dst->sa_family) {
  struct ifaddr *oifa = ifa;
  ifa = ifaof_ifpforaddr(dst, ifa->ifa_ifp);

Reply | Threaded
Open this post in threaded view
|

Re: routes get assigned to a wrong interface, openbsd 5.9

Mart Tõnso
Thank you! Assigning a proper ppp netmask solved this issue. I'll see
if I can get arround to testing the patch. Is there a chance of
including it in the "current"?

> By the way why do you use the same src and dst address?

I've been wondering about that myself, but that's how the client
interface ends up when server is running with "subnet" topology. It
seems to work well with bsd/linux/macos/windows clients, so I haven't
gone digging deeper.

Regards,

Mart

On Wed, Apr 13, 2016 at 10:45 AM, Martin Pieuchot <[hidden email]> wrote:
> Hello Mart,
>
> On 13/04/16(Wed) 09:22, Mart Tõnso wrote:
>> Ah, yes, sorry about that. Here's the full routing info with ifconfig
output:

>>
>> # ifconfig
>> [...]
>> ppp0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1500
>>         priority: 0
>>         groups: ppp egress
>>         inet 10.128.195.179 --> 10.64.64.64 netmask 0xff000000
>                                                       ^^^^^^^^^^
> Here is the problem.  For historical reasons the code that finds
> a matching interface to attach your route matches your gateway
> with ppp0's address/netmask.
>
> A workaround would be to change your ppp0 setup to use a /32 mask.
>
> A correct fix is included below, I'll be interested to hear if it
> works for you.
>
>> tun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1500
>>         priority: 0
>>         groups: tun
>>         status: active
>>         inet 10.88.0.124 --> 10.88.0.124 netmask 0xffffff00
>                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
> By the way why do you use the same src and dst address?
>
> Index: net/route.c
> ===================================================================
> RCS file: /cvs/src/sys/net/route.c,v
> retrieving revision 1.298
> diff -u -p -r1.298 route.c
> --- net/route.c 26 Mar 2016 21:56:04 -0000      1.298
> +++ net/route.c 13 Apr 2016 07:38:11 -0000
> @@ -740,20 +740,16 @@ ifa_ifwithroute(int flags, struct sockad
>                                 ifa = ifaof_ifpforaddr(dst, ifp);
>                         if_put(ifp);
>                 } else {
> -                       ifa = ifa_ifwithnet(gateway, rtableid);
> -               }
> -       }
> -       if (ifa == NULL) {
> -               struct rtentry  *rt = rtalloc(gateway, 0, rtableid);
> -               /* The gateway must be local if the same address family. */
> -               if (!rtisvalid(rt) || ((rt->rt_flags & RTF_GATEWAY) &&
> -                   rt_key(rt)->sa_family == dst->sa_family)) {
> +                       struct rtentry *rt;
> +
> +                       rt = rtalloc(gateway, RT_RESOLVE, rtableid);
> +                       if (rt != NULL)
> +                               ifa = rt->rt_ifa;
>                         rtfree(rt);
> -                       return (NULL);
>                 }
> -               ifa = rt->rt_ifa;
> -               rtfree(rt);
>         }
> +       if (ifa == NULL)
> +               return (NULL);
>         if (ifa->ifa_addr->sa_family != dst->sa_family) {
>                 struct ifaddr   *oifa = ifa;
>                 ifa = ifaof_ifpforaddr(dst, ifa->ifa_ifp);

Reply | Threaded
Open this post in threaded view
|

Re: routes get assigned to a wrong interface, openbsd 5.9

Martin Pieuchot
On 13/04/16(Wed) 13:27, Mart Tõnso wrote:
> Thank you! Assigning a proper ppp netmask solved this issue. I'll see
> if I can get arround to testing the patch. Is there a chance of
> including it in the "current"?

I'm waiting for you report, if it is positive I'll ask for reviews.  If
the review are ok, it will be included.

Reply | Threaded
Open this post in threaded view
|

Re: routes get assigned to a wrong interface, openbsd 5.9

Mart Tõnso
Hello,

It took a while, but I got the patch tested with current source.

I can confirm that even with the gaping /8 netmask on ppp0 interface,
routes end up on tun0 interface as expected.

Are there any specific tests you would like me to do or extra
information I could provide?

Regards,

Mart


On Wed, Apr 13, 2016 at 1:50 PM, Martin Pieuchot <[hidden email]> wrote:
> On 13/04/16(Wed) 13:27, Mart Tõnso wrote:
>> Thank you! Assigning a proper ppp netmask solved this issue. I'll see
>> if I can get arround to testing the patch. Is there a chance of
>> including it in the "current"?
>
> I'm waiting for you report, if it is positive I'll ask for reviews.  If
> the review are ok, it will be included.