trunk-ing + carp-ing

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

trunk-ing + carp-ing

Ganguin Michel
Hi misc,

I have the following setup:

* O1 and O2: two openbsd firewall/routers with:

  * carp-ed client side interface
  * trunked server side interfaces + carp-ed trunk device

* SRV: server with trunked interfaces (for testing I'm using one server, but there will be 2 servers cross connected to the switches)

* s: two switches with no "inter switch link" capability

             +----+         +---+
          +--+ O1 +---------+ s +--+
          |  |    +--\   /--+   |  |  +-----+
+-----+   |  +----+   \ /   +---+  +--+ SRV |
| CLI +---+            X           +--+     |
+-----+   |  +----+   / \   +---+  |  +-----+
          |  | O2 +--/   \--+ s |  |
          +--+    +---------+   +--+
             +----+         +---+

So SRV uses carp ip address as route and CLI uses the external carp ip address as route to reach SRV. And my high availability setup is transparent for the client. Each hardware (cables, routers, switches) may fail and my service is still reachable.

My questions are:

* what trunk to use? Without inter switch aggregation support I tried failover and broadcast

  * failover works fine, but active/failover interfaces needs to be switched manually in case of failure to select to correct active path, if think it could be automated with ifstated but I didn't test it yet.
  * broadcast works fine too, but the client receives packets duplicated 4 times, I don't know if it is normal behavior, shouldn't duplicates be filtered by the trunk device? Or is there a way to filter them?

* Is my design correct, or did I miss something? Is there another (simpler) way to achieve my high availability goal?

Config:

O1> ifconfig                                                  
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 33196
        priority: 0
        groups: lo
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
        inet 127.0.0.1 netmask 0xff000000
vr0: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:0d:b9:2d:0f:54
        priority: 0
        groups: egress
        media: Ethernet autoselect (100baseTX full-duplex)
        status: active
        inet6 fe80::20d:b9ff:fe2d:f54%vr0 prefixlen 64 scopeid 0x1
vr1: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:0d:b9:2d:0f:55
        priority: 0
        trunk: trunkdev trunk0
        media: Ethernet autoselect (100baseTX full-duplex)
        status: active
        inet6 fe80::20d:b9ff:fe2d:f54%vr1 prefixlen 64 scopeid 0x2
vr2: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:0d:b9:2d:0f:55
        priority: 0
        trunk: trunkdev trunk0
        media: Ethernet autoselect (100baseTX full-duplex)
        status: active
        inet6 fe80::20d:b9ff:fe2d:f54%vr2 prefixlen 64 scopeid 0x3
enc0: flags=0<>
        priority: 0
        groups: enc
        status: active
pflog0: flags=141<UP,RUNNING,PROMISC> mtu 33196
        priority: 0
        groups: pflog
trunk0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:0d:b9:2d:0f:55
        priority: 0
        trunk: trunkproto broadcast
                trunkport vr2 active
                trunkport vr1 master,active
        groups: trunk
        media: Ethernet autoselect
        status: active
        inet 192.168.10.1 netmask 0xffffff00 broadcast 192.168.10.255
        inet6 fe80::20d:b9ff:fe2d:f55%trunk0 prefixlen 64 scopeid 0x7
carp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:00:5e:00:01:01
        priority: 0
        carp: BACKUP carpdev trunk0 vhid 1 advbase 1 advskew 0
        groups: carp
        status: backup
        inet6 fe80::200:5eff:fe00:101%carp0 prefixlen 64 scopeid 0x8
        inet 192.168.10.10 netmask 0xffffff00 broadcast 192.168.10.255
carp1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:00:5e:00:01:02
        priority: 0
        carp: BACKUP carpdev vr0 vhid 2 advbase 1 advskew 0
        groups: carp
        status: backup
        inet6 fe80::200:5eff:fe00:102%carp1 prefixlen 64 scopeid 0x9
        inet 10.0.0.10 netmask 0xffffff00 broadcast 10.0.0.255

O1> netstat -rn -f inet
Routing tables

Internet:
Destination        Gateway            Flags   Refs      Use   Mtu  Prio Iface
default            10.0.0.1           UGS        0        2     -     8 vr0
10.0.0/24          link#1             UC         0        0     -     4 vr0
127/8              127.0.0.1          UGRS       0        0 33196     8 lo0
127.0.0.1          127.0.0.1          UH         1        0 33196     4 lo0
192.168.10/24      link#7             UC         0        0     -     4 trunk0
224/4              127.0.0.1          URS        0        0 33196     8 lo0

O2> ifconfig                                                
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 33196
        priority: 0
        groups: lo
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
        inet 127.0.0.1 netmask 0xff000000
vr0: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:0d:b9:2c:e5:3c
        priority: 0
        groups: egress
        media: Ethernet autoselect (100baseTX full-duplex)
        status: active
        inet6 fe80::20d:b9ff:fe2c:e53c%vr0 prefixlen 64 scopeid 0x1
vr1: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:0d:b9:2c:e5:3d
        priority: 0
        trunk: trunkdev trunk0
        media: Ethernet autoselect (100baseTX full-duplex)
        status: active
        inet6 fe80::20d:b9ff:fe2c:e53c%vr1 prefixlen 64 scopeid 0x2
vr2: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:0d:b9:2c:e5:3d
        priority: 0
        trunk: trunkdev trunk0
        media: Ethernet autoselect (100baseTX full-duplex)
        status: active
        inet6 fe80::20d:b9ff:fe2c:e53c%vr2 prefixlen 64 scopeid 0x3
enc0: flags=0<>
        priority: 0
        groups: enc
        status: active
pflog0: flags=141<UP,RUNNING,PROMISC> mtu 33196
        priority: 0
        groups: pflog
trunk0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:0d:b9:2c:e5:3d
        priority: 0
        trunk: trunkproto broadcast
                trunkport vr2 active
                trunkport vr1 master,active
        groups: trunk
        media: Ethernet autoselect
        status: active
        inet 192.168.10.2 netmask 0xffffff00 broadcast 192.168.10.255
        inet6 fe80::20d:b9ff:fe2c:e53d%trunk0 prefixlen 64 scopeid 0x7
carp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:00:5e:00:01:01
        priority: 0
        carp: MASTER carpdev trunk0 vhid 1 advbase 1 advskew 0
        groups: carp
        status: master
        inet6 fe80::200:5eff:fe00:101%carp0 prefixlen 64 scopeid 0x8
        inet 192.168.10.10 netmask 0xffffff00 broadcast 192.168.10.255
carp1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:00:5e:00:01:02
        priority: 0
        carp: MASTER carpdev vr0 vhid 2 advbase 1 advskew 0
        groups: carp
        status: master
        inet6 fe80::200:5eff:fe00:102%carp1 prefixlen 64 scopeid 0x9
        inet 10.0.0.10 netmask 0xffffff00 broadcast 10.0.0.255

O2> netstat -rn -f inet
Routing tables

Internet:
Destination        Gateway            Flags   Refs      Use   Mtu  Prio Iface
default            10.0.0.1           UGS        0        1     -     8 vr0
10.0.0/24          link#1             UC         0        0     -     4 vr0
10.0.0.10          10.0.0.10          UH         0        0     -     4 carp1
127/8              127.0.0.1          UGRS       0        0 33196     8 lo0
127.0.0.1          127.0.0.1          UH         1        0 33196     4 lo0
192.168.10/24      link#7             UC         0        0     -     4 trunk0
192.168.10.10      192.168.172.50     UH         0        0     -     4 carp0
224/4              127.0.0.1          URS        0        0 33196     8 lo0

SRV> ifconfig                                    
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 33196
        priority: 0
        groups: lo
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
        inet 127.0.0.1 netmask 0xff000000
vr0: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:0d:b9:2c:ff:ad
        priority: 0
        trunk: trunkdev trunk0
        media: Ethernet autoselect (100baseTX full-duplex)
        status: active
        inet6 fe80::20d:b9ff:fe2c:ffac%vr1 prefixlen 64 scopeid 0x2
vr1: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:0d:b9:2c:ff:ad
        priority: 0
        trunk: trunkdev trunk0
        media: Ethernet autoselect (100baseTX full-duplex)
        status: active
        inet6 fe80::20d:b9ff:fe2c:ffac%vr2 prefixlen 64 scopeid 0x3
enc0: flags=0<>
        priority: 0
        groups: enc
        status: active
pflog0: flags=141<UP,RUNNING,PROMISC> mtu 33196
        priority: 0
        groups: pflog
trunk0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:0d:b9:2c:ff:ad
        priority: 0
        trunk: trunkproto broadcast
                trunkport vr1 active
                trunkport vr0 master,active
        groups: trunk
        media: Ethernet autoselect
        status: active
        inet 192.168.10.100 netmask 0xffffff00 broadcast 192.168.10.255
        inet6 fe80::20d:b9ff:fe2c:ffad%trunk0 prefixlen 64 scopeid 0x7

SRV> netstat -rn -f inet
Routing tables

Internet:
Destination        Gateway            Flags   Refs      Use   Mtu  Prio Iface
default            192.168.10.10      GS         0    14754     -     8 trunk0
127/8              127.0.0.1          UGRS       0        0 33196     8 lo0
127.0.0.1          127.0.0.1          UH         1        0 33196     4 lo0
192.168.10/24      link#7             UC         0        0     -     4 trunk0
224/4              127.0.0.1          URS        0        0 33196     8 lo0

Reply | Threaded
Open this post in threaded view
|

Re: trunk-ing + carp-ing

Ganguin Michel
Hi again,

Thanks for your responses.

I found a setup that satisfies me:

* SRV: trunk in failover mode
* O1, O2: trunk in broadcast mode

As only one link at a time is active on SRV, I don't get duplicates. And as it is the only one being in failover mode, no need of ifstated.

Best Regards
Michel

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Ganguin Michel
Sent: mercredi, 19. juin 2013 11:45
To: [hidden email]
Subject: trunk-ing + carp-ing

Hi misc,

I have the following setup:

* O1 and O2: two openbsd firewall/routers with:

  * carp-ed client side interface
  * trunked server side interfaces + carp-ed trunk device

* SRV: server with trunked interfaces (for testing I'm using one server, but there will be 2 servers cross connected to the switches)

* s: two switches with no "inter switch link" capability

             +----+         +---+
          +--+ O1 +---------+ s +--+
          |  |    +--\   /--+   |  |  +-----+
+-----+   |  +----+   \ /   +---+  +--+ SRV |
| CLI +---+            X           +--+     |
+-----+   |  +----+   / \   +---+  |  +-----+
          |  | O2 +--/   \--+ s |  |
          +--+    +---------+   +--+
             +----+         +---+

So SRV uses carp ip address as route and CLI uses the external carp ip address as route to reach SRV. And my high availability setup is transparent for the client. Each hardware (cables, routers, switches) may fail and my service is still reachable.

My questions are:

* what trunk to use? Without inter switch aggregation support I tried failover and broadcast

  * failover works fine, but active/failover interfaces needs to be switched manually in case of failure to select to correct active path, if think it could be automated with ifstated but I didn't test it yet.
  * broadcast works fine too, but the client receives packets duplicated 4 times, I don't know if it is normal behavior, shouldn't duplicates be filtered by the trunk device? Or is there a way to filter them?

* Is my design correct, or did I miss something? Is there another (simpler) way to achieve my high availability goal?

Config:

O1> ifconfig                                                  
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 33196
        priority: 0
        groups: lo
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
        inet 127.0.0.1 netmask 0xff000000
vr0: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:0d:b9:2d:0f:54
        priority: 0
        groups: egress
        media: Ethernet autoselect (100baseTX full-duplex)
        status: active
        inet6 fe80::20d:b9ff:fe2d:f54%vr0 prefixlen 64 scopeid 0x1
vr1: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:0d:b9:2d:0f:55
        priority: 0
        trunk: trunkdev trunk0
        media: Ethernet autoselect (100baseTX full-duplex)
        status: active
        inet6 fe80::20d:b9ff:fe2d:f54%vr1 prefixlen 64 scopeid 0x2
vr2: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:0d:b9:2d:0f:55
        priority: 0
        trunk: trunkdev trunk0
        media: Ethernet autoselect (100baseTX full-duplex)
        status: active
        inet6 fe80::20d:b9ff:fe2d:f54%vr2 prefixlen 64 scopeid 0x3
enc0: flags=0<>
        priority: 0
        groups: enc
        status: active
pflog0: flags=141<UP,RUNNING,PROMISC> mtu 33196
        priority: 0
        groups: pflog
trunk0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:0d:b9:2d:0f:55
        priority: 0
        trunk: trunkproto broadcast
                trunkport vr2 active
                trunkport vr1 master,active
        groups: trunk
        media: Ethernet autoselect
        status: active
        inet 192.168.10.1 netmask 0xffffff00 broadcast 192.168.10.255
        inet6 fe80::20d:b9ff:fe2d:f55%trunk0 prefixlen 64 scopeid 0x7
carp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:00:5e:00:01:01
        priority: 0
        carp: BACKUP carpdev trunk0 vhid 1 advbase 1 advskew 0
        groups: carp
        status: backup
        inet6 fe80::200:5eff:fe00:101%carp0 prefixlen 64 scopeid 0x8
        inet 192.168.10.10 netmask 0xffffff00 broadcast 192.168.10.255
carp1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:00:5e:00:01:02
        priority: 0
        carp: BACKUP carpdev vr0 vhid 2 advbase 1 advskew 0
        groups: carp
        status: backup
        inet6 fe80::200:5eff:fe00:102%carp1 prefixlen 64 scopeid 0x9
        inet 10.0.0.10 netmask 0xffffff00 broadcast 10.0.0.255

O1> netstat -rn -f inet
Routing tables

Internet:
Destination        Gateway            Flags   Refs      Use   Mtu  Prio Iface
default            10.0.0.1           UGS        0        2     -     8 vr0
10.0.0/24          link#1             UC         0        0     -     4 vr0
127/8              127.0.0.1          UGRS       0        0 33196     8 lo0
127.0.0.1          127.0.0.1          UH         1        0 33196     4 lo0
192.168.10/24      link#7             UC         0        0     -     4 trunk0
224/4              127.0.0.1          URS        0        0 33196     8 lo0

O2> ifconfig                                                
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 33196
        priority: 0
        groups: lo
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
        inet 127.0.0.1 netmask 0xff000000
vr0: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:0d:b9:2c:e5:3c
        priority: 0
        groups: egress
        media: Ethernet autoselect (100baseTX full-duplex)
        status: active
        inet6 fe80::20d:b9ff:fe2c:e53c%vr0 prefixlen 64 scopeid 0x1
vr1: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:0d:b9:2c:e5:3d
        priority: 0
        trunk: trunkdev trunk0
        media: Ethernet autoselect (100baseTX full-duplex)
        status: active
        inet6 fe80::20d:b9ff:fe2c:e53c%vr1 prefixlen 64 scopeid 0x2
vr2: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:0d:b9:2c:e5:3d
        priority: 0
        trunk: trunkdev trunk0
        media: Ethernet autoselect (100baseTX full-duplex)
        status: active
        inet6 fe80::20d:b9ff:fe2c:e53c%vr2 prefixlen 64 scopeid 0x3
enc0: flags=0<>
        priority: 0
        groups: enc
        status: active
pflog0: flags=141<UP,RUNNING,PROMISC> mtu 33196
        priority: 0
        groups: pflog
trunk0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:0d:b9:2c:e5:3d
        priority: 0
        trunk: trunkproto broadcast
                trunkport vr2 active
                trunkport vr1 master,active
        groups: trunk
        media: Ethernet autoselect
        status: active
        inet 192.168.10.2 netmask 0xffffff00 broadcast 192.168.10.255
        inet6 fe80::20d:b9ff:fe2c:e53d%trunk0 prefixlen 64 scopeid 0x7
carp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:00:5e:00:01:01
        priority: 0
        carp: MASTER carpdev trunk0 vhid 1 advbase 1 advskew 0
        groups: carp
        status: master
        inet6 fe80::200:5eff:fe00:101%carp0 prefixlen 64 scopeid 0x8
        inet 192.168.10.10 netmask 0xffffff00 broadcast 192.168.10.255
carp1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:00:5e:00:01:02
        priority: 0
        carp: MASTER carpdev vr0 vhid 2 advbase 1 advskew 0
        groups: carp
        status: master
        inet6 fe80::200:5eff:fe00:102%carp1 prefixlen 64 scopeid 0x9
        inet 10.0.0.10 netmask 0xffffff00 broadcast 10.0.0.255

O2> netstat -rn -f inet
Routing tables

Internet:
Destination        Gateway            Flags   Refs      Use   Mtu  Prio Iface
default            10.0.0.1           UGS        0        1     -     8 vr0
10.0.0/24          link#1             UC         0        0     -     4 vr0
10.0.0.10          10.0.0.10          UH         0        0     -     4 carp1
127/8              127.0.0.1          UGRS       0        0 33196     8 lo0
127.0.0.1          127.0.0.1          UH         1        0 33196     4 lo0
192.168.10/24      link#7             UC         0        0     -     4 trunk0
192.168.10.10      192.168.172.50     UH         0        0     -     4 carp0
224/4              127.0.0.1          URS        0        0 33196     8 lo0

SRV> ifconfig                                    
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 33196
        priority: 0
        groups: lo
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
        inet 127.0.0.1 netmask 0xff000000
vr0: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:0d:b9:2c:ff:ad
        priority: 0
        trunk: trunkdev trunk0
        media: Ethernet autoselect (100baseTX full-duplex)
        status: active
        inet6 fe80::20d:b9ff:fe2c:ffac%vr1 prefixlen 64 scopeid 0x2
vr1: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:0d:b9:2c:ff:ad
        priority: 0
        trunk: trunkdev trunk0
        media: Ethernet autoselect (100baseTX full-duplex)
        status: active
        inet6 fe80::20d:b9ff:fe2c:ffac%vr2 prefixlen 64 scopeid 0x3
enc0: flags=0<>
        priority: 0
        groups: enc
        status: active
pflog0: flags=141<UP,RUNNING,PROMISC> mtu 33196
        priority: 0
        groups: pflog
trunk0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:0d:b9:2c:ff:ad
        priority: 0
        trunk: trunkproto broadcast
                trunkport vr1 active
                trunkport vr0 master,active
        groups: trunk
        media: Ethernet autoselect
        status: active
        inet 192.168.10.100 netmask 0xffffff00 broadcast 192.168.10.255
        inet6 fe80::20d:b9ff:fe2c:ffad%trunk0 prefixlen 64 scopeid 0x7

SRV> netstat -rn -f inet
Routing tables

Internet:
Destination        Gateway            Flags   Refs      Use   Mtu  Prio Iface
default            192.168.10.10      GS         0    14754     -     8 trunk0
127/8              127.0.0.1          UGRS       0        0 33196     8 lo0
127.0.0.1          127.0.0.1          UH         1        0 33196     4 lo0
192.168.10/24      link#7             UC         0        0     -     4 trunk0
224/4              127.0.0.1          URS        0        0 33196     8 lo0