HA: pair of firewalls, 2 switches and 1 server

classic Classic list List threaded Threaded
27 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: HA: pair of firewalls, 2 switches and 1 server

Reyk Floeter-2
On Thu, May 20, 2010 at 11:31:22PM +0300, Jussi Peltola wrote:
> I do this too. In addition to the previously mentioned problems with
> cheap switches losing their configs (and vlans) you should make sure the
> active interfaces are all on one switch so that the link between them
> isn't uselessly used; this will also avoid an unpleasant split brain
> event if that link ever happens to fail. But in this case you will also
> have to very carefully check the other switch stays properly configured so
> the backup interfaces will actually pass the traffic you want.
>

don't mix up cheap switches with crap switches.  actually, some very
expensive switches are really crappy indeed.  but i don't see your
"problems", you just have to take care a little bit and don't try to
run your highly redundant high-performance firewall cluster with a
bunch of SOHO linksys switches (oh wait, they're cisco now).

but there is no real problem, trunk failover with carp + pfsync and
redundant switches works very well and i have installed it in many
different highly available production sites.  it is hard to make it
not work unless you configure your switches wrong - eg. by cascading
the redundant switches to other uplink switches and creating some
weird loops.

> Linux's bonding module has an arp monitor which solves some of these
> problems, but the implementation is so hackish (as usual there...) that
> I'd rather not use it in production. arping and ifstated might do the
> same on openbsd, but I'm not sure if that will work when the interfaces
> are trunk ports. I'll need to check this when I have time.
>

why not?  trunk is just a "normal" ethernet interface.

the linux bondage trick sounds hackish, but link detection protocols
like udld or bfd should help here on the ethernet level.  many managed
switches support one of these protocols and i'd like to do this on the
openbsd side at some point to alter the link state based on optional
uni-/bidirectional link detection.

reyk

Reply | Threaded
Open this post in threaded view
|

Re: HA: pair of firewalls, 2 switches and 1 server

Tomoyuki Sakurai-5
In reply to this post by Axel Rau
On Tue, May 18, 2010 at 10:32 PM, Axel Rau <[hidden email]> wrote:

> Yes, but what carps/trunks do I need?

I'm doing carp(4)+pfsync(4)+bridge(4)+vether(4)+trunk(4)+ospfd(8) for
L3/L2 redundancy.

Part of my config can be found at:
http://cvs.openbsd.org/cgi-bin/query-pr-wrapper?full=yes&numbers=6318

You need additional two OSPF routers for L3 redundancy (claudio@
explained why in a paper).

--
Tomoyuki Sakurai

Reply | Threaded
Open this post in threaded view
|

Re: HA: pair of firewalls, 2 switches and 1 server

Olivier Cherrier
In reply to this post by Reyk Floeter-2
On Fri, May 21, 2010 at 12:22:10AM +0200, [hidden email] wrote:

> > Linux's bonding module has an arp monitor which solves some of these
> > problems, but the implementation is so hackish (as usual there...) that
> > I'd rather not use it in production. arping and ifstated might do the
> > same on openbsd, but I'm not sure if that will work when the interfaces
> > are trunk ports. I'll need to check this when I have time.
> >
>
> why not?  trunk is just a "normal" ethernet interface.
>
> the linux bondage trick sounds hackish, but link detection protocols
> like udld or bfd should help here on the ethernet level.  many managed
> switches support one of these protocols and i'd like to do this on the
> openbsd side at some point to alter the link state based on optional
> uni-/bidirectional link detection.

If one of the 2 ports of the switch where you are connected is badly
configured (wrong VLAN, ...), you may have problems.  The link can be
up but being unusable.  In that case, having something like the arp
monitor seems to not be so stupid; doesn't it ?

--
Olivier Cherrier - Symacx.com
mailto:[hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: HA: pair of firewalls, 2 switches and 1 server

Jussi Peltola
In reply to this post by Reyk Floeter-2
On Fri, May 21, 2010 at 12:22:10AM +0200, Reyk Floeter wrote:
> > Linux's bonding module has an arp monitor which solves some of these
> > problems, but the implementation is so hackish (as usual there...) that
> > I'd rather not use it in production. arping and ifstated might do the
> > same on openbsd, but I'm not sure if that will work when the interfaces
> > are trunk ports. I'll need to check this when I have time.
> >
>
> why not?  trunk is just a "normal" ethernet interface.
 
the monitoring should be done on the ports/slaves/child interfaces, not
the trunk itself. I don't see why arping wouldn't work on those, either,
but I haven't tested it.

> the linux bondage trick sounds hackish, but link detection protocols
> like udld or bfd should help here on the ethernet level.  many managed
> switches support one of these protocols and i'd like to do this on the
> openbsd side at some point to alter the link state based on optional
> uni-/bidirectional link detection.
 
This would be a pretty good "out of the box" solution. "end to end"
monitoring with ifstated would still be useful especially on the end
hosts, which can just (ar)ping the carp gateway and kick out interfaces
that can't reach it. That would work against config mistakes (missing
vlans) and all kinds of subtle switch failures. For the routers this is
not so easy, they would need to ping an assortment of end hosts to get a
really useful "end to end" check. And there is always relayd et al that
solve the problem even better (in the cases where it can be used.)

Reply | Threaded
Open this post in threaded view
|

Re: HA: pair of firewalls, 2 switches and 1 server

Axel Rau
In reply to this post by Reyk Floeter-2
Am 20.05.2010 um 22:07 schrieb Reyk Floeter:

I will try the following with unmanaged switches, no RST:

      +---+                      +------+
      |fw1|        +-----+       |      |
  ----+em1+--------+ sw1 +-------+      |
carp0|em2+--+     +-+-+-+    em0|      |
      |   |  |       | |         |      |
      +-+-+  |  +----+ |         |      |
        |    |  |      |         |Server|
      +-+-+  +--|----+ |         | fbsd |
      |fw2|     |    | |         |      |
      |em1+-----+  +-+-+-+       |      |
  ----+em2+--------+ sw2 +-------+      |
carp0|   |        +-----+    em1|      |
      +---+                      +------+
           vlan1+vlan2      vlan2

fw1# ifconfig em0 up
fw1# ifconfig em1 up
fw1# ifconfig trunk0 trunkport em0 trunkport em1 trunkproto failover up
fw1# ifconfig vlan1 vlandev trunk0 descr UPLINK 10.1.1.2/24
fw1# ifconfig vlan2 vlandev trunk0 descr SERVERLAN 10.1.2.2/24
fw1# ifconfig carp1 vhid 1 carpdev vlan1 10.1.1.1/24
fw1# ifconfig carp2 vhid 2 carpdev vlan2 10.1.2.1/24

fw2# ifconfig em0 up
fw2# ifconfig em1 up
fw2# ifconfig trunk0 trunkport em0 trunkport em1 trunkproto failover up
fw2# ifconfig vlan1 vlandev trunk0 descr UPLINK 10.1.1.3/24
fw2# ifconfig vlan2 vlandev trunk0 descr SERVERLAN 10.1.2.3/24
fw2# ifconfig carp1 vhid 1 carpdev vlan1 advskew 100 10.1.1.1/24
fw2# ifconfig carp2 vhid 2 carpdev vlan2 advskew 100 10.1.2.1/24


On fbsd:

fbsd# ifconfig em0 up
fbsd# ifconfig em1 up
fbsd# ifconfig lagg0 create
fbsd# ifconfig lagg0 laggproto failover laggport em0 laggport em1 up
fbsd# ifconfig vlan2 create
fbsd# ifconfig vlan2 vlan 2 vlandev lagg0 10.1.2.10 netmask
255.255.255.0 up

fbsd# route add default 10.1.2.1

Axel
---
[hidden email]  PGP-Key:29E99DD6  +49 151 2300 9283  computing @
chaos claudius

Reply | Threaded
Open this post in threaded view
|

Re: HA: pair of firewalls, 2 switches and 1 server

Axel Rau
Am 21.05.2010 um 12:55 schrieb Axel Rau:

> Am 20.05.2010 um 22:07 schrieb Reyk Floeter:
>
> I will try the following with unmanaged switches, no RST:
>

> On fbsd:
>
> fbsd# ifconfig em0 up
> fbsd# ifconfig em1 up
> fbsd# ifconfig lagg0 create
> fbsd# ifconfig lagg0 laggproto failover laggport em0 laggport em1 up
> fbsd# ifconfig vlan2 create
> fbsd# ifconfig vlan2 vlan 2 vlandev lagg0 10.1.2.10 netmask
> 255.255.255.0 up
This started working with 2 unmanaged switches after applying a patch
to fbsd.8.0 (bug with vlan on top of lagg).

Thanks again Reyk for your help,
Axel
---
[hidden email]  PGP-Key:29E99DD6  +49 151 2300 9283  computing @
chaos claudius

Reply | Threaded
Open this post in threaded view
|

Re: HA: pair of firewalls, 2 switches and 1 server

Axel Rau
In reply to this post by Tomoyuki Sakurai-5
Am 21.05.2010 um 01:53 schrieb Tomoyuki Sakurai:

> You need additional two OSPF routers for L3 redundancy (claudio@
> explained why in a paper).
Thanks for the hint, Tomoyuki.
I have now ospfd running on both firewalls, which was one necessary
stop towards success.

Axel
---
[hidden email]  PGP-Key:29E99DD6  +49 151 2300 9283  computing @
chaos claudius

12