carp: send only IPv4 carp packets on dual stack interface

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

carp: send only IPv4 carp packets on dual stack interface

Christopher Zimmermann-2
Hi,

as far as I can see a dual stack carp interface does not care whether it
receives advertisements addressed to IPv4 or IPv6. Any one will do.
So I propose to send IPv6 advertisements only when IPv4 is not possible.

Why?

- Noise can be reduced by using unicast advertisements.
   This is only possible for IPv4 by ``ifconfig carppeer``.
   I don't like flooding the whole network with carp advertisements when
   I may also unicast them.
- breaking IPv6 connectivity (for example by running iked without -6)
   will start a preempt-war, because failing ip6_output will cause the
   demote counter to be increased. That's what hit me.

I would suggest a change like this:

Index: ip_carp.c
===================================================================
RCS file: /cvs/src/sys/netinet/ip_carp.c,v
retrieving revision 1.342
diff -u -p -r1.342 ip_carp.c
--- ip_carp.c 8 Nov 2019 07:51:41 -0000 1.342
+++ ip_carp.c 15 Jan 2020 10:45:56 -0000
@@ -1175,7 +1175,7 @@ carp_send_ad(struct carp_vhost_entry *vh
  }
  }
  #ifdef INET6
- if (sc->sc_naddrs6) {
+ else if (sc->sc_naddrs6) {
  struct ip6_hdr *ip6;
 
  MGETHDR(m, M_DONTWAIT, MT_HEADER);


one could also use a slightly smaller hammer and only avoid sending IPv6
if the user requested an IPv4 unicast address:

- if (sc->sc_naddrs6) {
+ if (sc->sc_naddrs6 &&
+    (! sc->sc_naddrs ||
+    sc->sc_peer.s_addr == INADDR_CARP_GROUP) ) {


Christopher


--
http://gmerlin.de
OpenPGP: http://gmerlin.de/christopher.pub
CB07 DA40 B0B6 571D 35E2  0DEF 87E2 92A7 13E5 DEE1

Reply | Threaded
Open this post in threaded view
|

Re: carp: send only IPv4 carp packets on dual stack interface

Janne Johansson-3
Den ons 15 jan. 2020 kl 11:57 skrev Christopher Zimmermann <
[hidden email]>:

> So I propose to send IPv6 advertisements only when IPv4 is not possible.
>
> Why?
>
> - Noise can be reduced by using unicast advertisements.
>    This is only possible for IPv4 by ``ifconfig carppeer``.
>    I don't like flooding the whole network with carp advertisements when
>    I may also unicast them.
>

This sentence seems to assume people would only ever use two members in a
carp group, whereas having more than that is not impossible.
In those cases full meshed unicast would cause more packets than
multicasting one packet every second (+advskew).

--
May the most significant bit of your life be positive.
Reply | Threaded
Open this post in threaded view
|

Re: carp: send only IPv4 carp packets on dual stack interface

Christopher Zimmermann-5


On January 15, 2020 12:04:56 PM GMT+01:00, Janne Johansson <[hidden email]> wrote:

>Den ons 15 jan. 2020 kl 11:57 skrev Christopher Zimmermann <
>[hidden email]>:
>
>> So I propose to send IPv6 advertisements only when IPv4 is not
>possible.
>>
>> Why?
>>
>> - Noise can be reduced by using unicast advertisements.
>>    This is only possible for IPv4 by ``ifconfig carppeer``.
>>    I don't like flooding the whole network with carp advertisements
>when
>>    I may also unicast them.
>>
>
>This sentence seems to assume people would only ever use two members in
>a
>carp group

Well, even though it may have seemed like it, I did not assume this.

My concern is more about sending out through two network stacks, one of which (IPv6) may fail and produce unwanted effects in carp's demote logic.
Is there something to gain by sending out though both stacks?
http://gmerlin.de
OpenPGP: http://gmerlin.de/christopher.pub
CB07 DA40 B0B6 571D 35E2  0DEF 87E2 92A7 13E5 DEE1

Reply | Threaded
Open this post in threaded view
|

Re: carp: send only IPv4 carp packets on dual stack interface

Sebastian Benoit-3
In reply to this post by Christopher Zimmermann-2
Christopher Zimmermann([hidden email]) on 2020.01.15 11:55:43 +0100:

> Hi,
>
> as far as I can see a dual stack carp interface does not care whether it
> receives advertisements addressed to IPv4 or IPv6. Any one will do.
> So I propose to send IPv6 advertisements only when IPv4 is not possible.
>
> Why?
>
> - Noise can be reduced by using unicast advertisements.
>   This is only possible for IPv4 by ``ifconfig carppeer``.
>   I don't like flooding the whole network with carp advertisements when
>   I may also unicast them.

Maybe i'm getting confused, but in the problem description you were talking
about v6 vs v4, and here you argue about unicast (vs multicast?) being
better. Thats orthogonal, isnt it?

> - breaking IPv6 connectivity (for example by running iked without -6)
>   will start a preempt-war, because failing ip6_output will cause the
>   demote counter to be increased. That's what hit me.

But the whole point of carp is to notice broken connectivity. If you run v6
on an interface, you want to know if its working, no?

At the very least, this needs some more thought and testing in all the ways
carp can be configured.

> I would suggest a change like this:
>
> Index: ip_carp.c
> ===================================================================
> RCS file: /cvs/src/sys/netinet/ip_carp.c,v
> retrieving revision 1.342
> diff -u -p -r1.342 ip_carp.c
> --- ip_carp.c 8 Nov 2019 07:51:41 -0000 1.342
> +++ ip_carp.c 15 Jan 2020 10:45:56 -0000
> @@ -1175,7 +1175,7 @@ carp_send_ad(struct carp_vhost_entry *vh
>   }
>   }
>  #ifdef INET6
> - if (sc->sc_naddrs6) {
> + else if (sc->sc_naddrs6) {
>   struct ip6_hdr *ip6;
>  
>   MGETHDR(m, M_DONTWAIT, MT_HEADER);
>
>
> one could also use a slightly smaller hammer and only avoid sending IPv6
> if the user requested an IPv4 unicast address:
>
> - if (sc->sc_naddrs6) {
> + if (sc->sc_naddrs6 &&
> +    (! sc->sc_naddrs ||
> +    sc->sc_peer.s_addr == INADDR_CARP_GROUP) ) {
>
>
> Christopher
>
>
> --
> http://gmerlin.de
> OpenPGP: http://gmerlin.de/christopher.pub
> CB07 DA40 B0B6 571D 35E2  0DEF 87E2 92A7 13E5 DEE1
>

Reply | Threaded
Open this post in threaded view
|

Re: carp: send only IPv4 carp packets on dual stack interface

Stuart Henderson
In reply to this post by Janne Johansson-3
On 2020/01/15 12:04, Janne Johansson wrote:

> Den ons 15 jan. 2020 kl 11:57 skrev Christopher Zimmermann <
> [hidden email]>:
>
> > So I propose to send IPv6 advertisements only when IPv4 is not possible.
> >
> > Why?
> >
> > - Noise can be reduced by using unicast advertisements.
> >    This is only possible for IPv4 by ``ifconfig carppeer``.
> >    I don't like flooding the whole network with carp advertisements when
> >    I may also unicast them.
> >
>
> This sentence seems to assume people would only ever use two members in a
> carp group, whereas having more than that is not impossible.
> In those cases full meshed unicast would cause more packets than
> multicasting one packet every second (+advskew).
>
> --
> May the most significant bit of your life be positive.

I just filter the MACs on my switches..

Reply | Threaded
Open this post in threaded view
|

Re: carp: send only IPv4 carp packets on dual stack interface

Christopher Zimmermann-2
In reply to this post by Sebastian Benoit-3
On Wed, Jan 15, 2020 at 12:47:28PM +0100, Sebastian Benoit wrote:

>Christopher Zimmermann([hidden email]) on 2020.01.15 11:55:43 +0100:
>> Hi,
>>
>> as far as I can see a dual stack carp interface does not care whether it
>> receives advertisements addressed to IPv4 or IPv6. Any one will do.
>> So I propose to send IPv6 advertisements only when IPv4 is not possible.
>>
>> Why?
>>
>> - Noise can be reduced by using unicast advertisements.
>>   This is only possible for IPv4 by ``ifconfig carppeer``.
>>   I don't like flooding the whole network with carp advertisements when
>>   I may also unicast them.
>
>Maybe i'm getting confused, but in the problem description you were talking
>about v6 vs v4, and here you argue about unicast (vs multicast?) being
>better. Thats orthogonal, isnt it?

Yes, kind of. The point is we support ``carppeer`` for IPv4, but not for
IPv6.

>> - breaking IPv6 connectivity (for example by running iked without -6)
>>   will start a preempt-war, because failing ip6_output will cause the
>>   demote counter to be increased. That's what hit me.
>
>But the whole point of carp is to notice broken connectivity. If you run v6
>on an interface, you want to know if its working, no?

I grant you that much. But what kind of failures do you hope to detect
on the _sending_ carp master, that would not also affect the backup?

>At the very least, this needs some more thought and testing in all the ways
>carp can be configured.

Anyway, my main concern indeed is the broadcast noise generated by carp
and I would be equally happy if we had a ``carppeer6`` option. Would
that be considered?

>> I would suggest a change like this:
>>
>> Index: ip_carp.c
>> ===================================================================
>> RCS file: /cvs/src/sys/netinet/ip_carp.c,v
>> retrieving revision 1.342
>> diff -u -p -r1.342 ip_carp.c
>> --- ip_carp.c 8 Nov 2019 07:51:41 -0000 1.342
>> +++ ip_carp.c 15 Jan 2020 10:45:56 -0000
>> @@ -1175,7 +1175,7 @@ carp_send_ad(struct carp_vhost_entry *vh
>>   }
>>   }
>>  #ifdef INET6
>> - if (sc->sc_naddrs6) {
>> + else if (sc->sc_naddrs6) {
>>   struct ip6_hdr *ip6;
>>
>>   MGETHDR(m, M_DONTWAIT, MT_HEADER);
>>
>>
>> one could also use a slightly smaller hammer and only avoid sending IPv6
>> if the user requested an IPv4 unicast address:
>>
>> - if (sc->sc_naddrs6) {
>> + if (sc->sc_naddrs6 &&
>> +    (! sc->sc_naddrs ||
>> +    sc->sc_peer.s_addr == INADDR_CARP_GROUP) ) {
>>
>>
>> Christopher

--
http://gmerlin.de
OpenPGP: http://gmerlin.de/christopher.pub
CB07 DA40 B0B6 571D 35E2  0DEF 87E2 92A7 13E5 DEE1

Reply | Threaded
Open this post in threaded view
|

Re: carp: send only IPv4 carp packets on dual stack interface

Stefan Sperling-5
On Sat, Jan 18, 2020 at 06:18:21AM +0100, [hidden email] wrote:
> Anyway, my main concern indeed is the broadcast noise generated by carp and
> I would be equally happy if we had a ``carppeer6`` option. Would that be
> considered?

I could make use of carppeer6, too.

Reply | Threaded
Open this post in threaded view
|

Re: carp: send only IPv4 carp packets on dual stack interface

Stuart Henderson
In reply to this post by Christopher Zimmermann-2
On 2020/01/18 06:18, [hidden email] wrote:
> Anyway, my main concern indeed is the broadcast noise generated by carp and
> I would be equally happy if we had a ``carppeer6`` option. Would that be
> considered?

Adding carppeer6 seems a better/safer approach.

Reply | Threaded
Open this post in threaded view
|

Re: carp: send only IPv4 carp packets on dual stack interface

Claudio Jeker
On Sat, Jan 18, 2020 at 01:45:18PM +0000, Stuart Henderson wrote:
> On 2020/01/18 06:18, [hidden email] wrote:
> > Anyway, my main concern indeed is the broadcast noise generated by carp and
> > I would be equally happy if we had a ``carppeer6`` option. Would that be
> > considered?
>
> Adding carppeer6 seems a better/safer approach.
>

Is there a need to add a new command for this or could carppeer be exended
to support AF_INET6? In general it is better if there are less IPv6
specific commands.

--
:wq Claudio

Reply | Threaded
Open this post in threaded view
|

Re: carp: send only IPv4 carp packets on dual stack interface

Sebastian Benoit-3
In reply to this post by Christopher Zimmermann-2
[hidden email]([hidden email]) on 2020.01.18 06:18:21 +0100:

> On Wed, Jan 15, 2020 at 12:47:28PM +0100, Sebastian Benoit wrote:
> >Christopher Zimmermann([hidden email]) on 2020.01.15 11:55:43 +0100:
> >>Hi,
> >>
> >>as far as I can see a dual stack carp interface does not care whether it
> >>receives advertisements addressed to IPv4 or IPv6. Any one will do.
> >>So I propose to send IPv6 advertisements only when IPv4 is not possible.
> >>
> >>Why?
> >>
> >>- Noise can be reduced by using unicast advertisements.
> >>  This is only possible for IPv4 by ``ifconfig carppeer``.
> >>  I don't like flooding the whole network with carp advertisements when
> >>  I may also unicast them.
> >
> >Maybe i'm getting confused, but in the problem description you were talking
> >about v6 vs v4, and here you argue about unicast (vs multicast?) being
> >better. Thats orthogonal, isnt it?
>
> Yes, kind of. The point is we support ``carppeer`` for IPv4, but not for
> IPv6.
>
> >>- breaking IPv6 connectivity (for example by running iked without -6)
> >>  will start a preempt-war, because failing ip6_output will cause the
> >>  demote counter to be increased. That's what hit me.
> >
> >But the whole point of carp is to notice broken connectivity. If you run v6
> >on an interface, you want to know if its working, no?
>
> I grant you that much. But what kind of failures do you hope to detect
> on the _sending_ carp master, that would not also affect the backup?

sure: misconfigured pf. Missing routes. Buggy switch.
 
> >At the very least, this needs some more thought and testing in all the ways
> >carp can be configured.
>
> Anyway, my main concern indeed is the broadcast noise generated by carp
> and I would be equally happy if we had a ``carppeer6`` option. Would
> that be considered?

of course carppeer should work with v6, and as claudio says without an extra
keyword in ifconfig, but thats a trivial detail.

Reply | Threaded
Open this post in threaded view
|

Re: carp: send only IPv4 carp packets on dual stack interface

Stuart Henderson
On 2020/01/19 00:11, Sebastian Benoit wrote:

> [hidden email]([hidden email]) on 2020.01.18 06:18:21 +0100:
> > On Wed, Jan 15, 2020 at 12:47:28PM +0100, Sebastian Benoit wrote:
> > >Christopher Zimmermann([hidden email]) on 2020.01.15 11:55:43 +0100:
> > >>Hi,
> > >>
> > >>as far as I can see a dual stack carp interface does not care whether it
> > >>receives advertisements addressed to IPv4 or IPv6. Any one will do.
> > >>So I propose to send IPv6 advertisements only when IPv4 is not possible.
> > >>
> > >>Why?
> > >>
> > >>- Noise can be reduced by using unicast advertisements.
> > >>  This is only possible for IPv4 by ``ifconfig carppeer``.
> > >>  I don't like flooding the whole network with carp advertisements when
> > >>  I may also unicast them.
> > >
> > >Maybe i'm getting confused, but in the problem description you were talking
> > >about v6 vs v4, and here you argue about unicast (vs multicast?) being
> > >better. Thats orthogonal, isnt it?
> >
> > Yes, kind of. The point is we support ``carppeer`` for IPv4, but not for
> > IPv6.
> >
> > >>- breaking IPv6 connectivity (for example by running iked without -6)
> > >>  will start a preempt-war, because failing ip6_output will cause the
> > >>  demote counter to be increased. That's what hit me.
> > >
> > >But the whole point of carp is to notice broken connectivity. If you run v6
> > >on an interface, you want to know if its working, no?
> >
> > I grant you that much. But what kind of failures do you hope to detect
> > on the _sending_ carp master, that would not also affect the backup?
>
> sure: misconfigured pf. Missing routes. Buggy switch.

misconfigured mac address filter on switch.

> > >At the very least, this needs some more thought and testing in all the ways
> > >carp can be configured.
> >
> > Anyway, my main concern indeed is the broadcast noise generated by carp
> > and I would be equally happy if we had a ``carppeer6`` option. Would
> > that be considered?
>
> of course carppeer should work with v6, and as claudio says without an extra
> keyword in ifconfig, but thats a trivial detail.
>

Currently carp only handles one address per af, setting carppeer twice
changes the current peer address rather than adding another. A trivial
implementation that sets the v4 peer address if a v4 address is passed
in, and sets the v6 peer address if a v6 address is passed in, that
would mean things work differently with

ifconfig carp1 carppeer $foo
ifconfig carp1 carppeer $bar

depending on whether foo/bar are v4 or v6. Also removing a configured
carppeer address to reset to multicast is just done with -carppeer
with no way to indicate the af.

It would work pretty nicely if you could set multiple carppeer addresses
(of whatever af) and remove them individually. That's a more complex
change (carp would need to keep a list of peers per af rather than a
single address) but without something like that they can't really be
equals and it feels like shoehorning both afs into the same keyword
will just be confusing.

Reply | Threaded
Open this post in threaded view
|

Re: carp: send only IPv4 carp packets on dual stack interface

Christopher Zimmermann-2
On Sun, Jan 19, 2020 at 01:32:17PM +0000, Stuart Henderson wrote:

>On 2020/01/19 00:11, Sebastian Benoit wrote:
>> [hidden email]([hidden email]) on 2020.01.18 06:18:21 +0100:
>> > On Wed, Jan 15, 2020 at 12:47:28PM +0100, Sebastian Benoit wrote:
>> > >Christopher Zimmermann([hidden email]) on 2020.01.15 11:55:43 +0100:
>> > >>Hi,
>> > >>
>> > >>as far as I can see a dual stack carp interface does not care whether it
>> > >>receives advertisements addressed to IPv4 or IPv6. Any one will do.
>> > >>So I propose to send IPv6 advertisements only when IPv4 is not possible.
>> > >>
>> > >>Why?
>> > >>
>> > >>- Noise can be reduced by using unicast advertisements.
>> > >>  This is only possible for IPv4 by ``ifconfig carppeer``.
>> > >>  I don't like flooding the whole network with carp advertisements when
>> > >>  I may also unicast them.
>> > >
>> > >Maybe i'm getting confused, but in the problem description you were talking
>> > >about v6 vs v4, and here you argue about unicast (vs multicast?) being
>> > >better. Thats orthogonal, isnt it?
>> >
>> > Yes, kind of. The point is we support ``carppeer`` for IPv4, but not for
>> > IPv6.
>> >
>> > >>- breaking IPv6 connectivity (for example by running iked without -6)
>> > >>  will start a preempt-war, because failing ip6_output will cause the
>> > >>  demote counter to be increased. That's what hit me.
>> > >
>> > >But the whole point of carp is to notice broken connectivity. If you run v6
>> > >on an interface, you want to know if its working, no?
>> >
>> > I grant you that much. But what kind of failures do you hope to detect
>> > on the _sending_ carp master, that would not also affect the backup?
>>
>> sure: misconfigured pf. Missing routes. Buggy switch.
>
>misconfigured mac address filter on switch.

I'm afraid you guys haven't yet got the point I'm trying to make.

Current behaviour is that in a dual-stack carp setup failover only
happens when advertisements on _both_ AFs fail to reach the backup.
A node in backup state will stay in backup state as long as it receives
_any_ advertisements.
In my mind this is the only sensible way for a backup node to react.

If a backup node that fails to receive advertisements of only one AF
would transition to master it would in most cases start a preempt war.

So why do we even send dual-stack advertisements?
The only effect those dual-stack ipv6 advertisements currently have is
that they prevent failover when ipv4 connectivity breaks.

I would propose to choose one "sentinel" AF (in this case ipv4) and
failover whenever advertisements of this AF fail to reach the backup.

Monitoring multiple AFs is not helpful, because there is no good way in
which to react to a failure that affects only one AF.

>> > >At the very least, this needs some more thought and testing in all
>> > >the ways
>> > >carp can be configured.
>> >
>> > Anyway, my main concern indeed is the broadcast noise generated by carp
>> > and I would be equally happy if we had a ``carppeer6`` option. Would
>> > that be considered?
>>
>> of course carppeer should work with v6, and as claudio says without an extra
>> keyword in ifconfig, but thats a trivial detail.
>>
>
>Currently carp only handles one address per af, setting carppeer twice
>changes the current peer address rather than adding another. A trivial
>implementation that sets the v4 peer address if a v4 address is passed
>in, and sets the v6 peer address if a v6 address is passed in, that
>would mean things work differently with
>
>ifconfig carp1 carppeer $foo
>ifconfig carp1 carppeer $bar
>
>depending on whether foo/bar are v4 or v6. Also removing a configured
>carppeer address to reset to multicast is just done with -carppeer
>with no way to indicate the af.
>
>It would work pretty nicely if you could set multiple carppeer addresses
>(of whatever af) and remove them individually. That's a more complex
>change (carp would need to keep a list of peers per af rather than a
>single address) but without something like that they can't really be
>equals and it feels like shoehorning both afs into the same keyword
>will just be confusing.


--
http://gmerlin.de
OpenPGP: http://gmerlin.de/christopher.pub
CB07 DA40 B0B6 571D 35E2  0DEF 87E2 92A7 13E5 DEE1

Reply | Threaded
Open this post in threaded view
|

Re: carp: send only IPv4 carp packets on dual stack interface

David Gwynne-5


> On 23 May 2020, at 8:44 am, Christopher Zimmermann <[hidden email]> wrote:
>
> On Sun, Jan 19, 2020 at 01:32:17PM +0000, Stuart Henderson wrote:
>> On 2020/01/19 00:11, Sebastian Benoit wrote:
>>> [hidden email]([hidden email]) on 2020.01.18 06:18:21 +0100:
>>> > On Wed, Jan 15, 2020 at 12:47:28PM +0100, Sebastian Benoit wrote:
>>> > >Christopher Zimmermann([hidden email]) on 2020.01.15 11:55:43 +0100:
>>> > >>Hi,
>>> > >>
>>> > >>as far as I can see a dual stack carp interface does not care whether it
>>> > >>receives advertisements addressed to IPv4 or IPv6. Any one will do.
>>> > >>So I propose to send IPv6 advertisements only when IPv4 is not possible.
>>> > >>
>>> > >>Why?
>>> > >>
>>> > >>- Noise can be reduced by using unicast advertisements.
>>> > >>  This is only possible for IPv4 by ``ifconfig carppeer``.
>>> > >>  I don't like flooding the whole network with carp advertisements when
>>> > >>  I may also unicast them.
>>> > >
>>> > >Maybe i'm getting confused, but in the problem description you were talking
>>> > >about v6 vs v4, and here you argue about unicast (vs multicast?) being
>>> > >better. Thats orthogonal, isnt it?
>>> >
>>> > Yes, kind of. The point is we support ``carppeer`` for IPv4, but not for
>>> > IPv6.
>>> >
>>> > >>- breaking IPv6 connectivity (for example by running iked without -6)
>>> > >>  will start a preempt-war, because failing ip6_output will cause the
>>> > >>  demote counter to be increased. That's what hit me.
>>> > >
>>> > >But the whole point of carp is to notice broken connectivity. If you run v6
>>> > >on an interface, you want to know if its working, no?
>>> >
>>> > I grant you that much. But what kind of failures do you hope to detect
>>> > on the _sending_ carp master, that would not also affect the backup?
>>>
>>> sure: misconfigured pf. Missing routes. Buggy switch.
>>
>> misconfigured mac address filter on switch.
>
> I'm afraid you guys haven't yet got the point I'm trying to make.
>
> Current behaviour is that in a dual-stack carp setup failover only happens when advertisements on _both_ AFs fail to reach the backup.
> A node in backup state will stay in backup state as long as it receives _any_ advertisements.
> In my mind this is the only sensible way for a backup node to react.
>
> If a backup node that fails to receive advertisements of only one AF would transition to master it would in most cases start a preempt war.
> So why do we even send dual-stack advertisements?
> The only effect those dual-stack ipv6 advertisements currently have is that they prevent failover when ipv4 connectivity breaks.
>
> I would propose to choose one "sentinel" AF (in this case ipv4) and failover whenever advertisements of this AF fail to reach the backup.
>
> Monitoring multiple AFs is not helpful, because there is no good way in which to react to a failure that affects only one AF.

I don't know if this helps, but at work we use separate carp interfaces for v4 and v6. It ends up looking a bit like this:

# cat /etc/hostname.vlan871:
parent aggr0 vnetid 871
inet alias 192.0.2.2/24
inet6 alias 2001:db8:871::2/64
up

# cat /etc/hostname.carp40871
carpdev vlan871 vhid 47
-inet6
-group carp
group ipv4g
inet alias 192.0.2.1/24
up

# cat /etc/hostname.carp60871
carpdev vlan871 vhid 61
-group carp
group ipv6g
inet6 alias 2001:db8:871::1/64
up

This let's us run a pair of firewalls one active for v4 and the other for v6. We don't do any af-to in PF, so it works pretty well. But yeah, it means v4 and v6 fail separately.

>
>>> > >At the very least, this needs some more thought and testing in all > >the ways
>>> > >carp can be configured.
>>> >
>>> > Anyway, my main concern indeed is the broadcast noise generated by carp
>>> > and I would be equally happy if we had a ``carppeer6`` option. Would
>>> > that be considered?
>>>
>>> of course carppeer should work with v6, and as claudio says without an extra
>>> keyword in ifconfig, but thats a trivial detail.
>>>
>>
>> Currently carp only handles one address per af, setting carppeer twice
>> changes the current peer address rather than adding another. A trivial
>> implementation that sets the v4 peer address if a v4 address is passed
>> in, and sets the v6 peer address if a v6 address is passed in, that
>> would mean things work differently with
>>
>> ifconfig carp1 carppeer $foo
>> ifconfig carp1 carppeer $bar
>>
>> depending on whether foo/bar are v4 or v6. Also removing a configured
>> carppeer address to reset to multicast is just done with -carppeer
>> with no way to indicate the af.
>>
>> It would work pretty nicely if you could set multiple carppeer addresses
>> (of whatever af) and remove them individually. That's a more complex
>> change (carp would need to keep a list of peers per af rather than a
>> single address) but without something like that they can't really be
>> equals and it feels like shoehorning both afs into the same keyword
>> will just be confusing.
>
>
> --
> http://gmerlin.de
> OpenPGP: http://gmerlin.de/christopher.pub
> CB07 DA40 B0B6 571D 35E2  0DEF 87E2 92A7 13E5 DEE1

Reply | Threaded
Open this post in threaded view
|

Re: carp: send only IPv4 carp packets on dual stack interface

Christopher Zimmermann-2
On Sat, May 23, 2020 at 01:55:54PM +1000, David Gwynne wrote:

>
>
>> On 23 May 2020, at 8:44 am, Christopher Zimmermann <[hidden email]> wrote:
>>
>> On Sun, Jan 19, 2020 at 01:32:17PM +0000, Stuart Henderson wrote:
>>> On 2020/01/19 00:11, Sebastian Benoit wrote:
>>>> [hidden email]([hidden email]) on 2020.01.18 06:18:21 +0100:
>>>> > On Wed, Jan 15, 2020 at 12:47:28PM +0100, Sebastian Benoit wrote:
>>>> > >Christopher Zimmermann([hidden email]) on 2020.01.15 11:55:43 +0100:
>>>> > >>Hi,
>>>> > >>
>>>> > >>as far as I can see a dual stack carp interface does not care whether it
>>>> > >>receives advertisements addressed to IPv4 or IPv6. Any one will do.
>>>> > >>So I propose to send IPv6 advertisements only when IPv4 is not possible.
>>>> > >>
>>>> > >>Why?
>>>> > >>
>>>> > >>- Noise can be reduced by using unicast advertisements.
>>>> > >>  This is only possible for IPv4 by ``ifconfig carppeer``.
>>>> > >>  I don't like flooding the whole network with carp advertisements when
>>>> > >>  I may also unicast them.
>>>> > >
>>>> > >Maybe i'm getting confused, but in the problem description you were talking
>>>> > >about v6 vs v4, and here you argue about unicast (vs multicast?) being
>>>> > >better. Thats orthogonal, isnt it?
>>>> >
>>>> > Yes, kind of. The point is we support ``carppeer`` for IPv4, but not for
>>>> > IPv6.
>>>> >
>>>> > >>- breaking IPv6 connectivity (for example by running iked without -6)
>>>> > >>  will start a preempt-war, because failing ip6_output will cause the
>>>> > >>  demote counter to be increased. That's what hit me.
>>>> > >
>>>> > >But the whole point of carp is to notice broken connectivity. If you run v6
>>>> > >on an interface, you want to know if its working, no?
>>>> >
>>>> > I grant you that much. But what kind of failures do you hope to detect
>>>> > on the _sending_ carp master, that would not also affect the backup?
>>>>
>>>> sure: misconfigured pf. Missing routes. Buggy switch.
>>>
>>> misconfigured mac address filter on switch.
>>
>> I'm afraid you guys haven't yet got the point I'm trying to make.
>>
>> Current behaviour is that in a dual-stack carp setup failover only happens when advertisements on _both_ AFs fail to reach the backup.
>> A node in backup state will stay in backup state as long as it receives _any_ advertisements.
>> In my mind this is the only sensible way for a backup node to react.
>>
>> If a backup node that fails to receive advertisements of only one AF would transition to master it would in most cases start a preempt war.
>> So why do we even send dual-stack advertisements?
>> The only effect those dual-stack ipv6 advertisements currently have is that they prevent failover when ipv4 connectivity breaks.
>>
>> I would propose to choose one "sentinel" AF (in this case ipv4) and failover whenever advertisements of this AF fail to reach the backup.
>>
>> Monitoring multiple AFs is not helpful, because there is no good way in which to react to a failure that affects only one AF.
>
>I don't know if this helps, but at work we use separate carp interfaces for v4 and v6. It ends up looking a bit like this:
>
># cat /etc/hostname.vlan871:
>parent aggr0 vnetid 871
>inet alias 192.0.2.2/24
>inet6 alias 2001:db8:871::2/64
>up
>
># cat /etc/hostname.carp40871
>carpdev vlan871 vhid 47
>-inet6
>-group carp
>group ipv4g
>inet alias 192.0.2.1/24
>up
>
># cat /etc/hostname.carp60871
>carpdev vlan871 vhid 61
>-group carp
>group ipv6g
>inet6 alias 2001:db8:871::1/64
>up
>
>This let's us run a pair of firewalls one active for v4 and the other for v6. We don't do any af-to in PF, so it works pretty well. But yeah, it means v4 and v6 fail separately.

Yes that is a sensible and coherent setup where ipv4 and ipv6 are kept
completely separate.
But what I'm proposing to change is the dual stack mode of carp where
there is only one interface. There is no use for advertisements on two
address families, because in case of failure of only one address family
there is no sensible way to respond to that failure.

>>>> > >At the very least, this needs some more thought and testing in
>>>> > >all > >the ways
>>>> > >carp can be configured.
>>>> >
>>>> > Anyway, my main concern indeed is the broadcast noise generated by carp
>>>> > and I would be equally happy if we had a ``carppeer6`` option. Would
>>>> > that be considered?
>>>>
>>>> of course carppeer should work with v6, and as claudio says without an extra
>>>> keyword in ifconfig, but thats a trivial detail.
>>>>
>>>
>>> Currently carp only handles one address per af, setting carppeer twice
>>> changes the current peer address rather than adding another. A trivial
>>> implementation that sets the v4 peer address if a v4 address is passed
>>> in, and sets the v6 peer address if a v6 address is passed in, that
>>> would mean things work differently with
>>>
>>> ifconfig carp1 carppeer $foo
>>> ifconfig carp1 carppeer $bar
>>>
>>> depending on whether foo/bar are v4 or v6. Also removing a configured
>>> carppeer address to reset to multicast is just done with -carppeer
>>> with no way to indicate the af.
>>>
>>> It would work pretty nicely if you could set multiple carppeer addresses
>>> (of whatever af) and remove them individually. That's a more complex
>>> change (carp would need to keep a list of peers per af rather than a
>>> single address) but without something like that they can't really be
>>> equals and it feels like shoehorning both afs into the same keyword
>>> will just be confusing.

--
http://gmerlin.de
OpenPGP: http://gmerlin.de/christopher.pub
CB07 DA40 B0B6 571D 35E2  0DEF 87E2 92A7 13E5 DEE1