what would break arp on carp?

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

what would break arp on carp?

Devin Reade
I have an OpenBSD 5.8 stable carp setup where one of my upstream links
is serviced by a cable provider, a static IP is assigned, and I would
normally have no IP assigned to the carpdev:

# cat hostname.vr2
up
# cat hostname.carp2
inet aa.bb.cc.dd 255.255.255.248 NONE vhid 3 pass somepass
!/sbin/route -qn add -host -mpath default aa.bb.cc.ee

This has worked for a few years.  This past week, there was an equipment
change for the upstream router of that link, judging by the change in the
MAC of the gateway IP (aa.bb.cc.ee).

Since that change, that network segment hasn't been able to take any
carp-based traffic.  If I shut down one of the firewalls and revert that
link
on the remaining firewall to a non-carp link, then the traffic is
handled normally.

After some splunking (by using a hub on the upstream segment and sniffing
with tcpdump from another host with a static IP on the same segment) what
I can see is that when carp is NOT in use, the upstream router will do
the usual arp request and the firewall will do an arp reply with the MAC
of vr2, and everything is fine.

However, if carp IS in use, I can see the upstream router do the arp
request, followed by the firewall arp reply (with the carp MAC), however
the upstream router seems to ignore the answer and does continuous arp
requests.

I suspect that the fix for this is out of my control (since it seems
to be the upstream that is having problems), but I'm trying to understand
what would cause it.  Any thoughts?

I figured that the upstream might be blacklisting VRRP MACs, so I tried
assigning a non-VRRP MAC via lladdr in hostname.carp2, but that had no
effect on the outcome.  I also tried using a high-numbered vhid in case
vhid 3 was causing a conflict.

The cable company's troubleshooting procedure includes rebooting the
cable modem on MAC changes (presumably to clear the ARP cache), so that
was done on each configuration change.  A clear cache is confirmed in
that the original arp request from the upstream is sent to the ethernet
broadcast address.

Clues welcome.

Devin

Reply | Threaded
Open this post in threaded view
|

Re: what would break arp on carp?

Mihai Popescu-3
I'm a little bit interested by your setup, teoretically.

> However, if carp IS in use, I can see the upstream router do the arp
> request, followed by the firewall arp reply (with the carp MAC),

Is it the 'carp MAC' the MAC of vr2?

> however the upstream router seems to ignore the answer and does continuous arp
> requests.

Maybe that router is not receiving the arp response. Why would someone
ignore an ARP message?

Reply | Threaded
Open this post in threaded view
|

Re: what would break arp on carp?

Devin Reade
--On Monday, April 04, 2016 12:26:06 AM +0300 Mihai Popescu
<[hidden email]> wrote:

>> However, if carp IS in use, I can see the upstream router do the arp
>> request, followed by the firewall arp reply (with the carp MAC),
>
> Is it the 'carp MAC' the MAC of vr2?

No.  It is the lladdr shown in a `ifconfig carp2`, which would be
00:00:5e:00:01:03.  The lladdr of vr2 (the underlying carpdev) is
different.

>> however the upstream router seems to ignore the answer and does
>> continuous arp requests.
>
> Maybe that router is not receiving the arp response. Why would someone
> ignore an ARP message?

Indeed.  It's not clear why the arp replies are honoured in the non-carp
case but not in the carp case.

Devin