pf and ICMP in asymmetric routing setups

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

pf and ICMP in asymmetric routing setups

bernd-34
Hi list,

I've got two OpenBSD 5.1-stable/amd64 boxes employed which do all the
routing for our AS (OpenBGPd and OpenOSPFd). I see asymmetric traffic (I
thought it to be that way), which itself doesn't really create problems.
However, I see problems with ICMP. pf seems to drop all but the first
response from any of the hosts within our network (seen from the
Internet).

Any idea how to deal with this? As soon as I turn off pf, everything
runs smoothly.

Best,

Bernd

Reply | Threaded
Open this post in threaded view
|

Re: pf and ICMP in asymmetric routing setups

Simon Perreault-3
On 2012-06-12 14:08, Bernd wrote:
> I've got two OpenBSD 5.1-stable/amd64 boxes employed which do all the
> routing for our AS (OpenBGPd and OpenOSPFd). I see asymmetric traffic (I
> thought it to be that way), which itself doesn't really create problems.
> However, I see problems with ICMP. pf seems to drop all but the first
> response from any of the hosts within our network (seen from the Internet).
>
> Any idea how to deal with this? As soon as I turn off pf, everything
> runs smoothly.

Without having the details of your setup, the big principle is: pf is
stateful (by default). Statefulness doesn't play well with asymmetric
routing. I'm sure if you investigate a little bit more you'll discover
it's not limited to ICMP.

In the end the solution will be one of: remove statefulness, avoid
asymmetric routing, or share state with pfsync.

My two cents: try to avoid statefulness on core routers. Move stateful
elements to the edge, where routing is symmetric.

Simon

Reply | Threaded
Open this post in threaded view
|

Re: pf and ICMP in asymmetric routing setups

bernd-34
Am 2012-06-12 20:24, schrieb Simon Perreault:

> On 2012-06-12 14:08, Bernd wrote:
>> I've got two OpenBSD 5.1-stable/amd64 boxes employed which do all
>> the
>> routing for our AS (OpenBGPd and OpenOSPFd). I see asymmetric
>> traffic (I
>> thought it to be that way), which itself doesn't really create
>> problems.
>> However, I see problems with ICMP. pf seems to drop all but the
>> first
>> response from any of the hosts within our network (seen from the
>> Internet).
>>
>> Any idea how to deal with this? As soon as I turn off pf, everything
>> runs smoothly.
>
> Without having the details of your setup, the big principle is: pf is
> stateful (by default). Statefulness doesn't play well with asymmetric
> routing. I'm sure if you investigate a little bit more you'll
> discover
> it's not limited to ICMP.
>
> In the end the solution will be one of: remove statefulness, avoid
> asymmetric routing, or share state with pfsync.

I thought of removing statefulness or using pfsync. I run quite a few
load balancer setups that use, of course, pfsync and it runs like a
charm. However, removing statefulness seems the more appropriate
solution to me. Removing asymmetry isn't really an option, I guess, as
there's more infrastructure than just my two core routers.

> My two cents: try to avoid statefulness on core routers. Move
> stateful elements to the edge, where routing is symmetric.

What might be the easiest solution to have pf not care about states any
longer -- using 'keep state sloppy'? Or disabling statefulness entirely
(how?)?

> Simon

Thanks,

Bernd

Reply | Threaded
Open this post in threaded view
|

Re: pf and ICMP in asymmetric routing setups

Simon Perreault-3
On 2012-06-12 15:55, Bernd wrote:
> What might be the easiest solution to have pf not care about states any
> longer -- using 'keep state sloppy'? Or disabling statefulness entirely
> (how?)?

If you don't need it, just disable pf. echo pf=NO >>/etc/rc.conf.local

Sloppy tracking could work. Also check out "flags any".

Tagging "no state" at the end of each rule could incur a performance hit
because the rule set will need to be traversed for each packet instead
of relying on the state table. Only do it if performance doesn't matter
or you have few rules. I would definitely try it for the kind of
coarse-grain ACL we usually see on core routers, i.e. a single pass rule
with a table of allowed source/destination addresses.

Simon

Reply | Threaded
Open this post in threaded view
|

Re: pf and ICMP in asymmetric routing setups

Stuart Henderson
In reply to this post by Simon Perreault-3
On 2012-06-12, Simon Perreault <[hidden email]> wrote:

> On 2012-06-12 14:08, Bernd wrote:
>> I've got two OpenBSD 5.1-stable/amd64 boxes employed which do all the
>> routing for our AS (OpenBGPd and OpenOSPFd). I see asymmetric traffic (I
>> thought it to be that way), which itself doesn't really create problems.
>> However, I see problems with ICMP. pf seems to drop all but the first
>> response from any of the hosts within our network (seen from the Internet).
>>
>> Any idea how to deal with this? As soon as I turn off pf, everything
>> runs smoothly.
>
> Without having the details of your setup, the big principle is: pf is
> stateful (by default). Statefulness doesn't play well with asymmetric
> routing. I'm sure if you investigate a little bit more you'll discover
> it's not limited to ICMP.
>
> In the end the solution will be one of: remove statefulness, avoid
> asymmetric routing, or share state with pfsync.

If using pfsync for this, you would want to look at "defer", see pfsync(4).

Sloppy states might be more appropriate for this scenario though, and
would let you use other things which require state tracking, e.g. pflow(4).

Reply | Threaded
Open this post in threaded view
|

Re: pf and ICMP in asymmetric routing setups

Insan Praja SW
Hi,

On Wed, 13 Jun 2012 08:07:31 +0700, Stuart Henderson <[hidden email]>  
wrote:

> On 2012-06-12, Simon Perreault <[hidden email]> wrote:
>> On 2012-06-12 14:08, Bernd wrote:
>>> I've got two OpenBSD 5.1-stable/amd64 boxes employed which do all the
>>> routing for our AS (OpenBGPd and OpenOSPFd). I see asymmetric traffic  
>>> (I
>>> thought it to be that way), which itself doesn't really create  
>>> problems.
>>> However, I see problems with ICMP. pf seems to drop all but the first
>>> response from any of the hosts within our network (seen from the  
>>> Internet).
>>>
>>> Any idea how to deal with this? As soon as I turn off pf, everything
>>> runs smoothly.
>>
>> Without having the details of your setup, the big principle is: pf is
>> stateful (by default). Statefulness doesn't play well with asymmetric
>> routing. I'm sure if you investigate a little bit more you'll discover
>> it's not limited to ICMP.
>>
>> In the end the solution will be one of: remove statefulness, avoid
>> asymmetric routing, or share state with pfsync.
>
> If using pfsync for this, you would want to look at "defer", see  
> pfsync(4).
>

I think I had the same problem. Please visit

http://marc.info/?l=openbsd-misc&m=133957370427451&w=2

> Sloppy states might be more appropriate for this scenario though, and
> would let you use other things which require state tracking, e.g.  
> pflow(4).
>

Thanks,


Insan Praja

--
Using Opera's revolutionary email client: http://www.opera.com/mail/

Reply | Threaded
Open this post in threaded view
|

Re: pf and ICMP in asymmetric routing setups

bernd-34
Am 2012-06-13 09:55, schrieb Insan Praja SW:

> Hi,
>
> On Wed, 13 Jun 2012 08:07:31 +0700, Stuart Henderson
> <[hidden email]>  wrote:
>
>> On 2012-06-12, Simon Perreault <[hidden email]> wrote:
>>> On 2012-06-12 14:08, Bernd wrote:
>>>> I've got two OpenBSD 5.1-stable/amd64 boxes employed which do all
>>>> the
>>>> routing for our AS (OpenBGPd and OpenOSPFd). I see asymmetric
>>>> traffic  (I
>>>> thought it to be that way), which itself doesn't really create  
>>>> problems.
>>>> However, I see problems with ICMP. pf seems to drop all but the
>>>> first
>>>> response from any of the hosts within our network (seen from the  
>>>> Internet).
>>>>
>>>> Any idea how to deal with this? As soon as I turn off pf,
>>>> everything
>>>> runs smoothly.
>>>
>>> Without having the details of your setup, the big principle is: pf
>>> is
>>> stateful (by default). Statefulness doesn't play well with
>>> asymmetric
>>> routing. I'm sure if you investigate a little bit more you'll
>>> discover
>>> it's not limited to ICMP.
>>>
>>> In the end the solution will be one of: remove statefulness, avoid
>>> asymmetric routing, or share state with pfsync.
>>
>> If using pfsync for this, you would want to look at "defer", see  
>> pfsync(4).
>>
>
> I think I had the same problem. Please visit
>
> http://marc.info/?l=openbsd-misc&m=133957370427451&w=2

I saw it and instantly wished I'd have seen your mail about 24 hours
earlier... ;)

>> Sloppy states might be more appropriate for this scenario though,
>> and
>> would let you use other things which require state tracking, e.g.  
>> pflow(4).
>>
>
> Thanks,
>
>
> Insan Praja

Best,

Bernd