PF Config problem

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

PF Config problem

Gordon Ross-3
I've got two OBSD 4.1 boxes. They are setup identically, and I'm using CARP (&
pfsync) to obtain a redundant firewall. I appear to have CARP working fine. My
problem is when I enable pf. The initial TCP packet goes through fine, but the
return packet gets blocked. (I have verified this by putting "LOG" entries in
my ruleset) If I disable PF, everything works fine

Cutting down the pf ruleset to the bare minimum, I have:

adsl_if="em2"
int_if="em0"
pfsync_if="bge0"

scrub in
set skip on lo

block in

#These three lines allow the failover mechanisms to work
pass on { $int_if } proto carp keep state
pass on { $adsl_if } proto carp keep state
pass quick on { $pfsync_if} proto pfsync

#Allow internal people to SSH in.
pass in on $int_if proto tcp to ($int_if) port ssh keep state

#ICMP
pass in proto icmp to <me>

pass in on $int_if proto tcp from 172.16.2.34 to 192.168.249.3 keep state

With this config, 172.16.2.34 cannot make a TCP connection to 192.168.249.3.

What stupid thing have I missed ?

For reference, below are the details of the carp & em interfaces. If anything
else is needed, let me know.

Thanks,

GTG


# ifconfig carp0
carp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:00:5e:00:01:01
        carp: MASTER carpdev em0 vhid 1 advbase 1 advskew 0
        groups: carp
        inet 192.168.253.253 netmask 0xffffff00 broadcast 192.168.253.255
        inet6 fe80::200:5eff:fe00:101%carp0 prefixlen 64 scopeid 0xb
# ifconfig carp2
carp2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:00:5e:00:01:03
        carp: MASTER carpdev em2 vhid 3 advbase 1 advskew 0
        groups: carp
        inet6 fe80::200:5eff:fe00:103%carp2 prefixlen 64 scopeid 0xc
        inet 192.168.249.253 netmask 0xffffff00 broadcast 192.168.249.255
# ifconfig em0
em0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:1b:21:01:c8:30
        media: Ethernet autoselect (100baseTX full-duplex)
        status: active
        inet 192.168.253.20 netmask 0xffffff00 broadcast 192.168.253.255
        inet6 fe80::21b:21ff:fe01:c830%em0 prefixlen 64 scopeid 0x3
# ifconfig em2
em2: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:1b:21:01:c8:32
        media: Ethernet autoselect (100baseTX full-duplex,rxpause,txpause)
        status: active
        inet 192.168.249.251 netmask 0xffffff00 broadcast 192.168.249.255
        inet6 fe80::21b:21ff:fe01:c832%em2 prefixlen 64 scopeid 0x5

Reply | Threaded
Open this post in threaded view
|

Re: PF Config problem

Dag Richards
I think you will find that since carp is communicated with multicast
that your rules are not behaving as you think.

They are allowing the outbound transmissions, but since you are not
establishing tcp sessions the keep state does not do what you want.

Try explicitly allowing in protocol carp

What I do is this:

pass out quick proto carp
pass in  quick proto carp




Gordon Ross wrote:

> I've got two OBSD 4.1 boxes. They are setup identically, and I'm using CARP (&
> pfsync) to obtain a redundant firewall. I appear to have CARP working fine. My
> problem is when I enable pf. The initial TCP packet goes through fine, but the
> return packet gets blocked. (I have verified this by putting "LOG" entries in
> my ruleset) If I disable PF, everything works fine
>
> Cutting down the pf ruleset to the bare minimum, I have:
>
> adsl_if="em2"
> int_if="em0"
> pfsync_if="bge0"
>
> scrub in
> set skip on lo
>
> block in
>
> #These three lines allow the failover mechanisms to work
> pass on { $int_if } proto carp keep state
> pass on { $adsl_if } proto carp keep state
> pass quick on { $pfsync_if} proto pfsync
>
> #Allow internal people to SSH in.
> pass in on $int_if proto tcp to ($int_if) port ssh keep state
>
> #ICMP
> pass in proto icmp to <me>
>
> pass in on $int_if proto tcp from 172.16.2.34 to 192.168.249.3 keep state
>
> With this config, 172.16.2.34 cannot make a TCP connection to 192.168.249.3.
>
> What stupid thing have I missed ?
>
> For reference, below are the details of the carp & em interfaces. If anything
> else is needed, let me know.
>
> Thanks,
>
> GTG
>
>
> # ifconfig carp0
> carp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
>         lladdr 00:00:5e:00:01:01
>         carp: MASTER carpdev em0 vhid 1 advbase 1 advskew 0
>         groups: carp
>         inet 192.168.253.253 netmask 0xffffff00 broadcast 192.168.253.255
>         inet6 fe80::200:5eff:fe00:101%carp0 prefixlen 64 scopeid 0xb
> # ifconfig carp2
> carp2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
>         lladdr 00:00:5e:00:01:03
>         carp: MASTER carpdev em2 vhid 3 advbase 1 advskew 0
>         groups: carp
>         inet6 fe80::200:5eff:fe00:103%carp2 prefixlen 64 scopeid 0xc
>         inet 192.168.249.253 netmask 0xffffff00 broadcast 192.168.249.255
> # ifconfig em0
> em0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
>         lladdr 00:1b:21:01:c8:30
>         media: Ethernet autoselect (100baseTX full-duplex)
>         status: active
>         inet 192.168.253.20 netmask 0xffffff00 broadcast 192.168.253.255
>         inet6 fe80::21b:21ff:fe01:c830%em0 prefixlen 64 scopeid 0x3
> # ifconfig em2
> em2: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
>         lladdr 00:1b:21:01:c8:32
>         media: Ethernet autoselect (100baseTX full-duplex,rxpause,txpause)
>         status: active
>         inet 192.168.249.251 netmask 0xffffff00 broadcast 192.168.249.255
>         inet6 fe80::21b:21ff:fe01:c832%em2 prefixlen 64 scopeid 0x5

Reply | Threaded
Open this post in threaded view
|

Re: PF Config problem

Gordon Ross-3
In reply to this post by Gordon Ross-3
So why is this different to what I put ?

#These three lines allow the failover mechanisms to work
pass on { $int_if } proto carp keep state
pass on { $adsl_if } proto carp keep state
pass quick on { $pfsync_if} proto pfsync

The only difference I can see, is that your lines would allow CARP on the
pfsync (and loopback) interface.

GTG

>>> Dag Richards <[hidden email]> 07/19/07 4:55 PM >>>
I think you will find that since carp is communicated with multicast
that your rules are not behaving as you think.

They are allowing the outbound transmissions, but since you are not
establishing tcp sessions the keep state does not do what you want.

Try explicitly allowing in protocol carp

What I do is this:

pass out quick proto carp
pass in  quick proto carp

Reply | Threaded
Open this post in threaded view
|

Re: PF Config problem

Dag Richards
Gordon Ross wrote:

> So why is this different to what I put ?
>
> #These three lines allow the failover mechanisms to work
> pass on { $int_if } proto carp keep state
> pass on { $adsl_if } proto carp keep state
> pass quick on { $pfsync_if} proto pfsync
>
> The only difference I can see, is that your lines would allow CARP on the
> pfsync (and loopback) interface.
>
> GTG
>
>>>> Dag Richards <[hidden email]> 07/19/07 4:55 PM >>>
> I think you will find that since carp is communicated with multicast
> that your rules are not behaving as you think.
>
> They are allowing the outbound transmissions, but since you are not
> establishing tcp sessions the keep state does not do what you want.
>
> Try explicitly allowing in protocol carp
>
> What I do is this:
>
> pass out quick proto carp
> pass in  quick proto carp
>


The difference is you were paying attention.
I really thought I saw pass out not just pass on your lines.

When you do

tcpdump -n -e -ttt -i pflog0

with rules enables to you see inbound carp being blocked?

Reply | Threaded
Open this post in threaded view
|

Re: PF Config problem

Stuart Henderson
In reply to this post by Gordon Ross-3
On 2007/07/19 15:38, Gordon Ross wrote:
> Cutting down the pf ruleset to the bare minimum, I have:

Might be below the minimum; there's no explicit "pass out".
There's an implicit one, but I suspect it might not be keeping
state (though the default as of 4.1 is to keep state, I suspect
this _may_ apply only to rules configured by pfctl and not implicit
ones). And if that's the case it won't permit the return traffic.

I would have a look at http://www.openbsd.org/faq/pf/tagging.html 
before you start writing much more. I also find the ruleset a lot
easier to read when I leave out "keep state" keywords.

Reply | Threaded
Open this post in threaded view
|

Re: PF Config problem

Gordon Ross-3
>>> On 19 July 2007 at 23:52, in message
<[hidden email]>, Stuart Henderson
<[hidden email]> wrote:
> On 2007/07/19 15:38, Gordon Ross wrote:
>> Cutting down the pf ruleset to the bare minimum, I have:
>
> Might be below the minimum; there's no explicit "pass out".

No, the packets get out the "other side" of the OBSD box to the destination,
it's the return packets that get blocked.

> There's an implicit one, but I suspect it might not be keeping
> state (though the default as of 4.1 is to keep state, I suspect
> this _may_ apply only to rules configured by pfctl and not implicit
> ones). And if that's the case it won't permit the return traffic.

This is my problem - the return traffic is not being allowed back in.

Surely I don't need to write explicit "pass in" rules for the return packets ?
Or have I missed something really silly/obvious ?

> I would have a look at http://www.openbsd.org/faq/pf/tagging.html
> before you start writing much more.

Noted. However, it's not going to help me right now :-(

Thanks,

GTG

Reply | Threaded
Open this post in threaded view
|

Re: PF Config problem

Gordon Ross-3
In reply to this post by Dag Richards
>>> On 19 July 2007 at 18:55, in message <[hidden email]>,
Dag
Richards <[hidden email]> wrote:
> Gordon Ross wrote:
>> So why is this different to what I put ?
>>
>> #These three lines allow the failover mechanisms to work
>> pass on { $int_if } proto carp keep state
>> pass on { $adsl_if } proto carp keep state
>> pass quick on { $pfsync_if} proto pfsync
[snip]
> The difference is you were paying attention.

;-)

> I really thought I saw pass out not just pass on your lines.
>
> When you do
>
> tcpdump -n -e -ttt -i pflog0
>
> with rules enables to you see inbound carp being blocked?

No CARP packets are being blocked.

GTG

Reply | Threaded
Open this post in threaded view
|

Re: PF Config problem

Gordon Ross-3
In reply to this post by Stuart Henderson
>>> On 19 July 2007 at 23:52, in message
<[hidden email]>, Stuart Henderson
<[hidden email]> wrote:
> On 2007/07/19 15:38, Gordon Ross wrote:
>> Cutting down the pf ruleset to the bare minimum, I have:
>
> Might be below the minimum; there's no explicit "pass out".
> There's an implicit one, but I suspect it might not be keeping
> state (though the default as of 4.1 is to keep state, I suspect
> this _may_ apply only to rules configured by pfctl and not implicit
> ones). And if that's the case it won't permit the return traffic.

Made a little bit of progress..

If I change

pass in on $int_if proto tcp from 172.16.2.34 to 192.168.249.3 keep state

to:

pass proto tcp from 172.16.2.34 to 192.168.249.3 keep state

Then that works fine. Now I can half see why this does work: I've not
specified a direction or interface for the rule. For a simple two-interface
firewall, that's should be OK. My thoughts turn to when I have a firewall with
more than two interfaces: What would happen to a spoofed packet appearing on a
"wrong" interface ? As the rule no longer specifies interfaces, I could see
that PF would allow the packet through... Would the solution be to create
rules that only allow "valid" addresses to come in to interfaces ? Or am I
being paranoid ?

GTG

Reply | Threaded
Open this post in threaded view
|

Re: PF Config problem

Stuart Henderson
In reply to this post by Gordon Ross-3
On 2007/07/20 08:45, Gordon Ross wrote:
> > Might be below the minimum; there's no explicit "pass out".
>
> No, the packets get out the "other side" of the OBSD box to the destination,
> it's the return packets that get blocked.

Yes, exactly. Your implicit 'pass out' will allow the outbound
packets but it looks like this isn't stateful so it won't permit
the return packets (current behaviour doesn't match pf.conf(5)
docs; the diff below should address this).

Can you try just adding 'pass out' to the top of the ruleset
please? I guess it will help, you could then refine it by tagging
incoming packets and 'pass out on XX tagged FOO' which is much
easier than doing each rule individually.

Index: pf_ioctl.c
===================================================================
RCS file: /cvs/src/sys/net/pf_ioctl.c,v
retrieving revision 1.182
diff -u -p -r1.182 pf_ioctl.c
--- pf_ioctl.c 24 Jun 2007 11:17:13 -0000 1.182
+++ pf_ioctl.c 20 Jul 2007 08:56:32 -0000
@@ -177,6 +177,7 @@ pfattach(int num)
  /* default rule should never be garbage collected */
  pf_default_rule.entries.tqe_prev = &pf_default_rule.entries.tqe_next;
  pf_default_rule.action = PF_PASS;
+ pf_default_rule.keep_state = PF_STATE_NORMAL;
  pf_default_rule.nr = -1;
  pf_default_rule.rtableid = -1;

Reply | Threaded
Open this post in threaded view
|

Re: PF Config problem

Gordon Ross-3
>>> On 20 July 2007 at 10:04, in message
<[hidden email]>, Stuart Henderson
<[hidden email]> wrote:
> On 2007/07/20 08:45, Gordon Ross wrote:
>> > Might be below the minimum; there's no explicit "pass out".
>>
>> No, the packets get out the "other side" of the OBSD box to the
destination,
>> it's the return packets that get blocked.
>
> Yes, exactly. Your implicit 'pass out' will allow the outbound
> packets but it looks like this isn't stateful so it won't permit
> the return packets (current behaviour doesn't match pf.conf(5)
> docs; the diff below should address this).

Phew ! I thought my brain had gone the same way as my hair... ;-)

> Can you try just adding 'pass out' to the top of the ruleset
> please?

I did:

pass in on $int_if proto tcp from 172.16.2.34 to 192.168.249.3 keep state
pass out on $out_if

and that worked.

> I guess it will help, you could then refine it by tagging
> incoming packets and 'pass out on XX tagged FOO' which is much
> easier than doing each rule individually.

I then did:

pass in on $int_if proto tcp from 172.16.2.34 to 192.168.249.3 tag TEST_TAG
keep state
pass out on $out_if tagged TEST_TAG

and that worked as well - and (I believe) is tighter than just a "pass out".
(Certainly solves my paranoid problem in my previous posting)

Going off on a tangent here: Why is it that I've just picked this up and
no-one else has ? Is it because I'm running in full paranoia mode and blocking
*everything* unless explicitly allowed ?

I haven't tried your diff - let me know if you want me to.

Thanks for your help, much appreciated.

GTG

Reply | Threaded
Open this post in threaded view
|

Re: PF Config problem

Stuart Henderson
On 2007/07/20 10:46, Gordon Ross wrote:
> Going off on a tangent here: Why is it that I've just picked this up and
> no-one else has ?

I think because you had no rules (pass or block) affecting outgoing
packets - it's quite common to start things off with just 'block'
(without specifying the direction) or 'block log' which would give
more clues about what's going wrong when you tcpdump -netti pflog0.

It's possibly also connected with the change to defaulting to
'flags S/SA' (done to avoid sequence number problems with TCP
window-scaling without requiring people to change rulesets) -
though I didn't work through your rules to check that.

> I haven't tried your diff - let me know if you want me to.

It just changes the implicit rule to keep state so shouldn't
affect things for you now you've added specific rules; I was more
throwing it out for discussion. Actually looking at it again,
flags probably need to be addressed too, maybe with

        pf_default_rule.flags = 1; /* SYN */
        pf_default_rule.flagset = 18; /* SYN+ACK */

but I'm not so sure about that.