pfsync(4) mangles prio in master/slave setup

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

pfsync(4) mangles prio in master/slave setup

Alexey Suslikov
Hi.

This is on 5.4-stable. Trivial master/slave carp(4) setup. vlan(4) is to
make picture clear wrt prio.

Test 1 (without using match).

pf.conf (BOX1 and BOX2).

ext_if="vlan101"
dmz_if="vlan10"
pf_sync="vlan50"
block log all
pass quick on $pf_sync proto pfsync keep state (no-sync) set prio 7
pass quick on { $ext_if, $dmz_if } proto carp keep state (no-sync)
pass quick on $dmz_if inet proto icmp all icmp-type echoreq set prio 5
pass quick on $dmz_if
pass out quick on $ext_if inet proto icmp all icmp-type echoreq set prio 5
pass out quick on $ext_if

BOX1 is Master, BOX2 is Slave.

BOX1:
00:07:36.108948 802.1Q vid 10 pri 3 X.X.185.145 > X.X.36.14: icmp: echo request
00:07:36.109281 802.1Q vid 101 pri 5 X.X.185.145 > X.X.36.14: icmp: echo request
00:07:36.110013 802.1Q vid 101 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply
00:07:36.110030 802.1Q vid 10 pri 5 X.X.36.14 > X.X.185.145: icmp: echo reply

BOX1 is Slave, BOX2 is Master.

BOX2:
00:12:43.981979 802.1Q vid 10 pri 3 X.X.185.145 > X.X.36.14: icmp: echo request
00:12:43.982013 802.1Q vid 101 pri 0 X.X.185.145 > X.X.36.14: icmp: echo request
00:12:43.982693 802.1Q vid 101 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply
00:12:43.982713 802.1Q vid 10 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply

Test 2 (using match).

pf.conf (BOX1 and BOX2).

ext_if="vlan101"
dmz_if="vlan10"
pf_sync="vlan50"
block log all
match quick on { $ext_if, $dmz_if } inet proto icmp all icmp-type
echoreq set prio 5
pass quick on $pf_sync proto pfsync keep state (no-sync) set prio 7
pass quick on { $ext_if, $dmz_if } proto carp keep state (no-sync)
pass quick on $dmz_if
pass out quick on $ext_if

BOX1 is Master, BOX2 is Slave.

BOX1:
00:27:47.442820 802.1Q vid 10 pri 3 X.X.185.145 > X.X.36.14: icmp: echo request
00:27:47.442839 802.1Q vid 101 pri 5 X.X.185.145 > X.X.36.14: icmp: echo request
00:27:48.468709 802.1Q vid 101 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply
00:27:47.443523 802.1Q vid 10 pri 5 X.X.36.14 > X.X.185.145: icmp: echo reply

BOX1 is Slave, BOX2 is Master.

BOX2:
00:30:35.317329 802.1Q vid 10 pri 3 X.X.185.145 > X.X.36.14: icmp: echo request
00:30:35.317354 802.1Q vid 101 pri 0 X.X.185.145 > X.X.36.14: icmp: echo request
00:30:35.318065 802.1Q vid 101 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply
00:30:35.318084 802.1Q vid 10 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply

Maybe ICMP is not a sort of traffic which makes difference, but think
about TCP ACKs are prioritized. Switching to Slave in production setup
makes things *REALLY* bad.

Should I configure something, or this is an issue?

(Speaking of pfsync code, I'm unable to find where prio is set inside
pfsync_state_import).

Thanks,
Alexey

Reply | Threaded
Open this post in threaded view
|

Re: pfsync(4) mangles prio in master/slave setup

Mike Belopuhov-5
could you please add more description to this report since
it's very hard to follow and interpret your mail.

On 20 November 2013 12:11, Alexey Suslikov <[hidden email]> wrote:

> Hi.
>
> This is on 5.4-stable. Trivial master/slave carp(4) setup. vlan(4) is to
> make picture clear wrt prio.
>
> Test 1 (without using match).
>
> pf.conf (BOX1 and BOX2).
>
> ext_if="vlan101"
> dmz_if="vlan10"
> pf_sync="vlan50"
> block log all
> pass quick on $pf_sync proto pfsync keep state (no-sync) set prio 7
> pass quick on { $ext_if, $dmz_if } proto carp keep state (no-sync)
> pass quick on $dmz_if inet proto icmp all icmp-type echoreq set prio 5
> pass quick on $dmz_if
> pass out quick on $ext_if inet proto icmp all icmp-type echoreq set prio 5
> pass out quick on $ext_if
>
> BOX1 is Master, BOX2 is Slave.
>
> BOX1:
> 00:07:36.108948 802.1Q vid 10 pri 3 X.X.185.145 > X.X.36.14: icmp: echo request
> 00:07:36.109281 802.1Q vid 101 pri 5 X.X.185.145 > X.X.36.14: icmp: echo request
> 00:07:36.110013 802.1Q vid 101 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply
> 00:07:36.110030 802.1Q vid 10 pri 5 X.X.36.14 > X.X.185.145: icmp: echo reply
>
> BOX1 is Slave, BOX2 is Master.
>
> BOX2:
> 00:12:43.981979 802.1Q vid 10 pri 3 X.X.185.145 > X.X.36.14: icmp: echo request
> 00:12:43.982013 802.1Q vid 101 pri 0 X.X.185.145 > X.X.36.14: icmp: echo request
> 00:12:43.982693 802.1Q vid 101 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply
> 00:12:43.982713 802.1Q vid 10 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply
>
> Test 2 (using match).
>
> pf.conf (BOX1 and BOX2).
>
> ext_if="vlan101"
> dmz_if="vlan10"
> pf_sync="vlan50"
> block log all
> match quick on { $ext_if, $dmz_if } inet proto icmp all icmp-type
> echoreq set prio 5
> pass quick on $pf_sync proto pfsync keep state (no-sync) set prio 7
> pass quick on { $ext_if, $dmz_if } proto carp keep state (no-sync)
> pass quick on $dmz_if
> pass out quick on $ext_if
>
> BOX1 is Master, BOX2 is Slave.
>
> BOX1:
> 00:27:47.442820 802.1Q vid 10 pri 3 X.X.185.145 > X.X.36.14: icmp: echo request
> 00:27:47.442839 802.1Q vid 101 pri 5 X.X.185.145 > X.X.36.14: icmp: echo request
> 00:27:48.468709 802.1Q vid 101 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply
> 00:27:47.443523 802.1Q vid 10 pri 5 X.X.36.14 > X.X.185.145: icmp: echo reply
>
> BOX1 is Slave, BOX2 is Master.
>
> BOX2:
> 00:30:35.317329 802.1Q vid 10 pri 3 X.X.185.145 > X.X.36.14: icmp: echo request
> 00:30:35.317354 802.1Q vid 101 pri 0 X.X.185.145 > X.X.36.14: icmp: echo request
> 00:30:35.318065 802.1Q vid 101 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply
> 00:30:35.318084 802.1Q vid 10 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply
>
> Maybe ICMP is not a sort of traffic which makes difference, but think
> about TCP ACKs are prioritized. Switching to Slave in production setup
> makes things *REALLY* bad.
>
> Should I configure something, or this is an issue?
>
> (Speaking of pfsync code, I'm unable to find where prio is set inside
> pfsync_state_import).
>
> Thanks,
> Alexey
>

Reply | Threaded
Open this post in threaded view
|

Re: pfsync(4) mangles prio in master/slave setup

Alexey Suslikov
On Wed, Nov 20, 2013 at 1:32 PM, Mike Belopuhov <[hidden email]> wrote:
> could you please add more description to this report since
> it's very hard to follow and interpret your mail.

basically, when setup switches to slave, packets (matching
given state) have wrong prio set ("wrong" means they were
"right" when state was created on master).

I will be glad to provide more information/tests/etc - just say
what is needed.

>
> On 20 November 2013 12:11, Alexey Suslikov <[hidden email]> wrote:
>> Hi.
>>
>> This is on 5.4-stable. Trivial master/slave carp(4) setup. vlan(4) is to
>> make picture clear wrt prio.
>>
>> Test 1 (without using match).
>>
>> pf.conf (BOX1 and BOX2).
>>
>> ext_if="vlan101"
>> dmz_if="vlan10"
>> pf_sync="vlan50"
>> block log all
>> pass quick on $pf_sync proto pfsync keep state (no-sync) set prio 7
>> pass quick on { $ext_if, $dmz_if } proto carp keep state (no-sync)
>> pass quick on $dmz_if inet proto icmp all icmp-type echoreq set prio 5
>> pass quick on $dmz_if
>> pass out quick on $ext_if inet proto icmp all icmp-type echoreq set prio 5
>> pass out quick on $ext_if
>>
>> BOX1 is Master, BOX2 is Slave.
>>
>> BOX1:
>> 00:07:36.108948 802.1Q vid 10 pri 3 X.X.185.145 > X.X.36.14: icmp: echo request
>> 00:07:36.109281 802.1Q vid 101 pri 5 X.X.185.145 > X.X.36.14: icmp: echo request
>> 00:07:36.110013 802.1Q vid 101 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply
>> 00:07:36.110030 802.1Q vid 10 pri 5 X.X.36.14 > X.X.185.145: icmp: echo reply
>>
>> BOX1 is Slave, BOX2 is Master.
>>
>> BOX2:
>> 00:12:43.981979 802.1Q vid 10 pri 3 X.X.185.145 > X.X.36.14: icmp: echo request
>> 00:12:43.982013 802.1Q vid 101 pri 0 X.X.185.145 > X.X.36.14: icmp: echo request
>> 00:12:43.982693 802.1Q vid 101 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply
>> 00:12:43.982713 802.1Q vid 10 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply
>>
>> Test 2 (using match).
>>
>> pf.conf (BOX1 and BOX2).
>>
>> ext_if="vlan101"
>> dmz_if="vlan10"
>> pf_sync="vlan50"
>> block log all
>> match quick on { $ext_if, $dmz_if } inet proto icmp all icmp-type
>> echoreq set prio 5
>> pass quick on $pf_sync proto pfsync keep state (no-sync) set prio 7
>> pass quick on { $ext_if, $dmz_if } proto carp keep state (no-sync)
>> pass quick on $dmz_if
>> pass out quick on $ext_if
>>
>> BOX1 is Master, BOX2 is Slave.
>>
>> BOX1:
>> 00:27:47.442820 802.1Q vid 10 pri 3 X.X.185.145 > X.X.36.14: icmp: echo request
>> 00:27:47.442839 802.1Q vid 101 pri 5 X.X.185.145 > X.X.36.14: icmp: echo request
>> 00:27:48.468709 802.1Q vid 101 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply
>> 00:27:47.443523 802.1Q vid 10 pri 5 X.X.36.14 > X.X.185.145: icmp: echo reply
>>
>> BOX1 is Slave, BOX2 is Master.
>>
>> BOX2:
>> 00:30:35.317329 802.1Q vid 10 pri 3 X.X.185.145 > X.X.36.14: icmp: echo request
>> 00:30:35.317354 802.1Q vid 101 pri 0 X.X.185.145 > X.X.36.14: icmp: echo request
>> 00:30:35.318065 802.1Q vid 101 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply
>> 00:30:35.318084 802.1Q vid 10 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply
>>
>> Maybe ICMP is not a sort of traffic which makes difference, but think
>> about TCP ACKs are prioritized. Switching to Slave in production setup
>> makes things *REALLY* bad.
>>
>> Should I configure something, or this is an issue?
>>
>> (Speaking of pfsync code, I'm unable to find where prio is set inside
>> pfsync_state_import).
>>
>> Thanks,
>> Alexey
>>

Reply | Threaded
Open this post in threaded view
|

Re: pfsync(4) mangles prio in master/slave setup

Alexey Suslikov
On Wed, Nov 20, 2013 at 1:38 PM, Alexey Suslikov
<[hidden email]> wrote:

> On Wed, Nov 20, 2013 at 1:32 PM, Mike Belopuhov <[hidden email]> wrote:
>> could you please add more description to this report since
>> it's very hard to follow and interpret your mail.
>
> basically, when setup switches to slave, packets (matching
> given state) have wrong prio set ("wrong" means they were
> "right" when state was created on master).
>
> I will be glad to provide more information/tests/etc - just say
> what is needed.
>
>>
>> On 20 November 2013 12:11, Alexey Suslikov <[hidden email]> wrote:
>>> Hi.
>>>
>>> This is on 5.4-stable. Trivial master/slave carp(4) setup. vlan(4) is to
>>> make picture clear wrt prio.
>>>
>>> Test 1 (without using match).
>>>
>>> pf.conf (BOX1 and BOX2).
>>>
>>> ext_if="vlan101"
>>> dmz_if="vlan10"
>>> pf_sync="vlan50"
>>> block log all
>>> pass quick on $pf_sync proto pfsync keep state (no-sync) set prio 7
>>> pass quick on { $ext_if, $dmz_if } proto carp keep state (no-sync)
>>> pass quick on $dmz_if inet proto icmp all icmp-type echoreq set prio 5
>>> pass quick on $dmz_if
>>> pass out quick on $ext_if inet proto icmp all icmp-type echoreq set prio 5
>>> pass out quick on $ext_if
>>>
>>> BOX1 is Master, BOX2 is Slave.
>>>
>>> BOX1:
>>> 00:07:36.108948 802.1Q vid 10 pri 3 X.X.185.145 > X.X.36.14: icmp: echo request
>>> 00:07:36.109281 802.1Q vid 101 pri 5 X.X.185.145 > X.X.36.14: icmp: echo request
>>> 00:07:36.110013 802.1Q vid 101 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply
>>> 00:07:36.110030 802.1Q vid 10 pri 5 X.X.36.14 > X.X.185.145: icmp: echo reply
>>>
>>> BOX1 is Slave, BOX2 is Master.
>>>
>>> BOX2:
>>> 00:12:43.981979 802.1Q vid 10 pri 3 X.X.185.145 > X.X.36.14: icmp: echo request
>>> 00:12:43.982013 802.1Q vid 101 pri 0 X.X.185.145 > X.X.36.14: icmp: echo request
>>> 00:12:43.982693 802.1Q vid 101 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply
>>> 00:12:43.982713 802.1Q vid 10 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply

While on Slave, having all zeroes prio in output path (echo request in
vlan101 and reply in vlan10), imo, indicates a state being crafted by
pfsync_state_import without a prio took in account.

In contrast to set_tos, min_ttl and such, pf_state_export too isn't doing
anything about prio, so I think prio neither exported to pfsync packet,
nor imported from.

>>>
>>> Test 2 (using match).
>>>
>>> pf.conf (BOX1 and BOX2).
>>>
>>> ext_if="vlan101"
>>> dmz_if="vlan10"
>>> pf_sync="vlan50"
>>> block log all
>>> match quick on { $ext_if, $dmz_if } inet proto icmp all icmp-type
>>> echoreq set prio 5
>>> pass quick on $pf_sync proto pfsync keep state (no-sync) set prio 7
>>> pass quick on { $ext_if, $dmz_if } proto carp keep state (no-sync)
>>> pass quick on $dmz_if
>>> pass out quick on $ext_if
>>>
>>> BOX1 is Master, BOX2 is Slave.
>>>
>>> BOX1:
>>> 00:27:47.442820 802.1Q vid 10 pri 3 X.X.185.145 > X.X.36.14: icmp: echo request
>>> 00:27:47.442839 802.1Q vid 101 pri 5 X.X.185.145 > X.X.36.14: icmp: echo request
>>> 00:27:48.468709 802.1Q vid 101 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply
>>> 00:27:47.443523 802.1Q vid 10 pri 5 X.X.36.14 > X.X.185.145: icmp: echo reply
>>>
>>> BOX1 is Slave, BOX2 is Master.
>>>
>>> BOX2:
>>> 00:30:35.317329 802.1Q vid 10 pri 3 X.X.185.145 > X.X.36.14: icmp: echo request
>>> 00:30:35.317354 802.1Q vid 101 pri 0 X.X.185.145 > X.X.36.14: icmp: echo request
>>> 00:30:35.318065 802.1Q vid 101 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply
>>> 00:30:35.318084 802.1Q vid 10 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply
>>>
>>> Maybe ICMP is not a sort of traffic which makes difference, but think
>>> about TCP ACKs are prioritized. Switching to Slave in production setup
>>> makes things *REALLY* bad.
>>>
>>> Should I configure something, or this is an issue?
>>>
>>> (Speaking of pfsync code, I'm unable to find where prio is set inside
>>> pfsync_state_import).
>>>
>>> Thanks,
>>> Alexey
>>>

Reply | Threaded
Open this post in threaded view
|

Re: pfsync(4) mangles prio in master/slave setup

Florian Obser-2
In reply to this post by Alexey Suslikov
On Wed, Nov 20, 2013 at 01:38:11PM +0200, Alexey Suslikov wrote:

> On Wed, Nov 20, 2013 at 1:32 PM, Mike Belopuhov <[hidden email]> wrote:
> > could you please add more description to this report since
> > it's very hard to follow and interpret your mail.
>
> basically, when setup switches to slave, packets (matching
> given state) have wrong prio set ("wrong" means they were
> "right" when state was created on master).
>
> I will be glad to provide more information/tests/etc - just say
> what is needed.

Do you have the same ruleset checksum on both machines? check with
pfctl -vs info | fgrep Checksum

>
> >
> > On 20 November 2013 12:11, Alexey Suslikov <[hidden email]> wrote:
> >> Hi.
> >>
> >> This is on 5.4-stable. Trivial master/slave carp(4) setup. vlan(4) is to
> >> make picture clear wrt prio.
> >>
> >> Test 1 (without using match).
> >>
> >> pf.conf (BOX1 and BOX2).
> >>
> >> ext_if="vlan101"
> >> dmz_if="vlan10"
> >> pf_sync="vlan50"
> >> block log all
> >> pass quick on $pf_sync proto pfsync keep state (no-sync) set prio 7
> >> pass quick on { $ext_if, $dmz_if } proto carp keep state (no-sync)
> >> pass quick on $dmz_if inet proto icmp all icmp-type echoreq set prio 5
> >> pass quick on $dmz_if
> >> pass out quick on $ext_if inet proto icmp all icmp-type echoreq set prio 5
> >> pass out quick on $ext_if
> >>
> >> BOX1 is Master, BOX2 is Slave.
> >>
> >> BOX1:
> >> 00:07:36.108948 802.1Q vid 10 pri 3 X.X.185.145 > X.X.36.14: icmp: echo request
> >> 00:07:36.109281 802.1Q vid 101 pri 5 X.X.185.145 > X.X.36.14: icmp: echo request
> >> 00:07:36.110013 802.1Q vid 101 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply
> >> 00:07:36.110030 802.1Q vid 10 pri 5 X.X.36.14 > X.X.185.145: icmp: echo reply
> >>
> >> BOX1 is Slave, BOX2 is Master.
> >>
> >> BOX2:
> >> 00:12:43.981979 802.1Q vid 10 pri 3 X.X.185.145 > X.X.36.14: icmp: echo request
> >> 00:12:43.982013 802.1Q vid 101 pri 0 X.X.185.145 > X.X.36.14: icmp: echo request
> >> 00:12:43.982693 802.1Q vid 101 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply
> >> 00:12:43.982713 802.1Q vid 10 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply
> >>
> >> Test 2 (using match).
> >>
> >> pf.conf (BOX1 and BOX2).
> >>
> >> ext_if="vlan101"
> >> dmz_if="vlan10"
> >> pf_sync="vlan50"
> >> block log all
> >> match quick on { $ext_if, $dmz_if } inet proto icmp all icmp-type
> >> echoreq set prio 5
> >> pass quick on $pf_sync proto pfsync keep state (no-sync) set prio 7
> >> pass quick on { $ext_if, $dmz_if } proto carp keep state (no-sync)
> >> pass quick on $dmz_if
> >> pass out quick on $ext_if
> >>
> >> BOX1 is Master, BOX2 is Slave.
> >>
> >> BOX1:
> >> 00:27:47.442820 802.1Q vid 10 pri 3 X.X.185.145 > X.X.36.14: icmp: echo request
> >> 00:27:47.442839 802.1Q vid 101 pri 5 X.X.185.145 > X.X.36.14: icmp: echo request
> >> 00:27:48.468709 802.1Q vid 101 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply
> >> 00:27:47.443523 802.1Q vid 10 pri 5 X.X.36.14 > X.X.185.145: icmp: echo reply
> >>
> >> BOX1 is Slave, BOX2 is Master.
> >>
> >> BOX2:
> >> 00:30:35.317329 802.1Q vid 10 pri 3 X.X.185.145 > X.X.36.14: icmp: echo request
> >> 00:30:35.317354 802.1Q vid 101 pri 0 X.X.185.145 > X.X.36.14: icmp: echo request
> >> 00:30:35.318065 802.1Q vid 101 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply
> >> 00:30:35.318084 802.1Q vid 10 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply
> >>
> >> Maybe ICMP is not a sort of traffic which makes difference, but think
> >> about TCP ACKs are prioritized. Switching to Slave in production setup
> >> makes things *REALLY* bad.
> >>
> >> Should I configure something, or this is an issue?
> >>
> >> (Speaking of pfsync code, I'm unable to find where prio is set inside
> >> pfsync_state_import).
> >>
> >> Thanks,
> >> Alexey
> >>
>

--
I'm not entirely sure you are real.

Reply | Threaded
Open this post in threaded view
|

Re: pfsync(4) mangles prio in master/slave setup

Mike Belopuhov-5
In reply to this post by Alexey Suslikov
On 20 November 2013 13:10, Alexey Suslikov <[hidden email]> wrote:

> On Wed, Nov 20, 2013 at 1:38 PM, Alexey Suslikov
> <[hidden email]> wrote:
>> On Wed, Nov 20, 2013 at 1:32 PM, Mike Belopuhov <[hidden email]> wrote:
>>> could you please add more description to this report since
>>> it's very hard to follow and interpret your mail.
>>
>> basically, when setup switches to slave, packets (matching
>> given state) have wrong prio set ("wrong" means they were
>> "right" when state was created on master).
>>
>> I will be glad to provide more information/tests/etc - just say
>> what is needed.
>>
>>>
>>> On 20 November 2013 12:11, Alexey Suslikov <[hidden email]> wrote:
>>>> Hi.
>>>>
>>>> This is on 5.4-stable. Trivial master/slave carp(4) setup. vlan(4) is to
>>>> make picture clear wrt prio.
>>>>
>>>> Test 1 (without using match).
>>>>
>>>> pf.conf (BOX1 and BOX2).
>>>>
>>>> ext_if="vlan101"
>>>> dmz_if="vlan10"
>>>> pf_sync="vlan50"
>>>> block log all
>>>> pass quick on $pf_sync proto pfsync keep state (no-sync) set prio 7
>>>> pass quick on { $ext_if, $dmz_if } proto carp keep state (no-sync)
>>>> pass quick on $dmz_if inet proto icmp all icmp-type echoreq set prio 5
>>>> pass quick on $dmz_if
>>>> pass out quick on $ext_if inet proto icmp all icmp-type echoreq set prio 5
>>>> pass out quick on $ext_if
>>>>
>>>> BOX1 is Master, BOX2 is Slave.
>>>>
>>>> BOX1:
>>>> 00:07:36.108948 802.1Q vid 10 pri 3 X.X.185.145 > X.X.36.14: icmp: echo request
>>>> 00:07:36.109281 802.1Q vid 101 pri 5 X.X.185.145 > X.X.36.14: icmp: echo request
>>>> 00:07:36.110013 802.1Q vid 101 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply
>>>> 00:07:36.110030 802.1Q vid 10 pri 5 X.X.36.14 > X.X.185.145: icmp: echo reply
>>>>
>>>> BOX1 is Slave, BOX2 is Master.
>>>>
>>>> BOX2:
>>>> 00:12:43.981979 802.1Q vid 10 pri 3 X.X.185.145 > X.X.36.14: icmp: echo request
>>>> 00:12:43.982013 802.1Q vid 101 pri 0 X.X.185.145 > X.X.36.14: icmp: echo request
>>>> 00:12:43.982693 802.1Q vid 101 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply
>>>> 00:12:43.982713 802.1Q vid 10 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply
>
> While on Slave, having all zeroes prio in output path (echo request in
> vlan101 and reply in vlan10), imo, indicates a state being crafted by
> pfsync_state_import without a prio took in account.
>
> In contrast to set_tos, min_ttl and such, pf_state_export too isn't doing
> anything about prio, so I think prio neither exported to pfsync packet,
> nor imported from.
>

correct.  henning?

>>>>
>>>> Test 2 (using match).
>>>>
>>>> pf.conf (BOX1 and BOX2).
>>>>
>>>> ext_if="vlan101"
>>>> dmz_if="vlan10"
>>>> pf_sync="vlan50"
>>>> block log all
>>>> match quick on { $ext_if, $dmz_if } inet proto icmp all icmp-type
>>>> echoreq set prio 5
>>>> pass quick on $pf_sync proto pfsync keep state (no-sync) set prio 7
>>>> pass quick on { $ext_if, $dmz_if } proto carp keep state (no-sync)
>>>> pass quick on $dmz_if
>>>> pass out quick on $ext_if
>>>>
>>>> BOX1 is Master, BOX2 is Slave.
>>>>
>>>> BOX1:
>>>> 00:27:47.442820 802.1Q vid 10 pri 3 X.X.185.145 > X.X.36.14: icmp: echo request
>>>> 00:27:47.442839 802.1Q vid 101 pri 5 X.X.185.145 > X.X.36.14: icmp: echo request
>>>> 00:27:48.468709 802.1Q vid 101 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply
>>>> 00:27:47.443523 802.1Q vid 10 pri 5 X.X.36.14 > X.X.185.145: icmp: echo reply
>>>>
>>>> BOX1 is Slave, BOX2 is Master.
>>>>
>>>> BOX2:
>>>> 00:30:35.317329 802.1Q vid 10 pri 3 X.X.185.145 > X.X.36.14: icmp: echo request
>>>> 00:30:35.317354 802.1Q vid 101 pri 0 X.X.185.145 > X.X.36.14: icmp: echo request
>>>> 00:30:35.318065 802.1Q vid 101 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply
>>>> 00:30:35.318084 802.1Q vid 10 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply
>>>>
>>>> Maybe ICMP is not a sort of traffic which makes difference, but think
>>>> about TCP ACKs are prioritized. Switching to Slave in production setup
>>>> makes things *REALLY* bad.
>>>>
>>>> Should I configure something, or this is an issue?
>>>>
>>>> (Speaking of pfsync code, I'm unable to find where prio is set inside
>>>> pfsync_state_import).
>>>>
>>>> Thanks,
>>>> Alexey
>>>>

Reply | Threaded
Open this post in threaded view
|

Re: pfsync(4) mangles prio in master/slave setup

Alexey Suslikov
In reply to this post by Florian Obser-2
On Wed, Nov 20, 2013 at 2:15 PM, Florian Obser <[hidden email]> wrote:

> On Wed, Nov 20, 2013 at 01:38:11PM +0200, Alexey Suslikov wrote:
>> On Wed, Nov 20, 2013 at 1:32 PM, Mike Belopuhov <[hidden email]> wrote:
>> > could you please add more description to this report since
>> > it's very hard to follow and interpret your mail.
>>
>> basically, when setup switches to slave, packets (matching
>> given state) have wrong prio set ("wrong" means they were
>> "right" when state was created on master).
>>
>> I will be glad to provide more information/tests/etc - just say
>> what is needed.
>
> Do you have the same ruleset checksum on both machines? check with
> pfctl -vs info | fgrep Checksum

yes. checksums are same.

>
>>
>> >
>> > On 20 November 2013 12:11, Alexey Suslikov <[hidden email]> wrote:
>> >> Hi.
>> >>
>> >> This is on 5.4-stable. Trivial master/slave carp(4) setup. vlan(4) is to
>> >> make picture clear wrt prio.
>> >>
>> >> Test 1 (without using match).
>> >>
>> >> pf.conf (BOX1 and BOX2).
>> >>
>> >> ext_if="vlan101"
>> >> dmz_if="vlan10"
>> >> pf_sync="vlan50"
>> >> block log all
>> >> pass quick on $pf_sync proto pfsync keep state (no-sync) set prio 7
>> >> pass quick on { $ext_if, $dmz_if } proto carp keep state (no-sync)
>> >> pass quick on $dmz_if inet proto icmp all icmp-type echoreq set prio 5
>> >> pass quick on $dmz_if
>> >> pass out quick on $ext_if inet proto icmp all icmp-type echoreq set prio 5
>> >> pass out quick on $ext_if
>> >>
>> >> BOX1 is Master, BOX2 is Slave.
>> >>
>> >> BOX1:
>> >> 00:07:36.108948 802.1Q vid 10 pri 3 X.X.185.145 > X.X.36.14: icmp: echo request
>> >> 00:07:36.109281 802.1Q vid 101 pri 5 X.X.185.145 > X.X.36.14: icmp: echo request
>> >> 00:07:36.110013 802.1Q vid 101 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply
>> >> 00:07:36.110030 802.1Q vid 10 pri 5 X.X.36.14 > X.X.185.145: icmp: echo reply
>> >>
>> >> BOX1 is Slave, BOX2 is Master.
>> >>
>> >> BOX2:
>> >> 00:12:43.981979 802.1Q vid 10 pri 3 X.X.185.145 > X.X.36.14: icmp: echo request
>> >> 00:12:43.982013 802.1Q vid 101 pri 0 X.X.185.145 > X.X.36.14: icmp: echo request
>> >> 00:12:43.982693 802.1Q vid 101 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply
>> >> 00:12:43.982713 802.1Q vid 10 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply
>> >>
>> >> Test 2 (using match).
>> >>
>> >> pf.conf (BOX1 and BOX2).
>> >>
>> >> ext_if="vlan101"
>> >> dmz_if="vlan10"
>> >> pf_sync="vlan50"
>> >> block log all
>> >> match quick on { $ext_if, $dmz_if } inet proto icmp all icmp-type
>> >> echoreq set prio 5
>> >> pass quick on $pf_sync proto pfsync keep state (no-sync) set prio 7
>> >> pass quick on { $ext_if, $dmz_if } proto carp keep state (no-sync)
>> >> pass quick on $dmz_if
>> >> pass out quick on $ext_if
>> >>
>> >> BOX1 is Master, BOX2 is Slave.
>> >>
>> >> BOX1:
>> >> 00:27:47.442820 802.1Q vid 10 pri 3 X.X.185.145 > X.X.36.14: icmp: echo request
>> >> 00:27:47.442839 802.1Q vid 101 pri 5 X.X.185.145 > X.X.36.14: icmp: echo request
>> >> 00:27:48.468709 802.1Q vid 101 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply
>> >> 00:27:47.443523 802.1Q vid 10 pri 5 X.X.36.14 > X.X.185.145: icmp: echo reply
>> >>
>> >> BOX1 is Slave, BOX2 is Master.
>> >>
>> >> BOX2:
>> >> 00:30:35.317329 802.1Q vid 10 pri 3 X.X.185.145 > X.X.36.14: icmp: echo request
>> >> 00:30:35.317354 802.1Q vid 101 pri 0 X.X.185.145 > X.X.36.14: icmp: echo request
>> >> 00:30:35.318065 802.1Q vid 101 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply
>> >> 00:30:35.318084 802.1Q vid 10 pri 0 X.X.36.14 > X.X.185.145: icmp: echo reply
>> >>
>> >> Maybe ICMP is not a sort of traffic which makes difference, but think
>> >> about TCP ACKs are prioritized. Switching to Slave in production setup
>> >> makes things *REALLY* bad.
>> >>
>> >> Should I configure something, or this is an issue?
>> >>
>> >> (Speaking of pfsync code, I'm unable to find where prio is set inside
>> >> pfsync_state_import).
>> >>
>> >> Thanks,
>> >> Alexey
>> >>
>>
>
> --
> I'm not entirely sure you are real.