vlan(4), native vlan/vlan1, OpenBSD v.s. NetBSD behavior

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

vlan(4), native vlan/vlan1, OpenBSD v.s. NetBSD behavior

Brian A. Seklecki
All:

I've got a NetBSD 2.x box acting as a VLAN router for two Cisco 2924's
running IOS Version 12.0(5)WC10.

And I've also got an OpenBSD 3.7 box acting as a VLAN router for two Cisco
2924s running IOS Version 12.0(5)WC11

Both have a VLAN1 interface configured where I stick insecure services
(like telnet to the switchs, SNMP to UPSs, Hardware IPMI, Dell DRAC cards,

nbsd$ ifconfig vlan1
vlan1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
         vlan: 1 parent: fxp0
         address: 00:90:27:5d:f3:ee
         inet 10.0.0.1 netmask 0xffffff00 broadcast 10.0.0.255
         inet6 fe80::290:27ff:fe5d:f3ee%vlan1 prefixlen 64 scopeid 0x9

obsd% ifconfig vlan1
vlan1: flags=8943<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
         address: 00:50:da:28:37:7f
         vlan: 1 parent interface: xl0
         inet6 fe80::250:daff:fe28:377f%vlan1 prefixlen 64 scopeid 0x7
         inet 10.0.0.3 netmask 0xfffffff8 broadcast 10.0.0.7


Is there some major behavior difference in handling of the native VLAN
and/or Vlan#1 between the two platforms?  I recently had it out pretty
badly with a support person from Dell regarding a PowerConnect switch that
refused to trunk Vlan1 with a 3com switch.

3com's switches use a vlan-a-la-cart model, e.g., "this port has Vlan X as
trunked, and Vlans Y,Z as untrunked", where as most "other" swithes take
the Cisco VLAN model: either Access (1 vlan untrunked) or Trunk (all VLANs
trunked, or limited trunk (and *yes*, I know about hybrid mode)).

Previously I had only worked with Cisco, so I supposed I made the
assumption that it was Cisco/general practice to tag Vlan1 (or whatever
the native VLAN is) on Trunk ports.  I made this assumption based on what
I thought was known-working-behavior with my NetBSD VLAN Router, but after
taking a closer look, I'm seeing some odd behavior that may contradict
that.  I may owe the Dell guy an appology.


NetBSD Behavior:

When I ping from the Mgmnt./Vlan1 Interface on a c2924 to the IP assigned
to the Vlan1 interface on the NetBSD box in the same subnet, I see the

netbsd% tcpdump -i fxp0 'proto \icmp' -- Physical Interface:
14:24:21.337228 10.0.0.2 > mrvlan-vlan1: icmp: echo request seq 3271
14:24:21.340518 10.0.0.2 > mrvlan-vlan1: icmp: echo request seq 3271
14:24:21.343818 10.0.0.2 > mrvlan-vlan1: icmp: echo request seq 3271
14:24:21.346420 10.0.0.2 > mrvlan-vlan1: icmp: echo request seq 3271
14:24:21.349690 10.0.0.2 > mrvlan-vlan1: icmp: echo request seq 3271

The packets coming from the Cisco are untagged, but the subnet matches
that assigned to int vlan1.

But responses seem to leave as tagged w/ Vlan1 and as long as I have
static ARP entries setup for the master/base MAC address of the switch
(not the MAC of the physical interface port of the trunk).

nbsd% arp -an
? (10.0.0.2) at 00:02:fd:e8:11:40 on vlan1 permanent
? (10.0.0.3) at 00:04:27:95:6a:80 on vlan1 permanent

...and on the Cisco side:

# sh run
arp 10.0.0.1 0090.275d.f3ee ARPA

Return packets:

netbsd% tcpdump -i vlan 'proto \icmp' -- Vlan Logical 1:
14:24:50.199901 mrvlan-vlan1 > 10.0.0.2: icmp: echo reply seq 738
14:24:50.205955 mrvlan-vlan1 > 10.0.0.2: icmp: echo reply seq 738
14:24:50.210370 mrvlan-vlan1 > 10.0.0.2: icmp: echo reply seq 738
14:24:50.212962 mrvlan-vlan1 > 10.0.0.2: icmp: echo reply seq 738
14:24:50.216269 mrvlan-vlan1 > 10.0.0.2: icmp: echo reply seq 738

The packets do indeed make it back to the Switch.

Where tcpdump(8) would normally show packets on the physical interface
with
a VLAN tag:

14:28:03.508656 802.1Q vlan#10 P0 host-x-y-z-0.pit.choiceone.net.58394 >
ns4.choiceone.net.domain:  53401+ PTR?

The above ICMP packets on vlan1 are matched on tcpdump -i fxp0 with tagged
equiv.

14:43:10.072794 802.1Q vlan#1 P0 10.0.0.1 > 10.0.0.3: icmp: echo reply seq
5183

So it seems that NetBSD has some "magic code"(r) to deal with the native
VLAN, because most admins assume that a VLAN router can see a VLAN1
interface on a trunk regardless if the packets are tagged or not.


However, unless tcpdump(8) is lying, then Cisco must have a similar hook
because it's recieving the tagged return packet and accepting it as a
response to untagged send.

This this practicle in practice because it complies with the behavior most
vendors practice, and it avoids the need to assign an IPv4 address to the
physical interface (which is confusing and misleading).




OpenBSD Behavior:

When I run the ping on the OpenBSD system, vlan1 never sees the untagged
traffic.

$ sudo tcpdump -n -i vlan1
tcpdump: listening on vlan1, link-type EN10MB

[nothing]

However the physical interface does see it.  Unless, however, I configure
an IPv4 address on the physical interface which acts as the parent of all
the vlan interfaces on the router, the router will not ARP for VLAN1 (Even
with static ARP entries on both the Cisco and OpenBSD side.  )

It would seem that OpenBSD is more strictly following what I presume is
the published behavior/standard.

$ sudo tcpdump -n -i xl0
15:01:05.562906 arp who-has 10.0.0.1 tell 10.0.0.3
15:01:07.562869 arp who-has 10.0.0.1 tell 10.0.0.3
15:01:08.293927 arp who-has 10.0.0.2 tell 10.0.0.3
15:01:08.293905 10.0.0.2 > 10.0.0.3: icmp: echo request

-bash-3.00$ ifconfig xl0
xl0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
         address: 00:50:da:28:37:7f
         inet 10.0.0.3 netmask 0xfffffff8 broadcast 10.0.0.7

? (10.0.0.2) at 00:02:fd:0e:f3:80 on xl0

# sh arp
Internet  10.0.0.3                0   0050.da28.377f  ARPA   VLAN1


==================
==================

So now I'm wondering (I don't know how to change it, and I don't have set
of test
equipment to run a test on), but if I were to change the Native VLAN
of the trunk port on the Cisco side facing NetBSD/OpenBSD, would the
switch start tagging
VLAN1?

I know if I enter interface configuration for Vlan1 on the Cisco side,
there
exists a config directive to flag a Vlan as the management VLAN:

bs1#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
bs1(config)#int vla
bs1(config)#int vlaN 1
bs1(config-if)#man
bs1(config-if)#management ?
   <cr>

bs1(config-if)#management


Thoughts?

~BAS

l8*
  -lava

x.25 - minix - bitnet - plan9 - 110 bps - ASR 33 - base8

Reply | Threaded
Open this post in threaded view
|

Re: vlan(4), native vlan/vlan1, OpenBSD v.s. NetBSD behavior

Mike Belopuhov
On Wed, Dec 14, 2005 at 16:02 -0500, Brian A. Seklecki wrote:

> $ sudo tcpdump -n -i xl0
> 15:01:05.562906 arp who-has 10.0.0.1 tell 10.0.0.3
> 15:01:07.562869 arp who-has 10.0.0.1 tell 10.0.0.3
> 15:01:08.293927 arp who-has 10.0.0.2 tell 10.0.0.3
> 15:01:08.293905 10.0.0.2 > 10.0.0.3: icmp: echo request
>
> -bash-3.00$ ifconfig xl0
> xl0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
>         address: 00:50:da:28:37:7f
>         inet 10.0.0.3 netmask 0xfffffff8 broadcast 10.0.0.7
>

AFAIR, some (or most/all?) xl NICs do not support ethernet frames of
length more than MTU. So 802.1q packets just couldn't be processed.

And Intel ones (which are supported by the fxp(4) driver) do support
such long frames.

Sorry, if I mistaken.

Reply | Threaded
Open this post in threaded view
|

Re: vlan(4), native vlan/vlan1, OpenBSD v.s. NetBSD behavior

Brian A. Seklecki
On Thu, 15 Dec 2005, Mike Belopuhov wrote:

> AFAIR, some (or most/all?) xl NICs do not support ethernet frames of

That's strange, because I just got OpenBSD to perform like my NetBSD
config by forcing ARP entries everywhere.  ARP on VLAN1 is definately
broken in this config:


Physical IF tcpdump:
16:09:11.913598 802.1Q vid 1 pri 0 10.0.0.3 > 10.0.0.2: icmp: echo reply
16:09:11.913584 10.0.0.2 > 10.0.0.3: icmp: echo request

Replys:

Vlan1 tcpdump:
16:10:16.754775 10.0.0.3 > 10.0.0.2: icmp: echo reply
16:10:16.759987 10.0.0.3 > 10.0.0.2: icmp: echo reply

obsd# arp -an
? (10.0.0.2) at 00:02:fd:0e:f3:80 on vlan1 static

bs1#sh run | include arp
arp 10.0.0.3 0050.da28.377f ARPA

bs1# sh arp
Internet  10.0.0.2                -   0002.fd0e.f380  ARPA   VLAN1

bs1#sh mac-address-table  address ?
   H.H.H  48-bit hardware address

bs1#sh mac-address-table  address 0050.da28.377f
Non-static Address Table:
Destination Address  Address Type  VLAN  Destination Port
-------------------  ------------  ----  --------------------
0050.da28.377f       Dynamic          1  FastEthernet0/24
0050.da28.377f       Dynamic         20  FastEthernet0/24

bs1>ping 10.0.0.4

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.4, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/3/6 ms

seklecki@br1: ~# ping 10.0.0.2
PING 10.0.0.2 (10.0.0.2): 56 data bytes
64 bytes from 10.0.0.2: icmp_seq=0 ttl=255 time=1.299 ms


~BAS

> length more than MTU. So 802.1q packets just couldn't be processed.
>
> And Intel ones (which are supported by the fxp(4) driver) do support
> such long frames.
>
> Sorry, if I mistaken.
>

l8*
  -lava

x.25 - minix - bitnet - plan9 - 110 bps - ASR 33 - base8

Reply | Threaded
Open this post in threaded view
|

Re: vlan(4), native vlan/vlan1, OpenBSD v.s. NetBSD behavior

Brian A. Seklecki
In reply to this post by Brian A. Seklecki
> Create a real vlan ID for your mgmt/bad traffic. Trunk that one like the
> rest. Dont use native vlans as there is such poor vendor implementation.
> Or use one that is 'not vlan-id 1'. E.g. make all trunks native in vlan
> tag 99 or something.

Yea the Dell guy was trying to tell me that over my war cry.

In many established networks with pure public IP space, I've always seen
VLAN1 assigned a /24 for that facility's router loopback interfaces (for
OSPF, etc.) and switch managment interfaces.

In networks with combination private/public IP space, I tend to use VLAN1
to isolate crap. But I guess practices can change.

--

The Dell guy said that even Cisco recommends that as part of a S.A.F.E.
spec, but I wasn't buying it.

The catch is that Dell's low-end 27xx PowerConnect (which are great for
drop switches to workstations, btw) has a Web Interface:

BUT:

1) There's a *javascript* that prevents you from setting an interface as
tagged for VLAN1 (and I'm uplinking to an old 3com in the Core which
*does*, and lacks the concept of a native VLAN all together)

2) AND, you can't change the management VLAN on the PowerConnect from
anything other than VLAN1

Thus, I cant make the webmanagement interface in VLAN1 visible outside of
hosts with a port on the switch.

...unless I configure the 3com to send VLAN1 untagged on the dot1q trunk
to the Dell, and send VLAN1 tagged to OpenBSD, which I guess I visualize
as a security risk; but VLANs shouldn't be used as a security mechanism
alone.

It's all bad either way >:}

~BAS

Reply | Threaded
Open this post in threaded view
|

Re: vlan(4), native vlan/vlan1, OpenBSD v.s. NetBSD behavior

Jason Ackley
On Wed, 14 Dec 2005, Brian A. Seklecki wrote:

> In many established networks with pure public IP space, I've always seen
> VLAN1 assigned a /24 for that facility's router loopback interfaces (for
> OSPF, etc.) and switch managment interfaces.

   YMMV, but using tag-id of 1 has always been bad for me. I use 2 for
 this network at each POP.

> In networks with combination private/public IP space, I tend to use VLAN1
> to isolate crap. But I guess practices can change.

  Practice is good, just pick a tag that doesn't mean special things to
  some vendors..

> The Dell guy said that even Cisco recommends that as part of a S.A.F.E.
> spec, but I wasn't buying it.

 The SAFE blueprint sez:

 'Always use a dedicated VLAN ID for all trunk ports. '

 and a few items later:

 'Avoid using VLAN 1 for management purposes and eliminate native VLANs
from 802.1q trunks.'

 What that means is they want you to drop trunk ports into a vlan ID that
 is basically not used for anything, even traffic. This way everything
 gets tagged on 802.1q trunks (like ISL).

> 1) There's a *javascript* that prevents you from setting an interface as
> tagged for VLAN1 (and I'm uplinking to an old 3com in the Core which
> *does*, and lacks the concept of a native VLAN all together)
>
> 2) AND, you can't change the management VLAN on the PowerConnect from
> anything other than VLAN1

 Then your Dell guy needs to go back to the blueprint that he is banging
 over your head due to the special handling they have of 'VLAN1'.

> ...unless I configure the 3com to send VLAN1 untagged on the dot1q trunk
> to the Dell, and send VLAN1 tagged to OpenBSD, which I guess I visualize
> as a security risk; but VLANs shouldn't be used as a security mechanism
> alone.

 You can do some neat things with vlan(4) and bridge(4) with recent
 OpenBSD versions(post 3.7). E.g. transparently bridge everything and
 'peel off' specific vlan tags for termination on the openbsd box (or even
 to be bridged to other bridge group/vlan interface, hint hint).

I used this on a Gigabit-only cisco that I had and I was low on optical
ports. So I put in a Gigabit trunk between the openbsd box and cisco that
bridges to an fxp/dot1q and I peel off a few vlans for special treatment
on the openbsd box. (gif, etc)

It doesn't help having vendors do silly things like that tho.


cheers,
--
jason

Reply | Threaded
Open this post in threaded view
|

Re: vlan(4), native vlan/vlan1, OpenBSD v.s. NetBSD behavior

Brad Smith-14
In reply to this post by Mike Belopuhov
On Thu, Dec 15, 2005 at 12:45:55AM +0300, Mike Belopuhov wrote:

> On Wed, Dec 14, 2005 at 16:02 -0500, Brian A. Seklecki wrote:
> > $ sudo tcpdump -n -i xl0
> > 15:01:05.562906 arp who-has 10.0.0.1 tell 10.0.0.3
> > 15:01:07.562869 arp who-has 10.0.0.1 tell 10.0.0.3
> > 15:01:08.293927 arp who-has 10.0.0.2 tell 10.0.0.3
> > 15:01:08.293905 10.0.0.2 > 10.0.0.3: icmp: echo request
> >
> > -bash-3.00$ ifconfig xl0
> > xl0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
> >         address: 00:50:da:28:37:7f
> >         inet 10.0.0.3 netmask 0xfffffff8 broadcast 10.0.0.7
> >
>
> AFAIR, some (or most/all?) xl NICs do not support ethernet frames of
> length more than MTU. So 802.1q packets just couldn't be processed.
>
> And Intel ones (which are supported by the fxp(4) driver) do support
> such long frames.
>
> Sorry, if I mistaken.

You're mistaken. xl(4)'s with OpenBSD can support VLAN sized frames.

Reply | Threaded
Open this post in threaded view
|

Re: vlan(4), native vlan/vlan1, OpenBSD v.s. NetBSD behavior

Chris Cappuccio
In reply to this post by Brian A. Seklecki
You guys are making a simple mistake...  Brian, your static ARP entries
are a hack!

Vlan 1 is meaningless.  Most vendors use vlan 1 to refer to _untagged_
traffic.  However, when you create a vlan interface with OpenBSD, it is
always tagged, even if the id happens to be 1.  Your switches will never
generate traffic with a tag of 1.

The OpenBSD nomenclature for an untagged packet is to have that
IP address on the parent interface.  So, to utilize "vlan 1" you need to
put that IP address on the parent interface, not on a vlan with id of 1.

If you can, it's nice to move your management "networks" to vlans other than
1 so they are actually tagged.  That way, you are explicitly joining everything
that wants to talk on the management vlan, and there aren't going to be
random devices joining the management vlan that you didn't explicitly put
there.  (Naturally, it's rather easy to misconfigure any switch on any part
of your network to pass untagged traffic.)

Brian A. Seklecki [[hidden email]] wrote:

> On Thu, 15 Dec 2005, Mike Belopuhov wrote:
>
> >AFAIR, some (or most/all?) xl NICs do not support ethernet frames of
>
> That's strange, because I just got OpenBSD to perform like my NetBSD
> config by forcing ARP entries everywhere.  ARP on VLAN1 is definately
> broken in this config:
>
>
> Physical IF tcpdump:
> 16:09:11.913598 802.1Q vid 1 pri 0 10.0.0.3 > 10.0.0.2: icmp: echo reply
> 16:09:11.913584 10.0.0.2 > 10.0.0.3: icmp: echo request
>
> Replys:
>
> Vlan1 tcpdump:
> 16:10:16.754775 10.0.0.3 > 10.0.0.2: icmp: echo reply
> 16:10:16.759987 10.0.0.3 > 10.0.0.2: icmp: echo reply
>
> obsd# arp -an
> ? (10.0.0.2) at 00:02:fd:0e:f3:80 on vlan1 static
>
> bs1#sh run | include arp
> arp 10.0.0.3 0050.da28.377f ARPA
>
> bs1# sh arp
> Internet  10.0.0.2                -   0002.fd0e.f380  ARPA   VLAN1
>
> bs1#sh mac-address-table  address ?
>   H.H.H  48-bit hardware address
>
> bs1#sh mac-address-table  address 0050.da28.377f
> Non-static Address Table:
> Destination Address  Address Type  VLAN  Destination Port
> -------------------  ------------  ----  --------------------
> 0050.da28.377f       Dynamic          1  FastEthernet0/24
> 0050.da28.377f       Dynamic         20  FastEthernet0/24
>
> bs1>ping 10.0.0.4
>
> Type escape sequence to abort.
> Sending 5, 100-byte ICMP Echos to 10.0.0.4, timeout is 2 seconds:
> !!!!!
> Success rate is 100 percent (5/5), round-trip min/avg/max = 1/3/6 ms
>
> seklecki@br1: ~# ping 10.0.0.2
> PING 10.0.0.2 (10.0.0.2): 56 data bytes
> 64 bytes from 10.0.0.2: icmp_seq=0 ttl=255 time=1.299 ms
>
>
> ~BAS
>
> >length more than MTU. So 802.1q packets just couldn't be processed.
> >
> >And Intel ones (which are supported by the fxp(4) driver) do support
> >such long frames.
> >
> >Sorry, if I mistaken.
> >
>
> l8*
> -lava
>
> x.25 - minix - bitnet - plan9 - 110 bps - ASR 33 - base8

--
"You were about to change the channel when God healed you" -- Benny Hinn