Mixed internal network traffic, separated multiple ISPs, help?

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

Mixed internal network traffic, separated multiple ISPs, help?

I've struggled for a couple days configuring an OpenBSD
router/firewall and would like some help from the experts. I suspect
my difficulties stem from a my lack of route(8) knowledge.

Here's the basic setup and goal:

VOIP Phones (public 20.0.0.x/24)
     mixed with
Office PCs (private 192.168.1.x/24)
    HW switch
    $int_if OpenBSD router (
             |   +--- $ext_if for "PCs ISP" (configured via dhclient)
             +--- $voip_if for "Phone ISP" (either no addy or, route to

The OpenBSD router has 3 NICs- $int_if faces the single internal
switch that all the VOIP phones and office PCs connect to. $voip_if
faces an ISP that's assigned us public IPs for all the phones, and we
can use one of those for $voip_if itself. $ext_if faces another ISP,
and gets its address, gateway, and DNS servers via DHCP.

The VOIP phones have publicly routable addresses, all assigned from
the CIDR block. The office PCs get their addresses via
DHCP from the OpenBSD router in the private network.

All the VOIP traffic is to flow through the OpenBSD router, between
$voip_if and $int_if. All other external traffic is to travel over
$ext_if. The router itself needs to be ssh'able, serve DHCP to the
internal network, and provide other services later.

Plugging the VOIP ISP directly into our internal switch works, but
then we're bypassing our OpenBSD router for that traffic. We want to
tweak that traffic later, after the basic setup works, so bypassing is
not an option.

Tried so far:

A. http://www.openbsd.org/faq/pf/pools.html looked promising. I set up
pf so that "route-to" ($voip_if and
! "route-to" ($ext_if, with a nat on $ext_if
from -> ($ext_if)

This worked great for all the PCs, they could see the outside world
just fine. But the phones got no traffic at all. "tcpdump -i $int_if
net" showed no traffic. My guess is that the OpenBSD box
didn't advertise that $int_if was a route for that traffic. And why
should it? $int_if has the address, that's not on the net.

a smaller problem- how to specify what remote host to route-to in
pf-conf when the interface is configured via DHCP? ($ext_if) will
resolve to a changing interface address, but there's no way I can find
to symbolicly use that interface's remote router. I had to look up its
address in /var/db/dhclient.leases.$ext_if and hard-code that
( in our example). If our upstream ISP decides to change
what network it assigns to us, then the router on the other end
changes, and the "route-to" breaks

B. Add a bridge for the phone traffic. "ifconfig bridge0 create;
brconfig bridge0 add $int_if add $voip_if up" - created
/etc/bridgename.bridge0 to do just that at boot. Added rules to
pf.conf so only traffic would flow through $voip_if. Keep
the route-to for the PC traffic so it keeps going to $ext_if.

With the bridge the phones work great! Can call out, can recieve
incoming calls. And the PCs work too!  For a while... when the phones
are unused, everything is great. But pick up a phone, and some of PCs
lose all connections. They can't even get responses to "ping"  And hanging up/disconnecting the phones after doesn't
fix the problem. I can't predict which PCs will lose connections or
when, it seems random. Some PCs continue to work!

tcpdump shows some 192.168.1.x traffic "leaking" onto bridge0. Even
traffic for "ping" sometimes shows up on bridge0.

In act of desparation tried to add a rule to pf.conf by IP address on
bridge0 itself, but pfctl -vs rules showed that it never matched. I
know that brconfig can add rules at the bridge level to filter on MAC
address, but that seems difficult to maintain when adding/swapping
phones regularly.

I'm lost. A co-worker is building another OpenBSD box with 4 NICs, so
there can be one internal NIC for VOIP traffic, and another for the
office PCs. While that seems conceptually cleaner, all the traffic
will be going through the same HW switch- and I forsee similar issues.
If we could make all the phones go to one switch, connect that to one
internal NIC, and all the PCs go to another switch, and into the
second internal NIC, then this would be easy. I think. But we don't
have the space or the hardware.

So maybe I create a tun or two and use some pf wizardry to make it
look like two separate nics getting segregated traffic- sounds ugly
and I'm not sure I could get it to work...

And anyway, my gut feeling is that what I want to do is not all that
convoluted, and should have a not-so-convoluted solution, especially
with OpenBSD. If you can point me the right way I'll be mighty

thanks for your time