Change routing tables when ISP goes "down"

classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

Change routing tables when ISP goes "down"

cayuga2
I have a very unreliable ISP (approximately 97% uptime).  Many of the times that they go
down, I'm connected and can ping within their limited network, but can't get to the
"outside world".  In these cases, I have an alternate slow speed connection that I use.  
Right now, I manually change the default route and use pfctl to invoke an alternate
pf.conf file.  

I'm thinking that OpenOSPF, BIRD or one of the other routing oriented daemons might be a
way to automate switching back and forth.

Does anyone suggestions on effective ways to automate/manage this?  

Thanks!
        Jeff

Reply | Threaded
Open this post in threaded view
|

Re: Change routing tables when ISP goes "down"

Alan McKay
ifstated could do it ...

Reply | Threaded
Open this post in threaded view
|

Re: Change routing tables when ISP goes "down"

Stefan Sperling-8
In reply to this post by cayuga2
On Wed, Oct 01, 2014 at 11:10:12AM -0400, Jeff wrote:

> I have a very unreliable ISP (approximately 97% uptime).  Many of the times that they go
> down, I'm connected and can ping within their limited network, but can't get to the
> "outside world".  In these cases, I have an alternate slow speed connection that I use.  
> Right now, I manually change the default route and use pfctl to invoke an alternate
> pf.conf file.  
>
> I'm thinking that OpenOSPF, BIRD or one of the other routing oriented daemons might be a
> way to automate switching back and forth.
>
> Does anyone suggestions on effective ways to automate/manage this?  
>
> Thanks!
> Jeff

Have you considered using ifstated(8) with external tests (e.g. ping)?
See the ifstated.conf(5) man page.

Reply | Threaded
Open this post in threaded view
|

Re: Change routing tables when ISP goes "down"

Alucard
In reply to this post by cayuga2
On 2014-10-01 16:10, Jeff wrote:

> I have a very unreliable ISP (approximately 97% uptime).  Many of the
> times that they go
> down, I'm connected and can ping within their limited network, but
> can't get to the
> "outside world".  In these cases, I have an alternate slow speed
> connection that I use.
> Right now, I manually change the default route and use pfctl to invoke
> an alternate
> pf.conf file.
>
> I'm thinking that OpenOSPF, BIRD or one of the other routing oriented
> daemons might be a
> way to automate switching back and forth.
>
> Does anyone suggestions on effective ways to automate/manage this?
>
> Thanks!
> Jeff


Implementing a dynamic routing protocol will ensure the switch over but
would require either ISP cooperation or a server on the internet side.

the easiest way to achieve what you want is scripting default route
change.
Something like that should do the trick.



while true
do

         route1=$(ping -I $INTERFACE_TO_ISP1 $ISP1_GATEWAY -c 1 | tail
-n2 | head -1 | grep -c "1 received")
         route2=$(ping -I $INTERFACE_TO_ISP1 $ISP2_GATEWAY -c 1 | tail
-n2 | head -1 | grep -c "1 received")
         routa=$(ip route | grep "default" | cut -d' ' -f3 | tr -d ' ')

         if [ "$route1" != "1" ]
         then
                 route del default
                 route add default gw $ISP2_GATEWAY
         else
                 if [ "$routa" != "$ISP1_GATEWAY" ]
                 then
                      route del default
                      route add default gw $ISP1_GATEWAY
                 fi
         fi

         sleep $waittime //you may want to wait a bit between checks
done

Regards
Louis

Reply | Threaded
Open this post in threaded view
|

Re: Change routing tables when ISP goes "down"

cayuga2
This post was updated on .
It sounds like "ping -I" is what I was looking for, but when I use it, it seems
to be sending out the packet with the right source address, but sending it to
the wrong interface.....are there any tricks here?

Here's some data (edited) to show what I'm seeing:

fxp0: inet 10.16.100.1 netmask 0xfffffff0 broadcast 10.16.100.15

fxp1: inet 192.168.243.152 netmask 0xffffff00 broadcast 192.168.243.255

when I try "ping -I 192.168.243.152 ucla.edu", I see the following:

tcpdump -i fxp0 icmp and host ucla.edu
tcpdump: listening on fxp0, link-type EN10MB
13:06:36.478450 192.168.243.152 > 128.97.27.37: icmp: echo request
13:06:37.483393 192.168.243.152 > 128.97.27.37: icmp: echo request
13:06:38.493244 192.168.243.152 > 128.97.27.37: icmp: echo request

The routing table shows:

10.16.100.0/28     link#1             UC         4        0     -     4 fxp0
192.168.243/24     link#2             UC         1        0     -     4 fxp1


On Wed, Oct 01, 2014 at 05:23:43PM +0100, alucard@phangos.fr wrote:
> On 2014-10-01 16:10, Jeff wrote:
> >I have a very unreliable ISP (approximately 97% uptime).  Many of the
> >times that they go
> >down, I'm connected and can ping within their limited network, but
> >can't get to the
> >"outside world".  In these cases, I have an alternate slow speed
> >connection that I use.
> >Right now, I manually change the default route and use pfctl to invoke
> >an alternate
> >pf.conf file.
> >
> >I'm thinking that OpenOSPF, BIRD or one of the other routing oriented
> >daemons might be a
> >way to automate switching back and forth.
> >
> >Does anyone suggestions on effective ways to automate/manage this?
> >
> >Thanks!
> > Jeff
>
>
> Implementing a dynamic routing protocol will ensure the switch over but
> would require either ISP cooperation or a server on the internet side.
>
> the easiest way to achieve what you want is scripting default route change.
> Something like that should do the trick.
>
>
>
> while true
> do
>
>         route1=$(ping -I $INTERFACE_TO_ISP1 $ISP1_GATEWAY -c 1 | tail -n2 |
> head -1 | grep -c "1 received")
>         route2=$(ping -I $INTERFACE_TO_ISP1 $ISP2_GATEWAY -c 1 | tail -n2 |
> head -1 | grep -c "1 received")
>         routa=$(ip route | grep "default" | cut -d' ' -f3 | tr -d ' ')
>
>         if [ "$route1" != "1" ]
>         then
>                 route del default
>                 route add default gw $ISP2_GATEWAY
>         else
>                 if [ "$routa" != "$ISP1_GATEWAY" ]
>                 then
>                      route del default
>                      route add default gw $ISP1_GATEWAY
>                 fi
>         fi
>
>         sleep $waittime //you may want to wait a bit between checks
> done
>
> Regards
> Louis
>

--
===============================================================================
                        Jeff's Used Movie Finder    
                     http://www.usedmoviefinder.com
                    email: jeff@usedmoviefinder.com

Reply | Threaded
Open this post in threaded view
|

Re: Change routing tables when ISP goes "down"

cayuga2
This post was updated on .
In reply to this post by Alucard
It sounds like "ping -I" is what I was looking for, but when I use it, it seems
to be sending out the packet with the right source address, but sending it to
the wrong interface.....are there any tricks here?

Here's some data (edited) to show what I'm seeing:

fxp0: inet 10.16.100.1 netmask 0xfffffff0 broadcast 10.16.100.15

fxp1: inet 192.168.243.152 netmask 0xffffff00 broadcast 192.168.243.255

when I try "ping -I 192.168.243.152 ucla.edu", I see the following:

tcpdump -i fxp0 icmp and host ucla.edu
tcpdump: listening on fxp0, link-type EN10MB
13:06:36.478450 192.168.243.152 > 128.97.27.37: icmp: echo request
13:06:37.483393 192.168.243.152 > 128.97.27.37: icmp: echo request
13:06:38.493244 192.168.243.152 > 128.97.27.37: icmp: echo request

The routing table shows:

10.16.100.0/28     link#1             UC         4        0     -     4 fxp0
192.168.243/24     link#2             UC         1        0     -     4 fxp1
Reply | Threaded
Open this post in threaded view
|

Re: Change routing tables when ISP goes "down"

Gerald Chudyk
In reply to this post by cayuga2
On Wed, Oct 1, 2014 at 8:10 AM, Jeff <[hidden email]> wrote:

> I have a very unreliable ISP (approximately 97% uptime).  Many of the times that they go
> down, I'm connected and can ping within their limited network, but can't get to the
> "outside world".  In these cases, I have an alternate slow speed connection that I use.
> Right now, I manually change the default route and use pfctl to invoke an alternate
> pf.conf file.
>
> I'm thinking that OpenOSPF, BIRD or one of the other routing oriented daemons might be a
> way to automate switching back and forth.
>
> Does anyone suggestions on effective ways to automate/manage this?
>

Hi Jeff,

I have been casually working on this for some time now. I also have
two isp's. One more reliable than the other. The additional wish is to
load balance, since my backup isp is not that slow, so you can ignore
a few bits in the pf.conf files.

I almost have it working. I use ifstated, which calls a script called
manage-routes to do the heavy lifting. Multiple pf.conf files are
managed by anchors.

<Excuse>Something is wrong with what I have so far. Some quiet time is
needed to read through and trace the process, but I keep getting
interrupted by higher priorities. Plus my primary isp is very
reliable.</Excuse> Actually I am just lazy about most things until
there's an emergency.

Here are my files:

# cat ifstated.conf

shaw_linkup     = "vr1.link.up"
telus_linkup    = "vr2.link.up"

shaw_gate_test  = "( \"ping -q -c1 -w1 -I 199.71.129.170
199.71.129.169 > /dev/null \" every 15 )"
telus_gate_test = "( \"ping -q -c1 -w1 -I 200.116.7.41 200.116.7.1 >
/dev/null \" every 15 )"

init-state both

state both {
        init {
                run "/usr/local/sbin/manage-routes ALL"
        }
        if ! $telus_linkup {
                set-state shaw
        }
        if ! $shaw_linkup {
                set-state telus
        }
        if ! $telus_gate_test {
                set-state shaw
        }
        if !$shaw_gate_test {
                set-state telus
        }
}
state shaw {
        init {
                run "/usr/local/sbin/manage-routes SHAW"
        }
        if !$shaw_linkup {
                set-state telus
        }
        if !$shaw_gate_test {
                set-state telus
        }
        if $telus_gate_test {
                set-state both
        }
}

state telus {
        init {
                run "/usr/local/sbin/manage-routes TELUS"
        }
        if ! $telus_linkup {
                set-state none
        }
        if ! $telus_gate_test {
                set-state none
        }
        if $shaw_gate_test {
                set-state both
        }
}

state none {
        init {
                run "/usr/local/sbin/manage-routes NONE"
        }
        if $shaw_gate_test {
                set-state shaw
        }
        if $telus_gate_test {
                set-state telus
        }
}


I had a bit of fun with the led's on the front of the box, so you can
ignore that. Here is my route script:

# cat /usr/local/sbin/manage-routes
#!/bin/sh
#
# with help from Justin Jereza on [hidden email]
#

SCRIPT="$0";

function help {
    echo "Usage: $SCRIPT ALL | SHAW | TELUS | NONE";
}

function in_table {
    GW="$1";

    route -n show | grep '^default' | awk '{ print $2 }' | grep $GW
2>&1 > /dev/null;
}

function add_route {
    GW="$1";

    route add -mpath default $GW 2>&1 > /dev/null;
}

function delete_route {
    GW="$1";

    route delete default $GW 2>&1 > /dev/null;
}

function log_msg {
  SRV="$1";
  STATUS="$2";
  MSG="Unitow Network Status: $SRV is $STATUS";
  logger -p daemon.info -t ifstated $MSG ;
#  mail -s $MSG  -croot < "This is an automated message from gateway server";
}

function set_shaw_led {
        STATE="$1";
        gpioctl -q gpio0 shaw_led $STATE;
}
function set_telus_led {
        STATE="$1";
        gpioctl -q gpio0 telus_led $STATE;
}

function pf_all {
        pfctl -a isp_lan    -F rules;
        pfctl -a isp_egress -F rules;
        pfctl -a isp_lan    -f /etc/pf.all_lan.conf;
        pfctl -a isp_egress -f /etc/pf.all_egress.conf;
}
function pf_one {
        pfctl -a isp_lan    -F rules;
        pfctl -a isp_egress -F rules;
        pfctl -a isp_lan    -f /etc/pf.one_lan.conf;
        pfctl -a isp_egress -f /etc/pf.one_egress.conf;
}
function pf_none {
        pfctl -a isp_lan    -F rules;
        pfctl -a isp_egress -F rules;
}

if [ $# -ne 1 ]; then
    help;
    exit 1;
fi

STATE="$1";
SHAW_GW="184.71.129.169";
TELUS_GW="206.116.7.1";

case "$STATE" in
    ALL)
        if ! in_table $SHAW_GW; then
            add_route $SHAW_GW;
        fi
        if ! in_table $TELUS_GW; then
            add_route $TELUS_GW;
        fi
        pf_all;
        log_msg "SHAW" "UP";
        log_msg "TELUS" "UP";
        set_shaw_led "on";
        set_telus_led "on";
        ;;
    SHAW)
        if ! in_table $SHAW_GW; then
            add_route $SHAW_GW;
        fi
        if in_table $TELUS_GW; then
            delete_route $TELUS_GW;
        fi
        pf_one;
        log_msg "TELUS" "DOWN";
        set_shaw_led "on";
        set_telus_led "off";
        ;;
    TELUS)
        if in_table $SHAW_GW; then
            delete_route $SHAW_GW;
        fi
        if ! in_table $TELUS_GW; then
            add_route $TELUS_GW;
        fi
        pf_one;
        log_msg "SHAW" "DOWN";
        set_shaw_led "off";
        set_telus_led "on";
        ;;
    NONE)
        if in_table $SHAW_GW; then
            delete_route $SHAW_GW;
        fi
        if in_table $TELUS_GW; then
            delete_route $TELUS_GW;
        fi
        log_msg "SHAW" "DOWN";
        log_msg "TELUS" "DOWN";
        set_shaw_led "off";
        set_telus_led "off";
        pf_none;
        ;;
    *)
        help;
        exit 1;
        ;;
esac

pf.conf rules are pretty simple right now. I deleted a few variable
definitions because I am paranoid, but I hope their names are
descriptive enough:

# cat pf.all_lan.conf
#
# Used by ifstated to load balance services between shaw and telus
# These are the rules if both Shaw and Telus are up
#

# load balance www, but keep https on single connection
#
logit           = "log (all)"
logit           = " "

telus_if        = "vr2"
shaw_if         = "vr1"

telus_gw        ="200.116.7.1"
shaw_gw         ="199.71.129.169"
smtpIP          ="192.168.100.4"
webproxyIP      ="192.168.100.3"

pass in $logit on lan inet proto tcp from $webproxyIP to port www \
         route-to { ($shaw_if $shaw_gw) ($telus_if $telus_gw) } round-robin
pass in $logit on lan inet proto tcp from $webproxyIP to port https \
        route-to ($shaw_if $shaw_gw)
#
#
# email coming through
pass in log on lan inet proto tcp from $smtpIP to any port smtp \
        route-to ($shaw_if $shaw_gw)



# cat pf.one_lan.conf
#
# Used by ifstated to load balance services between shaw and telus
# These are the rules if one isp is down
#
logit           = "log (all)"
logit           = " "
smtpIP          ="192.168.100.4"
webproxyIP      ="192.168.100.3"

pass in $logit on lan inet proto tcp from $webproxyIP to port www
pass in $logit on lan inet proto tcp from $webproxyIP to port https

pass in $logit on lan inet proto tcp to port www
pass in $logit on lan inet proto tcp to port https
#
#
# email coming through
pass in log on lan inet proto tcp from $smtpIP to port smtp


# cat pf.all_egress.conf
#
# Used by ifstated to manage to egress routes
# This file is for load balancing two isp's
#
# we log outgoing smtp for spamd
#
# notice match statements above for NAT processing
#
logit           = "log (all)"
logit           = " "

telus_gw        = "200.116.7.1"
shaw_gw         = "199.71.129.169"

webproxyIP      ="192.168.100.3"
smtpIP          ="192.168.100.4"

telus_if        = "vr2"
shaw_if         = "vr1"

match out on $telus_if  from lan:network nat-to ($telus_if)
match out on $shaw_if  from lan:network nat-to ($shaw_if)

pass out $logit on shaw  inet
pass out $logit on telus inet
pass out $logit on telus inet from shaw  route-to ($shaw_if $shaw_gw)
pass out $logit on shaw  inet from telus route-to ($telus_if $telus_gw)

# cat pf.one_egress.conf
#
# Used by ifstated to change pf rules for one isp
#

#
# we log outgoing smtp for spamd
#
logit           = "log (all)"
logit           = " "
smtpIP          ="192.168.100.4"
webproxyIP      ="192.168.100.3"

match out on egress from lan:network nat-to (egress)

#
# notice match statements above for NAT processing
#
#pass out $logit   on egress inet
pass out log (all) on egress inet proto   tcp       to any port smtp
pass out $logit    on egress inet proto { tcp udp } to any port {
domain ftp ntp }
pass out $logit    on egress inet proto { tcp udp } to any port { www https }
#pass out $logit    on egress inet proto { tcp udp } from $webproxyIP
to any port { www https }


I am particularly accustomed to scorn and derision, but any comments
appreciated. I would happily make this simpler if someone will point
the way.

Gerald

Reply | Threaded
Open this post in threaded view
|

Re: Change routing tables when ISP goes "down"

Alan McKay
On Wed, Oct 1, 2014 at 2:10 PM, Gerald Chudyk <[hidden email]> wrote:
> I have been casually working on this for some time now.

Hey, nice work!


--
"Don't eat anything you've ever seen advertised on TV"
         - Michael Pollan, author of "In Defense of Food"

Reply | Threaded
Open this post in threaded view
|

Re: Change routing tables when ISP goes "down"

Giancarlo Razzolini-3
In reply to this post by cayuga2
On 01-10-2014 14:14, Jeff wrote:
> It sounds like "ping -I" is what I was looking for, but when I use it, it
seems
> to be sending out the packet with the right source address, but sending it
to
> the wrong interface.....are there any tricks here?
You must enforce through pf route-to the packets to go through the right
interface. Or, better yet, you should use multipath routing. Enable it
on your systctl.conf. It will allow you to have multiple default
gateways. If they both have the same priority the connections will go
out in a simple round-robin fashion.

Then you should use ifstated, as mentioned by others. If your ISP's
routers support SNMP, you could use it to check for the link status
instead of relying on external pinging. I only use it as last resort. On
some of my modems I even have a small script that connect with on the
administrative web interface to check if the link is up. On others I use
telnet and expect. I only use ping as a last resort.

I could help you with more elaborated examples, but I hope you got the idea.

Cheers

[demime 1.01d removed an attachment of type application/pkcs7-signature which had a name of smime.p7s]

Reply | Threaded
Open this post in threaded view
|

Re: Change routing tables when ISP goes "down"

cayuga2
Thanks to everyone for your help/suggestions.  I think that I'm headed in the
right direction.

I still can't seem to force a ping through a particular interface, even when I
have both interfaces as default routes (I've tried both with and without mpath).
If it matters, in both cases I used a lower priority (higher #) for our low speed
metered connection.

Here's my current routing information:

default            10.150.228.105     UGS        5   168287     -     8 fxp0
default            192.168.243.1      UGS        0        0     -    16 fxp1

and "ping -I 192.168.243.152 8.8.4.4" still sends traffic out through fxp0.

I have verified that if I swap the priorities that all traffic goes out through
fxp1 so I know that that connection works.

It feels like I'm missing something obvious here.  Can someone point me in the right
direction?

Thanks again!

Jeff




On Wed, Oct 01, 2014 at 07:35:41PM -0300, Giancarlo Razzolini wrote:

> On 01-10-2014 14:14, Jeff wrote:
> > It sounds like "ping -I" is what I was looking for, but when I use it, it
> seems
> > to be sending out the packet with the right source address, but sending it
> to
> > the wrong interface.....are there any tricks here?
> You must enforce through pf route-to the packets to go through the right
> interface. Or, better yet, you should use multipath routing. Enable it
> on your systctl.conf. It will allow you to have multiple default
> gateways. If they both have the same priority the connections will go
> out in a simple round-robin fashion.
>
> Then you should use ifstated, as mentioned by others. If your ISP's
> routers support SNMP, you could use it to check for the link status
> instead of relying on external pinging. I only use it as last resort. On
> some of my modems I even have a small script that connect with on the
> administrative web interface to check if the link is up. On others I use
> telnet and expect. I only use ping as a last resort.
>
> I could help you with more elaborated examples, but I hope you got the idea.
>
> Cheers
>
> [demime 1.01d removed an attachment of type application/pkcs7-signature which had a name of smime.p7s]

Reply | Threaded
Open this post in threaded view
|

Re: Change routing tables when ISP goes "down"

Giancarlo Razzolini-3
On 02-10-2014 10:11, Jeff wrote:
> I still can't seem to force a ping through a particular interface, even when
I
> have both interfaces as default routes (I've tried both with and without
mpath).
> If it matters, in both cases I used a lower priority (higher #) for our low
speed
> metered connection.
This is ok. The only thing is that a connection going out from the
firewall machine itself will never get routed to the fxp1 interface,
unless you force it through pf.
>
> Here's my current routing information:
>
> default            10.150.228.105     UGS        5   168287     -     8
fxp0
> default            192.168.243.1      UGS        0        0     -    16
fxp1
>
> and "ping -I 192.168.243.152 8.8.4.4" still sends traffic out through fxp0.
If I'm not mistaken, this should work. I'm guessing that your pf rules
are to blame.
>
> I have verified that if I swap the priorities that all traffic goes out
through
> fxp1 so I know that that connection works.
Good.
>
> It feels like I'm missing something obvious here.  Can someone point me in
the right
> direction?
Try disabling pf and see if it works. If it does, then you'll need to
change your rules to enforce the traffic to go to their respective
gateways. Some rule like this should do:

pass out on fxp0 from (fxp1) to any route-to fxp1
pass out on fxp1 from (fxp0) to any route-to fxp0

If that do not work, try including your gateways in the rule so it become:

pass out on fxp0 from (fxp1) to any route-to (fxp1 fxp1_gateway)
pass out on fxp1 from (fxp0) to any route-to (fxp0 fxp0_gateway)

Cheers,

[demime 1.01d removed an attachment of type application/pkcs7-signature which had a name of smime.p7s]

Reply | Threaded
Open this post in threaded view
|

Re: Change routing tables when ISP goes "down"

Boris Goldberg
In reply to this post by cayuga2
Hello Jeff,

Wednesday, October 1, 2014, 12:14:53 PM, you wrote:

J> It sounds like "ping -I" is what I was looking for, but when I use it, it seems
J> to be sending out the packet with the right source address, but sending it to
J> the wrong interface.....are there any tricks here?

J> Here's some data (edited) to show what I'm seeing:

J> fxp0: inet 10.16.100.1 netmask 0xfffffff0 broadcast 10.16.100.15

J> fxp1: inet 192.168.243.152 netmask 0xffffff00 broadcast 192.168.243.255

J> when I try "ping -I 192.168.243.152 ucla.edu", I see the following:

J> tcpdump -i fxp0 icmp and host ucla.edu
J> tcpdump: listening on fxp0, link-type EN10MB
J> 13:06:36.478450 192.168.243.152 > 128.97.27.37: icmp: echo request
J> 13:06:37.483393 192.168.243.152 > 128.97.27.37: icmp: echo request
J> 13:06:38.493244 192.168.243.152 > 128.97.27.37: icmp: echo request

J> The routing table shows:

J> 10.16.100.0/28     link#1             UC         4        0     -     4 fxp0
J> 192.168.243/24     link#2             UC         1        0     -     4 fpx1


  The output of "route -n get ucla.edu" would be helpful.
  It seems like you need more knowledge about routing, otherwise there is a
very big chance you "shoot yourself in the foot" messing around this. Been
there, probably still is.

--
Best regards,
 Boris                            mailto:[hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Change routing tables when ISP goes "down"

Marcus MERIGHI
In reply to this post by Giancarlo Razzolini-3
[hidden email] (Giancarlo Razzolini), 2014.10.02 (Thu) 15:39 (CEST):

> On 02-10-2014 10:11, Jeff wrote:
> > I still can't seem to force a ping through a particular interface, even when
> I
> > have both interfaces as default routes (I've tried both with and without
> mpath).
> > If it matters, in both cases I used a lower priority (higher #) for our low
> speed
> > metered connection.
> This is ok. The only thing is that a connection going out from the
> firewall machine itself will never get routed to the fxp1 interface,
> unless you force it through pf.
> >
> > Here's my current routing information:
> >
> > default            10.150.228.105     UGS        5   168287     -     8
> fxp0
> > default            192.168.243.1      UGS        0        0     -    16
> fxp1
> >
> > and "ping -I 192.168.243.152 8.8.4.4" still sends traffic out through fxp0.
> If I'm not mistaken, this should work. I'm guessing that your pf rules
> are to blame.
> >
> > I have verified that if I swap the priorities that all traffic goes out
> through
> > fxp1 so I know that that connection works.
> Good.
> >
> > It feels like I'm missing something obvious here.  Can someone point me in
> the right
> > direction?
> Try disabling pf and see if it works. If it does, then you'll need to
> change your rules to enforce the traffic to go to their respective
> gateways. Some rule like this should do:
>
> pass out on fxp0 from (fxp1) to any route-to fxp1
> pass out on fxp1 from (fxp0) to any route-to fxp0
>
> If that do not work, try including your gateways in the rule so it become:
>
> pass out on fxp0 from (fxp1) to any route-to (fxp1 fxp1_gateway)
> pass out on fxp1 from (fxp0) to any route-to (fxp0 fxp0_gateway)

quoting henning@:
(route-to and reply-to are stupid to begin with. Avoid at all cost.)
http://marc.info/?l=openbsd-misc&m=141053827907224

Bye, Marcus

Reply | Threaded
Open this post in threaded view
|

Re: Change routing tables when ISP goes "down"

Stuart Henderson
In reply to this post by cayuga2
On 2014-10-02, Jeff <[hidden email]> wrote:

> Thanks to everyone for your help/suggestions.  I think that I'm headed in the
> right direction.
>
> I still can't seem to force a ping through a particular interface, even when I
> have both interfaces as default routes (I've tried both with and without mpath).
> If it matters, in both cases I used a lower priority (higher #) for our low speed
> metered connection.
>
> Here's my current routing information:
>
> default            10.150.228.105     UGS        5   168287     -     8 fxp0
> default            192.168.243.1      UGS        0        0     -    16 fxp1
>
> and "ping -I 192.168.243.152 8.8.4.4" still sends traffic out through fxp0.

ping -I only selects the source address, not the outgoing route.

(With pf route-to rules suggested by others in the thread, that choice of
source address can *then* result in a different route being taken, but
it's not automatic).

To use your lower-priority default route, you need some way to take the
first route out of action. One possibility is to use something like
"ifconfig fxp0 down". Another is to have some kind of periodic check
that removes the "prio 8" default route.

There have been a few suggestions to use ifstated for this - that can
work - alternatives include a simple script run from cron, or relayd
has some code to handle this - check the "routers" section in relayd.conf(5).

Reply | Threaded
Open this post in threaded view
|

Re: Change routing tables when ISP goes "down"

Alucard
Or you can use a static route to force reaching the ip from an
interface.

Would be more secure than bringing down a working interface just to
check if another one is working ...

Cheers,
Louis


On 2014-10-02 17:09, Stuart Henderson wrote:

> On 2014-10-02, Jeff <[hidden email]> wrote:
>> Thanks to everyone for your help/suggestions.  I think that I'm headed
>> in the
>> right direction.
>>
>> I still can't seem to force a ping through a particular interface,
>> even when I
>> have both interfaces as default routes (I've tried both with and
>> without mpath).
>> If it matters, in both cases I used a lower priority (higher #) for
>> our low speed
>> metered connection.
>>
>> Here's my current routing information:
>>
>> default            10.150.228.105     UGS        5   168287     -    
>> 8 fxp0
>> default            192.168.243.1      UGS        0        0     -    
>> 16 fxp1
>>
>> and "ping -I 192.168.243.152 8.8.4.4" still sends traffic out through
>> fxp0.
>
> ping -I only selects the source address, not the outgoing route.
>
> (With pf route-to rules suggested by others in the thread, that choice
> of
> source address can *then* result in a different route being taken, but
> it's not automatic).
>
> To use your lower-priority default route, you need some way to take the
> first route out of action. One possibility is to use something like
> "ifconfig fxp0 down". Another is to have some kind of periodic check
> that removes the "prio 8" default route.
>
> There have been a few suggestions to use ifstated for this - that can
> work - alternatives include a simple script run from cron, or relayd
> has some code to handle this - check the "routers" section in
> relayd.conf(5).

Reply | Threaded
Open this post in threaded view
|

Re: Change routing tables when ISP goes "down"

Stuart Henderson
On 2014/10/02 17:21, [hidden email] wrote:
> Or you can use a static route to force reaching the ip from an interface.
> Would be more secure than bringing down a working interface just to check if
> another one is working ...

I didn't suggest that ;)

This would only be needed to spot the main link coming back when you're
running on the backup link, and what you would want to do is add a more-
specific static route to a certain address that you can trust to respond
reliably (ISP1's nameserver perhaps).

> >>default            10.150.228.105     UGS        5   168287     -     8
> >>fxp0
> >>default            192.168.243.1      UGS        0        0     -    16
> >>fxp1

I look after a site with very flaky connectivity, for them I have a bunch
of pppoe interfaces with default routes pointing at them with various
priorities. When pppoe(4) goes down the route is invalidated, things
seamlessly move to the next-lower priority..

Reply | Threaded
Open this post in threaded view
|

Re: Change routing tables when ISP goes "down"

cayuga2
In reply to this post by Stuart Henderson
Hi Everyone,

With the addition of a carefully constructed route-to rule I now have all of the
individual pieces working.  Now, with some careful plumbing and testing I should
be all set.  The final solution will be a combination of ifstated, multipath routing
(prioritized) and "ping -I"; thanks to everyone for your suggestions and patience!!!

Jeff

On Thu, Oct 02, 2014 at 04:09:12PM +0000, Stuart Henderson wrote:

> On 2014-10-02, Jeff <[hidden email]> wrote:
> > Thanks to everyone for your help/suggestions.  I think that I'm headed in the
> > right direction.
> >
> > I still can't seem to force a ping through a particular interface, even when I
> > have both interfaces as default routes (I've tried both with and without mpath).
> > If it matters, in both cases I used a lower priority (higher #) for our low speed
> > metered connection.
> >
> > Here's my current routing information:
> >
> > default            10.150.228.105     UGS        5   168287     -     8 fxp0
> > default            192.168.243.1      UGS        0        0     -    16 fxp1
> >
> > and "ping -I 192.168.243.152 8.8.4.4" still sends traffic out through fxp0.
>
> ping -I only selects the source address, not the outgoing route.
>
> (With pf route-to rules suggested by others in the thread, that choice of
> source address can *then* result in a different route being taken, but
> it's not automatic).
>
> To use your lower-priority default route, you need some way to take the
> first route out of action. One possibility is to use something like
> "ifconfig fxp0 down". Another is to have some kind of periodic check
> that removes the "prio 8" default route.
>
> There have been a few suggestions to use ifstated for this - that can
> work - alternatives include a simple script run from cron, or relayd
> has some code to handle this - check the "routers" section in relayd.conf(5).

Reply | Threaded
Open this post in threaded view
|

Re: Change routing tables when ISP goes "down"

Giancarlo Razzolini-3
On 02-10-2014 16:12, Jeff wrote:
> With the addition of a carefully constructed route-to rule I now have all of
the
> individual pieces working.  Now, with some careful plumbing and testing I
should
> be all set.  The final solution will be a combination of ifstated, multipath
routing
> (prioritized) and "ping -I"; thanks to everyone for your suggestions and
patience!!!
No problem. Just a few more points for you to be aware of. If your
firewall is also you dns server, the requests will go through the link
that is active. You might want to prioritize its packets to get the
minimum dns latency possible. As I mentioned before, if you can avoid
ping -I, avoid it. Most modems and routers support snmp or have another
method for determining link availability without the need for external
tests. Also, for making your life easier, use anchors for the rules that
direct traffic to the internet. This way you can easily make the
ifstated daemon to change the rules according the state of your links.
On some countries (mine included) there are some very poor quality links
that gets disconnected many times a day or show a lot of instability for
a few moments. I've developed a sqlite database and queries, so not only
I have logging, but I can also permanently disable a link if it is
unstable for the last few minutes. And only enable it again once it
remained stable for other determined amount of time. These are just a
few of the things you can do. You can send you e-mails when links go
down and get back up (of course if both go down you won't get any
notification, at all). The combination of multipath routing, with pf +
anchors and a carefully written machine state on ifstated can provide a
very good failover mechanism. I even managed firewalls with 4, 5
internet links that would failover each other and/or balance traffic
between them. Good luck with your setup.

Cheers,

[demime 1.01d removed an attachment of type application/pkcs7-signature which had a name of smime.p7s]