isakmpd question

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

isakmpd question

Neil Joseph Schelly
I'm having a problem with an IPSec tunnel I have configured connecting two
networks together.  Each firewall is running OpenBSD 3.9.  At one end, it's a
pair of firewalls running CARP and I've turned off sasyncd to troubleshoot
now, because I didn't want to have it interfering and I suspect it may have
been causing more problems.  Since the primary firewall is staying up without
issues, I'm ignoring the backup in my examples.

Essentially, the behavior I'm seeing is that communication over the tunnel is
interrupted whenever the Phase 1 SA is timed out.  When it hits the soft
timeout, a new SA is negotiated and looks fine.  As soon as the older Phase 1
SA times out, communication (even just pinging) is interrupted for a minute
or less.  To confirm that the behavior is related to the timeouts, I've
doubled all my timeout times in isakmpd.conf to 7200s for Phase 1 and 2400
seconds for Phase 2.  The outages happen roughly half as often now and still
correspond in timing to new Phase 1 SA establishments and changeover.

I have pf configured on both ends, with altq.  Altq isn't dropping any port
500 isakmpd packets (according to pfctl -vvs queue) on either side and only
occasional esp traffic under high loads.  Both have enough bandwidth reserved
and they're given the highest priority in CBQ mode.  pf is allowing isakmpd
traffic only from the other of our two locations at both sides.

I don't suspect that pf or altq is the problem here just because the SAs do
get recreated without any obvious problem and traffic is "allowed" at least
to proceed through the packet filter normally.  However, the problem is
exacerbated by higher throughput times during the business day - it usually
goes unnoticed (by Nagios) on a weekend or overnight, so altq could be a
factor if I need to reserve some bandwidth for more than port 500 and esp
traffic.

I've been watching SAs with the following procedure from isakmpd's man page:
# echo S>/var/run/isakmpd.fifo
# cat /var/run/isakmpd.result

The flows/routes as reported by ipsecctl -vvs all and netstat -rnf encap don't
appear to be interrupted ever - they are always present unless I clear the SA
table manually.  pflog is logging all dropped packets to /var/log/pflog and I
never find any esp or port 500 packets in there except obviously from servers
outside our networks.

Does anyone have any suggestions for points to investigate?   I can provide
configuration details about parts of this if anyone has a good place to look.  
I've already manually configured tunnels with isakmpd.conf (rather than
ipsec.conf) in hopes that something would show up in that process, but the
same behavior is noticed both ways.

--
Regards,
Neil Schelly
Senior Systems Administrator

W: 978-667-5115 x213
M: 508-410-4776

OASIS Open http://www.oasis-open.org
"Advancing E-Business Standards Since 1993"

Reply | Threaded
Open this post in threaded view
|

Re: isakmpd question

Jacob Yocom-Piatt
Neil Joseph Schelly wrote:
> Does anyone have any suggestions for points to investigate?   I can provide
> configuration details about parts of this if anyone has a good place to look.  
> I've already manually configured tunnels with isakmpd.conf (rather than
> ipsec.conf) in hopes that something would show up in that process, but the
> same behavior is noticed both ways.
>
>  

have you tried following this ipsecctl "howto"

http://www.securityfocus.com/infocus/1859

there are tons of things you could have wrong when not using ipsecctl.
you didn't post any of the relevant config files or debugging
information, so how do you expect anyone to help?

cheers,
jake

Reply | Threaded
Open this post in threaded view
|

Re: isakmpd question

Neil Joseph Schelly
On Thursday 11 January 2007 12:46 pm, Jacob Yocom-Piatt wrote:
> have you tried following this ipsecctl "howto"

Yes

> there are tons of things you could have wrong when not using ipsecctl.
> you didn't post any of the relevant config files or debugging
> information, so how do you expect anyone to help?

I was unclear in my original post.  These were running before with ipsec.conf,
as follows (with similar entries on the other end's firewall of course).

ike passive esp from 10.20.20.0/22 to 10.21.20.0/22 peer x.x.x.x

I've rebuilt them the long way in isakmpd.conf, but ultimately, they work just
as well either way.  I still have these occasional interruptions during SA
timeouts.  I've actually noticed today for the first time a Phase 2 SA
timeout caused a similar interruption, even though a new SA had already been
negotiated, so perhaps my initial observations are off still.

Anyway, I didn't submit debugging or config files before because attaching
every config file involved here would be overhwelming.  I'm hoping I can get
some direction to look for, more along the lines of generic isakmpd
troubleshooting.

I've been trying to make pf, altq, isakmpd, ipsec.conf, etc adjustments as
atomically as possible to see if I can at least affect the problem and get a
hint at where to look more closely.  The best I've got so far is that altq
may be related because it's under high loads in general that the connections
have more problems.  And isakmpd may be related because doubling the SA
timeouts makes it more reliable, in the sense that the behavior comes up half
as often.

#########################################
Here's the datacenter (dc0) side of my isakmpd.conf for example:

[General]
Listen-On = X.X.X.X (CARP)
Default-phase-1-lifetime = 7200,60:86400
Default-phase-2-lifetime = 2400,60:86400

[Phase 1]
X.X.X.X = ma0fw

[Phase 2]
Connections = dc0network-ma0network, dc0savvis-ma0network

[ma0fw]
Phase = 1
Transport = udp
Address = X.X.X.X
Configuration = Default-main-mode

[dc0network-ma0network]
Phase = 2
ISAKMP-peer = ma0fw
Configuration = Default-quick-mode
Local-ID = dc0network
Remote-ID = ma0network

[dc0savvis-ma0network]
Phase = 2
ISAKMP-peer = ma0fw
Configuration = Default-quick-mode
Local-ID = dc0savvis
Remote-ID = ma0network

[dc0network]
ID-type = IPV4_ADDR_SUBNET
Network = 10.20.20.0
Netmask = 255.255.252.0

[dc0savvis]
ID-type = IPV4_ADDR_SUBNET
Network = 10.1.1.0
Netmask = 255.255.255.0

[ma0network]
ID-type = IPV4_ADDR_SUBNET
Network = 10.21.20.0
Netmask = 255.255.252.0

[Default-main-mode]
EXCHANGE_TYPE = ID_PROT
Transforms = 3DES-SHA-GRP2-RSA_SIG

[Default-quick-mode]
EXCHANGE_TYPE = QUICK_MODE
Suites = QM-ESP-3DES-SHA-SUITE
#########################################

#########################################
Here's my datacenter side pf.conf, as applies to altq/IPSec
altq on fxp1 cbq bandwidth 6Mb queue { standard, admin, vpncontrol, carp }
queue standard bandwidth 82% { mail, std }
  queue mail bandwidth 25% priority 2 cbq(borrow)
  queue std bandwidth 75% priority 6 cbq(borrow, default)
queue admin bandwidth 10% { ssh, vpn }
  queue ssh bandwidth 20% { ssh_interactive, ssh_bulk }
    queue ssh_interactive bandwidth 25% priority 4 cbq(ecn, borrow)
    queue ssh_bulk bandwidth 75% cbq(ecn, borrow)
  queue vpn bandwidth 80% priority 6 cbq(borrow)
queue vpncontrol bandwidth 4% priority 7 cbq(borrow)
queue carp bandwidth 4% priority 7 cbq(borrow)

# Allow isakmpd control traffic between <isakmp_peers>
pass in quick on $ext_if proto udp from <isakmp_peers> to $extcarp_if:0 port
isakmp queue vpncontrol
pass out quick on $ext_if proto udp from $extcarp_if:0 to <isakmp_peers> port
isakmp queue vpncontrol

# Allow all isakmpd tunneled traffic (encoded with esp)
pass in quick on $ext_if proto esp from <isakmp_peers> to $extcarp_if:0 queue
vpn
pass out quick on $ext_if proto esp from $extcarp_if:0 to <isakmp_peers> queue
vpn
#########################################


#########################################
Here is the excerpts from /var/run/isakmpd.result on the office side firewal
during a Phase 2 SA timeout period.

SA name: dc0fw (Phase 1/Initiator)
src: MA0.X.X.X dst: DC0.X.X.X
Lifetime: 7200 seconds
Soft timeout in 4086 seconds
Hard timeout in 4468 seconds
icookie b83f99790cccd43a rcookie 0a4d6741d97c0d96

SA name: dc0savvis-ma0network (Phase 2)
src: MA0.X.X.X dst: DC0.X.X.X
Lifetime: 2400 seconds
Hard timeout in 41 seconds
SPI 0: 985404c3
SPI 1: 257a3144
Transform: IPsec ESP
Encryption key length: 24
Authentication key length: 20
Encryption algorithm: 3DES
Authentication algorithm: HMAC-SHA1

SA name: dc0network-ma0network (Phase 2)
src: MA0.X.X.X dst: DC0.X.X.X
Lifetime: 2400 seconds
Soft timeout in 149 seconds
Hard timeout in 296 seconds
SPI 0: 67f24a6f
SPI 1: e3f4896b
Transform: IPsec ESP
Encryption key length: 24
Authentication key length: 20
Encryption algorithm: 3DES
Authentication algorithm: HMAC-SHA1

SA name: dc0savvis-ma0network (Phase 2)
src: MA0.X.X.X dst: DC0.X.X.X
Lifetime: 2400 seconds
Soft timeout in 1852 seconds
Hard timeout in 2100 seconds
SPI 0: 3b9b6cc3
SPI 1: e4afeff8
Transform: IPsec ESP
Encryption key length: 24
Authentication key length: 20
Encryption algorithm: 3DES
Authentication algorithm: HMAC-SHA1
#########################################


#########################################
Here is the excerpts from /var/run/isakmpd.result on the office side firewal
during a Phase 2 SA timeout period.


SA name: ma0fw (Phase 1/Responder)
src: DC0.X.X.X dst: MA0.X.X.X
Lifetime: 7200 seconds
Soft timeout in 3629 seconds
Hard timeout in 4486 seconds
icookie b83f99790cccd43a rcookie 0a4d6741d97c0d96

SA name: dc0savvis-ma0network (Phase 2)
src: DC0.X.X.X dst: MA0.X.X.X
Lifetime: 2400 seconds
Hard timeout in 58 seconds
SPI 0: 257a3144
SPI 1: 985404c3
Transform: IPsec ESP
Encryption key length: 24
Authentication key length: 20
Encryption algorithm: 3DES
Authentication algorithm: HMAC-SHA1

SA name: dc0network-ma0network (Phase 2)
src: DC0.X.X.X dst: MA0.X.X.X
Lifetime: 2400 seconds
Soft timeout in 130 seconds
Hard timeout in 313 seconds
SPI 0: e3f4896b
SPI 1: 67f24a6f
Transform: IPsec ESP
Encryption key length: 24
Authentication key length: 20
Encryption algorithm: 3DES
Authentication algorithm: HMAC-SHA1

SA name: dc0savvis-ma0network (Phase 2)
src: DC0.X.X.X dst: MA0.X.X.X
Lifetime: 2400 seconds
Soft timeout in 1956 seconds
Hard timeout in 2117 seconds
SPI 0: e4afeff8
SPI 1: 3b9b6cc3
Transform: IPsec ESP
Encryption key length: 24
Authentication key length: 20
Encryption algorithm: 3DES
Authentication algorithm: HMAC-SHA1
#########################################

Again, I can provide more details if necessary - I've just run out of places
to look, so I'm really not sure what's useful or not.

--
Regards,
Neil Schelly
Senior Systems Administrator

W: 978-667-5115 x213
M: 508-410-4776

OASIS Open http://www.oasis-open.org
"Advancing E-Business Standards Since 1993"