Multiple web servers behind NAT

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Multiple web servers behind NAT

Radek
Hi,
I have one web_serwer_1 behind OpenBSD 5.9 router/NAT with single IP.

web_serwer_1 -apache,virtualhosts- (10.0.8.11):
1.domain.com
2.domain.com
3.domain.com

pf.conf:
pass in log quick on $ext_if inet proto tcp from any to $ext_if port 80 rdr-to $web_serwer_1 port 80 set prio (1, 6) keep state
pass in log quick on $ext_if inet proto tcp from any to $ext_if port 443 rdr-to $web_serwer_1 port 443 set prio (1, 6) keep state

Everything works fine.

Now, I need to add another web_serwer_2. It would be the "main" web server.
 
web_serwer_2 - native httpd,virtualhosts- (10.0.8.22):
4.domain.com
5.domain.com
6.domain.com

How can I make it work?
Any help appreciated.

--
radek

Reply | Threaded
Open this post in threaded view
|

Re: Multiple web servers behind NAT

Josh Grosse
On Fri, Sep 30, 2016 at 11:42:11AM +0200, Radek wrote:

> Hi,
> I have one web_serwer_1 behind OpenBSD 5.9 router/NAT with single IP.
>
> web_serwer_1 -apache,virtualhosts- (10.0.8.11):
> 1.domain.com
> 2.domain.com
> 3.domain.com
>
> pf.conf:
> pass in log quick on $ext_if inet proto tcp from any to $ext_if port 80 rdr-to $web_serwer_1 port 80 set prio (1, 6) keep state
> pass in log quick on $ext_if inet proto tcp from any to $ext_if port 443 rdr-to $web_serwer_1 port 443 set prio (1, 6) keep state
>
> Everything works fine.
>
> Now, I need to add another web_serwer_2. It would be the "main" web server.
>  
> web_serwer_2 - native httpd,virtualhosts- (10.0.8.22):
> 4.domain.com
> 5.domain.com
> 6.domain.com
>
> How can I make it work?
> Any help appreciated.

If the two web servers share the same external IP address, use relayd(8),
as it is designed to inspect HTTP URLs.

Reply | Threaded
Open this post in threaded view
|

Re: Multiple web servers behind NAT

Radek
Yes, my servers share the same ext IP.
It is 5.9. I am trying to configure relayd. I commented out previous "rdr-to" rules from /etc/pf.conf and added as below.
10.0.30.101, 10.0.30.201 - it is not a mistake - ( 10.0.8.11, 10.0.8.22 was just an exemplary IP)
All websites are unreachable now.

#grep relayd /etc/pf.conf
anchor "relayd/*"

#relayd -n
configuration OK

#cat /etc/relayd.conf
ext_addr="msk0"
host1="10.0.30.101"
host2="10.0.30.201"

table <www_101> { $host1 }
table <www_201> { $host2 }

http protocol "web_one" {
   return error
   pass
   match request header "Host" value "1.domain.com" forward to <www_101>
}

http protocol "web_two" {
   return error
   pass
   match request header "Host" value "4.domain.com" forward to <www_201>
}

relay relay_one {
   listen on $ext_addr port 80
   protocol "web_one"
   forward to <www_101> check tcp port 80
}

relay relay_two {
   listen on $ext_addr port 80
   protocol "web_two"
   forward to <www_201> check tcp port 80
}

#/etc/rc.d/relayd -df restart
doing _rc_parse_conf
doing _rc_quirks
relayd_flags empty, using default ><
doing _rc_read_runfile
doing _rc_parse_conf
doing _rc_quirks
relayd_flags empty, using default ><
doing _rc_read_runfile
doing rc_check
relayd
doing rc_stop
doing _rc_wait stop
doing rc_check
doing rc_check
doing _rc_rm_runfile
(ok)
doing _rc_parse_conf
doing _rc_quirks
relayd_flags empty, using default ><
doing _rc_read_runfile
doing rc_check
relayd
doing rc_pre
configuration OK
doing rc_start
doing _rc_wait start
doing rc_check
doing _rc_write_runfile
(ok)


On Fri, 30 Sep 2016 07:26:22 -0400
Josh Grosse <[hidden email]> wrote:

> On Fri, Sep 30, 2016 at 11:42:11AM +0200, Radek wrote:
> > Hi,
> > I have one web_serwer_1 behind OpenBSD 5.9 router/NAT with single IP.
> >
> > web_serwer_1 -apache,virtualhosts- (10.0.8.11):
> > 1.domain.com
> > 2.domain.com
> > 3.domain.com
> >
> > pf.conf:
> > pass in log quick on $ext_if inet proto tcp from any to $ext_if port 80 rdr-to $web_serwer_1 port 80 set prio (1, 6) keep state
> > pass in log quick on $ext_if inet proto tcp from any to $ext_if port 443 rdr-to $web_serwer_1 port 443 set prio (1, 6) keep state
> >
> > Everything works fine.
> >
> > Now, I need to add another web_serwer_2. It would be the "main" web server.
> >  
> > web_serwer_2 - native httpd,virtualhosts- (10.0.8.22):
> > 4.domain.com
> > 5.domain.com
> > 6.domain.com
> >
> > How can I make it work?
> > Any help appreciated.
>
> If the two web servers share the same external IP address, use relayd(8),
> as it is designed to inspect HTTP URLs.
>


--
radek

Reply | Threaded
Open this post in threaded view
|

Re: Multiple web servers behind NAT

trondd-2
On Wed, October 5, 2016 8:43 am, Radek wrote:

> Yes, my servers share the same ext IP.
> It is 5.9. I am trying to configure relayd. I commented out previous
> "rdr-to" rules from /etc/pf.conf and added as below.
> 10.0.30.101, 10.0.30.201 - it is not a mistake - ( 10.0.8.11, 10.0.8.22
> was just an exemplary IP)
> All websites are unreachable now.
>
> #grep relayd /etc/pf.conf
> anchor "relayd/*"
>
> #relayd -n
> configuration OK
>
> #cat /etc/relayd.conf
> ext_addr="msk0"
> host1="10.0.30.101"
> host2="10.0.30.201"
>
> table <www_101> { $host1 }
> table <www_201> { $host2 }
>
> http protocol "web_one" {
>    return error
>    pass
>    match request header "Host" value "1.domain.com" forward to <www_101>

I think you need "pass request header..."

> }
>
> http protocol "web_two" {
>    return error
>    pass
>    match request header "Host" value "4.domain.com" forward to <www_201>
> }

You should combine the two protocols into one.  You can have multiple pass
lines.  Last match wins, unless you use "quick".  You can define a default
that way.

>
> relay relay_one {
>    listen on $ext_addr port 80
>    protocol "web_one"
>    forward to <www_101> check tcp port 80
> }
>
> relay relay_two {
>    listen on $ext_addr port 80
>    protocol "web_two"
>    forward to <www_201> check tcp port 80
> }

You should have only one relay defined, you can't have two things
listening on the same port.  Just put the two "forward to" lines in the
same relay block.


>
> #/etc/rc.d/relayd -df restart
> doing _rc_parse_conf
> doing _rc_quirks
> relayd_flags empty, using default ><
> doing _rc_read_runfile
> doing _rc_parse_conf
> doing _rc_quirks
> relayd_flags empty, using default ><
> doing _rc_read_runfile
> doing rc_check
> relayd
> doing rc_stop
> doing _rc_wait stop
> doing rc_check
> doing rc_check
> doing _rc_rm_runfile
> (ok)
> doing _rc_parse_conf
> doing _rc_quirks
> relayd_flags empty, using default ><
> doing _rc_read_runfile
> doing rc_check
> relayd
> doing rc_pre
> configuration OK
> doing rc_start
> doing _rc_wait start
> doing rc_check
> doing _rc_write_runfile
> (ok)
>

relayctl is your friend here.  See if the relays are actually up:
'relayctl show relays' and 'relayctl show summary'

Reply | Threaded
Open this post in threaded view
|

Re: Multiple web servers behind NAT

Radek
Thank you for your precise explanation.

HTTP relay seems to work fine now.

#cat /etc/relayd.conf
ext_addr="msk0"
host1="10.0.30.101"
host2="10.0.30.201"

table <www_101> { $host1 }
table <www_201> { $host2 }

http protocol "web_one" {
   return error
   pass request header "Host" value "1.domain.com" forward to <www_101>
   pass request header "Host" value "2.domain.com" forward to <www_101>
   pass request header "Host" value "3.domain.com" forward to <www_101>

   pass request header "Host" value "4.domain.com" forward to <www_201>
   pass request header "Host" value "5.domain.com" forward to <www_201>  
   pass request header "Host" value "6.domain.com" forward to <www_201>
}

relay relay_one {
   listen on $ext_addr port 80
   protocol "web_one"
   forward to <www_101> check tcp port 80
   forward to <www_201> check tcp port 80
}

#relayctl show relays
Id      Type            Name                            Avlblty Status
1       relay           relay_one                               active

#relayctl show summary
Id      Type            Name                            Avlblty Status
1       relay           relay_one                               active
1       table           www_101:80                              active (1 hosts)
1       host            10.0.30.101                     100.00% up
2       table           www_201:80                              active (1 hosts)
2       host            10.0.30.201                     100.00% up


The second thing to do is enabling wesites' SSL/TLS certs.
Each website has its own certificate on its server. I suppose that I have to configure man-in-the-middle "TLS inspecion" mode to enable TLS connection using these certs again.
Am I right?

I did the following conf:

#grep divert /etc/pf.conf
pass in on $ext_if inet proto tcp to port 443 divert-to localhost port 8443

#openssl req -x509 -days 365 -newkey rsa:2048 -keyout /etc/ssl/private/ca.key -out /etc/ssl/ca.crt
#openssl req -nodes -x509 -days 365 -newkey rsa:2048 -keyout /etc/ssl/private/127.0.0.1.key -out /etc/ssl/127.0.0.1.crt

#ls -la /etc/ssl/*.crt
-rwxr-x---  1 root  _relayd  1298 Oct 10 09:29 /etc/ssl/127.0.0.1.crt
-rwxr-x---  1 root  _relayd  1371 Oct  6 13:11 /etc/ssl/ca.crt

#ls -la /etc/ssl/private/*.key
-rwxr-x---  1 root  _relayd  1704 Oct 10 09:29 /etc/ssl/private/127.0.0.1.key
-rwxr-x---  1 root  _relayd  1858 Oct  6 13:11 /etc/ssl/private/ca.key

#cat /etc/relayd.conf
ext_addr="msk0"
host1="10.0.30.101"
host2="10.0.30.201"

table <www_101> { $host1 }
table <www_201> { $host2 }

http protocol "web_one" {
   return error
   pass request header "Host" value "1.domain.com" forward to <www_101>
   pass request header "Host" value "2.domain.com" forward to <www_101>
   pass request header "Host" value "3.domain.com" forward to <www_101>

   pass request header "Host" value "4.domain.com" forward to <www_201>
   pass request header "Host" value "5.domain.com" forward to <www_201>  
   pass request header "Host" value "6.domain.com" forward to <www_201>
}

http protocol "web_tls" {
   return error
   pass request header "Host" value "1.domain.com" forward to <www_101>
   pass request header "Host" value "2.domain.com" forward to <www_101>
   pass request header "Host" value "3.domain.com" forward to <www_101>

   pass request header "Host" value "4.domain.com" forward to <www_201>
   pass request header "Host" value "5.domain.com" forward to <www_201>  
   pass request header "Host" value "6.domain.com" forward to <www_201>
   tls tlsv1
   tls ca key "/etc/ssl/private/ca.key" password "somepasshere"
   tls ca cert "/etc/ssl/ca.crt"
}
 
relay relay_one {
   listen on $ext_addr port 80
   protocol "web_one"
   forward to <www_101> check tcp port 80
   forward to <www_201> check tcp port 80
}

relay relay_tls {
   listen on 127.0.0.1 port 8443 tls
   protocol "web_tls"
   forward with tls to <www_101> check tcp port 443
   forward with tls to <www_201> check tcp port 443
}


#relayctl show relays
Id      Type            Name                            Avlblty Status
1       relay           relay_one                               active
2       relay           relay_tls                               active

#relayctl show summary
Id      Type            Name                            Avlblty Status
1       relay           relay_one                               active
1       table           www_101:80                              active (1 hosts)
1       host            10.0.30.101                     100.00% up
2       table           www_201:80                              active (1 hosts)
2       host            10.0.30.201                     100.00% up
2       relay           relay_tls                               active
3       table           www_101:443                             active (1 hosts)
3       host            10.0.30.101                     100.00% up
4       table           www_201:443                             active (1 hosts)
4       host            10.0.30.201                     100.00% up

Websites (https://4.domain, https://5.domain, https://6.domain) started to show the content of 1.domain.com

If I changed the order of "forward" websites (https://1.domain, https://2.domain, https://3.domain) started to show content of 4.domain.com

relay relay_tls {
   listen on 127.0.0.1 port 8443 tls
   protocol "web_tls"
   forward with tls to <www_201> check tcp port 443
   forward with tls to <www_101> check tcp port 443
}

All domains use relay_machine's certificate instead of the specific domain's cert.

What am I doing wrong?

On Wed, 5 Oct 2016 09:57:49 -0400

"trondd" <[hidden email]> wrote:

> On Wed, October 5, 2016 8:43 am, Radek wrote:
> > Yes, my servers share the same ext IP.
> > It is 5.9. I am trying to configure relayd. I commented out previous
> > "rdr-to" rules from /etc/pf.conf and added as below.
> > 10.0.30.101, 10.0.30.201 - it is not a mistake - ( 10.0.8.11, 10.0.8.22
> > was just an exemplary IP)
> > All websites are unreachable now.
> >
> > #grep relayd /etc/pf.conf
> > anchor "relayd/*"
> >
> > #relayd -n
> > configuration OK
> >
> > #cat /etc/relayd.conf
> > ext_addr="msk0"
> > host1="10.0.30.101"
> > host2="10.0.30.201"
> >
> > table <www_101> { $host1 }
> > table <www_201> { $host2 }
> >
> > http protocol "web_one" {
> >    return error
> >    pass
> >    match request header "Host" value "1.domain.com" forward to <www_101>
>
> I think you need "pass request header..."
>
> > }
> >
> > http protocol "web_two" {
> >    return error
> >    pass
> >    match request header "Host" value "4.domain.com" forward to <www_201>
> > }
>
> You should combine the two protocols into one.  You can have multiple pass
> lines.  Last match wins, unless you use "quick".  You can define a default
> that way.
>
> >
> > relay relay_one {
> >    listen on $ext_addr port 80
> >    protocol "web_one"
> >    forward to <www_101> check tcp port 80
> > }
> >
> > relay relay_two {
> >    listen on $ext_addr port 80
> >    protocol "web_two"
> >    forward to <www_201> check tcp port 80
> > }
>
> You should have only one relay defined, you can't have two things
> listening on the same port.  Just put the two "forward to" lines in the
> same relay block.
>
>
> >
> > #/etc/rc.d/relayd -df restart
> > doing _rc_parse_conf
> > doing _rc_quirks
> > relayd_flags empty, using default ><
> > doing _rc_read_runfile
> > doing _rc_parse_conf
> > doing _rc_quirks
> > relayd_flags empty, using default ><
> > doing _rc_read_runfile
> > doing rc_check
> > relayd
> > doing rc_stop
> > doing _rc_wait stop
> > doing rc_check
> > doing rc_check
> > doing _rc_rm_runfile
> > (ok)
> > doing _rc_parse_conf
> > doing _rc_quirks
> > relayd_flags empty, using default ><
> > doing _rc_read_runfile
> > doing rc_check
> > relayd
> > doing rc_pre
> > configuration OK
> > doing rc_start
> > doing _rc_wait start
> > doing rc_check
> > doing _rc_write_runfile
> > (ok)
> >
>
> relayctl is your friend here.  See if the relays are actually up:
> 'relayctl show relays' and 'relayctl show summary'
>
>


--
radek

Reply | Threaded
Open this post in threaded view
|

Re: Multiple web servers behind NAT

Markus Hennecke
Am 10.10.2016 um 12:01 schrieb Radek:

> The second thing to do is enabling wesites' SSL/TLS certs.
> Each website has its own certificate on its server. I suppose that I have to configure man-in-the-middle "TLS inspecion" mode to enable TLS connection using these certs again.
> Am I right?
>
You can't do that. TLS exchange is done before the host name is send in
the request.
The only thing you can do is to use one certificate for all hosts and
terminate the TLS connection in relayd.

Regards
Markus

Reply | Threaded
Open this post in threaded view
|

Re: Multiple web servers behind NAT

trondd-2
In reply to this post by Radek
On Mon, October 10, 2016 6:01 am, Radek wrote:
>
> The second thing to do is enabling wesites' SSL/TLS certs.
> Each website has its own certificate on its server. I suppose that I have
> to configure man-in-the-middle "TLS inspecion" mode to enable TLS
> connection using these certs again.
> Am I right?
>

No.  TLS inspection doesn't work that way.  It's for LAN systems
connecting out through the relayd server to sites on the internet.  It
doesn't work in the other direction.  You would have needed to use
'forward to destination' in place of 'forward to <www_101>' but that
original destination will be the relayd machine again as it's IP based,
not domain name based.

You need one certificate that matches all of your web site hostnames and
configure relayd as a TLS server as you had it.

Tim.

> I did the following conf:
>
> #grep divert /etc/pf.conf
> pass in on $ext_if inet proto tcp to port 443 divert-to localhost port
> 8443
>
> #openssl req -x509 -days 365 -newkey rsa:2048 -keyout
> /etc/ssl/private/ca.key -out /etc/ssl/ca.crt
> #openssl req -nodes -x509 -days 365 -newkey rsa:2048 -keyout
> /etc/ssl/private/127.0.0.1.key -out /etc/ssl/127.0.0.1.crt
>
> #ls -la /etc/ssl/*.crt
> -rwxr-x---  1 root  _relayd  1298 Oct 10 09:29 /etc/ssl/127.0.0.1.crt
> -rwxr-x---  1 root  _relayd  1371 Oct  6 13:11 /etc/ssl/ca.crt
>
> #ls -la /etc/ssl/private/*.key
> -rwxr-x---  1 root  _relayd  1704 Oct 10 09:29
> /etc/ssl/private/127.0.0.1.key
> -rwxr-x---  1 root  _relayd  1858 Oct  6 13:11 /etc/ssl/private/ca.key
>
> #cat /etc/relayd.conf
> ext_addr="msk0"
> host1="10.0.30.101"
> host2="10.0.30.201"
>
> table <www_101> { $host1 }
> table <www_201> { $host2 }
>
> http protocol "web_one" {
>    return error
>    pass request header "Host" value "1.domain.com" forward to <www_101>
>    pass request header "Host" value "2.domain.com" forward to <www_101>
>    pass request header "Host" value "3.domain.com" forward to <www_101>
>
>    pass request header "Host" value "4.domain.com" forward to <www_201>
>    pass request header "Host" value "5.domain.com" forward to <www_201>
>    pass request header "Host" value "6.domain.com" forward to <www_201>
> }
>
> http protocol "web_tls" {
>    return error
>    pass request header "Host" value "1.domain.com" forward to <www_101>
>    pass request header "Host" value "2.domain.com" forward to <www_101>
>    pass request header "Host" value "3.domain.com" forward to <www_101>
>
>    pass request header "Host" value "4.domain.com" forward to <www_201>
>    pass request header "Host" value "5.domain.com" forward to <www_201>
>    pass request header "Host" value "6.domain.com" forward to <www_201>
>    tls tlsv1
>    tls ca key "/etc/ssl/private/ca.key" password "somepasshere"
>    tls ca cert "/etc/ssl/ca.crt"
> }
>
> relay relay_one {
>    listen on $ext_addr port 80
>    protocol "web_one"
>    forward to <www_101> check tcp port 80
>    forward to <www_201> check tcp port 80
> }
>
> relay relay_tls {
>    listen on 127.0.0.1 port 8443 tls
>    protocol "web_tls"
>    forward with tls to <www_101> check tcp port 443
>    forward with tls to <www_201> check tcp port 443
> }
>
>
> #relayctl show relays
> Id      Type            Name                            Avlblty Status
> 1       relay           relay_one                               active
> 2       relay           relay_tls                               active
>
> #relayctl show summary
> Id      Type            Name                            Avlblty Status
> 1       relay           relay_one                               active
> 1       table           www_101:80                              active (1
> hosts)
> 1       host            10.0.30.101                     100.00% up
> 2       table           www_201:80                              active (1
> hosts)
> 2       host            10.0.30.201                     100.00% up
> 2       relay           relay_tls                               active
> 3       table           www_101:443                             active (1
> hosts)
> 3       host            10.0.30.101                     100.00% up
> 4       table           www_201:443                             active (1
> hosts)
> 4       host            10.0.30.201                     100.00% up
>
> Websites (https://4.domain, https://5.domain, https://6.domain) started to
> show the content of 1.domain.com
>
> If I changed the order of "forward" websites (https://1.domain,
> https://2.domain, https://3.domain) started to show content of
> 4.domain.com
>
> relay relay_tls {
>    listen on 127.0.0.1 port 8443 tls
>    protocol "web_tls"
>    forward with tls to <www_201> check tcp port 443
>    forward with tls to <www_101> check tcp port 443
> }
>
> All domains use relay_machine's certificate instead of the specific
> domain's cert.
>
> What am I doing wrong?

Reply | Threaded
Open this post in threaded view
|

Re: Multiple web servers behind NAT

Florian Ermisch
In reply to this post by Markus Hennecke
Am 10. Oktober 2016 14:35:00 MESZ, schrieb Markus Hennecke <[hidden email]>:

> Am 10.10.2016 um 12:01 schrieb Radek:
>
> > The second thing to do is enabling
> > wesites' SSL/TLS certs.
> > Each website has its own certificate
> > on its server. […]
> >
> You can't do that. TLS exchange is done
> before the host name is send in  the
> request.
> The only thing you can do is to use one
> certificate for all hosts and terminate
> the TLS connection in relayd.
>
So relayd doesn't support SNI yet?
Not that SNI and having a cert for each
site on the relay covers the usecase but
httpd does support SNI, right?

Regards, Florian

Reply | Threaded
Open this post in threaded view
|

Re: Multiple web servers behind NAT

trondd-2
On Wed, October 12, 2016 1:38 am, Florian Ermisch wrote:
>
> So relayd doesn't support SNI yet?
> Not that SNI and having a cert for each
> site on the relay covers the usecase but
> httpd does support SNI, right?
>
> Regards, Florian
>

I think you are correct.  I think SNI was added to libtls and httpd around
mid August.  No one implemented it in relayd.  Though it's possible that I
just missed it.

Tim.