relayd forwarding order

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

relayd forwarding order

Nick Guenther-2
>Synopsis: relayd depends on the order of the 'forward' lines even when they should be unambiguous
>Category: system
>Environment:
        System      : OpenBSD 6.6
        Details     : OpenBSD 6.6 (GENERIC.MP) #372: Sat Oct 12 10:56:27 MDT 2019
                         [hidden email]:/usr/src/sys/arch/amd64/compile/GENERIC.MP

        Architecture: OpenBSD.amd64
        Machine     : amd64

>Description:

        Most options in relayd.conf are order insensitive (except for filter rules). But
        when using `forward to <table>`, the order matters, even when there's only a single
        possible forward for each table.

        (
        originally reported at
         * https://www.reddit.com/r/openbsd/comments/eh6ll8/relayd_as_a_reverse_proxy_help_due_to_uncertainty/
         * https://www.reddit.com/r/openbsd/comments/3qb2c4/some_observations_about_relayd/
        )

>How-To-Repeat:

        Set up two backend http servers:

        ```
        $ mkdir relayd-order-sensitive; cd relayd-order-sensitive
        $ mkdir -p site app
        $ mkdir -p logs # httpd insists on this
        $ echo "Static Site" > site/index.html
        $ echo "WebApp" > app/index.html
        $
        $ cat > httpd.conf <<EOF
        chroot "."

        server "site" {
                listen on localhost port 8080
                root "site"
                directory auto index
        }

        server "app" {
                listen on localhost port 8082
                root "app"
                request strip 1
                directory auto index
        }
        EOF
        $ doas httpd -f httpd.conf
        $
        $ # test:
        $ curl http://localhost:8080/     
        Static Site
        $ curl http://localhost:8082/app/
        WebApp
        ```

        Now, with this relayd.conf:

        ```
        table <web> { "127.0.0.1" }
        table <app> { "127.0.0.1" }

        http protocol web {
                match request path "/app/*" forward to <app>
        }

        relay P {
                listen on 0.0.0.0 port 80

                protocol web

                forward to <web> port 8080
                forward to <app> port 8082
        }
        ```

        The two backends successfully run on the same web origin:

        ```
        $ curl http://localhost/   
        Static Site
        $ curl http://localhost/app/
        WebApp
        ```

        And the httpd logs say / routed to "site" (i.e. :8080) and /app routed to "app" (:8082).

        ```
        $ tail -n 2 logs/access.log
        site 127.0.0.1 - - [11/Feb/2020:02:16:22 -0500] "GET / HTTP/1.1" 200 12
        app 127.0.0.1 - - [11/Feb/2020:02:16:24 -0500] "GET /app/ HTTP/1.1" 200 7
        ```

        But switch around those forward lines:

        ```
        table <web> { "127.0.0.1" }
        table <app> { "127.0.0.1" }

        http protocol web {
                match request path "/app/*" forward to <app>
        }

        relay P {
                listen on 0.0.0.0 port 80

                protocol web

                forward to <app> port 8082
                forward to <web> port 8080
        }
        ```

        /app still works

        ```
        $ curl http://localhost/app/
        WebApp
        ```

        but / fails

        ```
        $ curl -v http://localhost/ 
        *   Trying 127.0.0.1:80...
        * TCP_NODELAY set
        * Connected to localhost (127.0.0.1) port 80 (#0)
        > GET / HTTP/1.1
        > Host: localhost
        > User-Agent: curl/7.66.0
        > Accept: */*
        >
        * Mark bundle as not supporting multiuse
        * HTTP 1.0, assume close after body
        < HTTP/1.0 301 Moved Permanently
        < Connection: close
        < Content-Length: 443
        < Content-Type: text/html
        < Date: Tue, 11 Feb 2020 07:18:27 GMT
        < Location: http://localhost//
        < Server: OpenBSD httpd
        <
        <!DOCTYPE html>
        <html>
        <head>
        <meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
        <title>301 Moved Permanently</title>
        <style type="text/css"><!--
        body { background-color: white; color: black; font-family: 'Comic Sans MS', 'Chalkboard SE', 'Comic Neue', sans-serif; }
        hr { border: 0; border-bottom: 1px dashed; }

        --></style>
        </head>
        <body>
        <h1>301 Moved Permanently</h1>
        <hr>
        <address>OpenBSD httpd</address>
        </body>
        </html>
        * Closing connection 0
        ```

        looking in logs/access.log we see why: both are now routing to :8082.

        ```
        $ tail -n 2 logs/access.log
        app 127.0.0.1 - - [11/Feb/2020:02:18:01 -0500] "GET /app/ HTTP/1.1" 200 7
        app 127.0.0.1 - - [11/Feb/2020:02:18:27 -0500] "GET / HTTP/1.1" 301 0
        ```

        I can't understand why though. It seems like the rules are unambiguous.

Reply | Threaded
Open this post in threaded view
|

Re: relayd forwarding order

Sebastian Benoit-3
Nick([hidden email]) on 2020.02.11 07:29:44 +0000:

> >Synopsis: relayd depends on the order of the 'forward' lines even when they should be unambiguous
> >Category: system
> >Environment:
> System      : OpenBSD 6.6
> Details     : OpenBSD 6.6 (GENERIC.MP) #372: Sat Oct 12 10:56:27 MDT 2019
> [hidden email]:/usr/src/sys/arch/amd64/compile/GENERIC.MP
>
> Architecture: OpenBSD.amd64
> Machine     : amd64
>
> >Description:
>
> Most options in relayd.conf are order insensitive (except for filter rules). But
> when using `forward to <table>`, the order matters, even when there's only a single
> possible forward for each table.
>
> (
> originally reported at
> * https://www.reddit.com/r/openbsd/comments/eh6ll8/relayd_as_a_reverse_proxy_help_due_to_uncertainty/
> * https://www.reddit.com/r/openbsd/comments/3qb2c4/some_observations_about_relayd/
> )
>
> >How-To-Repeat:
...
> I can't understand why though. It seems like the rules are unambiguous.

Its a feature:

     forward to <table> [port number] options ...
             Specify the tables of target hosts to be used; see the TABLES
             section above for information about table options.  If the port
             option is not specified, the first port from the listen on
             directive will be used.  *This directive can be specified twice
             the second entry will be used as the backup table if all hosts in
             the main table are down.*  At least one entry for the main table
             is mandatory.

I.e. its just the way the configuration laguage works to specify backup
destinations.

Reply | Threaded
Open this post in threaded view
|

Re: relayd forwarding order

Nick Guenther-2
Thanks for taking a look. relayd is a complicated but still elegant solution to some messy parts of the web and I appreciate that its author took the time to look in on this.

On February 11, 2020 6:15:54 PM EST, Sebastian Benoit <[hidden email]> wrote:

>Nick([hidden email]) on 2020.02.11 07:29:44 +0000:
>...
>> I can't understand why though. It seems like the rules are
>unambiguous.
>
>Its a feature:
>
>...
>
>I.e. its just the way the configuration laguage works to specify backup
>destinations.

In this case they aren't backup tables though, they are separate destinations with different meanings. Things to /app should always go to one server, everything else to another.
--
Nick