Forward IP headers in HAProxy to get the real IP of the client

Help!
1

TL;DR – option forwardfor and http-request set-header X-Real-IP %[src] are not working.

My setup is slightly complicated. I have a homeserver, with HAProxy installed and some docker containers. My homeserver is, then, connected to a VPS via WireGuard which also has HAProxy installed. HAProxy on homeserver forwards the docker containers with an SSL certificate to the VPS. The VPS, then, just does TLS pass through to the clients.

The issue is, if I do not use option forwardfor in either of the 2 HAProxy configurations, I get the internal IP address of the docker container (172.XX.XX.1). If I add option forwardfor on the homeserver’s HAProxy config, I get the internal IP of the WireGuard of the home server (10.0.0.2). And if I add option forwardfor to the HAProxy config of the VPS as well, I get the internal IP of the WireGuard tunnel (10.0.0.1). And as far as I know, http-request set-header X-Real-IP %[src] has no impact. I have also tried using send-proxy and send-proxy-v2, but then the whole setup stops working.

HAProxy config on home server:

global
        log /dev/log    local0
        log /dev/log    local1 notice
        chroot /var/lib/haproxy
        stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd listeners
        stats timeout 30s
        user haproxy
        group haproxy
        daemon

        # Default SSL material locations
        ca-base /etc/ssl/certs
        crt-base /etc/ssl/private

        # See: https://ssl-config.mozilla.org/#server=haproxy&server-version=2.0.3&config=intermediate
        ssl-default-bind-ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20>
        ssl-default-bind-ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256
        ssl-default-bind-options ssl-min-ver TLSv1.2 no-tls-tickets

defaults
        log     global
        mode    http
        option  httplog
        option  dontlognull
        timeout connect 5000
        timeout client  50000
        timeout server  50000
        errorfile 400 /etc/haproxy/errors/400.http
        errorfile 403 /etc/haproxy/errors/403.http
        errorfile 408 /etc/haproxy/errors/408.http
        errorfile 500 /etc/haproxy/errors/500.http
        errorfile 502 /etc/haproxy/errors/502.http
        errorfile 503 /etc/haproxy/errors/503.http
        errorfile 504 /etc/haproxy/errors/504.http

listen rp
        bind *:443 ssl crt /path/to/cert.pem

        acl service1 hdr_sub(host) -i service1.domain.me
        acl service2 hdr_sub(host) -i service2.domain.me

        use_backend service1_backend if service1
        use_backend service2_backend if service2

backend service1_backend
        server service1_server 127.0.0.1:8080

backend service2_backend
#       option forwardfor
#       http-request set-header X-Real-IP %[src]
        server service2_server 127.0.0.1:9090

HAProxy config on VPS:

global
        log /dev/log    local0
        log /dev/log    local1 notice
        chroot /var/lib/haproxy
        stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd listeners
        stats timeout 30s
        user haproxy
        group haproxy
        daemon
        tune.ssl.default-dh-param       4096

defaults
        log     global
        mode    tcp
#       option  forwardfor
        timeout connect 5000
        timeout client  50000
        timeout server  50000

listen http
        bind *:80
        mode tcp
        server default 10.0.0.2:80
listen https
        bind *:443 alpn h2,http/1.1
        mode tcp
#       option forwardfor header X-Real-IP
#       http-request set-header X-Real-IP %[src]
        server main 10.0.0.2:443

I have to resort to this because I am behind CGNAT, and want TLS pass through on the VPS for privacy.

What am I doing wrong?

HA-Proxy version 2.2.9-2+deb11u5 2023/04/10

2

You can’t use anything HTTP related on your VPS haproxy instance, because you don’t have access to HTTP header: HTTP headers and content are encrypted, your are running a passthrough TLS configuration.

So between the first and the second layer you need to use the proxy protocol.

This means:

VPS haprox:

listen https
        bind *:443
        mode tcp
        server main 10.0.0.2:443 send-proxy

I also removed the alpn configuration there. It will just confuse you into thinking that it does something, when it doesn’t. There is no ALPN to negotiate, if there is no SSL/TLS negotiation.

In the second haproxy layer, use accept-proxy.

listen rp
        bind *:443 ssl crt /path/to/cert.pem accept-proxy

Then your second proxy layer actually has the public Client IP and then you can use it with something like forwardfor or X-Real-IP, if your backends trust it.

This makes port 443 unusable locally. So if you also want to access HTTPS locally, I’d suggest adding another port, 443 for your normal local clients and 444 or something for your proxy protocol port (of course the VPS configuration would have to point to the new port).

3

Hi. Thank you for your reply. I do understand it now. I did try send-proxy, but I did not know that I have to use accept-proxy on the home server as well.

However, I resorted to another solution, which is passing through the data using iptables on the VPS. I am still using HAProxy on the home server.

Marking your reply as the solution. Thanks again.