TIEB62
October 1, 2022, 8:35pm
1
Hi,
To give the context, I have an HAProxy instance behind multiple Cloudflare domains (with ACLs etc…)
I want the final servers to have the Real Client IP
Without anything the final server get HAProxy IP
However, Cloudflare pass a “CF-Connecting-IP” header with the real client IP
Thanks in advance,
If you are using 2.4 or older, use the workaround described here:
opened 05:12PM - 02 May 19 UTC
closed 04:00PM - 05 Nov 21 UTC
type: bug
1.6
1.7
1.8
severity: medium
subsystem: http
status: fixed
2.0
2.2
2.3
2.4
Michael Brown (@supermathie) reported this, further researched it and provided a… workaround on discourse:
https://discourse.haproxy.org/t/cloudflare-haproxy-is-using-the-wrong-ip-for-http-requests-after-the-first/3769
## Output of `haproxy -vv` and `uname -a`
```
lukas@dev:~/haproxy$ ./haproxy -vv
HA-Proxy version 2.0-dev2-a48237-261 2019/05/02 - https://haproxy.org/
Build options :
TARGET = linux2628
CPU = generic
CC = gcc
CFLAGS = -O2 -g -fno-strict-aliasing -Wdeclaration-after-statement -fwrapv -Wno-unused-label -Wno-sign-compare -Wno-unused-parameter -Wno-old-style-declaration -Wno-ignored-qualifiers -Wno-clobbered -Wno-missing-field-initializers -Wtype-limits
OPTIONS = USE_GETADDRINFO=1
Feature list : +EPOLL -KQUEUE -MY_EPOLL -MY_SPLICE +NETFILTER -PCRE -PCRE_JIT -PCRE2 -PCRE2_JIT +POLL -PRIVATE_CACHE +THREAD -PTHREAD_PSHARED -REGPARM -STATIC_PCRE -STATIC_PCRE2 +TPROXY +LINUX_TPROXY +LINUX_SPLICE +LIBCRYPT +CRYPT_H -VSYSCALL +GETADDRINFO -OPENSSL -LUA +FUTEX +ACCEPT4 -MY_ACCEPT4 -ZLIB -SLZ +CPU_AFFINITY -TFO -NS +DL +RT -DEVICEATLAS -51DEGREES -WURFL -SYSTEMD -OBSOLETE_LINKER +PRCTL
Default settings :
bufsize = 16384, maxrewrite = 1024, maxpollevents = 200
Built with multi-threading support (MAX_THREADS=64, default=2).
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
Built without compression support (neither USE_ZLIB nor USE_SLZ are set).
Compression algorithms supported : identity("identity")
Built without PCRE or PCRE2 support (using libc's regex instead)
Encrypted password support via crypt(3): yes
Available polling systems :
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use epoll.
Available multiplexer protocols :
(protocols marked as <default> cannot be specified using 'proto' keyword)
h2 : mode=HTX side=FE|BE
h2 : mode=HTTP side=FE
<default> : mode=HTX side=FE|BE
<default> : mode=TCP|HTTP side=FE|BE
Available services : none
Available filters :
[SPOE] spoe
[COMP] compression
[CACHE] cache
[TRACE] trace
lukas@dev:~/haproxy$ uname -a
Linux dev 4.4.0-109-generic #132-Ubuntu SMP Tue Jan 9 19:52:39 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
lukas@dev:~/haproxy$
```
Not a regression though, from the initial commit 2fbcafc9ce 4 years ago until latest HEAD the behavior is the same.
## What's the configuration?
```
global
maxconn 100
log 10.0.0.4 syslog debug
defaults
mode http
option httplog
log global
timeout connect 10s
timeout client 120s
timeout server 120s
option http-keep-alive
frontend myfrontend
bind :80
http-request set-src hdr(x-forwarded-for) #if { src 10.0.0.4 }
default_backend be_ip
backend be_ip
server www 10.0.0.33:80
```
## Steps to reproduce the behavior
In a keep-alived client connection, issue multiple subsequent requests with different x-forwarded-for values (certain CDNs or if another frontend haproxy layer has http-reuse enabled)
1. Opening a telnet window to haproxy and paste something like this:
```
GET / HTTP/1.1
Host: localhost
X-Forwarded-For: 192.168.0.1
GET / HTTP/1.1
Host: localhost
X-Forwarded-For: 192.168.0.2
GET / HTTP/1.1
Host: localhost
X-Forwarded-For: 192.168.0.3
GET / HTTP/1.1
Host: localhost
```
## Actual behavior 1
When set-src is unconditionally called (no src based ACL), the first three request will show up fine in haproxy logs. The forth request will keep the src from the last set-src call (`192.168.0.3`)
## Actual behavior 2
When set-src is conditioned by an ACL (only trust X-Forwarded-For from known trusted proxies or CDNs), then the behavior is worse, because after the first set-src call (`192.168.0.1`), the ACL stops matching, because we replaced the actual low level connection IP.
## Expected behavior
Expected behavior would be to only set the source IP for this specific request/transaction, `http-request` implies exactly that, without impacting subsequent transactions on the same connection.
## Do you have any idea what may have caused this?
set-src works on the connection layer.
## Do you have an idea how to solve the issue?
Either we are able to fix the underlying technical issue (I assume this is complex), or we document this behavior clearly along with some workarounds (already provided on discourse by Michael).
If using 2.5 or newer, you just need:
# allow cloudflare src ranges (https://www.cloudflare.com/ips-v4 + https://www.cloudflare.com/ips-v6)
acl is_cloudflare src -f /etc/cloudflare/ips-v4
acl is_cloudflare src -f /etc/cloudflare/ips-v6
http-request set-src hdr(cf-connecting-ip) if is_cloudflare
1 Like
TIEB62
October 2, 2022, 7:17am
3
I update HAProxy to 2.6.6 because I don’t want to workaround bugs and I try that
1 Like