Https Frontend to Http backend 400 (BAD REQUEST)

Hello I have a https / http frontend which accesses an http backend.
Basically I want to limit the frontent to https. The problem, however, is that the web service repeatedly generates a POST 400 (BAD REQUEST) error when called up via https when I open the page via my web browser.

Here is my configuration:

global
	maxconn			5000
	log			/var/run/log	local0	debug
	stats socket /tmp/haproxy.socket level admin  expose-fd listeners
	uid			80
	gid			80
	nbproc			1
	nbthread			1
	hard-stop-after		15m
	chroot				/tmp/haproxy_chroot
	daemon
	tune.ssl.default-dh-param	2048
	server-state-file /tmp/haproxy_server_state
	

listen HAProxyLocalStats
	bind 127.0.0.1:2200 name localstats
	mode http
	stats enable
	stats admin if TRUE
	stats show-legends
	stats uri /haproxy/haproxy_stats.php?haproxystats=1
	timeout client 5000
	timeout connect 5000
	timeout server 5000

frontend NachOpenVPN_HTTPS-aus_LAN
	bind			10.4.4.239:443 name 10.4.4.239:443  no-sslv3 ssl crt-list /var/etc/haproxy/NachOpenVPN_HTTPS-aus_LAN.crt_list
	bind			10.4.4.239:80 name 10.4.4.239:80	
	mode			http
	log			global
	option			socket-stats
	option			log-separate-errors
	option			httplog
	option			http-keep-alive
	option			forwardfor
	acl https ssl_fc
	http-request set-header		X-Forwarded-Proto http if !https
	http-request set-header		X-Forwarded-Proto https if https
	timeout client		7200000
	acl			DL_thorstenACL	var(txn.txnhost) -m str -i vpn.schulte-batenXXX.de
	http-request set-var(txn.txnhost) hdr(host)
	use_backend DLthorsten-ohne-passwort_ipvANY  if  DL_thorstenACL aclcrt_NachOpenVPN_HTTPS-aus_LAN


backend DLthorsten-ohne-passwort_ipvANY
	mode			http
	id			106
	log			global
	stick-table type ip size 50k expire 30m
	stick on src
	timeout connect		30000
	timeout server		30000
	retries			3
	server			DL_Thorsten 10.4.4.10:1731 id 111

Log from haproxy:

May 15 05:54:47 haproxy[92018]: 10.4.4.241:62950 [15/May/2020:05:54:47.424] NachOpenVPN_HTTPS-aus_LAN~ DLthorsten-ohne-passwort_ipvANY/DL_Thorsten 0/0/0/5/5 400 130 - - ---- 6/6/5/5/0 0/0 "POST /socket.io/?EIO=3&transport=polling&t=N8MU63L&sid=ec5b759b878a4dfcac0b96b29893255b HTTP/1.1"
May 15 05:54:47 haproxy[92018]: 10.4.4.241:62881 [15/May/2020:05:54:17.061] NachOpenVPN_HTTPS-aus_LAN~ DLthorsten-ohne-passwort_ipvANY/DL_Thorsten 0/0/0/-1/30027 504 218 - - sH-- 6/6/5/5/0 0/0 "GET /socket.io/?EIO=3&transport=polling&t=N8MT-fV&sid=2fc24b8c6e3d4a2d8ac1850799d3b1b7 HTTP/1.1"
May 15 05:54:46 haproxy[92018]: 10.4.4.241:62939 [15/May/2020:05:54:46.763] NachOpenVPN_HTTPS-aus_LAN~ DLthorsten-ohne-passwort_ipvANY/DL_Thorsten 0/0/0/5/5 400 130 - - ---- 6/6/5/5/0 0/0 "POST /socket.io/?EIO=3&transport=polling&t=N8MU5va&sid=ec5b759b878a4dfcac0b96b29893255b HTTP/1.1"
May 15 05:54:46 haproxy[92018]: 10.4.4.241:62948 [15/May/2020:05:54:46.652] NachOpenVPN_HTTPS-aus_LAN~ DLthorsten-ohne-passwort_ipvANY/DL_Thorsten 0/0/1/5/8 101 138 - - ---- 7/7/5/5/0 0/0 "GET /socket.io/?EIO=3&transport=websocket&sid=ec5b759b878a4dfcac0b96b29893255b HTTP/1.1"
May 15 05:54:46 haproxy[92018]: 10.4.4.241:62939 [15/May/2020:05:54:44.220] NachOpenVPN_HTTPS-aus_LAN~ DLthorsten-ohne-passwort_ipvANY/DL_Thorsten 0/0/0/2242/2245 200 59965 - - ---- 6/6/5/5/0 0/0 "GET /socket.io/?EIO=3&transport=polling&t=N8MU3d3 HTTP/1.1"
May 15 05:54:44 haproxy[92018]: 10.4.4.241:62939 [15/May/2020:05:54:44.205] NachOpenVPN_HTTPS-aus_LAN~ DLthorsten-ohne-passwort_ipvANY/DL_Thorsten 0/0/1/7/8 400 130 - - ---- 6/6/5/5/0 0/0 "POST /socket.io/?EIO=3&transport=polling&t=N8MU37u&sid=7fe4c58f5d7a4297a1f9ca307c0eef55 HTTP/1.1"
May 15 05:54:44 haproxy[92018]: 10.4.4.241:62868 [15/May/2020:05:54:14.142] NachOpenVPN_HTTPS-aus_LAN~ DLthorsten-ohne-passwort_ipvANY/DL_Thorsten 0/0/0/-1/30028 504 218 - - sH-- 6/6/5/5/0 0/0 "GET /socket.io/?EIO=3&transport=polling&t=N8MTzxt&sid=0c793cecac2245c3b28d6ee7901c2f7f HTTP/1.1"
May 15 05:54:35 haproxy[92018]: 10.4.4.241:62921 [15/May/2020:05:54:34.965] NachOpenVPN_HTTPS-aus_LAN~ DLthorsten-ohne-passwort_ipvANY/DL_Thorsten 0/0/0/4/44 400 130 - - ---- 6/6/5/5/0 0/0 "POST /socket.io/?EIO=3&transport=polling&t=N8MU31C&sid=7fe4c58f5d7a4297a1f9ca307c0eef55 HTTP/1.1"
May 15 05:54:34 haproxy[92018]: 10.4.4.241:62923 [15/May/2020:05:54:34.930] NachOpenVPN_HTTPS-aus_LAN~ DLthorsten-ohne-passwort_ipvANY/DL_Thorsten 0/0/1/6/8 101 138 - - ---- 7/7/5/5/0 0/0 "GET /socket.io/?EIO=3&transport=websocket&sid=7fe4c58f5d7a4297a1f9ca307c0eef55 HTTP/1.1"

haproxy -vv

[2.4.4-RELEASE][root@pfSense.localdomain]/root: haproxy -vv
HA-Proxy version 2.0.14 2020/04/02 - https://haproxy.org/
Build options :
  TARGET  = freebsd
  CPU     = generic
  CC      = cc
  CFLAGS  = -O2 -pipe -fstack-protector-strong -fno-strict-aliasing -fno-strict-                                                                                                             aliasing -Wdeclaration-after-statement -fwrapv -Wno-address-of-packed-member -Wn                                                                                                             o-unused-label -Wno-sign-compare -Wno-unused-parameter -Wno-ignored-qualifiers -                                                                                                             Wno-missing-field-initializers -Wno-implicit-fallthrough -Wtype-limits -Wshift-n                                                                                                             egative-value -Wnull-dereference -DFREEBSD_PORTS
  OPTIONS = USE_PCRE=1 USE_PCRE_JIT=1 USE_REGPARM=1 USE_STATIC_PCRE=1 USE_GETADD                                                                                                             RINFO=1 USE_OPENSSL=1 USE_LUA=1 USE_ACCEPT4=1 USE_ZLIB=1 USE_CPU_AFFINITY=1

Feature list : -EPOLL +KQUEUE -MY_EPOLL -MY_SPLICE -NETFILTER +PCRE +PCRE_JIT -P                                                                                                             CRE2 -PCRE2_JIT +POLL -PRIVATE_CACHE +THREAD -PTHREAD_PSHARED +REGPARM +STATIC_P                                                                                                             CRE -STATIC_PCRE2 +TPROXY -LINUX_TPROXY -LINUX_SPLICE +LIBCRYPT -CRYPT_H -VSYSCA                                                                                                             LL +GETADDRINFO +OPENSSL +LUA -FUTEX +ACCEPT4 -MY_ACCEPT4 +ZLIB -SLZ +CPU_AFFINI                                                                                                             TY -TFO -NS -DL -RT -DEVICEATLAS -51DEGREES -WURFL -SYSTEMD -OBSOLETE_LINKER -PR                                                                                                             CTL -THREAD_DUMP -EVPORTS

Default settings :
  bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with multi-threading support (MAX_THREADS=64, default=4).
Built with OpenSSL version : OpenSSL 1.0.2t-freebsd  10 Sep 2019
Running on OpenSSL version : OpenSSL 1.0.2o-freebsd  27 Mar 2018 (VERSIONS DIFFE                                                                                                             R!)
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : SSLv3 TLSv1.0 TLSv1.1 TLSv1.2
Built with Lua version : Lua 5.3.5
Built with transparent proxy support using: IP_BINDANY IPV6_BINDANY
Built with zlib version : 1.2.11
Running on zlib version : 1.2.11
Compression algorithms supported : identity("identity"), deflate("deflate"), raw                                                                                                             -deflate("deflate"), gzip("gzip")
Built with PCRE version : 8.43 2019-02-23
Running on PCRE version : 8.43 2019-02-23
PCRE library supports JIT : yes
Encrypted password support via crypt(3): yes

Available polling systems :
     kqueue : pref=300,  test result OK
       poll : pref=200,  test result OK
     select : pref=150,  test result OK
Total: 3 (3 usable), will use kqueue.

Available multiplexer protocols :
(protocols marked as <default> cannot be specified using 'proto' keyword)
              h2 : mode=HTTP       side=FE        mux=H2
              h2 : mode=HTX        side=FE|BE     mux=H2
       <default> : mode=HTX        side=FE|BE     mux=H1
       <default> : mode=TCP|HTTP   side=FE|BE     mux=PASS

Available services : none

Available filters :
        [SPOE] spoe
        [COMP] compression
        [CACHE] cache
        [TRACE] trace

Log from Chrome:
VM41:1 POST https://vpn.schulte-batenXXX.de/socket.io/?EIO=3&transport=polling&t=N8MUv9n&sid=546c263979164fca9e7eae654fa4d92c 400 (BAD REQUEST)

I only open the page via http there are no problems.

Looks like the error comes from the backend server, what’s in the backend server log?

Hey! This is the Backend-Server-Log:

INFO:geventwebsocket.handler:10.4.4.1 - - [2020-05-15 15:09:05] "POST /socket.io/?EIO=3&transport=polling&t=N8OuRdE&sid=afcaa211245749f88987e7f0599c289a HTTP/1.1" 400 122 0.003531


INFO:geventwebsocket.handler:10.4.4.1 - - [2020-05-15 15:09:08] "GET /socket.io/?EIO=3&transport=polling&t=N8OuSDM HTTP/1.1" 200 29100 0.563801


INFO:geventwebsocket.handler:10.4.4.1 - - [2020-05-15 15:09:09] "POST /socket.io/?EIO=3&transport=polling&t=N8OuSQG&sid=22a6bf2ea16c494291f2cf34533fe9f0 HTTP/1.1" 400 122 0.000371


INFO:geventwebsocket.handler:10.4.4.1 - - [2020-05-15 15:09:09] "POST /socket.io/?EIO=3&transport=polling&t=N8OuSSz&sid=22a6bf2ea16c494291f2cf34533fe9f0 HTTP/1.1" 400 122 0.021671


INFO:geventwebsocket.handler:10.4.4.1 - - [2020-05-15 15:09:11] "GET /socket.io/?EIO=3&transport=polling&t=N8OuSyD HTTP/1.1" 200 29100 0.556703


INFO:geventwebsocket.handler:10.4.4.1 - - [2020-05-15 15:09:12] "POST /socket.io/?EIO=3&transport=polling&t=N8OuT9-&sid=b3d625d887e9417eaece2b204fde9afd HTTP/1.1" 400 122 0.000404


INFO:geventwebsocket.handler:10.4.4.1 - - [2020-05-15 15:09:12] "POST /socket.io/?EIO=3&transport=polling&t=N8OuTBr&sid=b3d625d887e9417eaece2b204fde9afd HTTP/1.1" 400 122 0.000360


INFO:geventwebsocket.handler:10.4.4.1 - - [2020-05-15 15:09:15] "GET /socket.io/?EIO=3&transport=polling&t=N8OuTh5 HTTP/1.1" 200 29100 0.734430


INFO:geventwebsocket.handler:10.4.4.1 - - [2020-05-15 15:09:15] "POST /socket.io/?EIO=3&transport=polling&t=N8OuTyL&sid=342edd273ddc466d98a0cbb784f996d5 HTTP/1.1" 400 122 0.005293


INFO:geventwebsocket.handler:10.4.4.1 - - [2020-05-15 15:09:16] "POST /socket.io/?EIO=3&transport=polling&t=N8OuUAL&sid=342edd273ddc466d98a0cbb784f996d5 HTTP/1.1" 400 122 0.000360


INFO:geventwebsocket.handler:10.4.4.1 - - [2020-05-15 15:09:18] "GET /socket.io/?EIO=3&transport=polling&t=N8OuUP_ HTTP/1.1" 200 29090 0.994597


INFO:geventwebsocket.handler:10.4.4.1 - - [2020-05-15 15:09:18] "POST /socket.io/?EIO=3&transport=polling&t=N8OuUj-&sid=aa14530115e94ed8a2c5fe23ef985497 HTTP/1.1" 400 122 0.000351


INFO:geventwebsocket.handler:10.4.4.1 - - [2020-05-15 15:09:19] "POST /socket.io/?EIO=3&transport=polling&t=N8OuUvE&sid=aa14530115e94ed8a2c5fe23ef985497 HTTP/1.1" 400 122 0.000413


INFO:geventwebsocket.handler:10.4.4.1 - - [2020-05-15 15:09:22] "GET /socket.io/?EIO=3&transport=polling&t=N8OuVOT HTTP/1.1" 200 29100 1.430087


INFO:geventwebsocket.handler:10.4.4.1 - - [2020-05-15 15:09:23] "POST /socket.io/?EIO=3&transport=polling&t=N8OuVtR&sid=db0e8ef4a7c9417a9b0544d415713fce HTTP/1.1" 400 122 0.000341

Unfortunately I can not influence the loglevel, since it is a docker container with python code and I have no idea about it. I installed the Docker container on a different server as a test and the same problem. Everything is ok with a direct IP connection. When I go through HAProxy I get these problems.

If your backend is a blackbox, capture the traffic between haproxy and your backend server in a working and in a non-working situation and compare the two.

The Issue is, that socket.io is being blocked/not forwarded properly with a https frontent (http working fine).

photo_2020-05-23_12-06-08

We already know about the 400 Bad request response. What we need to know is how that requests actually looks like.

okay I understand. Since I have never done this before, here is my attempt. In pfsense I recorded the packets that are on port 1731 (the port of the web server of the backend). The internal IP of the backend is 10.4.4.10. The internal IP of the pfsense (of the HAProxy) is 10.4.4.1.

Passwort: haproxy

We can see that the backend rejects the POST request, it’s unclear why.

The “good” capture does not shows a POST request, but a GET request instead, so it’s not really comparable.

Can you provide a working POST capture please?

The Backendwebserver is a Flask-app which needs a Proxyfix.

No Problem with HAProxy… with this fix it works. Thanks for your help!

1 Like