Hi, I have a website running HTTPD and node.js behind a pfSense box and I configured HAProxy to enable high availability on these servers and everything is running fine. I need now to duplicate the pfSense box and enable a sync between the two boxes. One of the steps requires the IP alias to be turned into CARP for the synchronization to work but as soon as I change the type of the virtual IP address and reload the configuration the connection to the node.js backend server timeouts with the error below. Reverting the changes to an IP alias works fine. Any suggestion on how to determine where the issue is ?
[Wed Oct 21 11:35:04.333690 2020] [proxy_http:error] [pid 1385] [client 192.168.0.1:54640] AH01114: HTTP: failed to make connection to backend: node.mysite.net
Hosts file on all servers
192.168.0.100 web.mysite.com
192.168.0.101 node.mysite.com
-
Frontend web.mysite.com
- Virtual IP: 192.168.0.100/24
- Backend servers:
- 192.168.0.10 with HTTPD
- 192.168.0.11 with HTTPD
-
Frontend node.mysite.com
- Virtual IP: 192.168.0.101/24
- Backend servers:
- 192.168.0.12 with node.js listening on port 3000
- 192.168.0.13 with node.js listening on port 3000
HTTPD call to node.js. The return is supposed to be a HTTP 200
<Location /node/>
Require all granted
RequestHeader unset Origin
ProxyPreserveHost On
ProxyPass http://node.mysite.net:3000/node/
Share the entire configuration in working and non-working situations as well as the output of haproxy -vv please.
Sure, here is the configuration. Sorry I wasn’t sure how to extract it from the pfSense box. We have 3 environments DEV, PREPROD and PROD but I kept only the common and production settings. The other two environments have a similar configuration but with different virtual and physical IP addresses.
global
maxconn 2000
stats socket /tmp/haproxy.socket level admin expose-fd listeners
uid 80
gid 80
nbproc 1
nbthread 1
hard-stop-after 15m
chroot /tmp/haproxy_chroot
daemon
tune.ssl.default-dh-param 2048
server-state-file /tmp/haproxy_server_state
listen HAProxyLocalStats
bind 127.0.0.1:2200 name localstats
mode http
stats enable
stats admin if TRUE
stats show-legends
stats uri /haproxy/haproxy_stats.php?haproxystats=1
timeout client 5000
timeout connect 5000
timeout server 5000
frontend PRODWebServerHTTP
bind 192.168.0.100:7776 name 192.168.0.100:7776
mode http
log global
option http-keep-alive
option forwardfor
acl https ssl_fc
http-request set-header X-Forwarded-Proto http if !https
http-request set-header X-Forwarded-Proto https if https
timeout client 30000
default_backend PRODLBWebServerHTTP_ipvANY
frontend PRODWebServerHTTPS
bind 192.168.0.100:4443 name 192.168.0.100:4443
mode tcp
log global
timeout client 30000
default_backend PRODLBWebServerHTTPS_ipvANY
frontend PRODNodeWS
bind 192.168.0.101:8080 name 192.168.0.101:8080
mode tcp
log global
timeout client 30000
default_backend PRODLBNodeWS_ipvANY
frontend PRODNodeEmail
bind 192.168.0.101:3000 name 192.168.0.101:3000
mode http
log global
option http-keep-alive
option forwardfor
acl https ssl_fc
http-request set-header X-Forwarded-Proto http if !https
http-request set-header X-Forwarded-Proto https if https
timeout client 30000
default_backend PRODLBNodeEmail_ipvANY
frontend PRODExtWebServerHTTP
bind 111.22.333.4:80 name 111.22.333.4:80
mode http
log global
option http-keep-alive
option forwardfor
acl https ssl_fc
http-request set-header X-Forwarded-Proto http if !https
http-request set-header X-Forwarded-Proto https if https
timeout client 30000
default_backend PRODLBWebServerHTTP_ipvANY
frontend PRODExtWebServerHTTPS
bind 111.22.333.4:443 name 111.22.333.4:443
mode tcp
log global
timeout client 30000
default_backend PRODLBWebServerHTTPS_ipvANY
backend PRODLBWebServerHTTP_ipvANY
mode http
id 110
log global
balance leastconn
timeout connect 30000
timeout server 30000
retries 3
server websrv1 192.168.0.10:7776 id 101 check inter 1000
server websrv2 192.168.0.11:7776 id 101 check inter 1000
backend PRODLBWebServerHTTPS_ipvANY
mode tcp
id 111
log global
tcp-request inspect-delay 5s
stick-table type binary len 32 size 10m expire 10m
acl clienthello req.ssl_hello_type 1
acl serverhello res.ssl_hello_type 2
tcp-request content accept if clienthello
tcp-response content accept if serverhello
stick on payload_lv(43,1) if clienthello
stick store-response payload_lv(43,1) if serverhello
balance leastconn
timeout connect 30000
timeout server 30000
retries 3
server websrv1 192.168.0.10:4443 id 101 check inter 1000
server websrv2 192.168.0.11:4443 id 101 check inter 1000
backend PRODLBNodeWS_ipvANY
mode tcp
id 112
log global
balance leastconn
timeout connect 30000
timeout server 30000
retries 3
server nodesrv1 192.168.0.12:8080 id 101 check inter 1000
server nodesrv2 192.168.0.13:8080 id 102 check inter 1000
backend PRODLBNodeEmail_ipvANY
mode http
id 113
log global
balance leastconn
timeout connect 30000
timeout server 30000
retries 3
server nodesrv1 192.168.0.12:3000 id 101 check inter 1000
server nodesrv2 192.168.0.13:3000 id 102 check inter 1000
Hi again,
One thing maybe worth nothing is that all machines including the pfSense boxes are VirtualBox machines but most of the machines denied promiscuous mode. I set them all to “allow-all” but I am not sure if I should reboot the machines, including the master pfsense or not.
So what exactly is the difference between a working and non-working configuration on haproxy?
The different is the virtual IP being an IP alias (working) or a CARP (not working). Unfortunately I need to enable CARP for the virtual IP addresses to be synchronized with the second pfSense.
So in other words, nothing in the haproxy config changes. 192.168.0.101 would move from an interface alias to a CARP IP.
I would suggest you stop binding to IP addresses, and just bind to the ports instead.
OK. I have 3 external IPs listening on ports 443 and 80 redirecting to 3 sets of backend servers with HTTPD and Node.js. So if I understand correctly you suggest that the HAProxy frontend servers listens on different ports (e.g. 8080, 8081, 8082 for Node.js and another range for HTTPD) and then redirects to the proper ports on the backend servers? I would still need the bind on the external IP:port, correct ?
Something like this perhaps ?!
frontend PRODExtWebServerHTTPS
bind 111.22.333.4:443 name 111.22.333.4:443
mode tcp
log global
timeout client 30000
default_backend PRODLBWebServerHTTPS_ipvANY
frontend PRODNodeWS
bind *:8080
mode tcp
log global
timeout client 30000
default_backend PRODLBNodeWS_ipvANY
frontend DEVNodeWS
bind *:8081
mode tcp
log global
timeout client 30000
default_backend PRODLBNodeWS_ipvANY
backend PRODLBNodeWS_ipvANY
mode tcp
id 112
log global
balance leastconn
timeout connect 30000
timeout server 30000
retries 3
server nodesrv1 192.168.0.16:8080 id 101 check inter 1000
server nodesrv2 192.168.0.17:8080 id 102 check inter 1000
...