Binding port 443 to both HTTP and TCP

Help!
1

We’ve recently setup HAProxy as one of our application suppliers required it. They supplied a basic configuration which has been working fine. However, we now have another supplier who needs us to accept in traffic on port 443 and forward it to a server on port 6002. So that we wouldn’t have to port forward things we don’t want to, or move servers between networks, I was asked if I could add this into the existing HAProxy setup.

I initially configured another acl which would only trigger if it came from one of the IP ranges that the suppler had specified, but although we could see traffic coming in and being forwarded, they said that the packets coming in made no sense to their application. I then found out that the application is expecting a TCP connection, so I’ve tried adjusting the configuration to separate into 2 different frontends.

The traffic is still not getting through as expected, so I don’t know what else to try at the moment with my limited knowledge of HAProxy. I’ve searched around and it could be an issue where I have port 443 on both an http and tcp frontend?

Below is my current config. I’ve adjusted it to remove sensitive info. I’d appreciate any help that can be given on this issue.

global
log 127.0.0.1:514 local0 info
chroot /var/lib/haproxy
pidfile /var/run/haproxy.pid
maxconn 40000
user haproxy
group haproxy
daemon

tune.ssl.default-dh-param 2048
ssl-default-bind-ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256
ssl-default-bind-options no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets
ssl-default-server-ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256
ssl-default-server-options no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets

# turn on stats unix socket
stats socket /var/lib/haproxy/stats

#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#---------------------------------------------------------------------

defaults
log global
option dontlognull
option redispatch
retries 7
timeout http-request 50s
timeout queue 2m
timeout connect 50s
timeout client 2m
timeout server 2m
timeout http-keep-alive 50s
timeout check 50s

# statistics admin level depends on the authenticated user

userlist stats-auth
group admin users admin
user admin password password
group readonly users haproxy
user haproxy password password

#---------------------------------------------------------------------
# main frontend which proxys to the backends
#---------------------------------------------------------------------

frontend main
bind *:80
mode http
option forwardfor except 127.0.0.0/8
option httplog
stats enable
option http-no-delay

default_backend vendor1
acl h_1 path_beg /h/1/
acl server01_down nbsrv(server01) eq 0
use_backend server01 if h_1 !server01_down

frontend main_https
bind *:443 ssl crt /etc/haproxy/certs/our_cert.pem
mode http
option forwardfor except 127.0.0.0/8
option httplog
stats enable
acl AUTH http_auth(stats-auth)
acl AUTH_ADMIN http_auth_group(stats-auth) admin
stats http-request auth unless AUTH
stats admin if AUTH_ADMIN
stats uri /haproxy?stats
default_backend vendor1_ssl
acl h_1 path_beg /h/1/
acl server01_down nbsrv(server01_ssl) eq 0
use_backend server01_ssl if h_1 !server01_down

frontend vendor2_tcp
bind *:443
mode tcp
option tcplog
# Vendor2 access
acl vendor2_prod src -f /etc/haproxy/ipranges/vendor2_prod.subnets
acl vendor2_dev src -f /etc/haproxy/ipranges/vendor2_dev.subnets
use_backend vendor2_tcp if vendor2_prod || vendor2_dev

backend vendor1
mode http
balance leastconn
option httpchk GET /vendor1web/server/about
option http-no-delay
cookie h insert
server server01 server01:8080 cookie 1 check weight 100 minconn 3000 maxconn 5000

backend vendor1_ssl
mode http
balance leastconn
option httpchk GET /vendor1web/server/about
cookie h insert server server01 server01:8443 cookie 1 ssl verify none check weight 100 minconn 3000 maxconn 5000

backend server01
mode http
reqrep ^([^\ :]*)\ /h/[^/]+/(.*) \1\ /\2
cookie h insert
server server01 server01:8080 cookie 1 check

backend server01_ssl
mode http
reqrep ^([^\ :]*)\ /h/[^/]+/(.*) \1\ /\2
cookie h insert
server server01 server01:8443 cookie 1 check ssl verify none

backend vendor2_tcp
mode tcp
balance leastconn
option tcpka
option tcp-check
server server02 server02:6002 check weight 100 minconn 3000 maxconn 5000
2

I know my previous post is old now and I hate bumping up old posts, but, we’re still having issues with this.

We are now at a point where we are getting the frontend matches, but it’s intermittent.

For some reason we are getting connections match the tcp connections from Vendor2 match against the main_https frontend. The connections are not getting to the backend, but is there any way to say ‘if traffic is from x then go to y frontend’?

3

You cannot configure 2 frontends on port 443, without specifying different IP addresses.

For example, if you have 2 public IP address you CAN do this:

frontend a
 bind 192.168.1.5:443 ssl crt ...

frontend b
 bind 192.168.1.6:443

However you cannot bind to port 443, if any of those bind statements on port 443 doesn’t also specify a dedicated IP address, otherwise your kernel will randomly load-balance between the two.

You can add noreuseport to the global configuration temporarily, to check if haproxy is still able to start. That’s a good indication of whether the port configuration works fine, even without SO_REUSEPORT on the sockets. In this case, your kernel would reject binding to the same port twice, because of this missconfiguration.

So, how could you solve this with just one public IP? Move your SSL terminating frontend somewhere else and use a single frontend on port 443 which decides where the traffic needs to go.

Something like:

frontend 443
 mode tcp
 bind :443
 acl vendor2_prod src -f /etc/haproxy/ipranges/vendor2_prod.subnets
 acl vendor2_dev src -f /etc/haproxy/ipranges/vendor2_dev.subnets
 use_backend vendor2_tcp if vendor2_prod || vendor2_dev
 default_backend localhttps

backend localhttps
 mode tcp
 server localhost 127.0.0.1:1443 send-proxy

frontend main_https
 bind 127.0.0.1:1443 ssl crt /etc/haproxy/certs/our_cert.pem accept-proxy
 mode http
 option forwardfor except 127.0.0.0/8
 option httplog
 stats enable
 acl AUTH http_auth(stats-auth)
 acl AUTH_ADMIN http_auth_group(stats-auth) admin
 stats http-request auth unless AUTH
 stats admin if AUTH_ADMIN
 stats uri /haproxy?stats
 default_backend vendor1_ssl
 acl h_1 path_beg /h/1/
 acl server01_down nbsrv(server01_ssl) eq 0
 use_backend server01_ssl if h_1 !server01_down

The import point is that you only have a single frontend binding to port 443.

1 Like
4

Thanks for your help. I’ve just implemented this and we’re now getting the expected results!

1 Like