2 frontends SSL over HTTP and TCP?

mandevnull · June 30, 2017, 8:11am

Hello, can i use 2 frontends configured with ssl but one frontend in tcp mode and the other in http mode? In the same port (443)

I try this:

frontend http-in
mode http
bind 0.0.0.0:80
bind 0.0.0.0:443 ssl crt /etc/pki/tls/private/cert.pem
option httplog
option dontlognull
option contstats

acl host_host2 hdr(host) -i host2.domain.com   
use_backend host2_cluster if host_host2

frontend https-in
mode tcp
option tcplog
bind 0.0.0.0:443

tcp-request inspect-delay 5s
tcp-request content accept if { req.ssl_hello_type 1 }

acl host_host1 req.ssl_sni -i host1.domain.com  
use_backend host1_cluster if host_host1

I get this error in log (not found the server https://host1.domain.com and not found https://host2.domain.com)

Jun 30 10:04:57 localhost haproxy[4570]: 172.21.6.23:36510 [30/Jun/2017:10:04:57.885] http-in~ http-in/ -1/-1/-1/-1/23 503 212 - - SC-- 6/0/0/0/0 0/0 “GET / HTTP/1.1”

thanks in advance!

lukastribus · June 30, 2017, 8:40am

No, you cannot bind to “0.0.0.0:443” twice, that will lead either to a bind error on startup or to a nasty kernel-based load-balancing between the two sockets (when REUSEPORT is supported).

You can only have one 0.0.0.0:443 socket, and use TCP mode to SNI route it to different destination; use a dedicatated backend to respin this traffic to the correct (TLS terminating) frontend. Abstract namespace sockets (abns) lend itself to this purpose very well.

So this would look something like this:

backend respin-tls-term
 option http-server-close
 server loopback-for-tls abns@haproxy-tls-term send-proxy-v2
!
frontend http-in
 mode http
 bind 0.0.0.0:80
 bind abns@haproxy-tls-term accept-proxy ssl crt /etc/pki/tls/private/cert.pem 
 option httplog 
 option dontlognull 
 option contstats
 acl host_host2 hdr(host) -i host2.domain.com   
 use_backend host2_cluster if host_host2
!
frontend https-in
 mode tcp
 option tcplog
 bind 0.0.0.0:443
 
 tcp-request inspect-delay 5s
 tcp-request content accept if { req.ssl_hello_type 1 }
 
 acl host_host1 req.ssl_sni -i host1.domain.com  
 use_backend host1_cluster if host_host1
 use_backend respin-tls-term if { req.ssl_sni -i host2.domain.com }

mandevnull · June 30, 2017, 9:00am

thank you lukastribus.
I have tried this configuration, and https://host1.domain.com works well but https://host2.domain.com gets this error:

Jun 30 10:54:34 localhost haproxy[4616]: 172.21.6.23:36978 [30/Jun/2017:10:54:34.730] https-in respin-tls-term/ -1/-1/0 187 PR 2/0/0/0/3 0/0
Jun 30 10:54:44 localhost haproxy[4616]: 172.21.6.23:36982 [30/Jun/2017:10:54:44.890] https-in respin-tls-term/ -1/-1/0 187 PR 0/0/0/0/3 0/0
Jun 30 10:54:47 localhost haproxy[4616]: 172.21.6.23:36986 [30/Jun/2017:10:54:47.065] https-in respin-tls-term/ -1/-1/0 187 PR 2/2/0/0/3 0/0

thanks in advance

mandevnull · June 30, 2017, 9:07am

sorry, the log:

Jun 30 11:06:10 localhost haproxy[4616]: 172.21.6.23:37060 [30/Jun/2017:11:06:10.932] https-in respin-tls-term/ -1/-1/0 187 PR 10/10/5/0/3 0/0
Jun 30 11:06:10 localhost haproxy[4616]: 172.21.6.23:37070 [30/Jun/2017:11:06:10.932] https-in respin-tls-term/ -1/-1/0 187 PR 9/9/4/0/3 0/0
Jun 30 11:06:10 localhost haproxy[4616]: 172.21.6.23:37068 [30/Jun/2017:11:06:10.932] https-in respin-tls-term/ -1/-1/0 187 PR 8/8/3/0/3 0/0
Jun 30 11:06:10 localhost haproxy[4616]: 172.21.6.23:37066 [30/Jun/2017:11:06:10.932] https-in respin-tls-term/ -1/-1/0 187 PR 7/7/2/0/3 0/0
Jun 30 11:06:10 localhost haproxy[4616]: 172.21.6.23:37064 [30/Jun/2017:11:06:10.932] https-in respin-tls-term/ -1/-1/0 187 PR 6/6/1/0/3 0/0
Jun 30 11:06:10 localhost haproxy[4616]: 172.21.6.23:37062 [30/Jun/2017:11:06:10.932] https-in respin-tls-term/ -1/-1/0 187 PR 5/5/0/0/3 0/0
Jun 30 11:06:10 localhost haproxy[4616]: 172.21.6.23:37072 [30/Jun/2017:11:06:10.958] https-in respin-tls-term/ -1/-1/0 187 PR 5/5/0/0/3 0/0

lukastribus · June 30, 2017, 9:08am

What error do you see in the client? 400 or 403?

Not sure what happens here, are you sure you have “send-proxy-v2” in the new backend and “accept-proxy” in the the “bind abns” config line?

mandevnull · June 30, 2017, 9:09am

ok, i have a problem with copy&paste with this form
In the log i see “NOSRV”

thanks

lukastribus · June 30, 2017, 9:09am

Can you put it on pastebin.com please?

mandevnull · June 30, 2017, 9:12am

of course
https://pastebin.com/BJYNWg3K

In chrome i see

This site can’t provide a secure connection

host2.domain.com sent an invalid response.
ERR_SSL_PROTOCOL_ERROR

lukastribus · June 30, 2017, 9:34am

Probably your kernel doesn’t support abns (although I would expect a bind error).

Try replacing the two “abns@haproxy-tls-term” declarations with 127.0.0.1:1443, so we use a localhost IP socket instead.

mandevnull · June 30, 2017, 9:38am

same error log

thanks

lukastribus · June 30, 2017, 10:04am

I need to know the complete configuration.

Try replacing “backend respin-tls-term” with “mode tcp” in “backend respin-tls-term”.

Also, do you see any other syslog messages when starting haproxy, like server down in the respin-tls-term backend?

mandevnull · June 30, 2017, 10:31am

there are no errors

Jun 30 12:14:56 hglb2 systemd: Starting HAProxy Load Balancer…
Jun 30 12:14:56 localhost haproxy[4784]: Proxy respin-tls-term started.
Jun 30 12:14:56 localhost haproxy[4784]: Proxy http-in started.
Jun 30 12:14:56 localhost haproxy[4784]: Proxy https-in started.
Jun 30 12:14:56 localhost haproxy[4784]: Proxy host1_cluster started.
Jun 30 12:14:56 localhost haproxy[4784]: Proxy host2_cluster started.

Jun 30 12:16:26 localhost haproxy[4785]: 172.21.6.23:38610 [30/Jun/2017:12:15:34.630] https-in respin-tls-term/loopback-for-tls 1/0/52018 2458 sD 9/7/1/1/0 0/0

chrome is waiting https://host2.domain.com and no occurs nothing

the config:

backend respin-tls-term
mode tcp
option http-server-close
server loopback-for-tls abns@haproxy-tls-term send-proxy-v2

frontend http-in
bind 0.0.0.0:80
bind abns@haproxy-tls-term accept-proxy ssl crt /etc/pki/tls/private/cert.pem
option httplog
option dontlognull
option contstats

acl host_host2 hdr(host) -i host2.domain.com
use_backend host2_cluster if host_host2

frontend https-in
mode tcp
option tcplog
bind 0.0.0.0:443

tcp-request inspect-delay 5s 
tcp-request content accept if { req.ssl_hello_type 1 }

acl host_host1 req.ssl_sni -i host1.domain.com
use_backend host1_cluster if host_host1   
use_backend respin-tls-term if { req.ssl_sni -i host2.domain.com }

backend host2_cluster
balance roundrobin
default-server inter 5s fall 2 on-marked-down shutdown-sessions
server node1 192.168.2.1:443 check cookie s1
server node2 192.168.2.2:443 check cookie s2

backend host1_cluster
balance leastconn
mode tcp
option log-health-checks
default-server inter 5s fall 2 on-marked-down shutdown-sessions
option tcplog
server node1 192.168.1.1:443 check
server node2 192.168.1.2:443 check

and the same replacing

server loopback-for-tls 127.0.0.1:1443 send-proxy-v2
and
bind 127.0.0.1:1443 accept-proxy ssl crt /etc/pki/tls/private/cert.pem

no errors

thanks!

lukastribus · June 30, 2017, 10:35am

I mean the default and global section of your configuration.

mandevnull · June 30, 2017, 10:45am

ok

global
daemon
ssl-server-verify none
tune.ssl.default-dh-param 2048
maxconn 1024
user haproxy
group haproxy
log 127.0.0.1 local0 debug

defaults
mode http
log global
timeout connect 5000ms
timeout client 50000ms
timeout server 50000ms
timeout queue 60s
retries 3

lukastribus · June 30, 2017, 10:43pm

I don’t know what’s wrong, the same configuration works fine for me.

Can you:

confirm your browser sends the TLS handshake with a SNI
post the output of “haproxy -vv”
post the output of “curl -vk https://host2.domain.com/asd”
run through “strace -tt” when the issue fails (a lot of output, put it on pastebin.com)

Topic		Replies	Views
Tcp mode and ssl confusion Help!	4	2845	September 20, 2022
Multiple port 443 backends - TCP mode Help!	3	2125	November 9, 2018
Mixing mode tcp and http - SSL termination and Passthrough Help!	17	37378	December 7, 2023
Multiple ssl/sni frontend configs without sharing config settings between sni names Help!	6	2952	July 29, 2020
FrontEnd HTTPS And TCP over TLS Help!	9	2495	August 4, 2018

2 frontends SSL over HTTP and TCP?

Related Topics