What is wrong with this config?

Help!
1

Hi,

During the week-end, I re-configured the HAProxy module in my pfSense firewall. HAProxy is version 1.7.11 and pfSense is 2.4.3. The idea is this :

A first frontend, SSL Mux, is listening the WAN IP ; TCP 443 and is sorting the sockets according to the CN of the certificate the client is looking for. According to the name, HAProxy uses a backend that loop to a specific frontend offering the service the client is looking for. All of the sorting is done using complete DNS names as expected from the clients.

One of this frontend is an SSL accelerator. Another one is a reverse proxy enforcing authentication with client certificates. I also have a pseudo STunnel server. For now, I would like to focus on the SSL accelerator.

I have 2 certificates from Let’s Encrypt, one with an RSA key and a complete CN and one as a wildcard and ECDSA. I also have a local CA in my pfSense which I used to produce home-made certificates (RSA and complete CN).

If I configure HAProxy to use the Let’s Encrypt certificate with a complete name and home-made certificates also with complete names, HAProxy does as expected. Everything is routed properly and clients are presented with the proper certificate and services.

If I try to use the wildcard certificate, either as the “main” certificate for the SSL Accelerator frontend or as an extra certificate for it, HAProxy refuses to use it. The clients are never presented the wildcard certificates and the access fails. Config No2 pasted below shows an example of one of these non-working configs. ServerA is stil working but ServerB is unaccessible.

I reviewed the difference between the configs but I have not been able to identify how / why HAProxy does not recognize the wildcard certificate as usable for the access. I think the problem is around ACLs (HAProxy thinks the name must be .*.domain.com instead of *.domain.com and / or is looking for an actual * instead of considering it as a wildcard) but I failed to manually create the extra ACL HAProxy would need to handle the wildcard certificate properly. I also tried to define the ACL as “contains .domain.com” as well as “ends by .domain.com” without any success.

Any idea on that ?

I tried to post my entire config but the stupid forum engine thinks that the dns names / IPs in the config are links and so refuse to let me post it…

Thanks in advance,

Hercales

±±+ Extract from working config±±±±+
acl aclcrt_SSL_Accelerator var(txn.txnhost) -m reg -i ^serverA.domain.com(:([0-9]){1,5})? acl aclcrt_SSL_Accelerator var(txn.txnhost) -m reg -i ^serverB\.domain\.com(:([0-9]){1,5})?

±±±+ Extract from non-working config ±±±±+
acl aclcrt_SSL_Accelerator var(txn.txnhost) -m reg -i ^serverA.domain.com(:([0-9]){1,5})? acl aclcrt_SSL_Accelerator var(txn.txnhost) -m reg -i ^([^\.]*)\.domain\.com(:([0-9]){1,5})?

2

Post the config with code formatting (</>), so it won’t recognize the links (and it will be readable).

I have also bumped your privileges in the forum so you don’t have any problems going forward either way, new users are strongly restricted by default.

I am gonna need the full configuration, because based on your description I’m not sure I have the full picture.

Also, explain which exact hostname the client request that haproxy does not match to the certificate?

cheers,
lukas

Can you elaborate what you mean by “main” certificate and

3

Hi,

Thanks for your reply.

Full config no1 below is working properly. When I connect to https://serverA.domain.org, I am presented with the proper certificate and I connect to the service. Same for serverB.

Under config no2, I can still connect serverA but can not connect to serverB anymore.

What I refer to as the “main” certificate is the first one configured in pfSense’s web interface for that frontend. Lower in that windows, there is a table in which we can add extra certificates.

In all cases, thanks for your support.

Heracles31

Config No1

Automaticaly generated, dont edit manually.

Generated on: 2018-09-05 09:46

global
maxconn 1000
stats socket /tmp/haproxy.socket level admin
uid 80
gid 80
nbproc 1
hard-stop-after 15m
chroot /tmp/haproxy_chroot
daemon
tune.ssl.default-dh-param 4096
server-state-file /tmp/haproxy_server_state

listen HAProxyLocalStats
bind 127.0.0.1:2200 name localstats
mode http
stats enable
stats admin if TRUE
stats show-legends
stats uri /haproxy/haproxy_stats.php?haproxystats=1
timeout client 5000
timeout connect 5000
timeout server 5000

frontend SSL_Accelerator
bind 127.0.0.1:8443 name 127.0.0.1:8443 ecdhe secp384r1 force-tlsv12 ciphers ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA:DHE-RSA-AES256-SHA ssl crt-list /var/etc/haproxy/SSL_Accelerator.crt_list
mode http
log global
option http-keep-alive
option forwardfor
acl https ssl_fc
http-request set-header X-Forwarded-Proto http if !https
http-request set-header X-Forwarded-Proto https if https
timeout client 30000
rspadd Referrer-Policy:\ no-referrer-when-downgrade
acl For_ServerA var(txn.txnhost) -m str -i cloud.domain.org
acl For_ServerB var(txn.txnhost) -m str -i ServerB.domain.org
acl aclcrt_SSL_Accelerator var(txn.txnhost) -m reg -i ^cloud.domain.org(:([0-9]){1,5})? acl aclcrt_SSL_Accelerator var(txn.txnhost) -m reg -i ^ServerB\.domain\.org(:([0-9]){1,5})?
http-request set-var(txn.txnhost) hdr(host)
use_backend To_ServerA_ipvANY if For_ServerA aclcrt_SSL_Accelerator
use_backend To_ServerB_ipvANY if For_ServerB aclcrt_SSL_Accelerator
use_backend Trash_ipvANY if aclcrt_SSL_Accelerator

frontend SSL_Mux
bind 1.2.3.4:443 name 1.2.3.4:443
mode tcp
log global
timeout client 30000
tcp-request inspect-delay 5s
tcp-request content accept if { req_ssl_hello_type 1 }
acl SNI_FOR_ServerA req_ssl_sni cloud.domain.org
acl SNI_FOR_ServerB req_ssl_sni ServerB.domain.org
acl SNI_FOR_domain req_ssl_sni domain.ddns.net
acl SNI_FOR_ServerC req_ssl_sni ServerC.domain.org
acl SNI_FOR_ServerD req_ssl_sni ServerD.domain.org
acl SNI_FOR_ServerE req_ssl_sni ServerE.domain.org
use_backend Backloop_SSL_Accelerator_ipvANY if SNI_FOR_ServerA
use_backend Backloop_SSL_Accelerator_ipvANY if SNI_FOR_ServerB
use_backend Backloop_SSH_Over_SSL_ipvANY if SNI_FOR_domain
use_backend Backloop_SSL_Authenticator_ipvANY if SNI_FOR_ServerC
use_backend Backloop_SSL_Authenticator_ipvANY if SNI_FOR_ServerD
use_backend Backloop_SSL_Authenticator_ipvANY if SNI_FOR_ServerE
default_backend Trash_ipvANY

frontend SSH_Over_SSL
bind 127.0.0.1:8445 name 127.0.0.1:8445 ecdhe secp384r1 force-tlsv12 ciphers ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA:DHE-RSA-AES256-SHA ssl crt-list /var/etc/haproxy/SSH_Over_SSL.crt_list ca-file /var/etc/haproxy/clientca_SSH_Over_SSL.pem verify required crl-file /var/etc/haproxy/clientcrl_SSH_Over_SSL.pem
mode tcp
log global
timeout client 30000
default_backend To_SSH_Management_ipvANY

frontend SSL_Authenticator
bind 127.0.0.1:8447 name 127.0.0.1:8447 ecdhe secp384r1 force-tlsv12 ciphers ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA:DHE-RSA-AES256-SHA ssl crt-list /var/etc/haproxy/SSL_Authenticator.crt_list ca-file /var/etc/haproxy/clientca_SSL_Authenticator.pem verify required crl-file /var/etc/haproxy/clientcrl_SSL_Authenticator.pem
mode http
log global
option http-keep-alive
option forwardfor
acl https ssl_fc
http-request set-header X-Forwarded-Proto http if !https
http-request set-header X-Forwarded-Proto https if https
timeout client 30000
acl For_ServerC var(txn.txnhost) -m str -i ServerC.domain.org
acl For_ServerD var(txn.txnhost) -m str -i ServerD.domain.org
acl For_ServerE var(txn.txnhost) -m str -i ServerE.domain.org
acl aclcrt_SSL_Authenticator var(txn.txnhost) -m reg -i ^domain.ddns.net(:([0-9]){1,5})? acl aclcrt_SSL_Authenticator var(txn.txnhost) -m reg -i ^ServerC\.domain\.org(:([0-9]){1,5})?
acl aclcrt_SSL_Authenticator var(txn.txnhost) -m reg -i ^ServerD.domain.org(:([0-9]){1,5})? acl aclcrt_SSL_Authenticator var(txn.txnhost) -m reg -i ^ServerE\.domain\.org(:([0-9]){1,5})?
http-request set-var(txn.txnhost) hdr(host)
use_backend To_ServerC_ipvANY if For_ServerC aclcrt_SSL_Authenticator
use_backend To_ServerD_ipvANY if For_ServerD aclcrt_SSL_Authenticator
use_backend To_ServerE_ipvANY if For_ServerE aclcrt_SSL_Authenticator
use_backend Trash_ipvANY if aclcrt_SSL_Authenticator

backend To_ServerA_ipvANY
mode http
id 100
log global
http-response set-header Strict-Transport-Security max-age=15780000;
timeout connect 30000
timeout server 30000
retries 3
option httpchk OPTIONS /
acl ACL_ServerA var(txn.txnhost) -m str -i cloud.domain.org
http-request set-var(txn.txnhost) hdr(host)
use-server ServerA if ACL_ServerA
server ServerA 12.34.4.64:80 id 101 check inter 1000

backend To_ServerB_ipvANY
mode http
id 106
log global
http-response set-header Strict-Transport-Security max-age=15780000;
timeout connect 30000
timeout server 30000
retries 3
option httpchk OPTIONS /
acl ACL_ServerB var(txn.txnhost) -m str -i ServerB.domain.org
http-request set-var(txn.txnhost) hdr(host)
use-server ServerB if ACL_ServerB
server ServerB 12.34.5.64:80 id 101 check inter 1000

backend Trash_ipvANY
mode http
id 104
log global
timeout connect 30000
timeout server 30000
retries 3
server null 127.0.0.1:55243 id 105 disabled

backend Backloop_SSL_Accelerator_ipvANY
mode tcp
id 102
log global
timeout connect 30000
timeout server 30000
retries 3
server HAProxy_SSL_Accelerator 127.0.0.1:8443 id 103

backend Backloop_SSH_Over_SSL_ipvANY
mode tcp
id 109
log global
timeout connect 7200000
timeout server 30000
retries 2
server HAProxy_STunnel 127.0.0.1:8445 id 110

backend Backloop_SSL_Authenticator_ipvANY
mode tcp
id 113
log global
timeout connect 30000
timeout server 30000
retries 3
server HAProxy_SSL_Auth 127.0.0.1:8447 id 114

backend To_SSH_Management_ipvANY
mode tcp
id 107
log global
timeout connect 7200000
timeout server 30000
retries 2
server Management 12.34.3.64:22 id 108

backend To_ServerC_ipvANY
mode http
id 111
log global
timeout connect 30000
timeout server 30000
retries 3
server ServerC 12.34.5.80:443 id 112 ssl check-ssl check inter 1000 verify none

backend To_ServerD_ipvANY
mode http
id 115
log global
timeout connect 30000
timeout server 30000
retries 3
server ServerD 12.34.3.80:443 id 112 ssl check-ssl check inter 1000 verify none

backend To_ServerE_ipvANY
mode http
id 116
log global
http-response set-header Strict-Transport-Security max-age=15780000;
timeout connect 30000
timeout server 30000
retries 3
server ServerE 127.0.0.1:443 id 117 ssl check inter 1000 ca-file /var/etc/haproxy/ca_5aee3bef38c80.pem crl-file /var/etc/haproxy/crl_5afc79e446b82.pem verifyhost ServerE.domain.org

And this is config no2 :

# Automaticaly generated, dont edit manually. # Generated on: 2018-09-05 09:58 global maxconn 1000 stats socket /tmp/haproxy.socket level admin uid 80 gid 80 nbproc 1 hard-stop-after 15m chroot /tmp/haproxy_chroot daemon tune.ssl.default-dh-param 4096 server-state-file /tmp/haproxy_server_state

listen HAProxyLocalStats
bind 127.0.0.1:2200 name localstats
mode http
stats enable
stats admin if TRUE
stats show-legends
stats uri /haproxy/haproxy_stats.php?haproxystats=1
timeout client 5000
timeout connect 5000
timeout server 5000

frontend SSL_Accelerator
bind 127.0.0.1:8443 name 127.0.0.1:8443 ecdhe secp384r1 force-tlsv12 ciphers ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA:DHE-RSA-AES256-SHA ssl crt-list /var/etc/haproxy/SSL_Accelerator.crt_list
mode http
log global
option http-keep-alive
option forwardfor
acl https ssl_fc
http-request set-header X-Forwarded-Proto http if !https
http-request set-header X-Forwarded-Proto https if https
timeout client 30000
rspadd Referrer-Policy:\ no-referrer-when-downgrade
acl For_ServerA var(txn.txnhost) -m str -i cloud.domain.org
acl For_ServerB var(txn.txnhost) -m str -i ServerB.domain.org
acl aclcrt_SSL_Accelerator var(txn.txnhost) -m reg -i ^cloud.domain.org(:([0-9]){1,5})? acl aclcrt_SSL_Accelerator var(txn.txnhost) -m reg -i ^([^\.]*)\.domain\.org(:([0-9]){1,5})?
http-request set-var(txn.txnhost) hdr(host)
use_backend To_ServerA_ipvANY if For_ServerA aclcrt_SSL_Accelerator
use_backend To_ServerB_ipvANY if For_ServerB aclcrt_SSL_Accelerator
use_backend Trash_ipvANY if aclcrt_SSL_Accelerator

frontend SSL_Mux
bind 1.2.3.4:443 name 1.2.3.4:443
mode tcp
log global
timeout client 30000
tcp-request inspect-delay 5s
tcp-request content accept if { req_ssl_hello_type 1 }
acl SNI_FOR_ServerA req_ssl_sni cloud.domain.org
acl SNI_FOR_ServerB req_ssl_sni ServerB.domain.org
acl SNI_FOR_domain req_ssl_sni domain.ddns.net
acl SNI_FOR_ServerC req_ssl_sni ServerC.domain.org
acl SNI_FOR_ServerD req_ssl_sni ServerD.domain.org
acl SNI_FOR_ServerE req_ssl_sni ServerE.domain.org
use_backend Backloop_SSL_Accelerator_ipvANY if SNI_FOR_ServerA
use_backend Backloop_SSL_Accelerator_ipvANY if SNI_FOR_ServerB
use_backend Backloop_SSH_Over_SSL_ipvANY if SNI_FOR_domain
use_backend Backloop_SSL_Authenticator_ipvANY if SNI_FOR_ServerC
use_backend Backloop_SSL_Authenticator_ipvANY if SNI_FOR_ServerD
use_backend Backloop_SSL_Authenticator_ipvANY if SNI_FOR_ServerE
default_backend Trash_ipvANY

frontend SSH_Over_SSL
bind 127.0.0.1:8445 name 127.0.0.1:8445 ecdhe secp384r1 force-tlsv12 ciphers ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA:DHE-RSA-AES256-SHA ssl crt-list /var/etc/haproxy/SSH_Over_SSL.crt_list ca-file /var/etc/haproxy/clientca_SSH_Over_SSL.pem verify required crl-file /var/etc/haproxy/clientcrl_SSH_Over_SSL.pem
mode tcp
log global
timeout client 30000
default_backend To_SSH_Management_ipvANY

frontend SSL_Authenticator
bind 127.0.0.1:8447 name 127.0.0.1:8447 ecdhe secp384r1 force-tlsv12 ciphers ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA:DHE-RSA-AES256-SHA ssl crt-list /var/etc/haproxy/SSL_Authenticator.crt_list ca-file /var/etc/haproxy/clientca_SSL_Authenticator.pem verify required crl-file /var/etc/haproxy/clientcrl_SSL_Authenticator.pem
mode http
log global
option http-keep-alive
option forwardfor
acl https ssl_fc
http-request set-header X-Forwarded-Proto http if !https
http-request set-header X-Forwarded-Proto https if https
timeout client 30000
acl For_ServerC var(txn.txnhost) -m str -i ServerC.domain.org
acl For_ServerD var(txn.txnhost) -m str -i ServerD.domain.org
acl For_ServerE var(txn.txnhost) -m str -i ServerE.domain.org
acl aclcrt_SSL_Authenticator var(txn.txnhost) -m reg -i ^domain.ddns.net(:([0-9]){1,5})? acl aclcrt_SSL_Authenticator var(txn.txnhost) -m reg -i ^ServerC\.domain\.org(:([0-9]){1,5})?
acl aclcrt_SSL_Authenticator var(txn.txnhost) -m reg -i ^ServerD.domain.org(:([0-9]){1,5})? acl aclcrt_SSL_Authenticator var(txn.txnhost) -m reg -i ^ServerE\.domain\.org(:([0-9]){1,5})?
http-request set-var(txn.txnhost) hdr(host)
use_backend To_ServerC_ipvANY if For_ServerC aclcrt_SSL_Authenticator
use_backend To_ServerD_ipvANY if For_ServerD aclcrt_SSL_Authenticator
use_backend To_ServerE_ipvANY if For_ServerE aclcrt_SSL_Authenticator
use_backend Trash_ipvANY if aclcrt_SSL_Authenticator

backend To_ServerA_ipvANY
mode http
id 100
log global
http-response set-header Strict-Transport-Security max-age=15780000;
timeout connect 30000
timeout server 30000
retries 3
option httpchk OPTIONS /
acl ACL_ServerA var(txn.txnhost) -m str -i cloud.domain.org
http-request set-var(txn.txnhost) hdr(host)
use-server ServerA if ACL_ServerA
server ServerA 12.34.4.64:80 id 101 check inter 1000

backend To_ServerB_ipvANY
mode http
id 106
log global
http-response set-header Strict-Transport-Security max-age=15780000;
timeout connect 30000
timeout server 30000
retries 3
option httpchk OPTIONS /
acl ACL_ServerB var(txn.txnhost) -m str -i ServerB.domain.org
http-request set-var(txn.txnhost) hdr(host)
use-server ServerB if ACL_ServerB
server ServerB 12.34.5.64:80 id 101 check inter 1000

backend Trash_ipvANY
mode http
id 104
log global
timeout connect 30000
timeout server 30000
retries 3
server null 127.0.0.1:55243 id 105 disabled

backend Backloop_SSL_Accelerator_ipvANY
mode tcp
id 102
log global
timeout connect 30000
timeout server 30000
retries 3
server HAProxy_SSL_Accelerator 127.0.0.1:8443 id 103

backend Backloop_SSH_Over_SSL_ipvANY
mode tcp
id 109
log global
timeout connect 7200000
timeout server 30000
retries 2
server HAProxy_STunnel 127.0.0.1:8445 id 110

backend Backloop_SSL_Authenticator_ipvANY
mode tcp
id 113
log global
timeout connect 30000
timeout server 30000
retries 3
server HAProxy_SSL_Auth 127.0.0.1:8447 id 114

backend To_SSH_Management_ipvANY
mode tcp
id 107
log global
timeout connect 7200000
timeout server 30000
retries 2
server Management 12.34.3.64:22 id 108

backend To_ServerC_ipvANY
mode http
id 111
log global
timeout connect 30000
timeout server 30000
retries 3
server ServerC 12.34.5.80:443 id 112 ssl check-ssl check inter 1000 verify none

backend To_ServerD_ipvANY
mode http
id 115
log global
timeout connect 30000
timeout server 30000
retries 3
server ServerD 12.34.3.80:443 id 112 ssl check-ssl check inter 1000 verify none

backend To_ServerE_ipvANY
mode http
id 116
log global
http-response set-header Strict-Transport-Security max-age=15780000;
timeout connect 30000
timeout server 30000
retries 3
server ServerE 127.0.0.1:443 id 117 ssl check inter 1000 ca-file /var/etc/haproxy/ca_5aee3bef38c80.pem crl-file /var/etc/haproxy/crl_5afc79e446b82.pem verifyhost ServerE.domain.org
4

The certificate and everything TLS/SSL related has nothing to do with your modified ACL, because the ACL you modified works at HTTP level (your are looking at the host header).

Can you clarify what “can not connect to serverB anymore” means exactly? Do you see the wrong certificate presented to you? Do you see an error message? What happens exactly when you can not connect to serverB anymore?

Note that if you are using a wildcard certificate, you will break the SNI routing in your SSL_Mux frontend: overlapping certificates don’t work with SNI routing, because a browser will reuse existing TLS sessions where the certificate covers the new request, causing the request to be routed to the wrong backend.

5

Hi again,

Thanks a lot for your answer. The error message I and SSLLabs.com see is that the client can not handshake a session. No common secure protocol can be identified.

As for the problem a wildcard will create in SSL_Mux, thanks for the info. It may well be either the source of the problem or it will hit me later down the road. From what you described, I will never be able to use a wildcard certificate the way I tried because some servers must use the accelerator while others must use the authenticator.

I will keep working my config keeping in mind that everything covered by a wildcard certificate must end up on the same backend from SSL_Mux’s point of view, that is either SSL_Accelerator or SSL_Authenticator but no mix between the two.

Does the error of no common secure protocol gives you more idea of what can be wrong ?

In all cases, a great thank for your support

6

The overlapping certificate issue would still allow the TLS connection to be successfully establish; it would definitely hit you further than down the road though, yes.

Sounds like that ECC wildcard certificate is bogus. Can you show the file with the ECC wildcard cert, without showing any content (just the BEGIN and END lines)?

7

Hi,

Thanks again for your valuable info. No need to send you the certificate; it is fine. The problem is in the config and I found it after you pointed me in the right direction :

There are only 4 ciphers selected, the same everywhere, and all of them are RSA. Not a single cipher for ECDSA, so HAProxy has no cipher usable with that certificate, so can not use it. All I need is to add at least one cipher usable for that certificate and HAProxy will use it.

Despite this, I will discard that certificate because as you told me, a wildcard certificate can not work in my setup and will confuse SSL_Mux. As I need SSL_Mux to work properly, I have to provide certificates with complete CNs instead of a wildcard.

Million thanks for your help,

1 Like