HAProxy with IIS CCS

Help!
1

Hi there. I’ve been trying to configure HAProxy to balance sadly old IIS sites using CCS (Centralized Certificate Store) feature without success. I get an SSL handshake failure.
Disabling CCS on the same site binding and selecting the same certificate manually all works fine.

_version=2187
Dataplaneapi managed File
changing file directly can cause a conflict if dataplaneapi is running

global
daemon
chroot /var/lib/haproxy
user haproxy
group haproxy
master-worker
nbproc 1
nbthread 2
cpu-map auto:1/1-2 0-1
maxconn 100000
stats socket /run/haproxy/admin.sock user haproxy group haproxy mode 660 level admin
stats timeout 2m
tune.ssl.default-dh-param 2048
ssl-default-bind-options no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets
ssl-default-bind-ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384
ssl-default-server-options no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets
ssl-default-server-ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384
log 127.0.0.1:514 local0
description HAProxy Balancer
ca-base /etc/ssl/certs
crt-base /etc/ssl/private
tune.ssl.cachesize 1000000

defaults
mode http
maxconn 100000
log global
option httplog
cookie serverid indirect nocache insert
no option http-use-htx
option redispatch
option dontlognull
option forwardfor
timeout connect 5s
timeout client 50s
timeout server 50s
retries 3
errorfile 400 /etc/haproxy/errors/400.http
errorfile 403 /etc/haproxy/errors/403.http
errorfile 408 /etc/haproxy/errors/408.http
errorfile 500 /etc/haproxy/errors/500.http
errorfile 502 /etc/haproxy/errors/502-custom.http
errorfile 503 /etc/haproxy/errors/503-custom.http
errorfile 504 /etc/haproxy/errors/504.http

userlist controller
user user password ###

peers lb-peers
peer haproxy1 haproxy1:1024
peer haproxy2 haproxy2:1024

frontend admin-statistics-frontend
mode http
bind :9099
stats enable
stats uri /
stats realm Haproxy\ Statistics
stats auth ##:##
stats show-node
stats show-legends
stats refresh 300s
stats admin if TRUE
stats show-desc

frontend f1
bind 192.168.20.101:80
redirect scheme https code 301 if !{ ssl_fc }

frontend f2
bind 192.168.20.101:443 ssl crt /etc/ssl/certificates/pem alpn h2,http/1.1
acl site1-acl hdr(host) -i site1
acl site2-acl hdr(host) -i site2
acl site3-acl hdr(host) -i site3
http-request set-header x-forwarded-proto https
http-request set-header X-ARR-SSL greatscott
use_backend site1 if site1-acl
use_backend site2 if site2-acl
use_backend site3 if site3-acl

backend site1
mode http
option httpchk GET https://site1.domain.com
server server1 server1:443 enabled weight 1 cookie dQBhAHQAMQAtADcA check ssl verify required ca-file /etc/ssl/certs/ca-certificates.crt

backend site2
mode http
option httpchk GET https://site2.domain.com
server server2 server2:443 enabled weight 1 cookie dQBhAHQAMQAtADcA check ssl verify required ca-file /etc/ssl/certs/ca-certificates.crt

backend site3
mode http
option httpchk GET https://site3.domain.com
server server3 server3:443 enabled weight 1 cookie dQBhAHQAMQAtADcA check ssl sni req.hdr(host) verify required ca-file /etc/ssl/certs/ca-certificates.crt

The configuration of site3 is the one I’m trying with CCS.

Thanks in advance for any help you could give me.

2

Hi.

Skimming the information, I am a bit confused by
‘http-request set-header X-ARR-SSL greatscott’.
The X-ARP-SSL header usually contains the information about the SSL certificate used at the frontend e.g.
‘2048|256|C=US, S=Washington, L=Redmond, O=Microsoft Corporation, OU=Microsoft IT, CN=Microsoft IT SSL SHA2|CN=*.azurewebsites.net’

Although I am not an expert for IIS, I don’t think that it’ll make a difference using CCS over local certs from the proxys point of view. Are you sure you are using the same certificate via CCS?

3

That header is requested by those old applications since, as far as i remember, the ARR helper rewrite HTTPS to on if X-ARR-SSL contains the certificate subject info. Was a workaround to avoid refactoring.
And yes, setting the same certificate that CCS should pick up everything works fine.

4

Yeah, that is the point. Did you check it does pick that certificate?

5

The site is currently working fine with CCS but is not balanced, requests are made directly to the backend server and CCS picks up the correct certificate.

How can I check what certificate it picks up when requests pass through HAProxy?

Thx, lhr :wink:

6

Ok, I had not read out that you have already checked that the page without HAPROXY delivers the correct certificate via CCS. So we can safely rule out the certificate itself as the cause.

My suggestion would be to look at the handshake from IIS with and without CCS using wireshark/tcpdump. Maybe there is a clear difference there.

7

Apologize for the lateness.
This is working for me. CCS picks up the correct certificate
server server01 server01:443 enabled weight 1 cookie cookie01 check check-sni site.domain.com ssl sni req.hdr(host) verify required ca-file /etc/ssl/certs/ca-certificates.crt

Thanks again.

8

Hi Martin, you are welcome.
Of course, if you are using a completely different configuration in IIS with CCS (SNI vs not), then we can only make fun guesses here. But this has nothing to do with CCS but with a deviating IIS configuration!

You could have also used CCS without SNI which would not needed to change the HAPROXY config.