My SNI routing does not seem to work - is there some option to debug ACL rule?

Hello all,
I am trying to configure SNI routing using OPNSense HAProxy plugin (based on HAProxy 2.8).

After configuration in the GUI, it gives me this configuration:

# Automatically generated configuration.
# Do not edit this file manually.

    uid                         80
    gid                         80
    chroot                      /var/haproxy
    stats                       socket /var/run/haproxy.socket group proxy mode 775 level admin
    nbthread                    1
    hard-stop-after             60s
    no strict-limits
    tune.ssl.default-dh-param   2048
    spread-checks               2
    tune.bufsize                16384
    tune.lua.maxmem             0
    log                         /var/run/log local0 debug
    lua-prepend-path            /tmp/haproxy/lua/?.lua

    log     global
    option redispatch -1
    timeout client 30s
    timeout connect 30s
    timeout server 30s
    retries 3
    default-server init-addr last,libc

# autogenerated entries for ACLs

# autogenerated entries for config in backends/frontends

# autogenerated entries for stats

# Frontend: public_https (HTTPS Port)
frontend public_https
    bind name 
    mode tcp
    default_backend backend_https

    # logging options
    option tcplog

# Frontend: public_http (HTTP Port)
frontend public_http
    bind name 
    mode http
    option http-keep-alive
    default_backend backend_http

    # logging options
    option httplog

# Backend: backend_https (Backend HTTPS)
backend backend_https
    # health checking is DISABLED
    mode tcp
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m  
    stick on src
    # ACL: iot_myserver_org_condition_sni
    acl acl_65a84ebeebab75.87915974 req.ssl_sni -m sub -i
    # ACL: server_myserver_org_condition_sni
    acl acl_65a84ed2d27713.21832231 req.ssl_sni -m sub -i

    # ACTION: iot_myserver_org_https_sni_rules
    use-server iot_myserver_org_server_https if acl_65a84ebeebab75.87915974
    # ACTION: server_myserver_org_https_sni_rules
    use-server server_myserver_org_server_https if acl_65a84ed2d27713.21832231
    server iot_myserver_org_server_https 
    server server_myserver_org_server_https 

# Backend: backend_http (Backend HTTP)
backend backend_http
    # health checking is DISABLED
    mode http
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m  
    stick on src
    # ACL: iot_myserver_org_condition_host
    acl acl_65a851411fc3d3.35763680 hdr_sub(host) -i
    # ACL: server_myserver_org_condition_host
    acl acl_65a85167495161.41598361 hdr_sub(host) -i

    # ACTION: iot_myserver_org_http_host_rules
    use-server iot_myserver_org_server_http if acl_65a851411fc3d3.35763680
    # ACTION: server_myserver_org_http_host_rules
    use-server server_myserver_org_server_http if acl_65a85167495161.41598361
    http-reuse safe
    server iot_myserver_org_server_http 
    server server_myserver_org_server_http 

listen local_statistics
    mode            http
    stats uri       /haproxy?stats
    stats realm     HAProxy\ statistics
    stats admin     if TRUE

# remote statistics are DISABLED

When I am trying to access both servers, it always redirect me to the same server (yesterday it actually always redirect me to while today is even I have not changed anything in the conf). I would suspect my ACL rule are not working and it is just doing load balancing.
Both servers are running nginx. I enabled debug logging on both. I can confirm the SNI is sent through the TLS connections.

Example from this morning, I am trying to access curl and on nginx logs:

2024/01/18 08:53:27 [debug] 4317#4317: epoll: fd:6 ev:0001 d:00007FB850AF2100
2024/01/18 08:53:27 [debug] 4317#4317: accept on, ready: 0
2024/01/18 08:53:27 [debug] 4317#4317: posix_memalign: 000055678966F830:512 @16
2024/01/18 08:53:27 [debug] 4317#4317: *2 accept: fd:15
2024/01/18 08:53:27 [debug] 4317#4317: *2 event timer add: 15: 60000:126347578
2024/01/18 08:53:27 [debug] 4317#4317: *2 reusable connection: 1
2024/01/18 08:53:27 [debug] 4317#4317: *2 epoll add event: fd:15 op:1 ev:80002001
2024/01/18 08:53:27 [debug] 4317#4317: epoll del event: fd:6 op:2 ev:00000000
2024/01/18 08:53:27 [debug] 4317#4317: epoll add event: fd:6 op:1 ev:10000001
2024/01/18 08:53:27 [debug] 4317#4317: timer delta: 314684
2024/01/18 08:53:27 [debug] 4317#4317: worker cycle
2024/01/18 08:53:27 [debug] 4317#4317: epoll timer: 60000
2024/01/18 08:53:27 [debug] 4317#4317: epoll: fd:15 ev:0001 d:00007FB850AF24C0
2024/01/18 08:53:27 [debug] 4317#4317: *2 http check ssl handshake
2024/01/18 08:53:27 [debug] 4317#4317: *2 http recv(): 1
2024/01/18 08:53:27 [debug] 4317#4317: *2 https ssl handshake: 0x16
2024/01/18 08:53:27 [debug] 4317#4317: *2 tcp_nodelay
2024/01/18 08:53:27 [debug] 4317#4317: *2 reusable connection: 0
2024/01/18 08:53:27 [debug] 4317#4317: *2 SSL server name: ""
2024/01/18 08:53:27 [debug] 4317#4317: *2 SSL ALPN supported by client: h2
2024/01/18 08:53:27 [debug] 4317#4317: *2 SSL ALPN supported by client: http/1.1
2024/01/18 08:53:27 [debug] 4317#4317: *2 SSL ALPN selected: http/1.1
2024/01/18 08:53:27 [debug] 4317#4317: *2 SSL_do_handshake: -1
2024/01/18 08:53:27 [debug] 4317#4317: *2 SSL_get_error: 2

So the SNI seems to be correct (SSL server name: "")

The corresponding log entry in HAProxy is:

2024-01-18T08:53:27	Informational	haproxy [18/Jan/2024:08:53:27.851] public_https backend_https/iot_codeur_org_server_https 1/0/115 4302 -- 1/1/0/0/0 0/0

It would be nice to have a setting in HAProxy where the ACL rule can be debugged (for instance for each frontend/backend get the result of the rule). At the moment the debug seems to only for custom Lua code: Observability | Logging | Setting Log Levels | HAProxy Enterprise 2.8r1

… And if someone has an idea what is my issue.

The configuration is incomplete, you need to wait for the ssl client_hello to be in the buffer, otherwise you can’t reliable route based on the SNI value if sometimes it’s there and sometimes not.

Put this into your https frontend section:

# Wait for a client hello for at most 5 seconds
tcp-request inspect-delay 5s
tcp-request content accept if { req.ssl_hello_type 1 }
1 Like

Thanks a lot @lukastribus it works :slight_smile:

(the setting to debug ACL rules would still be nice and useful :wink: )

You can log values, for example if you are troubleshooting SNI routing, log SNI to the logs and then you will see that it’s empty.

Thanks @lukastribus , what is missing to my configuration file to see such logs? I could not see how to trace such things.


1 Like

Thanks, I also found this blog: Surfacing ACL Actions in HAProxy Logs | by Carlo Mencarelli | Medium