QUIC bind fails, TCP is OK

Help!
1

Hello, haproxy users,

I am trying to setup haproxy as a frontend instead of IPVS. I am not sure what is the recommended way of hardening the environment in which haproxy runs - whether to let systemd drop privileges and switch to a non-root user, or let haproxy itself to do the work. Systemd list of hardening features is much larger, so I prefer that. So far it works for me except for the QUIC socket. I am getting the following error:

Jan  9 21:44:22 localhost haproxy[1585673]: Permission error on QUIC socket binding for proxy myservice.fqdn. Consider using setcap cap_net_bind_service (Linux only) or running as root.

The problem is that I get this error only when the first HTTP3 request arrives, and that TCP sockets (HTTP 1 and 2) :80 and :443 work without problem. It seems that haproxy binds the TCP sockets before dropping privileges, but QUIC socket is created only later (I have verified this using strace).

One set of options which works for me is to let systemd switch to a non-root user, and add the CAP_NET_BIND_SERVICE capability to the process (not a file):

[Service]
...
Type=notify
User=haproxy
Group=haproxy
AmbientCapabilities=CAP_NET_BIND_SERVICE

So - why does the QUIC socket get bound so late? And what is the recommended way of dropping privileges of haproxy process?

Thanks,

-Yenya

2

You can find the suggested systemd unit file in the tarball at admin / systemd / haproxy.service.in.
Haproxy must be startet as root and will drop privileges based on the user/group haproxy configuration.

Dropping privileges in systemd is not supported.

3

As you may know, TCP and UDP does not have the same POSIX API. For TCP, each new connection is assigned to its owned socket thanks to accept() syscall on the server side. This syscall is not available for UDP, traditionnaly multiple connections are multiplexed over a single socket.

However, this is not scalable, in particular if there is a lot of parallel connections. For this reason, haproxy by default assign a new socket for each QUIC connection. This requires manual invocation of socket() + bind() at runtime, which is the reason why the capability net_admin is required if a privileged port is used. Note that this behavior can be changed with the following option to only use the listener socket for every connections :
https://docs.haproxy.org/dev/configuration.html#tune.quic.socket-owner

However, as I already mentionned, this is not scalable if the number of threads and/or parallel connections is important.

4

I’m experimenting with HTTP/3 on Ubuntu 24.04 and HAProxy 3.2 from the official -awslc packages: Download the Official HAProxy Performance Packages and got it working.

I didn’t need to add AmbientCapabilities=CAP_NET_BIND_SERVICE in the service file but did need to add setcap cap_net_bind_service under the “global” section in haproxy.cfg. AIUI it passes the capability to the non-root user when it switches from root to non-root user.

Another, unrelated gotcha I ran into was with incorrect alpn values. I originally had:

bind *:443 ssl crt-list /etc/haproxy/crt-list.txt
bind quic4@:443 ssl crt-list /etc/haproxy/crt-list.txt

And in crt-list.txt I had:

hostname1.pem.rsa [alpn h2,http/1.1,http/1.0 ssl-min-ver TLSv1.0]
hostname2.pem.rsa [alpn h2,http/1.1,http/1.0 ssl-min-ver TLSv1.2]
...

The reason I’m using crt-list like so is because I want to have different minumum TLS requirements for different hostnames. The issue is of course that the HTTP/3 connections will see the wrong alpn value.

The fix was to move the alpn stuff back to haproxy.cfg:

bind *:443 ssl crt-list /etc/haproxy/crt-list.txt alpn h2,http/1.1,http/1.0
bind quic4@:443 ssl crt-list /etc/haproxy/crt-list.txt

And crt-list.txt:

hostname1.pem.rsa [ssl-min-ver TLSv1.0]
hostname2.pem.rsa [ssl-min-ver TLSv1.2]
...