DNS changes in 1.7.3+ break UNIX socket stats when resolvers are configured


#1

Greetings,

With HAProxy 1.7.3 and later on FreeBSD, recent DNS-related code changes in HAProxy appear to have broken the UNIX socket in daemon mode when resolvers are present in the configuration.

How to reproduce:

  • Install HAProxy 1.7.x (where x > 2) on FreeBSD 10.3 or FreeBSD 11, even HAProxy 1.7.5

  • Configure HAProxy to provide a UNIX socket for stats:
    stats socket /tmp/haproxy.socket level admin

  • Configure HAProxy with resolvers:

      resolvers globalresolvers
      	nameserver localdns 10.6.0.1:53
      	resolve_retries 3
      	timeout retry 1s
      	hold valid 10s
    
  • Start haproxy

  • Attempt to grab some stats from the UNIX socket (ANY stats, not just resolver stats!):
    echo "show stat server" | nc -U /tmp/haproxy.socket

  • The request never completes, it hangs indefinitely. The above command is a shorthand way, using it interactively also fails.

If I revert 91a964aae7a405f2752f8be22d669745caa0c16f eaf96d7a0849b2883e98459f52489d555b6b013c from the HAProxy source and rebuild HAProxy, it works as expected. The command succeeds and it yields proper output.

If the resolvers section is removed, it works with and without those commits applied. The resolvers do not even have to be added to any backend, only defined in the configuration. Adding multiple nameserver entries does not change the behavior. Removing the resolver parameters other than nameserver also does not change the behavior.

If the daemon is started in the foreground (without -D, or with -V -db and so on), it also works. It appears to only be a problem when using daemon mode (-D).

Here is the full configuration. It’s quite basic, only a test setup.

global
	maxconn			1000
	stats socket /tmp/haproxy.socket level admin
	uid			80
	gid			80
	nbproc			1
	chroot			/tmp/haproxy_chroot
	daemon
	tune.ssl.default-dh-param	2048
	server-state-file /tmp/haproxy_server_state

listen HAProxyLocalStats
	bind 127.0.0.1:2200 name localstats
	mode http
	stats enable
	stats admin if TRUE
	stats uri /haproxy/haproxy_stats.php?haproxystats=1
	timeout client 5000
	timeout connect 5000
	timeout server 5000

resolvers globalresolvers
	nameserver localdns 10.6.0.1:53
	resolve_retries 3
	timeout retry 1s
	hold valid 10s

frontend doc-front
	bind			0.0.0.0:4443 name 0.0.0.0:4443 ssl  crt /var/etc/haproxy/doc-front.pem  
	mode			http
	log			global
	option			http-keep-alive
	timeout client		30000
	default_backend doc-back_http_ipvANY

backend doc-back_http_ipvANY
	mode			http
	log			global
	timeout connect		30000
	timeout server		30000
	retries			3
	option			httpchk GET / 
	server			doctor 10.6.0.10:80 check inter 1000  resolvers globalresolvers 

Thanks in advance for any insight!
Jim


#2

For the record, this was analyzed on the mailing list:
https://www.mail-archive.com/haproxy@formilux.org/msg26040.html

and fixed in 6492053 (“BUG/MAJOR: dns: Broken kqueue events handling (BSD systems)”):
http://git.haproxy.org/?p=haproxy.git;a=commitdiff;h=64920538fc19f3c7f94dfa1e84a9a6569b8c3d37

1.7.6 will have this bugfix.

You do wanna use uptodate releases for FreeBSD, as 1.7.6 will contain other relevant fixes (“BUG/MAJOR: Use -fwrapv” or “BUG/MAJOR: stream-int: do not depend on connection flags to detect connection”), as opposed to 1.7.2.