Stats not reporting any Requests to backends

Help!
1

Hello all,

I am using haproxy as a TCP load-balancer and TLS terminator, and I am trying to measure how many connections and requests are being served per domain and backend. The weird thing is that no matter how many requests it reports for the frontends, there are always non-values for the backends.

I am watching the req_rate (47th) req_rate_max (48th) columns.

It does report a few established connections (scur, 5th column) though. The backends (that are hosted in different servers) do receive requests from haproxy.

Here is how I am getting the stats:

while true; do echo "show stat" | sudo netcat -U /run/haproxy/admin.sock | tail -n +3 | awk -F ',' '{print $1","$2","$5","$47","$48}' ; sleep 1; done

which gives (trimmed):

stats,BACKEND,0,,
vip-217,FRONTEND,1056,130,1553
vip-218,FRONTEND,0,0,0
internal-203,FRONTEND,0,0,0
internal-204,FRONTEND,0,0,0
p2p-tracker-prd,multi1,1,,
ch-nl-7,ch94,0,,
ch-nl-7,ch95,0,,
ch-nl-7,BACKEND,0,,
ch-nl-8,ch27,3,,
ch-nl-8,ch28,2,,
...

I would expect more connections (aka: scur which is $5) too.

And in order to filter out the null values:

while true; do echo "show stat" | sudo netcat -U /run/haproxy/admin.sock | tail -n +3 | awk -F ',' '{if ($1 != "" && $2 != "" && $5 != "" && $47 != "" && $48 != "") print $1","$2","$5","$47","$48}' ; sleep 1; done

which reports only the frontends, because the backend have empty values for req_rate and max_req_rate:

vip-217,FRONTEND,1016,164,1553
vip-218,FRONTEND,0,0,0
internal-203,FRONTEND,0,0,0
internal-204,FRONTEND,0,0,0

The haproxy.conf has:

listen stats
        bind 10.0.19.3:8888
        stats uri /haproxy?stats
        stats auth user:pass
        stats refresh 5s
        acl network_allowed src 10.0.0.0/16
        tcp-request connection reject if !network_allowed

and

global
        master-worker

        nbthread 28

        maxconn 1000000
        tune.ssl.cachesize 5000000

	log /dev/log	local0
	log /dev/log	local1 notice
        no log
	chroot /var/lib/haproxy
	stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd listeners
	stats timeout 30s
	user haproxy
	group haproxy
	daemon

	# Default SSL material locations
	ca-base /etc/ssl/certs
	crt-base /etc/ssl/private

defaults
        no log
	mode	http
	option	dontlognull
        timeout connect 5000
        timeout client  500000
        timeout server  500000
        option redispatch
        retries 3
        maxconn 300000

Sample frontend conf:

frontend vip-217
        bind pubipv4:80
        bind pubipv4:443 ssl crt /etc/haproxy/certs transparent
        bind pubipv6:80 transparent
        bind pubipv6:443 ssl crt /etc/haproxy/certs transparent
        mode http
        option forwardfor
        http-request add-header X-Forwarded-Proto https
        maxconn 300000
        timeout client 370000
        compression algo gzip
        compression type text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript
        
        # ... some ACLs here like : ....
        acl isAdminAPI hdr(host) -i domain1
        acl isAdminAPIStage hdr(host) -i domain2
        acl isAdminAPIBeta hdr(host) -i domain3
        acl isDateToken urlp(dateToken) -m found
        use_backend back1 if isAdminAPI
        use_backend back2 if isAdminAPIStage
        use_backend back3 if isAdminAPIBeta
        use_backend back4 if isDateToken
        use_backend %[req.hdr(host),lower,map_dom(/etc/haproxy/domain2backend.map,bk_default)]

sample backend conf:

backend back1
        mode http
        option splice-auto
        option forwardfor
        option abortonclose

        option http-server-close #SCL: server close

        timeout connect 5000
        timeout server  90000

        errorfile 408 /dev/null

        balance source
        hash-type consistent

        server server1 10.0.18.1:8080 check
        server server2 10.0.19.9:8080 check

Any ideas to troubleshoot this?

2

That info is only available for Frontends :cry: :cry: :cry: