We check stats of the haproxy by using this command on the host it runs on
echo show stat | sudo nc -U /var/run/api.sock
This works well when using the backend server configuration. The current use for HAProxy is to only use it for sending logs, so we don’t use the backend section, and only use this configuration
log-forward syslog
# Listen on TCP port 514 for incoming TCP log messages
bind 0.0.0.0:6514
log ring@logbuffer local0
ring logbuffer
description "buffer for logs"
format rfc5424
maxlen 1500
size 65536
timeout connect 10s
timeout server 20s
server filebeat-1 127.0.0.1:6515 check
server filebeat-1 127.0.0.1:6516 check
server filebeat-1 127.0.0.1:6517 check
server filebeat-1 127.0.0.1:6518 check
This works well, however now when checking the stats with the first command, we see no servers or stats. I think this is expected as this tracks only backend servers, which are no longer configured.
Is there a way to see same type of stats output for the ringbuffer like I saw with the server backend using cli commands?
Indeed I don’t know about any stats exposed for ring servers
log backends were recently introduced in haproxy 3.0. Those are backends with “mode log”:
https://docs.haproxy.org/dev/configuration.html#4.2-mode in case you’re interested in that feature, although it looks like you rely on logs being duplicated to multiple servers, which log backends don’t offer (they are designed to distribute the load over multiple servers instead)
Such backends accept UDP or TCP servers (tcp servers are backed by the ring API). And they are made specifically for balancing the logs between multiple endpoints. It also opens the door to servers management and stats like for regular TCP/HTTP servers.
However for now only some stats are exposed because log traffic oriented stats are not accounted for.
After reading your response, I realized that ring buffer was not what I wanted to use at all. I was not aware of the log backends. log backends was exactly what I wanted. I want to distribute syslog data across nodes, not duplicate. That was my mistake. Your solution was exactly what I was looking for. Thank you for pointing that out. After switching to log backend I the backend stats show up again from /var/run/api.sock, perfect.
Curious, is there a reason why the bin and bout show 0 for all values when using log backend servers? I would think they should show aggregates of syslog bytes to each node, but they just stay at 0. I get that with mode tcp it would only show when a session ended, but I would think this would be showing me rates as logs are distributed. Is this expected?
1 Like
Yes indeed for now some stats are lacking because the log processing chain is completely different from the tcp/http one internally, we will need to collect more stats in the log backends, and either link them to existing metrics such as bin:bout on the general backend stats, or provide log-specific backends stats. We will get there eventually