Using HAProxy to forward syslogs from rsyslog causing TCPSendBuf error

Help!
1

Hi all,

I am trying to use HAProxy to receive syslogs from Linux servers running rsyslog (tcp) and send them load balanced to multiple backend Logstash servers. While this works and logs are making it to Logstash, I am having issues with rsyslog showing the following error and it drops logs. This does not happen if I connect directly to Logstash. The error is happening every 50 seconds to 1 min consistently. I need a solid connection as these logs are critical. It looks like the drop is coming from HAProxy.

I really need some help with this. Also I need to add in UDP.

The error I receive in rsyslog on the Linux source is shown below

rsyslogd[512557]: omfwd: TCPSendBuf error -2027, destruct TCP Connection to haproxy-fe.example.com:5066

My haproxy.cfg is as follows:

global
    log /dev/log local0
    log-tag haproxy
    chroot /var/lib/haproxy
    user haproxy
    group haproxy
    daemon

defaults
    log     global
    mode    tcp
    option  tcplog
    timeout connect 5000ms
    timeout client  50000ms
    timeout server  50000ms

frontend frontend_5066
    bind *:5066
    mode tcp
    default_backend backend_5066

backend backend_5066
    mode tcp
    balance roundrobin
    server logstash-s1 logstash-s1.example.com:5066 check
    server logstash-s2 logstash-s2.example.com:5066 check
    server logstash-s3 logstash-s3.example.com:5066 check
    server logstash-s4 logstash-s4.example.com:5066 check
2

Maybe you should consider using a combination of log-forward section and log backend (using “mode log” in a backend section) since they may better fit your needs for log-oriented balancing. They do support both TCP and UDP along the chain.