"Backend has no server available!" redirect to haproxy.log

Help!
1

I’ve enabled using the backend server with the “check” option to see when it’s down/up in the stats page. However, whenever the backend server is taken down for a restart or maintenance, the output is genereated on the main SSH console instead of redirecting it to the haproxy logfile:

Message from syslogd@localhost at Oct 31 06:16:13 ...
   haproxy[18509]: backend backend-server has no server available!

I’m using this option in the global configuration section:

log         127.0.0.1 local2 debug

In the rsyslog.conf I’ve made the following modifications to redirect haproxy logging to /var/log/haproxy.log - enabled UDP module to receive activity from Haproxy, and redirected local2 to the desired logfile. All other activity, like handshake errors, tcplog goes there, but this notification somehow ended up in the main SSH console :smile:

# Provides UDP syslog reception
$ModLoad imudp
$UDPServerRun 514
$AllowedSender UDP, 127.0.0.1

# Save HAProxy messages to haproxy.log
local2.*                                                /var/log/haproxy.log

Any help is appreciated.

2

Can you share the output of haproxy -vv and, very important, the startup script you use (whether it is init.d or systemd)?

Also do you have multiple servers per backend or just one? The message that servers X goes down, goes to the right place, only the message indicating that no servers are available goes somewhere else?

3

Hi Lukas,

I’m using the built-in version HAProxy version 1.5.18 that comes in the CentOS 7 repositories, installed via yum, I’m using the standard systemd unit file that this creates.

I only have one backend, and haproxy is only used in TLS termination proxy mode for encryption/decription for this only 1 backend server.

Here’s the output:

HA-Proxy version 1.5.18 2016/05/10
Copyright 2000-2016 Willy Tarreau <willy@haproxy.org>

Build options :
  TARGET  = linux2628
  CPU     = generic
  CC      = gcc
  CFLAGS  = -O2 -g -fno-strict-aliasing -DTCP_USER_TIMEOUT=18
  OPTIONS = USE_LINUX_TPROXY=1 USE_ZLIB=1 USE_REGPARM=1 USE_OPENSSL=1 USE_PCRE=1

Default settings :
  maxconn = 2000, bufsize = 16384, maxrewrite = 8192, maxpollevents = 200

Encrypted password support via crypt(3): yes
Built with zlib version : 1.2.7
Compression algorithms supported : identity, deflate, gzip
Built with OpenSSL version : OpenSSL 1.0.2k-fips  26 Jan 2017
Running on OpenSSL version : OpenSSL 1.0.2k-fips  26 Jan 2017
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports prefer-server-ciphers : yes
Built with PCRE version : 8.32 2012-11-30
PCRE library supports JIT : no (USE_PCRE_JIT not set)
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND

Available polling systems :
      epoll : pref=300,  test result OK
       poll : pref=200,  test result OK
     select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

Here’s the system.d unit file contents:

[Unit]
Description=HAProxy Load Balancer
After=syslog.target network.target

[Service]
EnvironmentFile=/etc/sysconfig/haproxy
ExecStart=/usr/sbin/haproxy-systemd-wrapper -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid $OPTIONS
ExecReload=/bin/kill -USR2 $MAINPID
KillMode=mixed

[Install]
WantedBy=multi-user.target
4

This message will be sent to stderr if one of the following is true:

  • haproxy is currently starting or
  • haproxy is configured with the verbose option or
  • haproxy is not configured with the quiet option

However, in every case, the same message is also send to syslog. So you should have the same message in the usual place also.

If you want to configure haproxy to not emit this message to stderr at runtime, put quiet into your global settings.

1 Like
5

Thanks a lot, will try try out this solution :). I’d better go with the quiet option, since we’re monitoring the backend instance separately anyway via Prometheus and Nagios, and logging to stderr seems to me appropriate only for kernel messages.