Haproxy throwing intermittent 503, Backend server not available for layer6 timeout

Help!
1

Hi Team
Please we need your help to resolve an issue we are having. We are getting 503 error intermittently for our servers.

haproxy configuration:
#log 127.0.0.1 local2 debug
log 127.0.0.1:514 local0 debug
chroot /var/lib/haproxy
pidfile /var/run/haproxy.pid
maxconn 10000
user haproxy
group haproxy
daemon
debug
tune.ssl.default-dh-param 2048

turn on stats unix socket

stats socket /var/lib/haproxy/stats
default

mode                    http
log                     global
option                  httplog
option                  dontlognull
option http-server-close
option forwardfor       except 127.0.0.0/8
option                  redispatch
retries                 3
timeout http-request    30s
timeout queue           1m
timeout connect         30s
timeout client          1m
timeout server          1m
timeout http-keep-alive 30s
timeout check           30s
maxconn                 10000

backend activation-backend
balance roundrobin
http-send-name-header Host
http-response add-header XX-powered-by “West-1”
server XXXXXXX.com:443 XXXXXXX.com:443 check ssl verify none

output : for haproxy -vv
HA-Proxy version 1.8.27-493ce0b 2020/11/06
Copyright 2000-2020 Willy Tarreau willy@haproxy.org

Build options :
TARGET = linux2628
CPU = generic
CC = gcc
CFLAGS = -O2 -g -fno-strict-aliasing -Wdeclaration-after-statement -fwrapv -W no-null-dereference -Wno-unused-label -Wno-stringop-overflow
OPTIONS = USE_LINUX_TPROXY=1 USE_CRYPT_H=1 USE_GETADDRINFO=1 USE_ZLIB=1 USE_RE GPARM=1 USE_OPENSSL=1 USE_LUA=1 USE_SYSTEMD=1 USE_PCRE=1

Default settings :
maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with OpenSSL version : OpenSSL 1.1.1g FIPS 21 Apr 2020
Running on OpenSSL version : OpenSSL 1.1.1k FIPS 25 Mar 2021
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
Built with Lua version : Lua 5.3.4
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_F REEBIND
Encrypted password support via crypt(3): yes
Built with multi-threading support.
Built with PCRE version : 8.42 2018-03-20
Running on PCRE version : 8.42 2018-03-20
PCRE library supports JIT : no (USE_PCRE_JIT not set)
Built with zlib version : 1.2.11
Running on zlib version : 1.2.11
Compression algorithms supported : identity(“identity”), deflate(“deflate”), raw -deflate(“deflate”), gzip(“gzip”)
Built with network namespace support.

Available polling systems :
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use epoll.

Available filters :
[SPOE] spoe
[COMP] compression
[TRACE] trace

errors: May 26 10:16:31 localhost haproxy[4083320]: Server xxxxx-backend/XXXXX.com:443 is DOWN, reason: Layer6 timeout, check duration: 2002ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue
Message from syslogd@localhost at May 30 02:32:33 …
haproxy[104161]: backend activation-backend has no server available!

any possible solutions to resolve the issue.

2

Hi,

From the error message, it seems HAProxy was not able to establish an SSL connection with the server. TCP worked, but not SSL. Maybe your server expect an SNI. In such case, you may want to try this: HAProxy version 2.4.15 - Configuration Manual
Use the value behind the XXXXXXX.com