Haproxy 1.7.10 upgrade to 1.8.15 fails to start

Hello,

We currently run Haproxy 1.7.10 on Debian 9.4 Stretch. We have used Haproxy for Load Balancing for about 4 to 5 years now without much of an issue. We recently read that starting in 1.8 band Haproxy can be multi threaded when ran vs single threaded, so we want to try to take advantage of this.

Unfortunately I am running into some errors when upgrading to 1.8. I do not get any configuration errors, so I believe our haproxy.cfg file is good, but I will post the rest of the errors I am getting below. I can go through the upgrade process like we have done many times, but at the end the Haproxy service fails to start.

Blockquote
root@lab-p02:/usr/src/haproxy-1.8.15# haproxy -vv
HA-Proxy version 1.8.15 2018/12/13
Copyright 2000-2018 Willy Tarreau willy@haproxy.org
Build options :
TARGET = linux2628
CPU = generic
CC = gcc
CFLAGS = -O2 -g -fno-strict-aliasing -Wdeclaration-after-statement -fwrapv -Wno-null-dereference -Wno-unused-label
OPTIONS = USE_ZLIB=1 USE_OPENSSL=1 USE_PCRE=1
Default settings :
maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200
Built with OpenSSL version : OpenSSL 1.0.2q 20 Nov 2018
Running on OpenSSL version : OpenSSL 1.0.2q 20 Nov 2018
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : SSLv3 TLSv1.0 TLSv1.1 TLSv1.2
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
Encrypted password support via crypt(3): yes
Built with multi-threading support.
Built with PCRE version : 8.39 2016-06-14
Running on PCRE version : 8.39 2016-06-14
PCRE library supports JIT : no (USE_PCRE_JIT not set)
Built with zlib version : 1.2.8
Running on zlib version : 1.2.8
Compression algorithms supported : identity(“identity”), deflate(“deflate”), raw-deflate(“deflate”), gzip(“gzip”)
Built with network namespace support.
Available polling systems :
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use epoll.
Available filters :
[SPOE] spoe
[COMP] compression
[TRACE] trace

Blockquote
root@lab-p02:/usr/src/haproxy-1.8.15# service haproxy start
root@lab-p02:/usr/src/haproxy-1.8.15# service haproxy status
● haproxy.service - HAProxy Load Balancer
Loaded: loaded (/etc/systemd/system/haproxy.service; enabled; vendor preset: enabled)
Active: failed (Result: start-limit-hit) since Wed 2019-01-09 09:35:56 EST; 3s ago
Docs: man:haproxy(1)
file:/usr/share/doc/haproxy/configuration.txt.gz
Process: 23113 ExecStart=/usr/local/sbin/haproxy-systemd-wrapper -f ${CONFIG} -p /run/haproxy/haproxy.pid $EXTRAOPTS (code=exited, status=0/SUCCESS)
Process: 23111 ExecStartPre=/usr/local/sbin/haproxy -f ${CONFIG} -c -q (code=exited, status=0/SUCCESS)
Main PID: 23113 (code=exited, status=0/SUCCESS)
Jan 09 09:35:56 lab-p02 systemd[1]: haproxy.service: Service hold-off time over, scheduling restart.
Jan 09 09:35:56 lab-p02 systemd[1]: Stopped HAProxy Load Balancer.
Jan 09 09:35:56 lab-p02 systemd[1]: haproxy.service: Start request repeated too quickly.
Jan 09 09:35:56 lab-p02 systemd[1]: Failed to start HAProxy Load Balancer.
Jan 09 09:35:56 lab-p02 systemd[1]: haproxy.service: Unit entered failed state.
Jan 09 09:35:56 lab-p02 systemd[1]: haproxy.service: Failed with result ‘start-limit-hit’.

Blockquote
– The start-up result is done.
Jan 09 09:35:56 lab-p02 haproxy-systemd-wrapper[23113]: haproxy-systemd-wrapper: executing /usr/local/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /run/haproxy/haproxy.pid -Ds
Jan 09 09:35:56 lab-p02 haproxy-systemd-wrapper[23113]: haproxy-systemd-wrapper: exit, haproxy RC=0
Jan 09 09:35:56 lab-p02 systemd[1]: haproxy.service: Killing process 23116 (haproxy) with signal SIGKILL.
Jan 09 09:35:56 lab-p02 systemd[1]: haproxy.service: Service hold-off time over, scheduling restart.
Jan 09 09:35:56 lab-p02 systemd[1]: Stopped HAProxy Load Balancer.
– Subject: Unit haproxy.service has finished shutting down
– Defined-By: systemd
– Support: Debian -- User Support

– Unit haproxy.service has finished shutting down.
Jan 09 09:35:56 lab-p02 systemd[1]: haproxy.service: Start request repeated too quickly.
Jan 09 09:35:56 lab-p02 systemd[1]: Failed to start HAProxy Load Balancer.
– Subject: Unit haproxy.service has failed
– Defined-By: systemd
– Support: Debian -- User Support

– Unit haproxy.service has failed.

Also if I roll back the build to 1.7.10, I do not have the starting error. So it appears something from 1.8 and on has changed.

Thanks

1.7 and 1.8 has a very different startup procedure, that’s why you need:

  • compile haproxy 1.8 with systemd support (USE_SYSTEMD=1)
  • update the systemd unit file (in your case, cp contrib/systemd/haproxy.service.in /etc/systemd/system/haproxy.service && systemctl daemon-reload)
  • remove the old haproxy-systemd-wrapper executable, which doesn’t exist in 1.8 anymore

Thank you for the reply. I am still having a few issues, so I am just trying to make sure I understand what your saying. I need to compile Haproxy with USE_SYSTEMD=1:

make TARGET=linux2628 USE_PCRE=1 USE_OPENSSL=1 USE_ZLIB=1 LDFLAGS=-ldl SSL_INC=/opt/openssl/include SSL_LIB=/opt/openssl/lib USE_SYSTEMD=1

Then you say to replace the Haproxy.service file from /etc/system/system/ then replace it from the contrib/systemd/haproxy.service.in?

Thanks

Yes.

Replace the old system unit file from haproxy 1.7, which in your case is in /etc/systemd/system/haproxy.service with the updated file from haproxy 1.8.

Also delete haproxy-systemd-wrapper in your path.

Thank you for the help. I was able to get Haproxy to start with these steps.

Thanks