Unable to use systemctl reload haproxy (help enabling systemd?)

Hello,

I installed HAProxy 2.2.2 using the below article on two CentOS 7 servers:

make TARGET=linux-glibc USE_PCRE=1 USE_OPENSSL=1 USE_ZLIB=1 USE_CRYPT_H=1 USE_LIBCRYPT=1
sudo make install
sudo mkdir -p /etc/haproxy
sudo mkdir -p /var/lib/haproxy
sudo touch /var/lib/haproxy/stats
sudo ln -s /usr/local/sbin/haproxy /usr/sbin/haproxy
sudo cp examples/haproxy.init /etc/init.d/haproxy
sudo chmod 755 /etc/init.d/haproxy
sudo systemctl daemon-reload
sudo chkconfig haproxy on
sudo useradd -r haproxy

And then clustered them (Pacemaker/Corosync) for high availability

However (months on) i’ve just found out the systemctl reload haproxy command does not work - I am unable to reload the config without restarting haproxy via systemctl restart haproxy.

I found another thread in this forum saying it’s due to HAProxy not being compiled using USE_SYSTEMD=1 and is using init.d which is also confirmed from here:

haproxy -vv
HA-Proxy version 2.2.2 2020/07/31 - https://haproxy.org/
Status: long-term supported branch - will stop receiving fixes around Q2 2025.
Known bugs: http://www.haproxy.org/bugs/bugs-2.2.2.html
Running on: Linux 3.10.0-1127.13.1.el7.x86_64 #1 SMP Tue Jun 23 15:46:38 UTC 202                                                                                                             0 x86_64
Build options :
  TARGET  = linux-glibc
  CPU     = generic
  CC      = gcc
  CFLAGS  = -O2 -g -Wall -Wextra -Wdeclaration-after-statement -fwrapv -Wno-unus                                                                                                             ed-label -Wno-sign-compare -Wno-unused-parameter -Wno-clobbered -Wno-missing-fie                                                                                                             ld-initializers -Wtype-limits
  OPTIONS = USE_PCRE=1 USE_LIBCRYPT=1 USE_CRYPT_H=1 USE_OPENSSL=1 USE_ZLIB=1

Feature list : +EPOLL -KQUEUE +NETFILTER +PCRE -PCRE_JIT -PCRE2 -PCRE2_JIT +POLL                                                                                                              -PRIVATE_CACHE +THREAD -PTHREAD_PSHARED +BACKTRACE -STATIC_PCRE -STATIC_PCRE2 +                                                                                                             TPROXY +LINUX_TPROXY +LINUX_SPLICE +LIBCRYPT +CRYPT_H +GETADDRINFO +OPENSSL -LUA                                                                                                              +FUTEX +ACCEPT4 +ZLIB -SLZ +CPU_AFFINITY +TFO +NS +DL +RT -DEVICEATLAS -51DEGRE                                                                                                             ES -WURFL **-SYSTEMD** -OBSOLETE_LINKER +PRCTL +THREAD_DUMP -EVPORTS

Default settings :
  bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with multi-threading support (MAX_THREADS=64, default=4).
Built with OpenSSL version : OpenSSL 1.0.2k-fips  26 Jan 2017
Running on OpenSSL version : OpenSSL 1.0.2k-fips  26 Jan 2017
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : SSLv3 TLSv1.0 TLSv1.1 TLSv1.2
Built with network namespace support.
Built with zlib version : 1.2.7
Running on zlib version : 1.2.7
Compression algorithms supported : identity("identity"), deflate("deflate"), raw                                                                                                             -deflate("deflate"), gzip("gzip")
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_F                                                                                                             REEBIND
Built with PCRE version : 8.32 2012-11-30
Running on PCRE version : 8.32 2012-11-30
PCRE library supports JIT : no (USE_PCRE_JIT not set)
Encrypted password support via crypt(3): yes
Built with gcc compiler version 4.8.5 20150623 (Red Hat 4.8.5-39)

Available polling systems :
      epoll : pref=300,  test result OK
       poll : pref=200,  test result OK
     select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

Available multiplexer protocols :
(protocols marked as <default> cannot be specified using 'proto' keyword)
            fcgi : mode=HTTP       side=BE        mux=FCGI
       <default> : mode=HTTP       side=FE|BE     mux=H1
              h2 : mode=HTTP       side=FE|BE     mux=H2
       <default> : mode=TCP        side=FE|BE     mux=PASS

Available services : none

Available filters :
        [SPOE] spoe
        [COMP] compression
        [TRACE] trace
        [CACHE] cache
        [FCGI] fcgi-app

# systemctl status haproxy
â—Ź haproxy.service - Cluster Controlled haproxy
   Loaded: loaded (/etc/rc.d/init.d/haproxy; bad; vendor preset: disabled)
  Drop-In: /run/systemd/system/haproxy.service.d
           └─50-pacemaker.conf
   Active: active (running) since Wed 2020-11-04 20:19:15 GMT; 29min ago
     Docs: man:systemd-sysv-generator(8)
  Process: 2226 ExecStart=/etc/rc.d/init.d/haproxy start (code=exited, status=0/SUCCESS)
 Main PID: 2238 (haproxy)
   CGroup: /system.slice/haproxy.service
           └─2238 /usr/sbin/haproxy -D -f /etc/haproxy/haproxy.cfg -p /var/run/haproxy.pid

Nov 04 20:19:15  XXXXXX Starting Cluster Controlled haproxy...
Nov 04 20:19:15 XXXXXX haproxy[2226]: Starting haproxy: [  OK  ]
Nov 04 20:19:15 XXXXXX systemd[1]: Started Cluster Controlled haproxy.

Is it possible to fix this (enable systemd haproxy?) without completely wiping the server and just doing a fresh install of HAProxy? What would be the steps/commands to run for this?

I’m a complete novice with LINUX so would appreciate any help (literally reddit-style explain like i’m five please!)

Thank you!

First of all, you need to revert the existing mess. So rollback all those manual changes you did, that includes stopping haproxy, removing the executables and most importantly, the init.d file. Also remove directories and the haproxy user.

Then you need to think about what the best way forward is.

Are you aware that you can just install packages like haproxy with two simple commands as root, e.g. in CentOs that should be:

yum update
yum install haproxy

This will take care of all the little details that you don’t want to care about and you certainly don’t wont to read about on a random blog posts on the internet, which as you noticed, is not always accurate.

Yes, you can compile haproxy yourself. But at the very least you need to read the entire INSTALL file in the tarball two or three times and have a basic understanding.

Thanks! @lukastribus

I guess there’s no way to enable the reload without rebuilding then!

Just to see what other options there are - are there any other implications of using initd instead of systemd that you’re aware of? The setup seems to be working fine so far, it’s just a bit incovenient that a restart is needed rather than a reload when changing the haproxy config

Everything is possible, but you are just adding to the mess then.

If that’s what you want:

  • delete the init.d file
  • copy the system.d unit file as per the thread you found
  • edit the file and change Type=notify to Type=forking and -Ws to -W (basically reverting systemd notify support in the unit file)
  • only then complete the other steps (systemctl daemon-reload et all)

Hey @lukastribus

Thanks for your tips on this. We didn’t use yum as we wanted to use v2+

I ended up using:
find / -name haproxy* to look for all the haproxy files and ended up removing these files:

**/run/systemd/generator.late/haproxy.service
**/run/systemd/generator.late/runlevel5.target.wants/haproxy.service
**/run/systemd/generator.late/runlevel4.target.wants/haproxy.service
**/run/systemd/generator.late/runlevel3.target.wants/haproxy.service
**/run/systemd/generator.late/runlevel2.target.wants/haproxy.service
**/run/systemd/system/haproxy.service.d
**/etc/rc.d/init.d/haproxy
**/usr/local/sbin/haproxy)

I then read the install file and installed the pre-reqs:
yum install gcc pcre-static pcre-devel openssl-devel systemd-devel -y

Compiled and reinstalled with:
make TARGET=linux-glibc USE_PCRE=1 USE_OPENSSL=1 USE_ZLIB=1 USE_CRYPT_H=1 USE_LIBCRYPT=1 USE_SYSTEMD=1

Created a symlink: /usr/local/sbin/haproxy /usr/sbin/haproxy

Then edited copied the haproxy.service.in file to /lib/systemd/system/haproxy.service before daemon-reload etc.

After this, I was able to reload HAproxy fine :smiley:

Are you able to constructively re-criticise this approach please in case I am missing something or still have a massive mess?

Kindly appreciated! Thanks!

HA-Proxy version 2.2.2 2020/07/31 - https://haproxy.org/
Status: long-term supported branch - will stop receiving fixes around Q2 2025.
Known bugs: http://www.haproxy.org/bugs/bugs-2.2.2.html
Running on: Linux 3.10.0-1127.13.1.el7.x86_64 #1 SMP Tue Jun 23 15:46:38 UTC 2020 x86_64
Build options :
  TARGET  = linux-glibc
  CPU     = generic
  CC      = gcc
  CFLAGS  = -O2 -g -Wall -Wextra -Wdeclaration-after-statement -fwrapv -Wno-unused-label -Wno-sign-compare -Wno-unused-parameter -Wno-clobbered -Wno-missing-field-initializers -Wtype-limits
  OPTIONS = USE_PCRE=1 USE_LIBCRYPT=1 USE_CRYPT_H=1 USE_OPENSSL=1 USE_ZLIB=1 USE_SYSTEMD=1

Feature list : +EPOLL -KQUEUE +NETFILTER +PCRE -PCRE_JIT -PCRE2 -PCRE2_JIT +POLL -PRIVATE_CACHE +THREAD -PTHREAD_PSHARED +BACKTRACE -STATIC_PCRE -STATIC_PCRE2 +TPROXY +LINUX_TPROXY +LINUX_SPLICE +LIBCRYPT +CRYPT_H +GETADDRINFO +OPENSSL -LUA +FUTEX +ACCEPT4 +ZLIB -SLZ +CPU_AFFINITY +TFO +NS +DL +RT -DEVICEATLAS -51DEGREES -WURFL +SYSTEMD -OBSOLETE_LINKER +PRCTL +THREAD_DUMP -EVPORTS

Default settings :
  bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with multi-threading support (MAX_THREADS=64, default=4).
Built with OpenSSL version : OpenSSL 1.0.2k-fips  26 Jan 2017
Running on OpenSSL version : OpenSSL 1.0.2k-fips  26 Jan 2017
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : SSLv3 TLSv1.0 TLSv1.1 TLSv1.2
Built with network namespace support.
Built with zlib version : 1.2.7
Running on zlib version : 1.2.7
Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
Built with PCRE version : 8.32 2012-11-30
Running on PCRE version : 8.32 2012-11-30
PCRE library supports JIT : no (USE_PCRE_JIT not set)
Encrypted password support via crypt(3): yes
Built with gcc compiler version 4.8.5 20150623 (Red Hat 4.8.5-39)

Available polling systems :
  epoll : pref=300,  test result OK
   poll : pref=200,  test result OK
 select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

Available multiplexer protocols :
(protocols marked as <default> cannot be specified using 'proto' keyword)
        fcgi : mode=HTTP       side=BE        mux=FCGI
   <default> : mode=HTTP       side=FE|BE     mux=H1
          h2 : mode=HTTP       side=FE|BE     mux=H2
   <default> : mode=TCP        side=FE|BE     mux=PASS

Available services : none

Available filters :
    [SPOE] spoe
    [COMP] compression
    [TRACE] trace
    [CACHE] cache
    [FCGI] fcgi-app


[root@ XXX]# systemctl status haproxy
â—Ź haproxy.service - Cluster Controlled haproxy
   Loaded: loaded (/usr/lib/systemd/system/haproxy.service; enabled; vendor preset: disabled)
  Drop-In: /run/systemd/system/haproxy.service.d
           └─50-pacemaker.conf
   Active: active (running) since Fri 2020-11-06 22:01:47 GMT; 2min 32s ago
  Process: 8318 ExecReload=/bin/kill -USR2 $MAINPID (code=exited, status=0/SUCCESS)
  Process: 8316 ExecReload=/usr/sbin/haproxy -f $CONFIG -c -q $EXTRAOPTS (code=exited, status=0/SUCCESS)
  Process: 8064 ExecStartPre=/usr/sbin/haproxy -f $CONFIG -c -q $EXTRAOPTS (code=exited, status=0/SUCCESS)
 Main PID: 8067 (haproxy)
   CGroup: /system.slice/haproxy.service
           ├─8067 /usr/sbin/haproxy -sf 8213 -Ws -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -S /run/haproxy-master.sock
           ├─8213 /usr/sbin/haproxy -sf 8154 -Ws -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -S /run/haproxy-master.sock
           └─8320 /usr/sbin/haproxy -sf 8213 -Ws -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -S /run/haproxy-master.sock

Nov 06 22:04:13 XXX haproxy[8067]: [WARNING] 310/220413 (8213) : Stopping backend api-servers in 0 ms.
Nov 06 22:04:13 XXX haproxy[8067]: [WARNING] 310/220413 (8213) : Stopping frontend stats in 0 ms.
Nov 06 22:04:13 XXX haproxy[8067]: [WARNING] 310/220413 (8213) : Stopping frontend GLOBAL in 0 ms.
Nov 06 22:04:13 XXX haproxy[8067]: [WARNING] 310/220413 (8213) : Proxy https-in stopped (cumulated conns: FE: 14, BE: 0).
Nov 06 22:04:13 XXX haproxy[8067]: [WARNING] 310/220413 (8213) : Proxy app-servers stopped (cumulated conns: FE: 0, BE: 58).
Nov 06 22:04:13 XXX haproxy[8067]: [WARNING] 310/220413 (8213) : Proxy api-servers stopped (cumulated conns: FE: 0, BE: 23).
Nov 06 22:04:13 XXX systemd[1]: Reloaded Cluster Controlled haproxy.
Nov 06 22:04:13 XXX haproxy[8067]: [NOTICE] 310/220413 (8067) : New worker #1 (8320) forked
Nov 06 22:04:13 XXX haproxy[8067]: [WARNING] 310/220413 (8213) : Proxy stats stopped (cumulated conns: FE: 0, BE: 0).
Nov 06 22:04:13 XXX haproxy[8067]: [WARNING] 310/220413 (8213) : Proxy GLOBAL stopped (cumulated conns: FE: 0, BE: 0).

So it seems you:

  • fully cleaned up up the old setup
  • actually read the INSTALL file
  • installed haproxy cleanly
  • setup systemd properly

Now you got a proper installation with the correct setup. Thumbs up from me. I’d say now you know more about setting up haproxy on linux than the average users here in this forum.

Just one thing I noticed: you should grab the latest 2.2.5 release as opposed to 2.2.2, so you have all the latest bugfixes. Upgrading should be a non-issue, a make install should overwrite the executable and with a subsequent reload you are done.

1 Like

Awesome, it was a good learning experience haha! All upgraded to 2.2.5 now, thanks again!! :smiley:

1 Like