Hi HAProxy folks,
I am excited to experiment with the server add features that are more fully supported in HAProxy 2.5 in the runtime API, and have found a segfault when attempting to specify the ‘id’ field.
I was able to add servers without any problems when not specifying the id field. However, my issues arise when I attempt to add a new server with a custom id. The haproxy worker segfaults immediately upon receiving the command.
So before getting into the details, my main question here is: how best to provide useful outputs to developers or the community when haproxy worker segfaults?. Is there a configuration or setup I could be in which would provide more helpful logs?
As a note, I am using 127.0.0.1:81 as filler in for a different IP.
First a successful run to show HAProxy is up and working as expected
echo 'experimental-mode on; add server nodes/s6 127.0.0.1:81' | sudo socat /var/run/haproxy/admin.sock stdio
New server registered.
The these next attempts fail with output as expected because I have specified an existing ID, so I know that the ID parsing part of this works and is checking the existing list.
echo 'experimental-mode on; add server nodes/s6 127.0.0.1:81 id 1' | sudo socat /var/run/haproxy/admin.sock stdio
'id' : custom id 1 already used at /etc/haproxy/haproxy.cfg:506 ('server dev-us-west-2a-s2').
echo 'experimental-mode on; add server nodes/s6 127.0.0.1:81 id 0' | sudo socat /var/run/haproxy/admin.sock stdio
'id' : custom id has to be > 0.
TL;DR; LOOK HERE FOR THE SEGFAULT EXAMPLE
Finally the real meat of this post, we attempt to add a node with an arbitrary id:
echo 'experimental-mode on; add server nodes/s7 127.0.0.1:81 id 7' | sudo socat /var/run/haproxy/admin.sock stdio
The shell exits here and we see a segfault in the worker and haproxy reloads with its configuration from file
The logs show several other attempts including other segfaults:
2021-12-06T23:14:21.349355+00:00 ip-10-19-3-133 haproxy[78828]: [NOTICE] (78828) : haproxy version is 2.5.0-1ppa1~focal
2021-12-06T23:14:21.349589+00:00 ip-10-19-3-133 haproxy[78828]: [NOTICE] (78828) : path to executable is /usr/sbin/haproxy
2021-12-06T23:14:21.349675+00:00 ip-10-19-3-133 haproxy[78828]: [WARNING] (78828) : config : server-state partially applied for server 'nodes/dev-us-west-2a-s2', agent checks are not enabled on this server
2021-12-06T23:14:21.517210+00:00 ip-10-19-3-133 haproxy[78828]: [NOTICE] (78828) : New worker (78830) forked
2021-12-06T23:14:21.517589+00:00 ip-10-19-3-133 haproxy[78828]: [NOTICE] (78828) : Loading success.
2021-12-06T23:14:32.171031+00:00 ip-10-19-3-133 haproxy[78830]: [ALERT] (78830) : CLI : 'server nodes/s3' : 'id' : custom id 1 already used at /etc/haproxy/haproxy.cfg:506 ('server dev-us-west-2a-s2').
2021-12-06T23:14:46.849811+00:00 ip-10-19-3-133 haproxy[78830]: [ALERT] (78830) : CLI : 'server nodes/s3' : 'id' : custom id has to be > 0.
2021-12-06T23:14:53.933396+00:00 ip-10-19-3-133 haproxy[78830]: [ALERT] (78830) : CLI : 'server nodes/s3' : 'id' : custom id 1 already used at /etc/haproxy/haproxy.cfg:506 ('server dev-us-west-2a-s2').
2021-12-06T23:15:03.759429+00:00 ip-10-19-3-133 haproxy[78828]: [NOTICE] (78828) : haproxy version is 2.5.0-1ppa1~focal
2021-12-06T23:15:03.759534+00:00 ip-10-19-3-133 haproxy[78828]: [NOTICE] (78828) : path to executable is /usr/sbin/haproxy
2021-12-06T23:15:03.759568+00:00 ip-10-19-3-133 haproxy[78828]: [ALERT] (78828) : Current worker (78830) exited with code 139 (Segmentation fault)
2021-12-06T23:15:03.759603+00:00 ip-10-19-3-133 haproxy[78828]: [ALERT] (78828) : exit-on-failure: killing every processes with SIGTERM
2021-12-06T23:15:03.759635+00:00 ip-10-19-3-133 haproxy[78828]: [WARNING] (78828) : All workers exited. Exiting... (139)
2021-12-06T23:15:04.095521+00:00 ip-10-19-3-133 haproxy[78855]: [NOTICE] (78855) : haproxy version is 2.5.0-1ppa1~focal
2021-12-06T23:15:04.095660+00:00 ip-10-19-3-133 haproxy[78855]: [NOTICE] (78855) : path to executable is /usr/sbin/haproxy
2021-12-06T23:15:04.095793+00:00 ip-10-19-3-133 haproxy[78855]: [WARNING] (78855) : config : server-state partially applied for server 'nodes/dev-us-west-2a-s2', agent checks are not enabled on this server
2021-12-06T23:15:04.251390+00:00 ip-10-19-3-133 haproxy[78855]: [NOTICE] (78855) : New worker (78857) forked
2021-12-06T23:15:04.251502+00:00 ip-10-19-3-133 haproxy[78855]: [NOTICE] (78855) : Loading success.
So if you made it this far, I would like advice on how to proceed in providing debug information. Let me know if anybody knows if I am doing something wrong in the above?