504 Timeouts not changing after restarting the service

Morning
I have been playing with HAProxy for a few months now, I set up proof of concept which clearly showed how amazing it is, followed by a minor production deployment, to now a full blown HAProxy active /passive deployment on our intranet.

However end users have been experiencing the 504 timeout issue, here is an example from the log:
haproxy[1865]: x.x.x.x:50479 [03/Jul/2019:08:27:09.382] http_front http_back/server.server.com 0/0/1/-1/50002 504 195 - - sH-- 42/40/1/0/0 0/0 “POST /cgi-bin/lansaweb?wam=G_UVE&webrtn=display&ml=LANSA:XHTML&partHTTP/1.1”

In the HAProxy config I had set 50000ms as the timeout for both the client and the server - the reason for this is that some requests require a significant amount of time to compile from a backend db - our sites are spread across the country…

So I increased the timeouts:

timeout connect 500
timeout client 1200000
timeout server 1200000

However even after restarting HAProxy - the 504 apperas to be still set at 50000 as evidenced above.

any help / advice would be greatly appreciated.

Many thanks

You probably have an old haproxy process running with the old configuration. Because haproxy uses REUSEPORT, parallel binds to the same port are allowed and the linux kernel then load-balances between the old and the new instance, causing inconsistent behavior.

thank you for that - how can I check that is the case, and if so how do I kill it off - or is it just a case of rebooting the HAProxy?

Stop haproxy and then killall haproxy. This of course will completely interrupt any services.

Otherwise you need to check ps, netstat or ss output.