I’m using some curl scripts on DataPlaneApi to add and remove servers to a haproxy backend. That works very well in all but one point. It seems I’m losing all open calls to haproxy when I change the backend. The clients gets a NoHttpResponseException (Apache Http Client for Java). That works with a single retry on the call, however it is an undesired behavior. Please note it is not the calls already processed by the server since they are maxconn 1 and I get the same error on 5 parallel clients.
Is there a better way to add/remove servers without losing open calls to other servers?
The backend is for testing pretty simple:
backend test_backend from my_defaults mode http server server12345 127.0.0.1:12345 check maxconn 1 weight 100 server server12346 127.0.0.1:12346 check maxconn 1 weight 100 program api command /etc/haproxy/dataplaneapi -f /etc/haproxy/dataplaneapi.hcl no option start-on-reload
Some further testing shows that the problem is the client connections are kept alive and cannot reestablish a connection to the new process.