Managing servers with DataPlaneApi


I’m using some curl scripts on DataPlaneApi to add and remove servers to a haproxy backend. That works very well in all but one point. It seems I’m losing all open calls to haproxy when I change the backend. The clients gets a NoHttpResponseException (Apache Http Client for Java). That works with a single retry on the call, however it is an undesired behavior. Please note it is not the calls already processed by the server since they are maxconn 1 and I get the same error on 5 parallel clients.

Is there a better way to add/remove servers without losing open calls to other servers?

The backend is for testing pretty simple:

backend test_backend from my_defaults
  mode http
  server server12345 check maxconn 1 weight 100
  server server12346 check maxconn 1 weight 100

program api
  command /etc/haproxy/dataplaneapi -f /etc/haproxy/dataplaneapi.hcl
  no option start-on-reload

Some further testing shows that the problem is the client connections are kept alive and cannot reestablish a connection to the new process.

Best regards


for all others running into such an issue:

The problem was on the client side. A client connection pool was used and the client connection did not know about the removed endpoint. The solution is to use a retry handler that catches at least the io exceptions and reestablished the connection to haproxy