Greetings and Salutations,
I noticed an odd behavior at a client site. They have one enterprise HAProxy for all their K8 applications/service offerings. For applications that use sticky affinity cookie based - you have the option to use round robin/least connections as your load balancing policy.
Whenever HAProxy reloads endpoints (an endpoint/pod is removed or added or it just refreshes) - The state of both round robin/least connections is reset. Since this can happen 500ms-5seconds a time - it effectively makes round robin/least connections useless since it resets to the first path for round robin and it resets the connection counts for least connections.
I consider this a bug.
The application using HAProxy is state-full - the end reset of the position/metric loss is profoundly unbalanced traffic since every 500m-5s - traffic is reset and the first endpoint receives the traffic. For a state-full applicatin - they are placed on that endpoint and will never leave it until it blows up by being driven past capacity.
Is there a setting that will stop this behavior - or is this a bug - or is this some sort of feature?
I see no practical advantage to resetting the load balancing statistics on every reload of endpoints.
At an enterprise level with PODS spinning up like crazy - this behavior destroys load balancing.