We’re using HA Proxy 2.2.23 and encountered a scenario where the HA Proxy CPU is pegged at 100% until it’s restarted. Sometimes it remains at that level until we restart the backend servers.
Here’s a brief overview of my config - let me know if more details are needed.
Global maxconn - 20,000
Frontend - maxconn is not defined but in the stats page, the slim (session limit) is 2000. I assume the default value is 10% of the global maxconn.
When CPU usage spikes, here’s my observations:
- before the spike, the scur (current sessions) is well below 2000 and all of a sudden, the scur spikes to 4000+
- at the same time, the backend servers have increased response times (from 50ms to 1-2 seconds) and increased connections
Would saturation in the backend servers cause the HA Proxy CPU to spike?
It seems that if backend servers are under heavy load, response will slow and will result in increased connections which HA Proxy wiill need to queue once it exceeds the 2000 limit. This leads to increased CPU usage.
As a first step, we’ve just added more instances of the backend servers to handle the load so that shouldn’t be an issue anymore.
Is there a workaround to avoid hitting 100% cpu usage on HA Proxy when backend servers are saturated? I was thinking of increasing the frontend maxconn, but I’m not sure what effect it will have.
Any suggestions are appreciated.
Thanks