Remove session stickiness based on server load

I’m having an application setup where there are multiple instances of application running (let’s say two instances/servers s1 and s2) balanced by HAProxy. We are using sticky session based on application session cookie. So if the first lands on s1 server, all the subsequent requests will land on s1 server. The configuration I’m using for stickiness is below:

backend app_server
   ...
   cookie appCookie prefix nocache
   server s1 <ip> cookie s1
   server s2 <ip> cookie s2

Now what I want to achieve is to remove the stickiness based on the load on the server. Let’s say there are two active sessions on s1 and if the cpu utilization reaches 90%, drop all the session redirection to s1 (including the active sessions) and redirect them on s2. And once the utilization settles down on s1, balance the requests in a normal fashion with stickiness enabled.

I looked around for using agent-check, to use agent utility to check for server cpu utilization and remove/delete cookie if the feedback from agent-check if more than 90%. But I faced 2 challenges:

  1. I found no way to remove/delete the cookie conditionally.
  2. I found no way to use last agent-check feedback to use in the condition.

Can anyone provide their views to achieve this?

That sounds a bit of an odd thing to want?

Normally if a server is under heavy load you would want to drain new sessions i.e. use the feedback agent to lower the weight of the server to zero.

If you KILL existing sessions, then you will break all your existing users (assuming you need persistence/sticky for the application).

IF you dont NEED persistence/sticky then why use it in the first place?

The only time I would kill sessions is if I need to force a server down for maintenance immediately. Which is why the agent lets you send a down/drain command from the server.

Maybe I’m totaly missing the point though?

Well that’s correct in normal scenarios but in my application, the session cookie I’m persisting, is working in reverse proxy manner with conjunction to another cookie (let’s say master cookie), being maintained by another application layer. So even if I break the stickiness and redirect the request to another server, my application will be able to identify the session reading the master cookie and will internally create a substitute session which will thereafter be persisted on that server for subsequent requests. The existing user will not experience any session termination.

Now with this setup, when I see there is a load on a particular application instance (sever), I want all the persisted sessions to go away on another available servers so that the load on this instance can get lower. And as I explained, while shifting the sessions from a server to another, the user experience will be seamless.

I hope this will clear few clouds on why I need this kind of configuration.