Custom Solution for Handling HTTP 429 Responses and Retries in HAProxy

Hello HAProxy Community,

I am working on a scenario where I need to handle HTTP 429 (Too Many Requests) responses from my backend servers. Specifically, I want to implement a retry mechanism that performs a secondary request to another backend server when a 429 response is encountered. Here’s a summary of my requirements and the challenges I’m facing:

Requirements:

  • Detect HTTP 429 Responses: When a backend server returns a 429 response, I need to identify it and perform a custom action.
  • Make a Secondary Request: After detecting a 429 response, I need to make a POST request to a different backend server, using the original request body and headers.
  • Return Response to Client: After making the secondary request, the response from this backend should be sent back to the client without disconnecting them.

Current Challenges:

  • HAProxy Limitations: HAProxy does not natively support retrying requests on HTTP 429 responses, and the redispatch feature does not apply to 429 responses.
  • Custom Logic Needed: I need a way to implement this retry logic within HAProxy, ideally using Lua scripting or other mechanisms available in HAProxy.

Questions:

  1. Is there a recommended approach within HAProxy to implement a retry mechanism for HTTP 429 responses?
  2. How can I handle making secondary POST requests from Lua while ensuring the client connection remains active?
  3. Are there any best practices or examples for managing such complex request handling scenarios in HAProxy?
  4. I would greatly appreciate any advice, examples, or guidance on how to achieve this functionality using HAProxy

I would greatly appreciate any advice, examples, or guidance on how to achieve this functionality using HAProxy.

Implemented in 3.0.4 as per: