HAProxy And Keycloak Integration

Hi,

I hope someone can help me with this issue!

I am running HAProxy 3 along with Keycloak 26 in my setup. I have an issue where by my clients are not able to make use of the traditional “application/x-www-form-urlencoded” content-type to generate a JWT token as a client towards Keycloak.

I would like to offer an “application/json” API on my HAProxy application, that will then convert the content-type over to “application/x-www-form-urlencoded” and also take the JSON body of ‘{ “client_id”: “test”, “client_secret”: “test”, “grant_type”: “client_credentials” }’ and convert it to ‘client_id=test&grant_type=client_credentials&client_secret=test’ before sending it to my Keycloak backend for processing, generating a JWT token, and then responding with the JWT token.

I am happy to make use of a LUA script for this functionality, but I am not able to get it to work after numerous attempts with ChatGPT as well.

Please let me know if I can provide any other information that might assist.

Cheers,
Roebou

I would recommend thoroughly reading haproxy Lua documentation and examples, it may give you some hints: How Lua runs in HAProxy — haproxy-lua 3.1-dev14 (Fri Nov 22 02:04:23 CET 2024) 1.0 documentation

Lua service may probably do the trick to implement some custom HTTP service from Lua:
https://www.arpalert.org/src/haproxy-lua-api/3.1/index.html#core.register_service

global
   lua-load-per-thread lua_service.lua

frontend test
      mode http
      http-request use-service lua.my-api if { path_beg /my-custom-api-endpoint-url }

example lua_service.lua:

core.register_service("my-api", "http", function(applet)
-- here you could analyze the input request
-- craft your custom http request based on the infos from the input one
-- leverage the httpclient class to perform the JWT gen request on Keycloack backend
-- respond with the JWT token
end)

Thanks for the suggestion @adarragon .

I have actually looked into the register_service with the applet as well as into register_action along with chatgpt. Unfortunately, these functions do not offer a way to simply re-work the content-type, content-length and body of the request, then handover back to my HAProxy configuration to carry on to my backend.

I was hoping to perform it in this logical flow:
Client request → HAProxy Frontend (application/json) → LUA Service re-writes (application/x-www-form-urlencoded) → HAProxy Frontend (continue processing) → HAProxy backend (keycloak) → Keycloak application JWT generation

Indeed, there is Lua filter which may be able to do that directly (alter input request) before it is sent to the backend but it is complex for your use-case

This is why I suggested you to leverage lua service with httpclient combination

Consider the service as an independent application that can perform requests on its own and return the custom data when ready

With httpclient, you can manually craft the request with the proper headers and body (plus it is pretty trivial)

When making use of the register service, when it sends back the altered data, it actually closes the HTTP request stream, so it is not possible to then carry on to the backend. Let me get the code snippet to showcase this, perhaps I was doing something incorrect.

I suspect to get this to work, I need to do the following as everything else has failed thus far.

Client request → HAProxy Frontend (application/json) → LUA Service re-writes (application/x-www-form-urlencoded) + perform HTTP API call into Keycloak application JWT generation → Respond to client from the LUA script with the JWT token without going to any backend for this.

Oh ok, then the only option I can think of is to write a Lua filter to do that, since as you noted you need to “hack” the client request on the fly without impacting the request flow

I was hoping that once the JWT token is returned to the client through Lua service, the client is happy with that and may able to perform following requests on the normal path (ie: to your backend, not the custom lua service)

Wait no, I insist, with Lua service I think you can actually do this workflow:

Client request → HAProxy Frontend (application/json) → LUA Service re-writes (application/x-www-form-urlencoded) + perform HTTP API call into Keycloak application JWT generation → Respond to client from the LUA script with the JWT token without going to any backend for this.

As long as you don’t require to stream a response or keep the socket opened after the JWT is returned, Lua service + httpclient should do the trick.

Alright, I have made a lot of progress, here, but also I have been stuck for a few days, really hoping someone can assist me here.

I have implemented a service which will effectively perform the API call from LUA into my new frontend and backend section as discussed earlier in the thread. This seems to be working most of the time, but I am running into very odd scenarios. I have tried to tune LUA memory etc. as well. I am running this as a standalone docker container as well as in a docker swarm, both having similar experiences, but docker swarm experiences it way more often.


The experience:
When I startup HAProxy and I call my frontend that calls my backend towards the service, the service receives it and then proceeds to start the API call towards my inner frontend that will redirect towards Keycloak. At this point, the conversion of the headers and body has been completed already before making the API call. This first, second, third, etc. API calls ALL fail with a connection timeout, even though my backend is up and running etc. I can’t seem to find why it is failing and “timing out”, but it is at the point the of outgoing API call in my script. When I proceeded with first making an API call into any frontend and backend (it can be the final Keycloak one or any other frontend and backend in my system), and then proceed with calling my frontend with my service, then it works every time.

When I test locally on a standalone docker container, this keeps on working after that first successful / unsuccessful call. When I run this inside of a swarm, it only works for around 3 or 4 attempts, then it hangs again, until I do the successful call again.

Another thing that I tried is to run a test into my service frontend, but to trigger a negative scenario, for example, not sending a JSON body or using the wrong content-type to trigger a 400 response as coded in the script, then after that attempt, my script starts working again. Similar to making an API call, successful or unsuccessful, into any other frontend and backend.

I am currently making use of luasocket, as a lot of other http clients etc. looks like they have been written for nginx or something else, or they can’t work with LUA 5.4.6 which I have in my setup, along with HAProxy 3.0.4.

I have other LUA scripts that I also run as a “lua-load-per-thread”, and none of them have this type of issue, but none of them perform API calls into a third-party system.

A final item that I attempted as well, is to ensure that it is not the Keycloak platform that is problematic, but rather the LUA script / the HTTP module that I am making use of. I have changed my Keycloak backend temporarily to point into webhook.site which is publicly available on the internet and then I also receive the same problem. This rules out my own components, and hones in for me in either my HAProxy configuration / setup or the script itself. Since everything else in my HAProxy setup is running smoothly, I am at a point where the script is almost ready for use, but I just need this bug to be resolved first.

If anyone perhaps has an alternative approach, library, examples, etc. that they could recommend, that would be highly appreciated.


HAproxy Version:

1541423d19d2:/opt/haproxy$ haproxy -vv
HAProxy version 3.0.4-7a59afa 2024/09/03 - https://haproxy.org/
Status: long-term supported branch - will stop receiving fixes around Q2 2029.
Known bugs: http://www.haproxy.org/bugs/bugs-3.0.4.html
Running on: Linux 6.10.14-linuxkit #1 SMP PREEMPT_DYNAMIC Thu Oct 24 19:30:56 UTC 2024 x86_64
Build options :
  TARGET  = linux-musl
  CC      = cc
  CFLAGS  = -O2 -g -fwrapv
  OPTIONS = USE_GETADDRINFO=1 USE_OPENSSL=1 USE_LUA=1 USE_ZLIB=1 USE_PCRE2=1 USE_PCRE2_JIT=1
  DEBUG   = 

Feature list : -51DEGREES +ACCEPT4 -BACKTRACE -CLOSEFROM +CPU_AFFINITY +CRYPT_H -DEVICEATLAS +DL -ENGINE +EPOLL -EVPORTS +GETADDRINFO -KQUEUE -LIBATOMIC +LIBCRYPT +LINUX_CAP +LINUX_SPLICE +LINUX_TPROXY +LUA +MATH -MEMORY_PROFILING +NETFILTER +NS -OBSOLETE_LINKER +OPENSSL -OPENSSL_AWSLC -OPENSSL_WOLFSSL -OT -PCRE +PCRE2 +PCRE2_JIT -PCRE_JIT +POLL +PRCTL -PROCCTL -PROMEX -PTHREAD_EMULATION -QUIC -QUIC_OPENSSL_COMPAT +RT +SHM_OPEN -SLZ +SSL -STATIC_PCRE -STATIC_PCRE2 -SYSTEMD +TFO +THREAD +THREAD_DUMP +TPROXY -WURFL +ZLIB

Default settings :
  bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with multi-threading support (MAX_TGROUPS=16, MAX_THREADS=256, default=12).
Built with OpenSSL version : OpenSSL 3.3.2 3 Sep 2024
Running on OpenSSL version : OpenSSL 3.3.2 3 Sep 2024
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
OpenSSL providers loaded : default
Built with Lua version : Lua 5.4.6
Built with network namespace support.
Built with zlib version : 1.3.1
Running on zlib version : 1.3.1
Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
Built with PCRE2 version : 10.43 2024-02-16
PCRE2 library supports JIT : yes
Encrypted password support via crypt(3): yes
Built with gcc compiler version 13.2.1 20240309

Available polling systems :
      epoll : pref=300,  test result OK
       poll : pref=200,  test result OK
     select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

Available multiplexer protocols :
(protocols marked as <default> cannot be specified using 'proto' keyword)
         h2 : mode=HTTP  side=FE|BE  mux=H2    flags=HTX|HOL_RISK|NO_UPG
  <default> : mode=HTTP  side=FE|BE  mux=H1    flags=HTX
         h1 : mode=HTTP  side=FE|BE  mux=H1    flags=HTX|NO_UPG
       fcgi : mode=HTTP  side=BE     mux=FCGI  flags=HTX|HOL_RISK|NO_UPG
  <default> : mode=TCP   side=FE|BE  mux=PASS  flags=
       none : mode=TCP   side=FE|BE  mux=PASS  flags=NO_UPG

Available services : none

Available filters :
        [BWLIM] bwlim-in
        [BWLIM] bwlim-out
        [CACHE] cache
        [COMP] compression
        [FCGI] fcgi-app
        [SPOE] spoe
        [TRACE] trace

LUA Version:

1541423d19d2:/opt/haproxy$ lua -v
Lua 5.4.6  Copyright (C) 1994-2023 Lua.org, PUC-Rio

HAProxy configuration:

global
  # General
  log stdout format timed local0 debug
  maxconn 10000
  max-spread-checks 500
  spread-checks 50

  # Lua Configuration
  tune.lua.maxmem 0
  tune.lua.session-timeout 60s
  tune.lua.burst-timeout 60s
  tune.lua.service-timeout 60s

  lua-load-per-thread /opt/haproxy/lua/data_converter.lua
  lua-load-per-thread /opt/haproxy/lua/extractor.lua
  lua-load-per-thread /opt/haproxy/lua/healthcheck.lua

### Defaults Configuration ###
defaults
  # General
  mode http

  # Default Server
  default-server check
  balance roundrobin

  # Default Logging
  log global
  unique-id-format %[uuid()]
  log-format "$LOG_FORMAT"
  option tcp-check
  option dontlognull
  option log-separate-errors
  option log-health-checks

  # Default Connection Config
  timeout http-request 121s
  option abortonclose

  timeout connect 5s
  timeout client 120s
  timeout server 120s
  retries 3

frontend http-keycloak-frontend-8087

  # Binding
  bind *:8087
  
  default_backend keycloak-json-converter

backend keycloak-json-converter

  http-request use-service lua.json_to_form_data

frontend http-keycloak-frontend-8088

  # Binding
  bind *:8088
  
  default_backend https-keycloak-8443

backend https-keycloak-8443

  default-server check check-ssl ssl verify none
  server keycloak-frontend-8443-01 172.31.3.171:8082

Logs: (Testing Script Without Testing Another Config Block First)

haproxy  | 2024-12-06 13:08:53 SAST INFO: Starting HaProxy!
haproxy  | [NOTICE]   (1) : New worker (27) forked
haproxy  | [NOTICE]   (1) : Loading success.
haproxy  | [WARNING]  (27) : Health check for server https-keycloak-8443/keycloak-frontend-8443-01 succeeded, reason: Layer6 check passed, check duration: 106ms, status: 3/3 UP.
haproxy  | <5>2024-12-06T13:08:53.577856+02:00 Health check for server https-keycloak-8443/keycloak-frontend-8443-01 succeeded, reason: Layer6 check passed, check duration: 106ms, status: 3/3 UP.
haproxy  | <5>2024-12-06T13:08:53.593001+02:00 Health check for server success_backend/testA-frontend succeeded, reason: Layer4 check passed, check duration: 0ms, status: 3/3 UP.
haproxy  | [WARNING]  (27) : Health check for server success_backend/testA-frontend succeeded, reason: Layer4 check passed, check duration: 0ms, status: 3/3 UP.
haproxy  | <5>2024-12-06T13:08:53.761183+02:00 Health check for server success_backend/testB-frontend succeeded, reason: Layer4 check passed, check duration: 0ms, status: 3/3 UP.
haproxy  | [WARNING]  (27) : Health check for server success_backend/testB-frontend succeeded, reason: Layer4 check passed, check duration: 0ms, status: 3/3 UP.
haproxy  | [2024-12-06 13:09:00] INFO - === New request received ===
haproxy  | [2024-12-06 13:09:00] DEBUG - Path: /realms/Sandbox/protocol/openid-connect/token
haproxy  | [2024-12-06 13:09:00] DEBUG - Processing JSON input
haproxy  | [2024-12-06 13:09:00] DEBUG - Parsed values - client_id: dJwWi*****VihFV, grant_type: client_credentials
haproxy  | [2024-12-06 13:09:00] DEBUG - Form data generated successfully
haproxy  | [2024-12-06 13:09:00] DEBUG - Forwarding request to: http://localhost:8088/realms/Sandbox/protocol/openid-connect/token
haproxy  | [2024-12-06 13:09:00] DEBUG - Sending request...
haproxy  | [2024-12-06 13:09:30] DEBUG - Request completed in 30.03 seconds - Status: timeout
haproxy  | [2024-12-06 13:09:30] ERROR - HTTP Request failed: timeout
haproxy  | [2024-12-06 13:09:30] ERROR - Service Unavailable: Connection error: timeout
haproxy  | [2024-12-06 13:09:30] ERROR - Error: Service Unavailable: Connection error: timeout
haproxy  | <3>2024-12-06T13:09:30.725409+02:00 172.18.0.1:56156 - http-keycloak-frontend-8087 keycloak-json-converter/<lua.json_to_form_data> 0/0/0/0/0 503 162 - - LR-- 2/1/0/0/0 0/0 "POST /realms/Sandbox/protocol/openid-connect/token HTTP/1.1" 2a7f681d-c5fb-444d-8e33-6eb66544deed -
haproxy  | <3>2024-12-06T13:09:30.726092+02:00 127.0.0.1:44164 - http-keycloak-frontend-8088 https-keycloak-8443/keycloak-frontend-8443-01 0/0/-1/-1/30034 -1 0 - - CC-- 2/1/0/0/0 0/0 "POST /realms/Sandbox/protocol/openid-connect/token HTTP/1.1" 5fc9bc27-e38a-4ab9-8e27-bb449455e991 -

Logs: (After Testing Script With A Failing Scenario On My Service - Similar To Testing Another Frontend / Backend Section)

haproxy  | [2024-12-06 13:10:39] INFO - === New request received ===
haproxy  | [2024-12-06 13:10:39] DEBUG - Path: /realms/Sandbox/protocol/openid-connect/token
haproxy  | [2024-12-06 13:10:39] ERROR - Expected application/json content type
haproxy  | [2024-12-06 13:10:39] ERROR - Error: Expected application/json content type
haproxy  | <6>2024-12-06T13:10:39.655113+02:00 172.18.0.1:56156 - http-keycloak-frontend-8087 keycloak-json-converter/<lua.json_to_form_data> 0/0/0/0/0 415 157 - - LR-- 1/1/0/0/0 0/0 "POST /realms/Sandbox/protocol/openid-connect/token HTTP/1.1" a841f778-1261-4d70-b568-dda95dd083ea -
haproxy  | [2024-12-06 13:10:53] INFO - === New request received ===
haproxy  | [2024-12-06 13:10:53] DEBUG - Path: /realms/Sandbox/protocol/openid-connect/token
haproxy  | [2024-12-06 13:10:53] DEBUG - Processing JSON input
haproxy  | [2024-12-06 13:10:53] DEBUG - Parsed values - client_id: dJwWi*****VihFV, grant_type: client_credentials
haproxy  | [2024-12-06 13:10:53] DEBUG - Form data generated successfully
haproxy  | [2024-12-06 13:10:53] DEBUG - Forwarding request to: http://localhost:8088/realms/Sandbox/protocol/openid-connect/token
haproxy  | [2024-12-06 13:10:53] DEBUG - Sending request...
haproxy  | <6>2024-12-06T13:10:53.490236+02:00 127.0.0.1:59268 - http-keycloak-frontend-8088 https-keycloak-8443/keycloak-frontend-8443-01 40/0/119/69/228 200 1495 - - ---- 2/1/0/0/0 0/0 "POST /realms/Sandbox/protocol/openid-connect/token HTTP/1.1" 4ed18e49-5915-481d-9a9f-ec9dfa3d2338 -
haproxy  | [2024-12-06 13:10:53] DEBUG - Request completed in 0.23 seconds - Status: 200
haproxy  | [2024-12-06 13:10:53] DEBUG - Successfully forwarding response to client
haproxy  | [2024-12-06 13:10:53] INFO - === Request completed ===
haproxy  | <6>2024-12-06T13:10:53.490626+02:00 172.18.0.1:56156 - http-keycloak-frontend-8087 keycloak-json-converter/<lua.json_to_form_data> 0/0/0/0/0 200 1510 - - LR-- 1/1/0/0/0 0/0 "POST /realms/Sandbox/protocol/openid-connect/token HTTP/1.1" 6ccb9c56-a8c6-4de0-9b75-000ecc3fc86d -

LUA Script: data_converter.lua

--- HAProxy JSON to Form Data Converter
-- @module json_to_form_converter
-- @author Rohan de Jongh
-- @license MIT
-- @description Converts JSON payloads to form-urlencoded data for backend authentication

local cjson = require("cjson")
local http = require("socket.http")
local ltn12 = require("ltn12")
local socket = require("socket")local cjson = require("cjson")
local http = require("socket.http")
local ltn12 = require("ltn12")
local socket = require("socket")

--- Enhanced Logging with Stack Trace
local function log_error(message, err)
    local trace = debug.traceback(message, 2)
    print(string.format("[ERROR] %s\n%s", message, trace))
end

--- Robust HTTP Request Function
local function robust_http_request(url, method, headers, body, timeout)
    local responseBody = {}
    local responseHeaders = {}

    -- Reset socket and http timeout for each request
    socket.TIMEOUT = timeout or 30
    http.TIMEOUT = timeout or 30

    local request = {
        url = url,
        method = method,
        headers = headers or {},
        source = body and ltn12.source.string(body) or nil,
        sink = ltn12.sink.table(responseBody),
        redirect = true,
        create = function()
            local conn = socket.tcp()
            -- Explicitly set connection timeout
            conn:settimeout(timeout or 30)
            return conn
        end
    }

    local startTime = socket.gettime()
    
    -- Explicit error handling with multiple attempts
    for attempt = 1, 3 do  -- 3 retry attempts
        local success, responseStatus, statusCode, responseHeaders = pcall(function()
            return http.request(request)
        end)

        if success and responseStatus then
            local endTime = socket.gettime()
            local responseBodyStr = table.concat(responseBody)
            
            return {
                success = true,
                status = statusCode,
                body = responseBodyStr,
                headers = responseHeaders,
                duration = endTime - startTime
            }
        else
            log_error(string.format("HTTP Request Attempt %d Failed", attempt), 
                success and responseStatus or "Unknown error")
            
            -- Brief pause between retries
            socket.sleep(0.5)
        end
    end

    -- If all attempts fail
    return {
        success = false,
        error = "All request attempts failed"
    }
end

--- Modify your existing forwardRequest function to use robust_http_request
local function forwardRequest(url, headers, body)
    log("Forwarding request to: " .. url, "DEBUG")

    local response = robust_http_request(url, "POST", headers, body, Config.timeouts.client)

    if not response.success then
        log("Request failed: " .. tostring(response.error), "ERROR")
        return nil, response.error
    end

    -- Optional field removal logic remains the same
    if Config.modes.remove_fields and #Config.remove_fields_list > 0 then
        local success, responseBodyTable = pcall(cjson.decode, response.body)
        if success and type(responseBodyTable) == "table" then
            for _, field in ipairs(Config.remove_fields_list) do
                if responseBodyTable[field] ~= nil then
                    log(string.format("Removing field: %s", field), "DEBUG")
                    responseBodyTable[field] = nil
                end
            end
            response.body = cjson.encode(responseBodyTable)
        end
    end

    log(string.format("Request completed in %.2f seconds", response.duration or 0), "DEBUG")

    return {
        status = response.status,
        headers = response.headers,
        body = response.body
    }
end

--- Configuration and Environment Management
-- Centralizes configuration with secure defaults and environment-based overrides
local Config = {
    --- Backend Server Configuration
    backend = {
        host = os.getenv("LUA_JSON_TO_FORM_BACKEND_HOST") or "127.0.0.1",
        port = os.getenv("LUA_JSON_TO_FORM_BACKEND_PORT") or "8080"
    },

    --- Operational Modes
    modes = {
        debug = os.getenv("LUA_JSON_TO_FORM_DEBUG_MODE") == "true" or false,
        remove_fields = os.getenv("LUA_JSON_TO_FORM_REMOVE_FIELDS") == "true" or false
    },

    --- Fields to Remove from Response
    remove_fields_list = (function()
        local fields_to_remove = os.getenv("LUA_JSON_TO_FORM_REMOVE_FIELDS_LIST")
        if not fields_to_remove then
            return {}  -- Empty Default list
        end

        local fields = {}

        for field in string.gmatch(fields_to_remove, "[^,]+") do
            local trimmed_field = field:gsub("^%s+", ""):gsub("%s+$", "")
            if trimmed_field ~= "" then
                fields[#fields + 1] = trimmed_field
            end
        end
        return fields
    end)(),

    --- Performance and Security Timeouts
    timeouts = {
        client = tonumber(os.getenv("LUA_JSON_TO_FORM_TIMEOUT_CLIENT")) or 30,
        connect = tonumber(os.getenv("LUA_JSON_TO_FORM_TIMEOUT_CONNECT")) or 5
    }
}

--- Logging Utility
-- Provides conditional logging based on debug mode
-- @param message string The message to log
-- @param level string Optional log level (default: "INFO")
local function log(message, level)
    if Config.modes.debug then
        level = level or "INFO"
        print(string.format("[%s] %s - %s", 
            os.date("%Y-%m-%d %H:%M:%S"), 
            level, 
            message
        ))
    end
end

--- Create a Standardized Error Response
-- Generates a JSON-formatted error message for consistent error handling
-- @param message string Error description
-- @return string JSON error response
local function createErrorResponse(message)
    log("Error: " .. message, "ERROR")
    return string.format(
        '{"error": "%s"}', 
        message:gsub('"', '\\"')
    )
end

--- URL Encoding Utility
-- Safely encodes strings for URL transmission, preserving certain characters
-- @param str string String to encode
-- @return string URL-encoded string
local function urlEncode(str)
    if not str then return "" end
    local preserve = { ['_'] = true, ['-'] = true, ['.'] = true }
    return str:gsub("[^%w]", function(c)
        return preserve[c] and c or string.format("%%%02X", string.byte(c))
    end)
end

--- JSON to Form Data Converter
-- Transforms JSON payload into form-urlencoded data with robust validation
-- @param jsonStr string JSON input string
-- @return string|nil Encoded form data, nil on failure
-- @return string|nil Error message if conversion fails
local function jsonToFormData(jsonStr)
    log("Processing JSON input", "DEBUG")

    -- Validate JSON structure
    if not jsonStr:match("^%s*{") or not jsonStr:match("}%s*$") then
        return nil, "Invalid JSON format"
    end

    -- Safely parse JSON
    local ok, parsedJson = pcall(cjson.decode, jsonStr)
    if not ok then
        return nil, "Failed to parse JSON: " .. tostring(parsedJson)
    end

    -- Validate required fields
    local requiredFields = {"client_id", "client_secret", "grant_type"}
    for _, field in ipairs(requiredFields) do
        if not parsedJson[field] then
            return nil, string.format("Missing %s field", field)
        end
    end

    -- Mask sensitive information in logs
    log(string.format("Parsed values - client_id: %s, grant_type: %s",
        parsedJson.client_id:sub(1, 5) .. "*****" .. parsedJson.client_id:sub(-5),
        parsedJson.grant_type), "DEBUG")

    -- Construct form-encoded data
    local formData = string.format(
        "client_id=%s&client_secret=%s&grant_type=%s",
        urlEncode(parsedJson.client_id),
        urlEncode(parsedJson.client_secret),
        urlEncode(parsedJson.grant_type)
    )

    log("Form data generated successfully", "DEBUG")
    return formData
end

--- Forward Request to Backend
-- Manages request forwarding with comprehensive error handling and timeout management
-- @param url string Target backend URL
-- @param headers table Request headers
-- @param body string Request body
-- @return table|nil Response object or nil on failure
-- @return string|nil Error message if request fails
local function forwardRequest(url, headers, body)
    log("Forwarding request to: " .. url, "DEBUG")

    local responseBody = {}
    local responseHeaders = {}

    -- Configure request with enhanced timeout handling
    local request = {
        url = url,
        method = "POST",
        headers = headers,
        source = ltn12.source.string(body),
        sink = ltn12.sink.table(responseBody),
        redirect = true,
        timeout = Config.timeouts.client
    }

    -- Set global timeout configurations
    socket.TIMEOUT = Config.timeouts.connect
    http.TIMEOUT = Config.timeouts.client

    log("Sending request...", "DEBUG")
    local startTime = socket.gettime()

    local responseStatus, statusCode, responseHeaders = http.request(request)

    local endTime = socket.gettime()

    log(string.format("Request completed in %.2f seconds - Status: %s",
        endTime - startTime, 
        statusCode or "N/A"), "DEBUG")

    if not responseStatus then
        log("HTTP Request failed: " .. tostring(statusCode), "ERROR")
        return nil, "Connection error: " .. tostring(statusCode)
    end

    local responseBodyStr = table.concat(responseBody)

    -- Optional field removal for security/privacy
    if Config.modes.remove_fields then
        local success, responseBodyTable = pcall(cjson.decode, responseBodyStr)
        if success and type(responseBodyTable) == "table" then
            -- Remove specified fields
            for _, field in ipairs(Config.remove_fields_list) do
                if responseBodyTable[field] ~= nil then
                    log(string.format("Removing field: %s", field), "DEBUG")
                    responseBodyTable[field] = nil
                end
            end
            responseBodyStr = cjson.encode(responseBodyTable)
        end
    end

    return {
        status = statusCode,
        headers = responseHeaders,
        body = responseBodyStr
    }
end

--- Error Handling Utility
-- Centralized error response generation
-- @param applet HAProxy applet instance
-- @param statusCode number HTTP status code
-- @param errorMessage string Error description
function handleError(applet, statusCode, errorMessage)
    log(errorMessage, "ERROR")
    applet:set_status(statusCode)
    applet:add_header("content-type", "application/json")
    applet:start_response()
    applet:send(createErrorResponse(errorMessage))
end

--- Response Sending Utility
-- Centralized response transmission
-- @param applet HAProxy applet instance
-- @param response table Response object
function sendResponse(applet, response)
    applet:set_status(response.status)
    for name, value in pairs(response.headers or {}) do
        applet:add_header(name, value)
    end
    applet:start_response()
    if response.body then
        applet:send(response.body)
    end
    log("=== Request completed ===", "INFO")
end

--- Main HAProxy Service Handler
-- Processes incoming requests, converts JSON to form data, and forwards to backend
core.register_service("json_to_form_data", "http", function(applet)
    log("=== New request received ===", "INFO")
    log("Path: " .. applet.path, "DEBUG")

    -- Read request body
    local body = applet:receive()
    if not body then
        return handleError(applet, 400, "Failed to read request body")
    end

    -- Validate content type
    local contentType = applet.headers["content-type"] and applet.headers["content-type"][0]
    if not contentType or not contentType:lower():match("application/json") then
        return handleError(applet, 415, "Expected application/json content type")
    end

    -- Convert JSON to form data
    local formData, errorMessage = jsonToFormData(body)
    if not formData then
        return handleError(applet, 400, errorMessage)
    end

    -- Prepare request headers
    local headersOut = {
        ["content-type"] = "application/x-www-form-urlencoded",
        ["content-length"] = tostring(#formData)
    }

    -- Preserve original non-content/length headers
    for name, values in pairs(applet.headers) do
        local lowerName = name:lower()
        if lowerName ~= "content-type" and lowerName ~= "content-length" then
            headersOut[name] = values[0]
        end
    end

    -- Backend URL construction
    local backendUrl = string.format("http://%s:%s%s", 
        Config.backend.host, 
        Config.backend.port, 
        applet.path
    )

    -- Forward request to backend
    local response, errorMsg = forwardRequest(backendUrl, headersOut, formData)

    if response then
        log("Successfully forwarding response to client", "DEBUG")
        return sendResponse(applet, response)
    else
        return handleError(applet, 503, 
            string.format("Service Unavailable: %s", tostring(errorMsg))
        )
    end
end)

I doubt the fact that the http/socket lua library you are using support yielding.

When using Lua script for haproxy, it is forbidden to use api/functions that can pause the process (ie: waiting for data or some event to occur). This is documented here: HAProxy: Lua architecture and first steps

Here you don’t follow this rule! For instance by using socket.sleep(0.5), you may end up freezing the whole haproxy process for .5 second! Same when you use socket.tcp() or http.request() as they most likely don’t support yielding: when they wait for data, they freeze the process (including haproxy) because they don’t give back the hand to haproxy.

To sleep in your Lua script, you should use the builtin core.sleep() function which is meant for that and gives the hand back to haproxy until the script needs to be resumed.

Now for the HTTP API request, when I suggested you to use the httpclient, I didn’t meant to use an external library that can do http requests, I meant to use the native httpclient. Indeed, the native httpclient (see also https://www.arpalert.org/src/haproxy-lua-api/3.1/index.html#httpclient-class) strictly follows haproxy event driven engine restrictions in order to be impactless on haproxy: if the httpclient needs to wait it yields (which means give back the hand to haproxy) until the condition is met. All of this is transparent from an user point of view (native httpclient is easy to use), but this matters a lot if you want your script to run properly and without impacting haproxy which is probably busy doing other things in parallel.

Of course this doesn’t mean that you can’t use external libraries, but when doing so you must ensure that the functions you’re using don’t block or wait the current thread (either for a definite or indefinite amount of time). A general rule is that string manipulation functions are rarely problematic, but socket or filesystem oriented ones often cause issues. When in doubt you can ask for help or advises on the haproxy mailing list or on the discourse.

Apologies, the LUA script with the sleep updated for debugging purposes.

Here is my LUA script that I attempted to make use of for the exteranl http client:

--- HAProxy JSON to Form Data Converter
-- @module json_to_form_converter
-- @author Rohan de Jongh
-- @license MIT
-- @description Converts JSON payloads to form-urlencoded data for backend authentication

local cjson = require("cjson")
local http = require("socket.http")
local ltn12 = require("ltn12")
local socket = require("socket")

--- Configuration and Environment Management
-- Centralizes configuration with secure defaults and environment-based overrides
local Config = {
    --- Backend Server Configuration
    backend = {
        host = os.getenv("LUA_JSON_TO_FORM_BACKEND_HOST") or "127.0.0.1",
        port = os.getenv("LUA_JSON_TO_FORM_BACKEND_PORT") or "8080"
    },

    --- Operational Modes
    modes = {
        debug = os.getenv("LUA_JSON_TO_FORM_DEBUG_MODE") == "true" or false,
        remove_fields = os.getenv("LUA_JSON_TO_FORM_REMOVE_FIELDS") == "true" or false
    },

    --- Fields to Remove from Response
    remove_fields_list = (function()
        local fields_to_remove = os.getenv("LUA_JSON_TO_FORM_REMOVE_FIELDS_LIST")
        if not fields_to_remove then
            return {}  -- Empty Default list
        end

        local fields = {}

        for field in string.gmatch(fields_to_remove, "[^,]+") do
            local trimmed_field = field:gsub("^%s+", ""):gsub("%s+$", "")
            if trimmed_field ~= "" then
                fields[#fields + 1] = trimmed_field
            end
        end
        return fields
    end)(),

    --- Performance and Security Timeouts
    timeouts = {
        client = tonumber(os.getenv("LUA_JSON_TO_FORM_TIMEOUT_CLIENT")) or 30,
        connect = tonumber(os.getenv("LUA_JSON_TO_FORM_TIMEOUT_CONNECT")) or 5
    }
}

--- Logging Utility
-- Provides conditional logging based on debug mode
-- @param message string The message to log
-- @param level string Optional log level (default: "INFO")
local function log(message, level)
    if Config.modes.debug then
        level = level or "INFO"
        print(string.format("[%s] %s - %s", 
            os.date("%Y-%m-%d %H:%M:%S"), 
            level, 
            message
        ))
    end
end

--- Create a Standardized Error Response
-- Generates a JSON-formatted error message for consistent error handling
-- @param message string Error description
-- @return string JSON error response
local function createErrorResponse(message)
    log("Error: " .. message, "ERROR")
    return string.format(
        '{"error": "%s"}', 
        message:gsub('"', '\\"')
    )
end

--- URL Encoding Utility
-- Safely encodes strings for URL transmission, preserving certain characters
-- @param str string String to encode
-- @return string URL-encoded string
local function urlEncode(str)
    if not str then return "" end
    local preserve = { ['_'] = true, ['-'] = true, ['.'] = true }
    return str:gsub("[^%w]", function(c)
        return preserve[c] and c or string.format("%%%02X", string.byte(c))
    end)
end

--- JSON to Form Data Converter
-- Transforms JSON payload into form-urlencoded data with robust validation
-- @param jsonStr string JSON input string
-- @return string|nil Encoded form data, nil on failure
-- @return string|nil Error message if conversion fails
local function jsonToFormData(jsonStr)
    log("Processing JSON input", "DEBUG")

    -- Validate JSON structure
    if not jsonStr:match("^%s*{") or not jsonStr:match("}%s*$") then
        return nil, "Invalid JSON format"
    end

    -- Safely parse JSON
    local ok, parsedJson = pcall(cjson.decode, jsonStr)
    if not ok then
        return nil, "Failed to parse JSON: " .. tostring(parsedJson)
    end

    -- Validate required fields
    local requiredFields = {"client_id", "client_secret", "grant_type"}
    for _, field in ipairs(requiredFields) do
        if not parsedJson[field] then
            return nil, string.format("Missing %s field", field)
        end
    end

    -- Mask sensitive information in logs
    log(string.format("Parsed values - client_id: %s, grant_type: %s",
        parsedJson.client_id:sub(1, 5) .. "*****" .. parsedJson.client_id:sub(-5),
        parsedJson.grant_type), "DEBUG")

    -- Construct form-encoded data
    local formData = string.format(
        "client_id=%s&client_secret=%s&grant_type=%s",
        urlEncode(parsedJson.client_id),
        urlEncode(parsedJson.client_secret),
        urlEncode(parsedJson.grant_type)
    )

    log("Form data generated successfully", "DEBUG")
    return formData
end

--- Forward Request to Backend
-- Manages request forwarding with comprehensive error handling and timeout management
-- @param url string Target backend URL
-- @param headers table Request headers
-- @param body string Request body
-- @return table|nil Response object or nil on failure
-- @return string|nil Error message if request fails
local function forwardRequest(url, headers, body)
    log("Forwarding request to: " .. url, "DEBUG")

    local responseBody = {}
    local responseHeaders = {}

    -- Configure request with enhanced timeout handling
    local request = {
        url = url,
        method = "POST",
        headers = headers,
        source = ltn12.source.string(body),
        sink = ltn12.sink.table(responseBody),
        redirect = true,
        timeout = Config.timeouts.client
    }

    -- Set global timeout configurations
    socket.TIMEOUT = Config.timeouts.connect
    http.TIMEOUT = Config.timeouts.client

    log("Sending request...", "DEBUG")
    local startTime = socket.gettime()

    local responseStatus, statusCode, responseHeaders = http.request(request)

    local endTime = socket.gettime()

    log(string.format("Request completed in %.2f seconds - Status: %s",
        endTime - startTime, 
        statusCode or "N/A"), "DEBUG")

    if not responseStatus then
        log("HTTP Request failed: " .. tostring(statusCode), "ERROR")
        return nil, "Connection error: " .. tostring(statusCode)
    end

    local responseBodyStr = table.concat(responseBody)

    -- Optional field removal for security/privacy
    if Config.modes.remove_fields then
        local success, responseBodyTable = pcall(cjson.decode, responseBodyStr)
        if success and type(responseBodyTable) == "table" then
            -- Remove specified fields
            for _, field in ipairs(Config.remove_fields_list) do
                if responseBodyTable[field] ~= nil then
                    log(string.format("Removing field: %s", field), "DEBUG")
                    responseBodyTable[field] = nil
                end
            end
            responseBodyStr = cjson.encode(responseBodyTable)
        end
    end

    return {
        status = statusCode,
        headers = responseHeaders,
        body = responseBodyStr
    }
end

--- Error Handling Utility
-- Centralized error response generation
-- @param applet HAProxy applet instance
-- @param statusCode number HTTP status code
-- @param errorMessage string Error description
function handleError(applet, statusCode, errorMessage)
    log(errorMessage, "ERROR")
    applet:set_status(statusCode)
    applet:add_header("content-type", "application/json")
    applet:start_response()
    applet:send(createErrorResponse(errorMessage))
end

--- Response Sending Utility
-- Centralized response transmission
-- @param applet HAProxy applet instance
-- @param response table Response object
function sendResponse(applet, response)
    applet:set_status(response.status)
    for name, value in pairs(response.headers or {}) do
        applet:add_header(name, value)
    end
    applet:start_response()
    if response.body then
        applet:send(response.body)
    end
    log("=== Request completed ===", "INFO")
end

--- Main HAProxy Service Handler
-- Processes incoming requests, converts JSON to form data, and forwards to backend
core.register_service("json_to_form_data", "http", function(applet)
    log("=== New request received ===", "INFO")
    log("Path: " .. applet.path, "DEBUG")

    -- Read request body
    local body = applet:receive()
    if not body then
        return handleError(applet, 400, "Failed to read request body")
    end

    -- Validate content type
    local contentType = applet.headers["content-type"] and applet.headers["content-type"][0]
    if not contentType or not contentType:lower():match("application/json") then
        return handleError(applet, 415, "Expected application/json content type")
    end

    -- Convert JSON to form data
    local formData, errorMessage = jsonToFormData(body)
    if not formData then
        return handleError(applet, 400, errorMessage)
    end

    -- Prepare request headers
    local headersOut = {
        ["content-type"] = "application/x-www-form-urlencoded",
        ["content-length"] = tostring(#formData)
    }

    -- Preserve original non-content/length headers
    for name, values in pairs(applet.headers) do
        local lowerName = name:lower()
        if lowerName ~= "content-type" and lowerName ~= "content-length" then
            headersOut[name] = values[0]
        end
    end

    -- Backend URL construction
    local backendUrl = string.format("http://%s:%s%s", 
        Config.backend.host, 
        Config.backend.port, 
        applet.path
    )

    -- Forward request to backend
    local response, errorMsg = forwardRequest(backendUrl, headersOut, formData)

    if response then
        log("Successfully forwarding response to client", "DEBUG")
        return sendResponse(applet, response)
    else
        return handleError(applet, 503, 
            string.format("Service Unavailable: %s", tostring(errorMsg))
        )
    end
end)

I have gone over your first comment again and referenced the core http client, and decided to convert the code snippet that I pasted above, over to the core httpclient.

LUA Script using core.httpclient: (Added some debugging to check where things are breaking)

--- HAProxy JSON to Form Data Converter
-- @module json_to_form_converter
-- @author Rohan de Jongh
-- @license MIT
-- @description Converts JSON payloads to form-urlencoded data for backend authentication

local cjson = require("cjson")

function dump(o)
    if type(o) == 'table' then
       local s = '{ '
       for k,v in pairs(o) do
          if type(k) ~= 'number' then k = '"'..k..'"' end
          s = s .. '['..k..'] = ' .. dump(v) .. ','
       end
       return s .. '} '
    else
       return tostring(o)
    end
 end

--- Configuration and Environment Management
local Config = {
    --- Backend Server Configuration
    backend = {
        host = os.getenv("LUA_JSON_TO_FORM_BACKEND_HOST") or "172.31.3.171",
        port = os.getenv("LUA_JSON_TO_FORM_BACKEND_PORT") or "8082"
    },

    --- Operational Modes
    modes = {
        debug = os.getenv("LUA_JSON_TO_FORM_DEBUG_MODE") == "true" or false,
        remove_fields = os.getenv("LUA_JSON_TO_FORM_REMOVE_FIELDS") == "true" or false
    },

    --- Fields to Remove from Response
    remove_fields_list = (function()
        local fields_to_remove = os.getenv("LUA_JSON_TO_FORM_REMOVE_FIELDS_LIST")
        if not fields_to_remove then
            return {}  -- Empty Default list
        end

        local fields = {}

        for field in string.gmatch(fields_to_remove, "[^,]+") do
            local trimmed_field = field:gsub("^%s+", ""):gsub("%s+$", "")
            if trimmed_field ~= "" then
                fields[#fields + 1] = trimmed_field
            end
        end
        return fields
    end)(),

    --- Performance and Security Timeouts
    timeouts = {
        client = tonumber(os.getenv("LUA_JSON_TO_FORM_TIMEOUT_CLIENT")) or 30,
        connect = tonumber(os.getenv("LUA_JSON_TO_FORM_TIMEOUT_CONNECT")) or 5
    }
}

--- Logging Utility
local function log(message, level)
    if Config.modes.debug then
        level = level or "INFO"
        print(string.format("[%s] %s - %s", 
            os.date("%Y-%m-%d %H:%M:%S"), 
            level, 
            message
        ))
    end
end

--- Create a Standardized Error Response
local function createErrorResponse(message)
    log("Error: " .. message, "ERROR")
    return string.format(
        '{"error": "%s"}', 
        message:gsub('"', '\\"')
    )
end

--- URL Encoding Utility
local function urlEncode(str)
    if not str then return "" end
    local preserve = { ['_'] = true, ['-'] = true, ['.'] = true }
    return str:gsub("[^%w]", function(c)
        return preserve[c] and c or string.format("%%%02X", string.byte(c))
    end)
end

--- JSON to Form Data Converter
local function jsonToFormData(jsonStr)
    log("Processing JSON input", "DEBUG")

    -- Validate JSON structure
    if not jsonStr:match("^%s*{") or not jsonStr:match("}%s*$") then
        return nil, "Invalid JSON format"
    end

    -- Safely parse JSON
    local ok, parsedJson = pcall(cjson.decode, jsonStr)
    if not ok then
        return nil, "Failed to parse JSON: " .. tostring(parsedJson)
    end

    -- Validate required fields
    local requiredFields = {"client_id", "client_secret", "grant_type"}
    for _, field in ipairs(requiredFields) do
        if not parsedJson[field] then
            return nil, string.format("Missing %s field", field)
        end
    end

    -- Mask sensitive information in logs
    log(string.format("Parsed values - client_id: %s, grant_type: %s",
        parsedJson.client_id:sub(1, 5) .. "*****" .. parsedJson.client_id:sub(-5),
        parsedJson.grant_type), "DEBUG")

    -- Construct form-encoded data
    local formData = string.format(
        "client_id=%s&client_secret=%s&grant_type=%s",
        urlEncode(parsedJson.client_id),
        urlEncode(parsedJson.client_secret),
        urlEncode(parsedJson.grant_type)
    )

    log("Form data generated successfully", "DEBUG")
    return formData
end

--- Forward Request to Backend with core.httpclient()
local function forwardRequest(applet, url, headers, body)
    log("Forwarding request to: " .. url, "DEBUG")

    -- Create HTTP client
    local httpclient = core.httpclient()

    -- Prepare the request
    local req = {
        url = url,
        headers = headers,
        body = body,
        timeout = Config.timeouts.client,
        -- Explicitly set the destination
        dst = "127.0.0.1:8088"
    }

    -- Send the request and wait for response
    local success, response = pcall(function()
        return httpclient:post(req)
    end)

    log("HTTP Client Request Details:")
    log("URL: " .. tostring(url))
    log("Headers: " .. dump(headers))
    log("Body: " .. tostring(body))
    log("Success: " .. tostring(success))
    log("Response: " .. dump(response))

    -- Check for request success
    if not success then
        log("HTTP Request failed: " .. tostring(response), "ERROR")
        return nil, "Connection error: " .. tostring(response)
    end

    -- Normalize headers (extract first value)
    local normalizedHeaders = {}
    for k, v in pairs(response.headers or {}) do
        normalizedHeaders[k] = v[0]
    end

    -- Optional field removal for security/privacy
    local responseBody = response.body
    if Config.modes.remove_fields then
        local ok, responseBodyTable = pcall(cjson.decode, responseBody)
        if ok and type(responseBodyTable) == "table" then
            -- Remove specified fields
            for _, field in ipairs(Config.remove_fields_list) do
                if responseBodyTable[field] ~= nil then
                    log(string.format("Removing field: %s", field), "DEBUG")
                    responseBodyTable[field] = nil
                end
            end
            responseBody = cjson.encode(responseBodyTable)
        end
    end

    log("Request completed successfully", "DEBUG")
    return {
        status = response.status,
        headers = normalizedHeaders,
        body = responseBody
    }
end

--- Error Handling Utility
local function handleError(applet, statusCode, errorMessage)
    log(errorMessage, "ERROR")
    applet:set_status(statusCode)
    applet:add_header("content-type", "application/json")
    applet:start_response()
    applet:send(createErrorResponse(errorMessage))
end

--- Response Sending Utility
local function sendResponse(applet, response)
    log("Sending Response:", "DEBUG")
    log("Status: " .. tostring(response.status), "DEBUG")
    
    applet:set_status(response.status)
    
    -- Safely add headers
    if response.headers then
        for name, value in pairs(response.headers) do
            log(string.format("Adding header: %s = %s", name, tostring(value)), "DEBUG")
            applet:add_header(name, tostring(value))
        end
    end
    
    applet:start_response()
    
    if response.body then
        applet:send(response.body)
    end
    
    log("=== Request completed ===", "INFO")
end

--- Main HAProxy Service Handler
core.register_service("json_to_form_data", "http", function(applet)
    log("=== New request received ===", "INFO")
    log("Path: " .. applet.path, "DEBUG")

    -- Read request body
    local body = applet:receive()
    if not body then
        return handleError(applet, 400, "Failed to read request body")
    end

    -- Validate content type
    local contentType = applet.headers["content-type"] and applet.headers["content-type"][0]
    if not contentType or not contentType:lower():match("application/json") then
        return handleError(applet, 415, "Expected application/json content type")
    end

    -- Convert JSON to form data
    local formData, errorMessage = jsonToFormData(body)
    if not formData then
        return handleError(applet, 400, errorMessage)
    end

    -- Prepare request headers
    local headersOut = {
        ["content-type"] = "application/x-www-form-urlencoded",
        ["content-length"] = tostring(#formData)
    }

    -- Preserve original non-content/length headers
    for name, values in pairs(applet.headers) do
        local lowerName = name:lower()
        if lowerName ~= "content-type" and lowerName ~= "content-length" then
            headersOut[name] = values[0]
        end
    end

    -- Backend URL construction
    local backendUrl = string.format("http://%s:%s%s", 
        Config.backend.host, 
        Config.backend.port, 
        applet.path
    )

    -- Forward request to backend
    local response, errorMsg = forwardRequest(applet, backendUrl, headersOut, formData)

    if response then
        log("Successfully forwarding response to client", "DEBUG")
        return sendResponse(applet, response)
    else
        return handleError(applet, 503, 
            string.format("Service Unavailable: %s", tostring(errorMsg))
        )
    end
end)

With the updated code snippet, I get the following errors now.

Without dst field being part of my request and my URL value = “http://localhost:8088/realms/Sandbox/protocol/openid-connect/token”, I get the following logs. It is interesting to note that a single API call is being performed here, and my 8088 backend is never called. It is like the HTTP client doesn’t know where exactly to go.

haproxy  | [2024-12-06 14:20:38] INFO - === New request received ===
haproxy  | [2024-12-06 14:20:38] DEBUG - Path: /realms/Sandbox/protocol/openid-connect/token
haproxy  | [2024-12-06 14:20:38] DEBUG - Processing JSON input
haproxy  | [2024-12-06 14:20:38] DEBUG - Parsed values - client_id: test, grant_type: client_credentials
haproxy  | [2024-12-06 14:20:38] DEBUG - Form data generated successfully
haproxy  | [2024-12-06 14:20:38] DEBUG - Forwarding request to: http://localhost:8088/realms/Sandbox/protocol/openid-connect/token
haproxy  | <6>2024-12-06T14:20:41.116428+02:00 -:- [06/Dec/2024:14:20:38.105] <HTTPCLIENT> -/- 5/0/-1/-1/3010 503 217 - - SC-- 1/0/0/0/3 0/0 {::1} "POST http://localhost:8088/realms/Sandbox/protocol/openid-connect/token HTTP/1.1"
haproxy  | [2024-12-06 14:20:41] INFO - HTTP Client Request Details:
haproxy  | [2024-12-06 14:20:41] INFO - URL: http://localhost:8088/realms/Sandbox/protocol/openid-connect/token
haproxy  | [2024-12-06 14:20:41] INFO - Headers: { ["host"] = localhost:8087,["content-length"] = 119,["cookie"] = _auth_verification=%7B%22nonce%22%3A%22B-hRR0m8kJnj-9ea2wywgNfUBahZxilvz7KfTF6ajZc%22%2C%22state%22%3A%22eyJyZXR1cm5UbyI6Ii9hcGkvdjEvaGVhbHRoQ2hlY2sifQ%22%7D.ACJ66m4TyjOjMQaBg0Qyvf1QKyoNxnQEqhvJotdgcaE; auth_verification=%7B%22nonce%22%3A%22B-hRR0m8kJnj-9ea2wywgNfUBahZxilvz7KfTF6ajZc%22%2C%22state%22%3A%22eyJyZXR1cm5UbyI6Ii9hcGkvdjEvaGVhbHRoQ2hlY2sifQ%22%7D.ShqCbQpYXr_UtEiQKHvcj-FXCUDxNH9ccM9_fqLEgww,["user-agent"] = insomnia/8.6.1,["accept"] = */*,["content-type"] = application/x-www-form-urlencoded,} 
haproxy  | [2024-12-06 14:20:41] INFO - Body: client_id=test&client_secret=test&grant_type=client_credentials
haproxy  | [2024-12-06 14:20:41] INFO - Success: true
haproxy  | [2024-12-06 14:20:41] INFO - Response: { ["status"] = 503,["body"] = <html><body><h1>503 Service Unavailable</h1>
haproxy  | No server is available to handle this request.
haproxy  | </body></html>
haproxy  | ,["reason"] = Service Unavailable,["headers"] = { ["content-length"] = { [0] = 107,} ,["content-type"] = { [0] = text/html,} ,["cache-control"] = { [0] = no-cache,} ,} ,} 
haproxy  | [2024-12-06 14:20:41] DEBUG - Request completed successfully
haproxy  | [2024-12-06 14:20:41] DEBUG - Successfully forwarding response to client
haproxy  | [2024-12-06 14:20:41] DEBUG - Sending Response:
haproxy  | [2024-12-06 14:20:41] DEBUG - Status: 503
haproxy  | [2024-12-06 14:20:41] DEBUG - Adding header: content-type = text/html
haproxy  | [2024-12-06 14:20:41] DEBUG - Adding header: content-length = 107
haproxy  | [2024-12-06 14:20:41] DEBUG - Adding header: cache-control = no-cache
haproxy  | [2024-12-06 14:20:41] INFO - === Request completed ===
haproxy  | <3>2024-12-06T14:20:41.116999+02:00 172.18.0.1:62582 - http-keycloak-frontend-8087 keycloak-json-converter/<lua.json_to_form_data> 0/0/0/3010/3010 503 217 - - LR-- 1/1/0/0/0 0/0 "POST /realms/Sandbox/protocol/openid-connect/token HTTP/1.1" 854e86c3-5bab-4e00-95d9-568622984e64 -

Adding the dst = "127.0.0.1:8088" field as part of my request and my URL value = "http://localhost:8088/realms/Sandbox/protocol/openid-connect/token", I get the following logs. It is interesting that it is performing numerous API calls this time towards the backend, but it keeps stating the backend has no servers available, even though the backend is available.
haproxy  | [2024-12-06 14:25:37] INFO - === New request received ===
haproxy  | [2024-12-06 14:25:37] DEBUG - Path: /realms/Sandbox/protocol/openid-connect/token
haproxy  | [2024-12-06 14:25:37] DEBUG - Processing JSON input
haproxy  | [2024-12-06 14:25:37] DEBUG - Parsed values - client_id: test, grant_type: client_credentials
haproxy  | [2024-12-06 14:25:37] DEBUG - Form data generated successfully
haproxy  | [2024-12-06 14:25:37] DEBUG - Forwarding request to: http://localhost:8088/realms/Sandbox/protocol/openid-connect/token
haproxy  | <3>2024-12-06T14:25:37.293074+02:00 127.0.0.1:42136 - http-keycloak-frontend-8088 https-keycloak-8443/keycloak-frontend-8443-01 0/0/-1/-1/67 -1 0 - - CC-- 3/2/0/0/0 0/0 "POST /realms/Sandbox/protocol/openid-connect/token HTTP/1.1" 55ddb940-f679-4ec2-96ff-147c9ff65057 -
haproxy  | <3>2024-12-06T14:25:37.324975+02:00 127.0.0.1:42146 - http-keycloak-frontend-8088 https-keycloak-8443/keycloak-frontend-8443-01 0/0/-1/-1/32 -1 0 - - CC-- 2/1/0/0/0 0/0 "POST /realms/Sandbox/protocol/openid-connect/token HTTP/1.1" 9e2c69f5-03a3-4596-af67-eee3856d16e3 -
haproxy  | <3>2024-12-06T14:25:37.357880+02:00 127.0.0.1:42156 - http-keycloak-frontend-8088 https-keycloak-8443/keycloak-frontend-8443-01 0/0/-1/-1/32 -1 0 - - CC-- 2/1/0/0/0 0/0 "POST /realms/Sandbox/protocol/openid-connect/token HTTP/1.1" 3614f8f0-2ac2-4054-bdb1-96d869dc85d1 -
haproxy  | <6>2024-12-06T14:25:37.389796+02:00 -:- [06/Dec/2024:14:25:37.222] <HTTPCLIENT> -/- 2/0/136/-1/167 504 198 - - sH-- 2/0/0/0/3 0/0 {} "POST http://localhost:8088/realms/Sandbox/protocol/openid-connect/token HTTP/1.1"
haproxy  | <3>2024-12-06T14:25:37.389989+02:00 127.0.0.1:42158 - http-keycloak-frontend-8088 https-keycloak-8443/keycloak-frontend-8443-01 0/0/-1/-1/31 -1 0 - - CC-- 2/1/0/0/0 0/0 "POST /realms/Sandbox/protocol/openid-connect/token HTTP/1.1" e4b8f557-12fd-4f45-b2b0-d13dd7887309 -
haproxy  | [2024-12-06 14:25:37] INFO - HTTP Client Request Details:
haproxy  | [2024-12-06 14:25:37] INFO - URL: http://localhost:8088/realms/Sandbox/protocol/openid-connect/token
haproxy  | [2024-12-06 14:25:37] INFO - Headers: { ["user-agent"] = insomnia/8.6.1,["content-type"] = application/x-www-form-urlencoded,["content-length"] = 119,["cookie"] = _auth_verification=%7B%22nonce%22%3A%22B-hRR0m8kJnj-9ea2wywgNfUBahZxilvz7KfTF6ajZc%22%2C%22state%22%3A%22eyJyZXR1cm5UbyI6Ii9hcGkvdjEvaGVhbHRoQ2hlY2sifQ%22%7D.ACJ66m4TyjOjMQaBg0Qyvf1QKyoNxnQEqhvJotdgcaE; auth_verification=%7B%22nonce%22%3A%22B-hRR0m8kJnj-9ea2wywgNfUBahZxilvz7KfTF6ajZc%22%2C%22state%22%3A%22eyJyZXR1cm5UbyI6Ii9hcGkvdjEvaGVhbHRoQ2hlY2sifQ%22%7D.ShqCbQpYXr_UtEiQKHvcj-FXCUDxNH9ccM9_fqLEgww,["accept"] = */*,["host"] = localhost:8087,} 
haproxy  | [2024-12-06 14:25:37] INFO - Body: client_id=test&client_secret=test&grant_type=client_credentials
haproxy  | [2024-12-06 14:25:37] INFO - Success: true
haproxy  | [2024-12-06 14:25:37] INFO - Response: { ["status"] = 504,["reason"] = Gateway Time-out,["body"] = <html><body><text>504 Gateway Time-out</text>
haproxy  | The server didn't respond in time.
haproxy  | </body></html>
haproxy  | ,["headers"] = { ["content-length"] = { [0] = 92,} ,["cache-control"] = { [0] = no-cache,} ,["content-type"] = { [0] = text/html,} ,} ,} 
haproxy  | [2024-12-06 14:25:37] DEBUG - Request completed successfully
haproxy  | [2024-12-06 14:25:37] DEBUG - Successfully forwarding response to client
haproxy  | [2024-12-06 14:25:37] DEBUG - Sending Response:
haproxy  | [2024-12-06 14:25:37] DEBUG - Status: 504
haproxy  | [2024-12-06 14:25:37] DEBUG - Adding header: content-length = 92
haproxy  | [2024-12-06 14:25:37] DEBUG - Adding header: cache-control = no-cache
haproxy  | [2024-12-06 14:25:37] DEBUG - Adding header: content-type = text/html
haproxy  | [2024-12-06 14:25:37] INFO - === Request completed ===

I have also tested converting my backend server over to webhook.site to double check if Keycloak is not rejecting something, but I also don’t receive anything on webhook.site, which tells me I might be missing a small details now between the LUA service and calling my next frontend / backend section and actually being able to go out to the specific connection on the backend.

Logs to webhook site:

haproxy  | [2024-12-06 14:30:32] INFO - === New request received ===
haproxy  | [2024-12-06 14:30:32] DEBUG - Path: /9b9713a5-aa83-46ed-b9b2-60d71a1b9270
haproxy  | [2024-12-06 14:30:32] DEBUG - Processing JSON input
haproxy  | [2024-12-06 14:30:32] DEBUG - Parsed values - client_id: test, grant_type: client_credentials
haproxy  | [2024-12-06 14:30:32] DEBUG - Form data generated successfully
haproxy  | [2024-12-06 14:30:32] DEBUG - Forwarding request to: http://localhost:8088/9b9713a5-aa83-46ed-b9b2-60d71a1b9270
haproxy  | <3>2024-12-06T14:30:32.088434+02:00 127.0.0.1:54154 - http-keycloak-frontend-8088 https-keycloak-8443/keycloak-frontend-8443-01 0/0/-1/-1/33 -1 0 - - CC-- 3/2/0/0/0 0/0 "POST /9b9713a5-aa83-46ed-b9b2-60d71a1b9270 HTTP/1.1" f036121e-9cf7-4ffc-87fc-b584be500d37 -
haproxy  | <3>2024-12-06T14:30:32.121015+02:00 127.0.0.1:54156 - http-keycloak-frontend-8088 https-keycloak-8443/keycloak-frontend-8443-01 0/0/-1/-1/32 -1 0 - - CC-- 2/1/0/0/0 0/0 "POST /9b9713a5-aa83-46ed-b9b2-60d71a1b9270 HTTP/1.1" 35896caf-37cf-4038-a3f7-d1cf762e05d8 -
haproxy  | <3>2024-12-06T14:30:32.153954+02:00 127.0.0.1:54170 - http-keycloak-frontend-8088 https-keycloak-8443/keycloak-frontend-8443-01 0/0/-1/-1/32 -1 0 - - CC-- 2/1/0/0/0 0/0 "POST /9b9713a5-aa83-46ed-b9b2-60d71a1b9270 HTTP/1.1" dba20f93-2dec-41ee-a878-e3f13dd69cbf -
haproxy  | <6>2024-12-06T14:30:32.186732+02:00 -:- [06/Dec/2024:14:30:32.054] <HTTPCLIENT> -/- 2/0/100/-1/132 504 198 - - sH-- 2/0/0/0/3 0/0 {} "POST http://localhost:8088/9b9713a5-aa83-46ed-b9b2-60d71a1b9270 HTTP/1.1"
haproxy  | <3>2024-12-06T14:30:32.186882+02:00 127.0.0.1:54180 - http-keycloak-frontend-8088 https-keycloak-8443/keycloak-frontend-8443-01 0/0/-1/-1/32 -1 0 - - CC-- 2/1/0/0/0 0/0 "POST /9b9713a5-aa83-46ed-b9b2-60d71a1b9270 HTTP/1.1" d88ca830-fa0a-49dd-8d76-71136c3a1e7d -
haproxy  | [2024-12-06 14:30:32] INFO - HTTP Client Request Details:
haproxy  | [2024-12-06 14:30:32] INFO - URL: http://localhost:8088/9b9713a5-aa83-46ed-b9b2-60d71a1b9270
haproxy  | [2024-12-06 14:30:32] INFO - Headers: { ["content-type"] = application/x-www-form-urlencoded,["user-agent"] = insomnia/8.6.1,["content-length"] = 119,["accept"] = */*,["host"] = localhost:8087,["cookie"] = _auth_verification=%7B%22nonce%22%3A%22B-hRR0m8kJnj-9ea2wywgNfUBahZxilvz7KfTF6ajZc%22%2C%22state%22%3A%22eyJyZXR1cm5UbyI6Ii9hcGkvdjEvaGVhbHRoQ2hlY2sifQ%22%7D.ACJ66m4TyjOjMQaBg0Qyvf1QKyoNxnQEqhvJotdgcaE; auth_verification=%7B%22nonce%22%3A%22B-hRR0m8kJnj-9ea2wywgNfUBahZxilvz7KfTF6ajZc%22%2C%22state%22%3A%22eyJyZXR1cm5UbyI6Ii9hcGkvdjEvaGVhbHRoQ2hlY2sifQ%22%7D.ShqCbQpYXr_UtEiQKHvcj-FXCUDxNH9ccM9_fqLEgww,} 
haproxy  | [2024-12-06 14:30:32] INFO - Body: client_id=test&client_secret=test&grant_type=client_credentials
haproxy  | [2024-12-06 14:30:32] INFO - Success: true
haproxy  | [2024-12-06 14:30:32] INFO - Response: { ["status"] = 504,["headers"] = { ["content-length"] = { [0] = 92,} ,["content-type"] = { [0] = text/html,} ,["cache-control"] = { [0] = no-cache,} ,} ,["reason"] = Gateway Time-out,["body"] = <html><body><text>504 Gateway Time-out</text>
haproxy  | The server didn't respond in time.
haproxy  | </body></html>
haproxy  | ,} 
haproxy  | [2024-12-06 14:30:32] DEBUG - Request completed successfully
haproxy  | [2024-12-06 14:30:32] DEBUG - Successfully forwarding response to client
haproxy  | [2024-12-06 14:30:32] DEBUG - Sending Response:
haproxy  | [2024-12-06 14:30:32] DEBUG - Status: 504
haproxy  | [2024-12-06 14:30:32] DEBUG - Adding header: content-type = text/html
haproxy  | [2024-12-06 14:30:32] DEBUG - Adding header: content-length = 92
haproxy  | [2024-12-06 14:30:32] DEBUG - Adding header: cache-control = no-cache
haproxy  | [2024-12-06 14:30:32] INFO - === Request completed ===
haproxy  | <3>2024-12-06T14:30:32.187730+02:00 172.18.0.1:61406 - http-keycloak-frontend-8087 keycloak-json-converter/<lua.json_to_form_data> 0/0/0/132/132 504 198 - - LR-- 1/1/0/0/0 0/0 "POST /9b9713a5-aa83-46ed-b9b2-60d71a1b9270 HTTP/1.1" 31d9bc5e-3bd0-440d-993e-702de5f1a7d3 -

Making some more progress by checking through the documentation pages. I had a timeout of “30” as well as “65”, but if I am not mistaken, it looks like this field is in milliseconds and not seconds. Changing this over to 30000 now does not close the connection from the client side any longer.

Unrelated but it doesn’t make sense to set the lua burst timeout to 60s, either you want to disable it (not recommended) by setting it to 0, else you want to keep it below 2seconds to prevent the current thread from impacting other threads. I assume probably set this to prevent the watchdog from kicking in when you didn’t use native httpclient previously with blocking functions.

yes indeed timeout is MS by default

Ok so does it work any better for you now ?

For the fact that it failed when you didn’t provide “dst=”, perhaps it’s because it tried to resolve “localhost” during runtime and you don’t have a resolver section so it fails. It’s better if you set an explicit “dst=” anyway since you route the request internally, this way you won’t have bad surprises

Finally got it fully working locally with using the standard core.httpclient.

LUA Script:

--- HAProxy JSON to Form Data Converter
-- @module json_to_form_converter
-- @author Rohan de Jongh
-- @license MIT
-- @description Converts JSON payloads to form-urlencoded data for backend authentication

local cjson = require("cjson")

--- Configuration and Environment Management
local Config = {
    --- Backend Server Configuration
    backend = {
        host = os.getenv("LUA_JSON_TO_FORM_BACKEND_HOST") or "127.0.0.1",
        port = os.getenv("LUA_JSON_TO_FORM_BACKEND_PORT") or "8080"
    },

    --- Operational Modes
    modes = {
        debug = os.getenv("LUA_JSON_TO_FORM_DEBUG_MODE") == "true" or false,
        remove_fields = os.getenv("LUA_JSON_TO_FORM_REMOVE_FIELDS") == "true" or false
    },

    --- Fields to Remove from Response
    remove_fields_list = (function()
        local fields_to_remove = os.getenv("LUA_JSON_TO_FORM_REMOVE_FIELDS_LIST")
        if not fields_to_remove then
            return {}  -- Empty Default list
        end

        local fields = {}

        for field in string.gmatch(fields_to_remove, "[^,]+") do
            local trimmed_field = field:gsub("^%s+", ""):gsub("%s+$", "")
            if trimmed_field ~= "" then
                fields[#fields + 1] = trimmed_field
            end
        end
        return fields
    end)(),

    --- Performance and Security Timeouts
    timeouts = {
        client = tonumber(os.getenv("LUA_JSON_TO_FORM_TIMEOUT_CLIENT")) or 30,
        connect = tonumber(os.getenv("LUA_JSON_TO_FORM_TIMEOUT_CONNECT")) or 5
    }
}

--- Logging Utility
local function log(message, level)
    if Config.modes.debug then
        level = level or "INFO"
        print(string.format("[%s] %s - %s",
            os.date("%Y-%m-%d %H:%M:%S"),
            level,
            message
        ))
    end
end

--- Create a Standardized Error Response
local function createErrorResponse(message)
    log("Error: " .. message, "ERROR")
    return string.format(
        '{"error": "%s"}',
        message:gsub('"', '\\"')
    )
end

--- URL Encoding Utility
local function urlEncode(str)
    if not str then return "" end
    local preserve = { ['_'] = true, ['-'] = true, ['.'] = true }
    return str:gsub("[^%w]", function(c)
        return preserve[c] and c or string.format("%%%02X", string.byte(c))
    end)
end

--- JSON to Form Data Converter
local function jsonToFormData(jsonStr)
    log("Processing JSON input", "DEBUG")

    -- Validate JSON structure
    if not jsonStr:match("^%s*{") or not jsonStr:match("}%s*$") then
        return nil, "Invalid JSON format"
    end

    -- Safely parse JSON
    local ok, parsedJson = pcall(cjson.decode, jsonStr)
    if not ok then
        return nil, "Failed to parse JSON: " .. tostring(parsedJson)
    end

    -- Validate required fields
    local requiredFields = {"client_id", "client_secret", "grant_type"}
    for _, field in ipairs(requiredFields) do
        if not parsedJson[field] then
            return nil, string.format("Missing %s field", field)
        end
    end

    -- Mask sensitive information in logs
    log(string.format("Parsed values - client_id: %s, grant_type: %s",
        parsedJson.client_id:sub(1, 5) .. "*****" .. parsedJson.client_id:sub(-5),
        parsedJson.grant_type), "DEBUG")

    -- Construct form-encoded data
    local formData = string.format(
        "client_id=%s&client_secret=%s&grant_type=%s",
        urlEncode(parsedJson.client_id),
        urlEncode(parsedJson.client_secret),
        urlEncode(parsedJson.grant_type)
    )

    log("Form data generated successfully", "DEBUG")
    return formData
end

--- Forward Request to Backend with core.httpclient()
local function forwardRequest(applet, url, headers, body)
    log("Forwarding request to: " .. url, "DEBUG")

    -- Create HTTP client
    local httpclient = core.httpclient()

    -- Prepare the request
    local req = {
        url = url,
        headers = headers,
        body = body,
        timeout = Config.timeouts.client,
        -- Explicitly set the destination
        dst = "127.0.0.1:8088"
    }

    -- Send the request and wait for response
    local success, response = pcall(function()
        return httpclient:post(req)
    end)

    -- Check for request success
    if not success then
        log("HTTP Request failed: " .. tostring(response), "ERROR")
        return nil, "Connection error: " .. tostring(response)
    end

    -- Normalize headers (extract first value)
    local normalizedHeaders = {}
    for k, v in pairs(response.headers or {}) do
        normalizedHeaders[k] = v[0]
    end

    -- Optional field removal for security/privacy
    local responseBody = response.body
    if Config.modes.remove_fields then
        local ok, responseBodyTable = pcall(cjson.decode, responseBody)
        if ok and type(responseBodyTable) == "table" then
            -- Remove specified fields
            for _, field in ipairs(Config.remove_fields_list) do
                if responseBodyTable[field] ~= nil then
                    log(string.format("Removing field: %s", field), "DEBUG")
                    responseBodyTable[field] = nil
                end
            end
            responseBody = cjson.encode(responseBodyTable)

            -- Update content-length header to match new body length
            normalizedHeaders["content-length"] = tostring(#responseBody)
        end
    end

    log("Request completed successfully", "DEBUG")
    return {
        status = response.status,
        headers = normalizedHeaders,
        body = responseBody
    }
end

--- Error Handling Utility
local function handleError(applet, statusCode, errorMessage)
    log(errorMessage, "ERROR")
    applet:set_status(statusCode)
    applet:add_header("content-type", "application/json")
    applet:start_response()
    applet:send(createErrorResponse(errorMessage))
end

--- Response Sending Utility
local function sendResponse(applet, response)
    log("Sending Response:", "DEBUG")
    log("Status: " .. tostring(response.status), "DEBUG")

    applet:set_status(response.status)

    -- Safely add headers
    if response.headers then
        for name, value in pairs(response.headers) do
            log(string.format("Adding header: %s = %s", name, tostring(value)), "DEBUG")
            applet:add_header(name, tostring(value))
        end
    end

    applet:start_response()
    if response.body then
        applet:send(response.body)
    end

    log("=== Request completed ===", "INFO")
end

--- Main HAProxy Service Handler
core.register_service("json_to_form_data", "http", function(applet)
    log("=== New request received ===", "INFO")
    log("Path: " .. applet.path, "DEBUG")

    -- Read request body
    local body = applet:receive()
    if not body then
        return handleError(applet, 400, "Failed to read request body")
    end

    -- Validate content type
    local contentType = applet.headers["content-type"] and applet.headers["content-type"][0]
    if not contentType or not contentType:lower():match("application/json") then
        return handleError(applet, 415, "Expected application/json content type")
    end

    -- Convert JSON to form data
    local formData, errorMessage = jsonToFormData(body)
    if not formData then
        return handleError(applet, 400, errorMessage)
    end

    -- Prepare request headers
    local headersOut = {
        ["content-type"] = { "application/x-www-form-urlencoded" },
        ["content-length"] = { tostring(#formData) }
    }

    -- Preserve original non-content/length headers
    for name, values in pairs(applet.headers) do
        local lowerName = name:lower()
        if lowerName ~= "content-type" and lowerName ~= "content-length" then
            headersOut[name] = values[0]
        end
    end

    -- Backend URL construction
    local backendUrl = string.format("http://%s:%s%s",
        Config.backend.host,
        Config.backend.port,
        applet.path
    )

    -- Forward request to backend
    local response, errorMsg = forwardRequest(applet, backendUrl, headersOut, formData)

    if response then
        log("Successfully forwarding response to client", "DEBUG")
        return sendResponse(applet, response)
    else
        return handleError(applet, 503,
            string.format("Service Unavailable: %s", tostring(errorMsg))
        )
    end
end)
1 Like