Haproxy2.6 memory leak with lua filters

Lua Scripts
1

Hello

Getting the below crash messages from haproxy , seems to be issue with my filter code, please help with debugging. I can find one potential problem that the “body“ variable hasnt been declared as local. Is there anything else thats wrong with the below code?

haproxy version : 2.6
Filter code is below, took it from documentation : How Lua runs in HAProxy — haproxy-lua 2.8.16-2dd8e3-17 (Mon Nov 3 02:03:22 CET 2025) 1.0 documentation

TestFilter = {}
TestFilter.id = “Lua Sts filter”
TestFilter.flags = filter.FLT_CFG_FL_HTX;
TestFilter.__index = TestFilter

function TestFilter:new()
local trace = {}
setmetatable(trace, TestFilter)
trace.res_len = 0
return trace
end

function is_sts_transaction(txn,chn)
if chn:is_resp() then
local is_test_setvar_analyze = txn:get_var(“txn.is_test”)
if is_test_setvar_analyze ~= nil then
return true
end
end
return false
end

function TestFilter:start_analyze(txn, chn)
if chn and chn:is_resp() then
filter.register_data_filter(self, chn)
end
end

function TestFilter:end_analyze(txn, chn)
–if chn and chn:is_resp() then
–    filter:unregister_data_filter()
–end
end

function TestFilter:http_payload(txn, http_msg)

if http_msg ~= nil and type(http_msg) == "table" then
    if http_msg.channel ~= nil  and type(http_msg.channel) == "table" then
        if http_msg.channel:is_resp()  then
            local is_test_setvar = txn:get_var("txn.is_test")
            if is_test_setvar ~= nil and type(http_msg.body) == "function" then
                body = http_msg:body(-930) -- >> Potential problem fixed this
                if body ~= nil and type(body) == "string" and #body > 0 then
                    core.Info("body="..body)
                end
            end
        end
    end
end

end

core.register_filter(“TestFilter”, TestFilter, function(TestFilter, args)
return TestFilter
end)

Crash log and trace is below


Downloading separate debug info for /lib64/libssl.so.1.1…
Downloading separate debug info for /lib64/libcrypto.so.1.1…
Downloading separate debug info for /lib64/libpcreposix.so.0…
[Thread debugging using libthread_db enabled]
Using host libthread_db library “/lib64/libthread_db.so.1”.
Core was generated by `/usr/bin/test3/haproxy/haproxy -f
/usr/bin/test3/haproxy/haproxy.conf’.
Program terminated with signal SIGABRT, Aborted.
Downloading source file
/usr/src/debug/glibc-2.28-251.el8_10.25.x86_64/signal/../sysdeps/unix/sysv/linux
/raise.c…
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50

warning: 50     ../sysdeps/unix/sysv/linux/raise.c: No such file or directory
[Current thread is 1 (Thread 0x7f2d38826700 (LWP 89671))]
(gdb) >>>>>>>>>>
Id   Target Id                         Frame

1    Thread 0x7f2d38826700 (LWP 89671) __GI_raise (sig=sig@entry=6)
at ../sysdeps/unix/sysv/linux/raise.c:50
2    Thread 0x7f2d3a029700 (LWP 89668) 0x00007f2d40c76487 in epoll_wait (
epfd=16, events=0x7f2d28027740, maxevents=200, timeout=timeout@entry=9321)
at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
3    Thread 0x7f2d3a82a700 (LWP 89667) 0x00007f2d40c76487 in epoll_wait (
epfd=13, events=0x7f2d30027740, maxevents=200, timeout=timeout@entry=9321)
at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
4    Thread 0x7f2d4091e700 (LWP 89665) 0x00007f2d40c3f178 in __GI___nanosleep
(requested_time=requested_time@entry=0x7f2d408fb3a0,
remaining=remaining@entry=0x0) at ../sysdeps/unix/sysv/linux/nanosleep.c:28
5    Thread 0x7f2d39828700 (LWP 89669) 0x00007f2d40c76487 in epoll_wait (
epfd=24, events=0x7f2d20027740, maxevents=200, timeout=timeout@entry=762)
at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
6    Thread 0x7f2d428f8240 (LWP 89663) pl_wait_unlock_long (
lock=lock@entry=0x769bd0 <hlua_global_lock>, mask=18446744069414584320)
at include/import/plock.h:96
7    Thread 0x7f2d39027700 (LWP 89670) 0x00007f2d40c76487 in epoll_wait (
epfd=19, events=0x7f2d2c027740, maxevents=200, timeout=timeout@entry=2)
at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
8    Thread 0x7f2d3b02b700 (LWP 89666) 0x00007f2d40c76487 in epoll_wait (
epfd=10, events=0x7f2d3c027e10, maxevents=200, timeout=timeout@entry=698)
at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
9    Thread 0x7f2d38025700 (LWP 89672) 0x00007f2d40c76487 in epoll_wait (
epfd=27, events=0x7f2d24027740, maxevents=200, timeout=timeout@entry=9321)
at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
rax            0x0                 0
rbx            0x6                 6
rcx            0x7f2d40b8552f      139832336078127
rdx            0x0                 0
rsi            0x7f2d38802c40      139832198179904
rdi            0x2                 2
rbp            0x7f2d38802f90      0x7f2d38802f90
rsp            0x7f2d38802c40      0x7f2d38802c40
r8             0x0                 0
r9             0x7f2d38802c40      139832198179904
r10            0x8                 8
r11            0x246               582
r12            0x7f2d42905000      139832367009792
r13            0x7f2d38802eb0      139832198180528
r14            0x10                16
r15            0x1000              4096
rip            0x7f2d40b8552f      0x7f2d40b8552f <__GI_raise+271>
eflags         0x246               [ PF ZF IF ]
cs             0x33                51
ss             0x2b                43
ds             0x0                 0
es             0x0                 0
fs             0x0                 0
gs             0x0                 0
k0             0x40000001          1073741825
k1             0xffffff            16777215
k2             0x0                 0
k3             0x0                 0
k4             0xffffffc0          4294967232
k5             0x12000000          301989888
k6             0x10000             65536
k7             0xffffffc0          4294967232
fs_base        0x7f2d38826700      139832198326016
gs_base        0x0                 0

BEGIN ACTIVE THREAD WALKBACK: /usr/bin/test3/haproxy/haproxy
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x00007f2d40b58e65 in __GI_abort () at abort.c:79
#2  0x00007f2d40bc6807 in __libc_message (action=action@entry=do_abort,
fmt=fmt@entry=0x7f2d40cd0f78 “%s\n”) at ../sysdeps/posix/libc_fatal.c:182
#3  0x00007f2d40bcdb0c in malloc_printerr (
str=str@entry=0x7f2d40cd2c98 “double free or corruption (fasttop)”)
at malloc.c:5449
#4  0x00007f2d40bcf7ed in int_free (av=0x7f2d2c000020, p=0x7f2d2c0f8be0,
have_lock=) at malloc.c:4343
#5  0x0000000000456f43 in hlua_alloc (ud=0x769c40 <hlua_global_allocator>,
ptr=, osize=, nsize=)
at src/hlua.c:13218
#6  0x00000000006177af in luaM_realloc ()
#7  0x000000000061dee0 in luaH_free ()
#8  0x0000000000616215 in singlestep ()
#9  0x000000000061739c in luaC_step ()
#10 0x000000000045737c in hlua_txn_new (L=0x7f2d1866aa08,
s=s@entry=0x7f2d1827c0b0, p=0x1f62a00, dir=dir@entry=0,
flags=flags@entry=48) at src/hlua.c:8044
#11 0x000000000046aad1 in hlua_filter_callback (s=,
filter=, fun=, dir=,
flags=) at src/hlua.c:12001
#12 0x00000000005a7985 in flt_start_analyze (s=s@entry=0x7f2d1827c0b0,
chn=chn@entry=0x7f2d1827c0d0, an_bit=an_bit@entry=1) at src/filters.c:747
#13 0x00000000004caa04 in process_stream (t=t@entry=0x7f2d2c5ef1d0,
context=0x7f2d1827c0b0, state=) at src/stream.c:2017
#14 0x00000000005a0ac3 in run_tasks_from_lists (
budgets=budgets@entry=0x7f2d38803330) at src/task.c:632
#15 0x00000000005a1358 in process_runnable_tasks () at src/task.c:876
#16 0x000000000056f637 in run_poll_loop () at src/haproxy.c:2968
#17 0x000000000056fc17 in run_thread_poll_loop (data=)
at src/haproxy.c:3167
#18 0x00007f2d40f161ca in start_thread (arg=)
at pthread_create.c:479
#19 0x00007f2d40b708d3 in clone ()
at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

END   ACTIVE THREAD WALKBACK: /usr/bin/test3/haproxy/haproxy
**** END   walkback for /usr/bin/test3/haproxy/haproxy
2

Actually, if someone has more working examples of filter code than the one in How Lua runs in HAProxy — haproxy-lua 2.8.16-2dd8e3-17 (Wed Nov 5 02:03:25 CET 2025) 1.0 documentation , that would be great.

3

What version are you using exactly?

4

Apologies, the crash scans were generated with haproxy 2.8-dev2 : https://bbgithub.dev.bloomberg.com/storageproduct/haproxy/commit/0f29b34e0a06cdd59ae2278d33c16f63ca435468

5

2.8-dev2 is a development version so it can suffer from a wide range of bugs, you need to use the latest 2.8 stable version instead (2.8.16 to this day)

1 Like
6

Thanks, appreciate the reply.
Update : I was trying different versions and upgraded to 3.2.0 , commit e134140d282c006417945d78e7964cc8fa14586a .. The crash seemed to go away

7

Another potential issue which I noted- There seems to be a significant hit to performance . Ill close this and open another topic for that with more specifics.

8

Always use the latest bugfix release in a branch.

If you want to use 3.2, use 3.2.9 (as of today), not 3.2.0.
If you want to use 2.8, use 2.8.16 as of today as Aurelien mentioned.

As an example, 3.2.0 has 229 bugs that are already fixed, only 118 of them are considered minor, the rest are medium/major/critical bugs.