Tcp mode keylog for TLS 1.3

Hello,

I am using haproxy (version 2.6.12) as a TLS proxy to serve a local TCP server.

I would like to log the TLS secret key as I was doing for TLS1.2 (with a lua on a tcp-request content and txn.sf:ssl_fc_session_key).

But for TLS1.3 I am getting nowhere, the variables are always empty.

Here is the extract of my configuration:

global
	log stdout format raw daemon
	user root
	group root
	tune.ssl.keylog on
	lua-load sslkeylogger.lua
    ssl-default-bind-ciphersuites TLS_AES_256_GCM_SHA384
    ssl-default-server-ciphersuites TLS_AES_256_GCM_SHA38

frontend remote_client_to_local_server
 	# try to log an element
	tcp-request session set-var(sess.early_secret) ssl_fc_client_early_traffic_secret
	log-format "ssl_fc_client_early_traffic_secret: %[var(sess.early_secret)]"
 	# tcp-request content lua.sslkeylog /tmp/key.log if { ssl_fc }
     
	bind 10.85.221.14:14443 ssl crt crt.pem ssl-min-ver TLSv1.3 
	default_backend backend_remote_client_to_local_server

backend backend_remote_client_to_local_server
	mode tcp
	tcp-request content lua.sslkeylog /tmp/key.log if { ssl_fc }
	server local_server 127.0.0.1:14443

The lua script is the one provided here: haproxy/dev/sslkeylogger/sslkeylogger.lua at e1c8bfd0ed960d3b3dec39e78ad75bec117912d0 · haproxy/haproxy · GitHub

I added the tcp to the action register : core.register_action('sslkeylog', { "tcp-req", "http-req" }, sslkeylog, 1)

None of the variable define in the documentation (such as ssl_fc_client_early_traffic_secret) contains an element, they are always empty.

Also, the session variable (sess.early_secret) is empty too.

I don’t know what I might be doing wrong or if it is even possible for TLS1.3…

Thanks for your replies !

Any news on this ?

Thanks

I would suggest you use the documented approach, as opposed to external LUA scripts:

http://docs.haproxy.org/3.0/configuration.html#3.2-tune.ssl.keylog

Hello,

Thank you for the reply.

I am using Haproxy version 2.6.12 and from the documentation (HAProxy version 2.6.19 - Configuration Manual), it is written to use ssl_fs_* options and all the ssl_bc_server options from the 3.0 documentation do not exist.

When I try to use a simple log-format as described in your link (and replacing all bc by fc):

  log-format "CLIENT_EARLY_TRAFFIC_SECRET %[ssl_bc_client_random,hex] %[ssl_bc_client_early_traffic_secret]\n
              CLIENT_HANDSHAKE_TRAFFIC_SECRET %[ssl_bc_client_random,hex] %[ssl_bc_client_handshake_traffic_secret]\n
              SERVER_HANDSHAKE_TRAFFIC_SECRET %[ssl_bc_client_random,hex] %[ssl_bc_server_handshake_traffic_secret]\n
              CLIENT_TRAFFIC_SECRET_0 %[ssl_bc_client_random,hex] %[ssl_bc_client_traffic_secret_0]\n
              SERVER_TRAFFIC_SECRET_0 %[ssl_bc_client_random,hex] %[ssl_bc_server_traffic_secret_0]\n
              EXPORTER_SECRET %[ssl_bc_client_random,hex] %[ssl_bc_exporter_secret]\n
              EARLY_EXPORTER_SECRET %[ssl_bc_client_random,hex] %[ssl_bc_early_exporter_secret]"

The output is empty during the configuration attempt (only the client random exist):

CLIENT_EARLY_TRAFFIC_SECRET A730A7A6A2E469197163AD8172DFA0825E5DE32F1BAFF2049238B4443F0F614D -
CLIENT_HANDSHAKE_TRAFFIC_SECRET A730A7A6A2E469197163AD8172DFA0825E5DE32F1BAFF2049238B4443F0F614D -
SERVER_HANDSHAKE_TRAFFIC_SECRET A730A7A6A2E469197163AD8172DFA0825E5DE32F1BAFF2049238B4443F0F614D -
CLIENT_TRAFFIC_SECRET_0 A730A7A6A2E469197163AD8172DFA0825E5DE32F1BAFF2049238B4443F0F614D -
SERVER_TRAFFIC_SECRET_0 A730A7A6A2E469197163AD8172DFA0825E5DE32F1BAFF2049238B4443F0F614D -
EXPORTER_SECRET A730A7A6A2E469197163AD8172DFA0825E5DE32F1BAFF2049238B4443F0F614D -
EARLY_EXPORTER_SECRET A730A7A6A2E469197163AD8172DFA0825E5DE32F1BAFF2049238B4443F0F614D -

Is there something else to do than tune.ssl.keylog on to activate the key logging or may be haproxy version 2.12 is doing something wrong in TCP mode ?

Indeed you need at least haproxy 3.0 for those variables. The reason they are not in the documentation for releases older than 3.0 is that they are not supported.

Yes, that’s why I used ssl_fc_client_early_traffic_secret and cie, as described in 2.6 documentation. But they are never set / contain empty data and my question is why so ?

Can you provide the full, unredacted output of haproxy -vv ?

Sure:

haproxy -vv
HAProxy version 2.6.12-1+deb12u1 2023/12/16 - https://haproxy.org/
Status: long-term supported branch - will stop receiving fixes around Q2 2027.
Known bugs: http://www.haproxy.org/bugs/bugs-2.6.12.html
Running on: Linux 6.1.0-18-arm64 #1 SMP PREEMPT @1710115200 aarch64
Build options :
  TARGET  = linux-glibc
  CPU     = generic
  CC      = cc
  CFLAGS  = -O2 -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -Wall -Wextra -Wundef -Wdeclaration-after-statement -Wfatal-errors -Wtype-limits -Wshift-negative-value -Wshift-overflow=2 -Wduplicated-cond -Wnull-dereference -fwrapv -Wno-address-of-packed-member -Wno-unused-label -Wno-sign-compare -Wno-unused-parameter -Wno-clobbered -Wno-missing-field-initializers -Wno-cast-function-type -Wno-string-plus-int -Wno-atomic-alignment
  OPTIONS = USE_PCRE2=1 USE_PCRE2_JIT=1 USE_OPENSSL=1 USE_LUA=1 USE_SLZ=1 USE_SYSTEMD=1 USE_OT=1 USE_PROMEX=1
  DEBUG   = -DDEBUG_STRICT -DDEBUG_MEMORY_POOLS

Feature list : -51DEGREES +ACCEPT4 +BACKTRACE -CLOSEFROM +CPU_AFFINITY +CRYPT_H -DEVICEATLAS +DL -ENGINE +EPOLL -EVPORTS +GETADDRINFO -KQUEUE +LIBCRYPT +LINUX_SPLICE +LINUX_TPROXY +LUA -MEMORY_PROFILING +NETFILTER +NS -OBSOLETE_LINKER +OPENSSL +OT -PCRE +PCRE2 +PCRE2_JIT -PCRE_JIT +POLL +PRCTL -PROCCTL +PROMEX -QUIC +RT +SLZ -STATIC_PCRE -STATIC_PCRE2 +SYSTEMD +TFO +THREAD +THREAD_DUMP +TPROXY -WURFL -ZLIB

Default settings :
  bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with multi-threading support (MAX_THREADS=64, default=1).
Built with OpenSSL version : OpenSSL 3.0.11 19 Sep 2023
Running on OpenSSL version : OpenSSL 3.0.11 19 Sep 2023
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
OpenSSL providers loaded : default
Built with Lua version : Lua 5.3.6
Built with the Prometheus exporter as a service
Built with network namespace support.
Built with OpenTracing support.
Support for malloc_trim() is enabled.
Built with libslz for stateless compression.
Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
Built with PCRE2 version : 10.42 2022-12-11
PCRE2 library supports JIT : yes
Encrypted password support via crypt(3): yes
Built with gcc compiler version 12.2.0

Available polling systems :
      epoll : pref=300,  test result OK
       poll : pref=200,  test result OK
     select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

Available multiplexer protocols :
(protocols marked as <default> cannot be specified using 'proto' keyword)
         h2 : mode=HTTP  side=FE|BE  mux=H2    flags=HTX|HOL_RISK|NO_UPG
       fcgi : mode=HTTP  side=BE     mux=FCGI  flags=HTX|HOL_RISK|NO_UPG
  <default> : mode=HTTP  side=FE|BE  mux=H1    flags=HTX
         h1 : mode=HTTP  side=FE|BE  mux=H1    flags=HTX|NO_UPG
  <default> : mode=TCP   side=FE|BE  mux=PASS  flags=
       none : mode=TCP   side=FE|BE  mux=PASS  flags=NO_UPG

Available services : prometheus-exporter
Available filters :
        [CACHE] cache
        [COMP] compression
        [FCGI] fcgi-app
        [  OT] opentracing
        [SPOE] spoe
        [TRACE] trace 

It works fine for me on Debian 12, ARM64, haproxy 2.6 as shipped with debian, I don’t see a reason why it doesn’t work for you.

root@debian-4gb-hel1-1:~# haproxy -vv
HAProxy version 2.6.12-1+deb12u1 2023/12/16 - https://haproxy.org/
Status: long-term supported branch - will stop receiving fixes around Q2 2027.
Known bugs: http://www.haproxy.org/bugs/bugs-2.6.12.html
Running on: Linux 6.1.0-26-arm64 #1 SMP Debian 6.1.112-1 (2024-09-30) aarch64
Build options :
  TARGET  = linux-glibc
  CPU     = generic
  CC      = cc
  CFLAGS  = -O2 -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -Wall -Wextra -Wundef -Wdeclaration-after-statement -Wfatal-errors -Wtype-limits -Wshift-negative-value -Wshift-overflow=2 -Wduplicated-cond -Wnull-dereference -fwrapv -Wno-address-of-packed-member -Wno-unused-label -Wno-sign-compare -Wno-unused-parameter -Wno-clobbered -Wno-missing-field-initializers -Wno-cast-function-type -Wno-string-plus-int -Wno-atomic-alignment
  OPTIONS = USE_PCRE2=1 USE_PCRE2_JIT=1 USE_OPENSSL=1 USE_LUA=1 USE_SLZ=1 USE_SYSTEMD=1 USE_OT=1 USE_PROMEX=1
  DEBUG   = -DDEBUG_STRICT -DDEBUG_MEMORY_POOLS

Feature list : -51DEGREES +ACCEPT4 +BACKTRACE -CLOSEFROM +CPU_AFFINITY +CRYPT_H -DEVICEATLAS +DL -ENGINE +EPOLL -EVPORTS +GETADDRINFO -KQUEUE +LIBCRYPT +LINUX_SPLICE +LINUX_TPROXY +LUA -MEMORY_PROFILING +NETFILTER +NS -OBSOLETE_LINKER +OPENSSL +OT -PCRE +PCRE2 +PCRE2_JIT -PCRE_JIT +POLL +PRCTL -PROCCTL +PROMEX -QUIC +RT +SLZ -STATIC_PCRE -STATIC_PCRE2 +SYSTEMD +TFO +THREAD +THREAD_DUMP +TPROXY -WURFL -ZLIB

Default settings :
  bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with multi-threading support (MAX_THREADS=64, default=2).
Built with OpenSSL version : OpenSSL 3.0.11 19 Sep 2023
Running on OpenSSL version : OpenSSL 3.0.14 4 Jun 2024
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
OpenSSL providers loaded : default
Built with Lua version : Lua 5.3.6
Built with the Prometheus exporter as a service
Built with network namespace support.
Built with OpenTracing support.
Support for malloc_trim() is enabled.
Built with libslz for stateless compression.
Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
Built with PCRE2 version : 10.42 2022-12-11
PCRE2 library supports JIT : yes
Encrypted password support via crypt(3): yes
Built with gcc compiler version 12.2.0

Available polling systems :
      epoll : pref=300,  test result OK
       poll : pref=200,  test result OK
     select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

Available multiplexer protocols :
(protocols marked as <default> cannot be specified using 'proto' keyword)
         h2 : mode=HTTP  side=FE|BE  mux=H2    flags=HTX|HOL_RISK|NO_UPG
       fcgi : mode=HTTP  side=BE     mux=FCGI  flags=HTX|HOL_RISK|NO_UPG
  <default> : mode=HTTP  side=FE|BE  mux=H1    flags=HTX
         h1 : mode=HTTP  side=FE|BE  mux=H1    flags=HTX|NO_UPG
  <default> : mode=TCP   side=FE|BE  mux=PASS  flags=
       none : mode=TCP   side=FE|BE  mux=PASS  flags=NO_UPG

Available services : prometheus-exporter
Available filters :
        [CACHE] cache
        [COMP] compression
        [FCGI] fcgi-app
        [  OT] opentracing
        [SPOE] spoe
        [TRACE] trace

root@debian-4gb-hel1-1:~#
root@debian-4gb-hel1-1:~#
root@debian-4gb-hel1-1:~# cat haproxy.cfg
global
        log stdout format raw local7 debug
        user root
        group root
        tune.ssl.keylog on
    ssl-default-bind-ciphersuites TLS_AES_256_GCM_SHA384
    ssl-default-server-ciphersuites TLS_AES_256_GCM_SHA38

defaults
 log global

frontend remote_client_to_local_server


        bind :443 ssl crt /root/ubuntuvm.crt ssl-min-ver TLSv1.3
log-format "CLIENT_EARLY_TRAFFIC_SECRET %[ssl_fc_client_random,hex] %[ssl_fc_client_early_traffic_secret] CLIENT_HANDSHAKE_TRAFFIC_SECRET %[ssl_fc_client_random,hex] %[ssl_fc_client_handshake_traffic_secret]SERVER_HANDSHAKE_TRAFFIC_SECRET %[ssl_fc_client_random,hex] %[ssl_fc_server_handshake_traffic_secret] CLIENT_TRAFFIC_SECRET_0 %[ssl_fc_client_random,hex] %[ssl_fc_client_traffic_secret_0]SERVER_TRAFFIC_SECRET_0 %[ssl_fc_client_random,hex] %[ssl_fc_server_traffic_secret_0]EXPORTER_SECRET %[ssl_fc_client_random,hex] %[ssl_fc_exporter_secret]EARLY_EXPORTER_SECRET %[ssl_fc_client_random,hex] %[ssl_fc_early_exporter_secret]"



        default_backend backend_remote_client_to_local_server

backend backend_remote_client_to_local_server
        mode tcp
        server s1 127.0.0.1:22



root@debian-4gb-hel1-1:~#
root@debian-4gb-hel1-1:~#
root@debian-4gb-hel1-1:~# sudo haproxy -f haproxy.cfg -d
[NOTICE]   (2804) : haproxy version is 2.6.12-1+deb12u1
[NOTICE]   (2804) : path to executable is /usr/sbin/haproxy
[WARNING]  (2804) : config : missing timeouts for frontend 'remote_client_to_local_server'.
   | While not properly invalid, you will certainly encounter various problems
   | with such a configuration. To fix this, please ensure that all following
   | timeouts are set to a non-zero value: 'client', 'connect', 'server'.
[WARNING]  (2804) : config : missing timeouts for backend 'backend_remote_client_to_local_server'.
   | While not properly invalid, you will certainly encounter various problems
   | with such a configuration. To fix this, please ensure that all following
   | timeouts are set to a non-zero value: 'client', 'connect', 'server'.
Note: setting global.maxconn to 524279.
Available polling systems :
      epoll : pref=300,  test result OK
       poll : pref=200,  test result OK
     select : pref=150,  test result FAILED
Total: 3 (2 usable), will use epoll.

Available filters :
        [CACHE] cache
        [COMP] compression
        [FCGI] fcgi-app
        [  OT] opentracing
        [SPOE] spoe
        [TRACE] trace
Using epoll() as the polling mechanism.
00000000:remote_client_to_local_server.accept(0004)=000a from [81.161.232.1:43472] ALPN=<none>
CLIENT_EARLY_TRAFFIC_SECRET A2E529B97C24F2D7427C0298DD6C3EAFE83E100CA2CBDD0390405B8550898413 - CLIENT_HANDSHAKE_TRAFFIC_SECRET A2E529B97C24F2D7427C0298DD6C3EAFE83E100CA2CBDD0390405B8550898413 17199972ff30d3bdf48fb92a929df929bc4002e857e2ad2d2b32377f853a7d57a93e7aaf33ae7b19aad7c1d07b7f6e92SERVER_HANDSHAKE_TRAFFIC_SECRET A2E529B97C24F2D7427C0298DD6C3EAFE83E100CA2CBDD0390405B8550898413 2a17facfa96fcf0062837707a93882d99dd597db2b90a50ed454fe11ae57a81617f5eaf7ab83acaa5ae4c4da7a78cfd9 CLIENT_TRAFFIC_SECRET_0 A2E529B97C24F2D7427C0298DD6C3EAFE83E100CA2CBDD0390405B8550898413 ad28030d2f3f0d237e755d84afe88a9a2265f54cf380324a649154e5b3587254c46207169de04759ba0a145a67b35020SERVER_TRAFFIC_SECRET_0 A2E529B97C24F2D7427C0298DD6C3EAFE83E100CA2CBDD0390405B8550898413 db9d6b7f72bad7a4f8c2e1d00d3f62d61f665c8dc4959a2d392818fd21b77ff6f59a09349248c73a1d2e13a657331769EXPORTER_SECRET A2E529B97C24F2D7427C0298DD6C3EAFE83E100CA2CBDD0390405B8550898413 611ee3e696974eb349a63a88c1c5deaa6ac6409eed2fd00fa7d8fa2955c670613a8b72f8e1adebb1d2897bb06f
00000000:backend_remote_client_to_local_server.srvcls[000a:ffff]
00000000:backend_remote_client_to_local_server.clicls[000a:ffff]
00000000:backend_remote_client_to_local_server.closed[000a:ffff]


    C
root@debian-4gb-hel1-1:~#

Sorry, I forgot to swap back to TLS1.3…

So it is working with a log now, I will not use the lua script.

The log-format:

frontend 
   	log-format "CLIENT_EARLY_TRAFFIC_SECRET %[ssl_fc_client_random,hex] %[ssl_fc_client_early_traffic_secret]\n CLIENT_HANDSHAKE_TRAFFIC_SECRET %[ssl_fc_client_random,hex] %[ssl_fc_client_handshake_traffic_secret]\n SERVER_HANDSHAKE_TRAFFIC_SECRET %[ssl_fc_client_random,hex] %[ssl_fc_server_handshake_traffic_secret]\n CLIENT_TRAFFIC_SECRET_0 %[ssl_fc_client_random,hex] %[ssl_fc_client_traffic_secret_0]\n SERVER_TRAFFIC_SECRET_0 %[ssl_fc_client_random,hex] %[ssl_fc_server_traffic_secret_0]\n EXPORTER_SECRET %[ssl_fc_client_random,hex] %[ssl_fc_exporter_secret]\n EARLY_EXPORTER_SECRET %[ssl_fc_client_random,hex] %[ssl_fc_early_exporter_secret]"

Thanks a lot

1 Like