HAProxy Logs for Redirects

We have some redirects set up in HAProxy and when the redirection happens, we can see it in the logs as expected, but sometimes it reports the correct status_code, and other times it doesn’t. It’s almost like something it overwriting the %ST log variable so that when the redirect happens it logs that value instead of the value sent to the client. I should note that the operation of the redirects is working as intended–this is purely a problem with what we are seeing in the logs.

Given the following log settings:

  nbthread 4
  log local0

  log global
  option dontlognull
  mode http

frontend http
  option httplog
  log-format %ci:%cp\ %T\ %ft\ %b/%s\ %Tq/%Tw/%Tc/%Tr/%Tt\ %ST\ %B\ %CC\ %CS\ %tsc\ %ac/%fc/%bc/%sc/%rc\ %sq/%bq\ %hr\ %H\ %{+Q}r
  redirect scheme https code 301

frontend default
  option httplog
  log-format %ci:%cp\ %T\ %ft\ %b/%s\ %Tq/%Tw/%Tc/%Tr/%Tt\ %ST\ %B\ %CC\ %CS\ %tsc\ %ac/%fc/%bc/%sc/%rc\ %sq/%bq\ %hr\ %H\ %{+Q}r
  redirect prefix https://location1 code 301 if is_acl_1 !is_acl_2

So when we send in an HTTP request, we correctly get a 301 to the client with the correct location and the logs show this as well

x.x.x.x:42973 06/Dec/2019:17:45:59 +0000 http http/<NOSRV> 0/-1/-1/-1/0 301 95 - - LR-- 7108/16/0/0/0 0/0 <hostname> "GET / HTTP/1.1"

When we send in an HTTPS request, we correctly get the 301 status_code to the client with the correct location, but our log looks like this

x.x.x.x:26080 06/Dec/2019:17:45:59 +0000 default default/<NOSRV> 60/-1/-1/-1/60 503 343 - - LR-- 7107/7090/0/0/0 0/0 {<headers>} <hostname> "GET <path> HTTP/1.1"

Please note that I have inserted the x.x.x.x, <headers>, <hostname>, and <path> strings to mask potentially sensitive data.

We do have a lot of other configuration values in there that I have left out, but we can start with this and I can add more configuration if needed to debug this further.

Haproxy version output

HA-Proxy version 2.0.7 2019/09/27 - https://haproxy.org/
Build options :
  TARGET  = linux-glibc
  CPU     = generic
  CC      = gcc
  CFLAGS  = -O2 -g -fno-strict-aliasing -Wdeclaration-after-statement -fwrapv -Wno-format-truncation -Wno-unused-label -Wno-sign-compare -Wno-unused-parameter -Wno-old-style-declaration -Wno-ignored-qualifiers -Wno-clobbered -Wno-missing-field-initializers -Wno-implicit-fallthrough -Wno-stringop-overflow -Wno-cast-function-type -Wtype-limits -Wshift-negative-value -Wshift-overflow=2 -Wduplicated-cond -Wnull-dereference


Default settings :
  bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with multi-threading support (MAX_THREADS=64, default=8).
Built with OpenSSL version : OpenSSL 1.1.1c  28 May 2019
Running on OpenSSL version : OpenSSL 1.1.1c  28 May 2019
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
Built with Lua version : Lua 5.3.3
Built with network namespace support.
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
Built with zlib version : 1.2.11
Running on zlib version : 1.2.11
Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with PCRE2 version : 10.32 2018-09-10
PCRE2 library supports JIT : yes
Encrypted password support via crypt(3): yes
Built with the Prometheus exporter as a service

Available polling systems :
      epoll : pref=300,  test result OK
       poll : pref=200,  test result OK
     select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

Available multiplexer protocols :
(protocols marked as <default> cannot be specified using 'proto' keyword)
              h2 : mode=HTX        side=FE|BE     mux=H2
              h2 : mode=HTTP       side=FE        mux=H2
       <default> : mode=HTX        side=FE|BE     mux=H1
       <default> : mode=TCP|HTTP   side=FE|BE     mux=PASS

Available services :

Available filters :
        [SPOE] spoe
        [COMP] compression
        [CACHE] cache
        [TRACE] trace