Haproxy not presenting intermediate certificate

Hi

I am having a problem with one haproxy 3.2 instance. It is presenting only the leaf certificate to clients, rather than the leaf + intermediate certificate.

I am getting ‘incomplete chain’ warnings, and I would like to force haproxy sending the intermediate certificate together with the leaf certificate.

The certificate file contains:

  • [ Private key ]
  • [ Leaf certificate ]
  • [ Intermediate certificate ]

openssl test (simulating client connection):

$ openssl s_client -showcerts -connect www.myorg.org:443 -servername www.myorg.org
CONNECTED(00000003)
depth=0 CN = www.myorg.org
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 CN = www.myorg.org
verify error:num=21:unable to verify the first certificate
verify return:1
depth=0 CN = www.myorg.org
verify return:1
---
Certificate chain
 0 s:CN = www.myorg.org
   i:C = US, O = DigiCert Inc, OU = www.digicert.com, CN = RapidSSL TLS RSA CA G1
   a:PKEY: rsaEncryption, 4096 (bit); sigalg: RSA-SHA256
   v:NotBefore: Apr  3 00:00:00 2025 GMT; NotAfter: Apr  9 23:59:59 2026 GMT
-----BEGIN CERTIFICATE-----
MIIHJTCCBg2gAwIBAgIQDzJBDt6dB45B6e+/3xdzqjANBgkqhkiG9w0BAQsFADBg
(---snip---)
mq1q8NeH8G4b
-----END CERTIFICATE-----
---
Server certificate
subject=CN = www.myorg.org
issuer=C = US, O = DigiCert Inc, OU = www.digicert.com, CN = RapidSSL TLS RSA CA G1
---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 2671 bytes and written 395 bytes
Verification error: unable to verify the first certificate
---
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
Server public key is 4096 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 21 (unable to verify the first certificate)
---
---
Post-Handshake New Session Ticket arrived:
SSL-Session:
    Protocol  : TLSv1.3
    Cipher    : TLS_AES_256_GCM_SHA384
    Session-ID: 8E(--snip--)D7
    Session-ID-ctx: 
    Resumption PSK: 8C(--snip--)D8
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 7200 (seconds)
    TLS session ticket:
    0000 - ad d0 40 8e d9 09 29 ba-e8 fd 36 b5 93 72 05 ad   ..@...)...6..r..
(--snip--)
    0140 - ac a2 a5 7e 04 53 bd 17-06 84 b1 06 9d 62         ...~.S.......b

    Start Time: 1766049169
    Timeout   : 7200 (sec)
    Verify return code: 21 (unable to verify the first certificate)
    Extended master secret: no
    Max Early Data: 0
---
read R BLOCK
---
Post-Handshake New Session Ticket arrived:
SSL-Session:
    Protocol  : TLSv1.3
    Cipher    : TLS_AES_256_GCM_SHA384
    Session-ID: 2F(--snip--)FB
    Session-ID-ctx: 
    Resumption PSK: 5ED(--snip--)96D
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 7200 (seconds)
    TLS session ticket:
    0000 - df 64 f2 d8 5b cc fc b0-f4 41 95 57 37 41 f9 d8   .d..[....A.W7A..
(--snip--)
    0140 - 7c 54 6e 19 eb eb e1 d7-ae 55 30 e8 f9 35         |Tn......U0..5

    Start Time: 1766049169
    Timeout   : 7200 (sec)
    Verify return code: 21 (unable to verify the first certificate)
    Extended master secret: no
    Max Early Data: 0
---
read R BLOCK

Querying the certificate file contents via haproxy CLI API:

Filename: /etc/haproxy/certs/myorg.org_202605.pem
Crt filename: /etc/haproxy/certs/myorg.org_202605.pem
Key filename: /etc/haproxy/certs/myorg.org_202605.pem
OCSP filename: /etc/haproxy/certs/myorg.org_202605.pem.ocsp
SCTL filename: /etc/haproxy/certs/myorg.org_202605.pem.sctl
Status: Used
Serial: 0FC9942D8B2C5C83BC945ADDD6FE106F
notBefore: Apr  7 00:00:00 2025 GMT
notAfter: May  8 23:59:59 2026 GMT
Subject Alternative Name: DNS:www.myorg.org, DNS:myorg.org
Algorithm: RSA4096
SHA1 FingerPrint: 19E3ACBFF0F559E3A442EC37B15E50D1CE9B42CD
Subject: /CN=www.myorg.org
Issuer: /C=US/O=DigiCert Inc/OU=www.digicert.com/CN=RapidSSL TLS RSA CA G1
Chain Subject: /C=US/O=DigiCert Inc/OU=www.digicert.com/CN=RapidSSL TLS RSA CA G1
Chain Issuer: /C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert Global Root G2
OCSP Response Key:

HAProxy version:

# haproxy -vvv
HAProxy version 3.2.9-1704369 2025/11/21 - https://haproxy.org/
Status: long-term supported branch - will stop receiving fixes around Q2 2030.
Known bugs: http://www.haproxy.org/bugs/bugs-3.2.9.html
Running on: Linux 4.18.0-553.89.1.el8_10.x86_64 #1 SMP Fri Dec 12 10:42:53 UTC 2025 x86_64
Build options :
  TARGET  = linux-glibc
  CC      = cc
  CFLAGS  = -O2 -g -fwrapv
  OPTIONS = USE_THREAD=1 USE_LINUX_TPROXY=1 USE_OPENSSL=1 USE_LUA=1 USE_ZLIB=1 USE_TFO=1 USE_NS=1 USE_QUIC=1 USE_PROMEX=1 USE_PCRE=1 USE_PCRE_JIT=1 USE_QUIC_OPENSSL_COMPAT=1
  DEBUG   =

Feature list : -51DEGREES +ACCEPT4 +BACKTRACE -CLOSEFROM +CPU_AFFINITY +CRYPT_H -DEVICEATLAS +DL -ENGINE +EPOLL -EVPORTS +GETADDRINFO -KQUEUE -LIBATOMIC +LIBCRYPT +LINUX_CAP +LINUX_SPLICE +LINUX_TPROXY +LUA +MATH -MEMORY_PROFILING +NETFILTER +NS -OBSOLETE_LINKER +OPENSSL -OPENSSL_AWSLC -OPENSSL_WOLFSSL -OT +PCRE -PCRE2 -PCRE2_JIT +PCRE_JIT +POLL +PRCTL -PROCCTL +PROMEX -PTHREAD_EMULATION +QUIC +QUIC_OPENSSL_COMPAT +RT -SLZ +SSL -STATIC_PCRE -STATIC_PCRE2 +TFO +THREAD +THREAD_DUMP +TPROXY -WURFL +ZLIB +ACME

Default settings :
  bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with multi-threading support (MAX_TGROUPS=32, MAX_THREADS=1024, default=1).
Built with SSL library version : OpenSSL 3.5.4 30 Sep 2025
Running on SSL library version : OpenSSL 3.5.4 30 Sep 2025
SSL library supports TLS extensions : yes
SSL library supports SNI : yes
SSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
OpenSSL providers loaded : default
QUIC: connection socket-owner mode support : yes
QUIC: GSO emission support : no
Built with Lua version : Lua 5.4.7
Built with the Prometheus exporter as a service
Built with network namespace support.
Built with zlib version : 1.2.11
Running on zlib version : 1.2.11
Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
Built with PCRE version : 8.42 2018-03-20
Running on PCRE version : 8.42 2018-03-20
PCRE library supports JIT : yes
Encrypted password support via crypt(3): yes
Built with gcc compiler version 8.5.0 20210514 (Red Hat 8.5.0-28)

Available polling systems :
      epoll : pref=300,  test result OK
       poll : pref=200,  test result OK
     select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

Available multiplexer protocols :
(protocols marked as <default> cannot be specified using 'proto' keyword)
       quic : mode=HTTP  side=FE     mux=QUIC  flags=HTX|NO_UPG|FRAMED
         h2 : mode=HTTP  side=FE|BE  mux=H2    flags=HTX|HOL_RISK|NO_UPG
  <default> : mode=HTTP  side=FE|BE  mux=H1    flags=HTX
         h1 : mode=HTTP  side=FE|BE  mux=H1    flags=HTX|NO_UPG
       fcgi : mode=HTTP  side=BE     mux=FCGI  flags=HTX|HOL_RISK|NO_UPG
  <default> : mode=SPOP  side=BE     mux=SPOP  flags=HOL_RISK|NO_UPG
       spop : mode=SPOP  side=BE     mux=SPOP  flags=HOL_RISK|NO_UPG
  <default> : mode=TCP   side=FE|BE  mux=PASS  flags=
       none : mode=TCP   side=FE|BE  mux=PASS  flags=NO_UPG

Available services : prometheus-exporter
Available filters :
        [BWLIM] bwlim-in
        [BWLIM] bwlim-out
        [CACHE] cache
        [COMP] compression
        [FCGI] fcgi-app
        [SPOE] spoe
        [TRACE] trace

haproxy is running on Alma Linux 8.

Any ideas?

I suggest the sequence:

  • leaf cert
  • intermediate cert
  • key

in the PEM file.

Also, verify that the intermediate certificate is actually the correct one.

I’m noticing something else though, the certificate returned from your openssl commands and the certificate returned from haproxy CLI are different.

One certificate (openssl) is valid from Apr 3rd 2025 until Apr 9th 2026.
The other certificate (haproxy) is valid from Apr 7th 2025 until May 8th 2026.

Those are completely different leaf certificates.

Either your openssl command access something other than the haproxy node (perhaps a external CDN?) or you have an old haproxy instance running in the background that still sometimes gets requests (this can happen with process manager mis-configuration or mississued manual starts or reloads).

To eliminate that possibilty stop and kill all remaining haproxy processes and then starting haproxy again normally using your normal process manager (for example systemctl start haproxy for systemd).

This is an excellent observation, thank you!

Yes, most likely, another proxy was introduced in front of haproxy, and nobody told me.

Luckily, the certs are different.

Many thanks again!