Issues with HAProxy 2.0.25 Upgrade and OpenSSL 3.0

Help!
1

I am trying to upgrade from HAProxy 1.5 to 2.0 and I ran into some SSL issues during installation. I updated my make command to the following which decreased the total errors by 80% but I am still getting SSL.sock errors. Any help would be greatly appreciated.

Dist: CentOS 7

OpenSSL Data
openssl version -a
OpenSSL 3.0.0 7 sep 2021 (Library: OpenSSL 3.0.0 7 sep 2021)
built on: Thu Oct 14 17:07:54 2021 UTC
platform: linux-x86_64
options: bn(64,64)
compiler: gcc -fPIC -pthread -m64 -Wa,–noexecstack -Wall -O3 -DOPENSSL_USE_NODELETE -DL_ENDIAN -DOPENSSL_PIC -DOPENSSL_BUILDING_OPENSSL -DZLIB -DNDEBUG
OPENSSLDIR: “/usr/local/ssl”
ENGINESDIR: “/usr/local/ssl/lib64/engines-3”
MODULESDIR: “/usr/local/ssl/lib64/ossl-modules”
Seeding source: os-specific
CPUINFO: OPENSSL_ia32cap=0x80202001478bfffd:0x0

HAProxy Version
haproxy -v
HA-Proxy version 2.0.25-6986403 2021/09/07 - https://haproxy.org/

ERROR
make TARGET=linux-glibc USE_PCRE=1 USE_OPENSSL=1 SSL_INC=/usr/local/ssl/include/openssl SSL_LIB=/usr/local/ssl/lib64 USE_ZLIB=1 USE_CRYPT_H=1 USE_LIBCRYPT=1

LD haproxy
src/ssl_sock.o: In function ssl_sock_ocsp_stapling_cbk': /usr/src/haproxy-2.0.25/src/ssl_sock.c:1141: undefined reference to EVP_PKEY_base_id’
src/ssl_sock.o: In function ssl_get_tmp_dh': /usr/src/haproxy-2.0.25/src/ssl_sock.c:2689: undefined reference to EVP_PKEY_base_id’
/usr/src/haproxy-2.0.25/src/ssl_sock.c:2695: undefined reference to EVP_PKEY_bits' src/ssl_sock.o: In function smp_fetch_ssl_c_used’:
/usr/src/haproxy-2.0.25/src/ssl_sock.c:7108: undefined reference to SSL_get_peer_certificate' src/ssl_sock.o: In function smp_fetch_ssl_x_sig_alg’:
/usr/src/haproxy-2.0.25/src/ssl_sock.c:7181: undefined reference to SSL_get_peer_certificate' src/ssl_sock.o: In function smp_fetch_ssl_x_serial’:
/usr/src/haproxy-2.0.25/src/ssl_sock.c:6813: undefined reference to SSL_get_peer_certificate' src/ssl_sock.o: In function smp_fetch_ssl_x_s_dn’:
/usr/src/haproxy-2.0.25/src/ssl_sock.c:7056: undefined reference to SSL_get_peer_certificate' src/ssl_sock.o: In function smp_fetch_ssl_x_version’:
/usr/src/haproxy-2.0.25/src/ssl_sock.c:7141: undefined reference to SSL_get_peer_certificate' src/ssl_sock.o:/usr/src/haproxy-2.0.25/src/ssl_sock.c:6861: more undefined references to SSL_get_peer_certificate’ follow
src/ssl_sock.o: In function ssl_sock_do_create_cert': /usr/src/haproxy-2.0.25/src/ssl_sock.c:1924: undefined reference to EVP_PKEY_base_id’
src/ssl_sock.o: In function ssl_sock_load_ocsp': /usr/src/haproxy-2.0.25/src/ssl_sock.c:1327: undefined reference to EVP_PKEY_base_id’
/usr/src/haproxy-2.0.25/src/ssl_sock.c:1293: undefined reference to EVP_PKEY_base_id' src/ssl_sock.o: In function ssl_sock_load_cert_chain_file’:
/usr/src/haproxy-2.0.25/src/ssl_sock.c:3446: undefined reference to EVP_PKEY_bits' /usr/src/haproxy-2.0.25/src/ssl_sock.c:3447: undefined reference to EVP_PKEY_base_id’
src/ssl_sock.o: In function ssl_sock_get_pkey_algo': /usr/src/haproxy-2.0.25/src/ssl_sock.c:6251: undefined reference to EVP_PKEY_bits’
/usr/src/haproxy-2.0.25/src/ssl_sock.c:6252: undefined reference to EVP_PKEY_base_id' src/ssl_sock.o: In function ssl_sock_get_remote_common_name’:
/usr/src/haproxy-2.0.25/src/ssl_sock.c:6603: undefined reference to SSL_get_peer_certificate' src/ssl_sock.o: In function ssl_sock_get_cert_used_sess’:
/usr/src/haproxy-2.0.25/src/ssl_sock.c:6630: undefined reference to `SSL_get_peer_certificate’
collect2: error: ld returned 1 exit status
make: *** [haproxy] Error 1

2

Haproxy is not yet compatible with the API changes in OpenSSL 3.0.

3

Thanks for response. Is HAProxy 2.4.7 compatible with OpenSSL 1.1.1l?

4

Yes, absolutely!

5

I tried to compile with OpenSSL 1.1.1l and got the following error.

OpenSSL Version
openssl version -a
OpenSSL 1.1.1l 24 Aug 2021
built on: Fri Oct 15 16:50:01 2021 UTC
platform: linux-x86_64
options: bn(64,64) rc4(16x,int) des(int) idea(int) blowfish(ptr)
compiler: gcc -fPIC -pthread -m64 -Wa,–noexecstack -Wall -O3 -DOPENSSL_USE_NODELETE -DL_ENDIAN -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DRC4_ASM -DMD5_ASM -DAESNI_ASM -DVPAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DX25519_ASM -DPOLY1305_ASM -DZLIB -DNDEBUG
OPENSSLDIR: “/usr/local/openssl”
ENGINESDIR: “/usr/local/openssl/lib/engines-1.1”
Seeding source: os-specific

Error
make TARGET=linux-glibc USE_NS=1 USE_TFO=1 USE_PCRE=1 USE_ZLIB=1 USE_CRYPT_H=1 USE_LIBCRYPT=1 USE_OPENSSL=1 SSL_INC=/usr/local/openssl/include/openssl SSL_LIB=/usr/local/openssl/lib

CC src/ssl_sock.o
src/ssl_sock.c: In function ‘ssl_sock_dump_errors’:
src/ssl_sock.c:656:12: warning: ‘ERR_func_error_string’ is deprecated (declared at /usr/local/include/openssl/err.h:447): Since OpenSSL 3.0 [-Wdeprecated-declarations]
ERR_func_error_string(ret), ERR_reason_error_string(ret));
^
src/ssl_sock.c: In function ‘ssl_init_single_engine’:
src/ssl_sock.c:670:2: warning: ‘ENGINE_by_id’ is deprecated (declared at /usr/local/include/openssl/engine.h:336): Since OpenSSL 3.0 [-Wdeprecated-declarations]
engine = ENGINE_by_id(engine_id);
^
src/ssl_sock.c:676:2: warning: ‘ENGINE_init’ is deprecated (declared at /usr/local/include/openssl/engine.h:620): Since OpenSSL 3.0 [-Wdeprecated-declarations]
if (!ENGINE_init(engine)) {
^
src/ssl_sock.c:682:2: warning: ‘ENGINE_set_default_string’ is deprecated (declared at /usr/local/include/openssl/engine.h:685): Since OpenSSL 3.0 [-Wdeprecated-declarations]
if (ENGINE_set_default_string(engine, def_algorithms) == 0) {
^
src/ssl_sock.c:700:2: warning: ‘ENGINE_finish’ is deprecated (declared at /usr/local/include/openssl/engine.h:628): Since OpenSSL 3.0 [-Wdeprecated-declarations]
ENGINE_finish(engine);
^
src/ssl_sock.c:704:2: warning: ‘ENGINE_free’ is deprecated (declared at /usr/local/include/openssl/engine.h:493): Since OpenSSL 3.0 [-Wdeprecated-declarations]
ENGINE_free(engine);
^
src/ssl_sock.c: In function ‘ssl_tlsext_ticket_key_cb’:
src/ssl_sock.c:1107:4: warning: ‘HMAC_Init_ex’ is deprecated (declared at /usr/local/include/openssl/hmac.h:43): Since OpenSSL 3.0 [-Wdeprecated-declarations]
HMAC_Init_ex(hctx, keys[head].key_128.hmac_key, 16, TLS_TICKET_HASH_FUNCT(), NULL);
^
src/ssl_sock.c:1115:4: warning: ‘HMAC_Init_ex’ is deprecated (declared at /usr/local/include/openssl/hmac.h:43): Since OpenSSL 3.0 [-Wdeprecated-declarations]
HMAC_Init_ex(hctx, keys[head].key_256.hmac_key, 32, TLS_TICKET_HASH_FUNCT(), NULL);
^
src/ssl_sock.c:1128:4: warning: ‘HMAC_Init_ex’ is deprecated (declared at /usr/local/include/openssl/hmac.h:43): Since OpenSSL 3.0 [-Wdeprecated-declarations]
HMAC_Init_ex(hctx, keys[(head + i) % TLS_TICKETS_NO].key_128.hmac_key, 16, TLS_TICKET_HASH_FUNCT(), NULL);
^
src/ssl_sock.c:1135:4: warning: ‘HMAC_Init_ex’ is deprecated (declared at /usr/local/include/openssl/hmac.h:43): Since OpenSSL 3.0 [-Wdeprecated-declarations]
HMAC_Init_ex(hctx, keys[(head + i) % TLS_TICKETS_NO].key_256.hmac_key, 32, TLS_TICKET_HASH_FUNCT(), NULL);
^
src/ssl_sock.c: In function ‘ssl_sock_do_create_cert’:
src/ssl_sock.c:2059:2: warning: ‘SSL_CTX_set_tmp_dh_callback’ is deprecated (declared at /usr/local/include/openssl/ssl.h:2213): Since OpenSSL 3.0 [-Wdeprecated-declarations]
SSL_CTX_set_tmp_dh_callback(ssl_ctx, ssl_get_tmp_dh);
^
src/ssl_sock.c:2069:3: warning: ‘EC_KEY_new_by_curve_name’ is deprecated (declared at /usr/local/include/openssl/ec.h:996): Since OpenSSL 3.0 [-Wdeprecated-declarations]
if (!(ecc = EC_KEY_new_by_curve_name(nid)))
^
src/ssl_sock.c:2072:3: warning: ‘EC_KEY_free’ is deprecated (declared at /usr/local/include/openssl/ec.h:1001): Since OpenSSL 3.0 [-Wdeprecated-declarations]
EC_KEY_free(ecc);
^
In file included from include/haproxy/openssl-compat.h:7:0,
from include/haproxy/listener-t.h:37,
from include/haproxy/server-t.h:36,
from include/haproxy/lb_map-t.h:26,
from include/haproxy/backend-t.h:30,
from include/haproxy/proxy-t.h:35,
from include/haproxy/applet-t.h:31,
from include/haproxy/action-t.h:25,
from include/haproxy/stream.h:25,
from include/haproxy/channel.h:30,
from src/ssl_sock.c:51:
src/ssl_sock.c: In function ‘ctx_set_TLSv13_func’:
src/ssl_sock.c:2273:5: error: missing binary operator before token “1”
#if SSL_OP_NO_TLSv1_3
^
src/ssl_sock.c: In function ‘ssl_set_TLSv13_func’:
src/ssl_sock.c:2279:5: error: missing binary operator before token “1”
#if SSL_OP_NO_TLSv1_3
^
src/ssl_sock.c: In function ‘ssl_get_dh_1024’:
src/ssl_sock.c:2697:2: warning: ‘DH_new’ is deprecated (declared at /usr/local/include/openssl/dh.h:199): Since OpenSSL 3.0 [-Wdeprecated-declarations]
DH *dh = DH_new();
^
src/ssl_sock.c:2703:4: warning: ‘DH_free’ is deprecated (declared at /usr/local/include/openssl/dh.h:200): Since OpenSSL 3.0 [-Wdeprecated-declarations]
DH_free(dh);
^
src/ssl_sock.c:2706:4: warning: ‘DH_set0_pqg’ is deprecated (declared at /usr/local/include/openssl/dh.h:255): Since OpenSSL 3.0 [-Wdeprecated-declarations]
DH_set0_pqg(dh, p, NULL, g);
^
src/ssl_sock.c: In function ‘ssl_get_dh_2048’:
src/ssl_sock.c:2744:2: warning: ‘DH_new’ is deprecated (declared at /usr/local/include/openssl/dh.h:199): Since OpenSSL 3.0 [-Wdeprecated-declarations]
DH *dh = DH_new();
^
src/ssl_sock.c:2750:4: warning: ‘DH_free’ is deprecated (declared at /usr/local/include/openssl/dh.h:200): Since OpenSSL 3.0 [-Wdeprecated-declarations]
DH_free(dh);
^
src/ssl_sock.c:2753:4: warning: ‘DH_set0_pqg’ is deprecated (declared at /usr/local/include/openssl/dh.h:255): Since OpenSSL 3.0 [-Wdeprecated-declarations]
DH_set0_pqg(dh, p, NULL, g);
^
src/ssl_sock.c: In function ‘ssl_get_dh_4096’:
src/ssl_sock.c:2812:2: warning: ‘DH_new’ is deprecated (declared at /usr/local/include/openssl/dh.h:199): Since OpenSSL 3.0 [-Wdeprecated-declarations]
DH *dh = DH_new();
^
src/ssl_sock.c:2818:4: warning: ‘DH_free’ is deprecated (declared at /usr/local/include/openssl/dh.h:200): Since OpenSSL 3.0 [-Wdeprecated-declarations]
DH_free(dh);
^
src/ssl_sock.c:2821:4: warning: ‘DH_set0_pqg’ is deprecated (declared at /usr/local/include/openssl/dh.h:255): Since OpenSSL 3.0 [-Wdeprecated-declarations]
DH_set0_pqg(dh, p, NULL, g);
^
src/ssl_sock.c: In function ‘ssl_sock_get_dh_from_file’:
src/ssl_sock.c:2872:2: warning: ‘PEM_read_bio_DHparams’ is deprecated (declared at /usr/local/include/openssl/pem.h:469): Since OpenSSL 3.0 [-Wdeprecated-declarations]
dh = PEM_read_bio_DHparams(in, NULL, NULL, NULL);
^
src/ssl_sock.c: In function ‘ssl_sock_load_dh_params’:
src/ssl_sock.c:3093:4: warning: ‘SSL_CTX_set_tmp_dh_callback’ is deprecated (declared at /usr/local/include/openssl/ssl.h:2213): Since OpenSSL 3.0 [-Wdeprecated-declarations]
SSL_CTX_set_tmp_dh_callback(ctx, ssl_get_tmp_dh);
^
src/ssl_sock.c: In function ‘ssl_sock_prepare_ctx’:
src/ssl_sock.c:4490:3: warning: ‘EC_KEY_new_by_curve_name’ is deprecated (declared at /usr/local/include/openssl/ec.h:996): Since OpenSSL 3.0 [-Wdeprecated-declarations]
if (!i || ((ecdh = EC_KEY_new_by_curve_name(i)) == NULL)) {
^
src/ssl_sock.c:4497:4: warning: ‘EC_KEY_free’ is deprecated (declared at /usr/local/include/openssl/ec.h:1001): Since OpenSSL 3.0 [-Wdeprecated-declarations]
EC_KEY_free(ecdh);
^
src/ssl_sock.c: In function ‘__ssl_sock_init’:
src/ssl_sock.c:7129:2: warning: ‘ENGINE_load_builtin_engines’ is deprecated (declared at /usr/local/include/openssl/engine.h:358): Since OpenSSL 3.0 [-Wdeprecated-declarations]
ENGINE_load_builtin_engines();
^
src/ssl_sock.c:7149:2: warning: ‘ERR_load_SSL_strings’ is deprecated (declared at /usr/local/include/openssl/sslerr_legacy.h:29): Since OpenSSL 3.0 [-Wdeprecated-declarations]
ERR_load_SSL_strings();
^
src/ssl_sock.c: In function ‘ssl_free_engines’:
src/ssl_sock.c:7225:3: warning: ‘ENGINE_finish’ is deprecated (declared at /usr/local/include/openssl/engine.h:628): Since OpenSSL 3.0 [-Wdeprecated-declarations]
ENGINE_finish(wl->e);
^
src/ssl_sock.c:7226:3: warning: ‘ENGINE_free’ is deprecated (declared at /usr/local/include/openssl/engine.h:493): Since OpenSSL 3.0 [-Wdeprecated-declarations]
ENGINE_free(wl->e);
^
src/ssl_sock.c: In function ‘ssl_free_dh’:
src/ssl_sock.c:7236:3: warning: ‘DH_free’ is deprecated (declared at /usr/local/include/openssl/dh.h:200): Since OpenSSL 3.0 [-Wdeprecated-declarations]
DH_free(local_dh_1024);
^
src/ssl_sock.c:7240:3: warning: ‘DH_free’ is deprecated (declared at /usr/local/include/openssl/dh.h:200): Since OpenSSL 3.0 [-Wdeprecated-declarations]
DH_free(local_dh_2048);
^
src/ssl_sock.c:7244:3: warning: ‘DH_free’ is deprecated (declared at /usr/local/include/openssl/dh.h:200): Since OpenSSL 3.0 [-Wdeprecated-declarations]
DH_free(local_dh_4096);
^
src/ssl_sock.c:7248:3: warning: ‘DH_free’ is deprecated (declared at /usr/local/include/openssl/dh.h:200): Since OpenSSL 3.0 [-Wdeprecated-declarations]
DH_free(global_dh);
^
make: *** [src/ssl_sock.o] Error 1

6

Yes, because you are not compiling against OpenSSL 1.1.1.

The error message literally say 34 times that OpenSSL 3.0 is used (“Since OpenSSL 3.0 …”).

7

Thanks for responding, not trying to be lazy, I saw that but OpenSSL 3.0 was removed and I installed OpenSSL 1.1.1l and checked the version. So I guess this is saying that there is some remnants of 3.0 hanging out somewhere in the server? Sorry, not trying to be a n00b

8

Correct. Unfortunately I cannot directly help you with this, because it would require a lot of time and back and forth.

9

Appreciate that, I ran a find and found some left over libs/paths/files that have been removed. I ran the following and have a new set of errors not related to OpenSSL 3.0.

make TARGET=linux-glibc USE_NS=1 USE_TFO=1 USE_PCRE=1 USE_ZLIB=1 USE_CRYPT_H=1 USE_LIBCRYPT=1 USE_OPENSSL=1 SSL_INC=/usr/local/openssl/include/openssl SSL_LIB=/usr/local/openssl/lib

src/ssl_sample.o: In function smp_fetch_ssl_x_version': /usr/src/haproxy-2.4.7/src/ssl_sample.c:608: undefined reference to SSL_get1_peer_certificate’
src/ssl_sample.o: In function smp_fetch_ssl_c_used': /usr/src/haproxy-2.4.7/src/ssl_sample.c:569: undefined reference to SSL_get1_peer_certificate’
src/ssl_sample.o: In function smp_fetch_ssl_x_sha1': /usr/src/haproxy-2.4.7/src/ssl_sample.c:289: undefined reference to SSL_get1_peer_certificate’
src/ssl_sample.o: In function smp_fetch_ssl_x_serial': /usr/src/haproxy-2.4.7/src/ssl_sample.c:235: undefined reference to SSL_get1_peer_certificate’
src/ssl_sample.o: In function smp_fetch_ssl_x_s_dn': /usr/src/haproxy-2.4.7/src/ssl_sample.c:512: undefined reference to SSL_get1_peer_certificate’
src/ssl_sample.o:/usr/src/haproxy-2.4.7/src/ssl_sample.c:654: more undefined references to `SSL_get1_peer_certificate’ follow
collect2: error: ld returned 1 exit status
make: *** [haproxy] Error 1

10

This is gonna be a never ending back and forth for which I do not have the time.

Quick suggestion: install the OS from scratch and follow the advice in the INSTALL haproxy tarball:

http://git.haproxy.org/?p=haproxy-2.4.git;a=blob;f=INSTALL;h=85e8e8f59b3c5032d15231577f71a5b47330e4ec;hb=HEAD#l225

Mainly:

  • don’t install openssl in standard paths or even /usr/local/
  • build it statically, not dynamically, otherwise things get even more complex
1 Like