Haproxy from 1.8.X to 2.3.1 [solved]

Hi to everyone … I upgrade to 2.3.1 latest haproxy stable… and without change ssl config… starting to getting error tunnel with windows 7… i’m pretty sure is about tls.1.0 issue so

i had 1 link to try fix it
https://support.microsoft.com/it-it/help/3140245/update-to-enable-tls-1-1-and-tls-1-2-as-default-secure-protocols-in-wi

BUT. we need to support tls 1.0 exist any change inside of binary to try to use it. without windows 7 get an error again… ?
or some verions between 1.8 and 2.3.1 to avoid this issue and try to get more update haproxy to not use anymore 1.8 ?

i don’t have control to send to our millions client with windows 7 to say “hey need to update to use the new tls 1.1 and 1.2.”

any idea?

What’s a tunnel error? Can you elaborate what you are seeing EXACTLY and what the error messages are EXACTLY?

Also:

  • which OS system and exact release did you use before and after the upgrade?
  • post haproxy -vv output before and after the upgrade
  • provide all the SSL/TLS related configuration
  • provide the steps that you used to upgrade

Hi lukas,
thanks for your time to reply

OS where is installed Haproxy is CentOS 7.8 on 3 balance servers,…(now we are trying testing with only one 2.3.1 and the rest 2 with 1.8.10 to avoid issues services) splitting 2.3.1 for test

HA-Proxy version 2.3.1-bdd7178 2020/11/13 - https://haproxy.org/
Status: stable branch - will stop receiving fixes around Q1 2022.
Known bugs: http://www.haproxy.org/bugs/bugs-2.3.1.html
Running on: Linux 3.10.0-1127.10.1.el7.x86_64 #1 SMP Wed Jun 3 14:28:03 UTC 2020 x86_64
Build options :
TARGET = linux-glibc
CPU = generic
CC = cc
CFLAGS = -O2 -g -Wall -Wextra -Wdeclaration-after-statement -fwrapv -Wno-unused-label -Wno-sign-compare -Wno-unused-parameter -Wno-clobbered -Wno-missing-field-initializers -Wtype-limits
OPTIONS = USE_PCRE=1 USE_PCRE_JIT=1 USE_LINUX_TPROXY=1 USE_OPENSSL=1 USE_ZLIB=1 USE_SYSTEMD=1

Feature list : +EPOLL -KQUEUE +NETFILTER +PCRE +PCRE_JIT -PCRE2 -PCRE2_JIT +POLL -PRIVATE_CACHE +THREAD -PTHREAD_PSHARED +BACKTRACE -STATIC_PCRE -STATIC_PCRE2 +TPROXY +LINUX_TPROXY +LINUX_SPLICE +LIBCRYPT +CRYPT_H +GETADDRINFO +OPENSSL -LUA +FUTEX +ACCEPT4 -CLOSEFROM +ZLIB -SLZ +CPU_AFFINITY +TFO +NS +DL +RT -DEVICEATLAS -51DEGREES -WURFL +SYSTEMD -OBSOLETE_LINKER +PRCTL +THREAD_DUMP -EVPORTS

Default settings :
bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with multi-threading support (MAX_THREADS=64, default=4).
Built with OpenSSL version : OpenSSL 1.0.2k-fips 26 Jan 2017
Running on OpenSSL version : OpenSSL 1.0.2k-fips 26 Jan 2017
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : SSLv3 TLSv1.0 TLSv1.1 TLSv1.2
Built with network namespace support.
Built with zlib version : 1.2.7
Running on zlib version : 1.2.7
Compression algorithms supported : identity(“identity”), deflate(“deflate”), raw-deflate(“deflate”), gzip(“gzip”)
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
Built with PCRE version : 8.32 2012-11-30
Running on PCRE version : 8.32 2012-11-30
PCRE library supports JIT : yes
Encrypted password support via crypt(3): yes
Built with gcc compiler version 4.8.5 20150623 (Red Hat 4.8.5-39)

Available polling systems :
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use epoll.

Available multiplexer protocols :
(protocols marked as cannot be specified using ‘proto’ keyword)
h2 : mode=HTTP side=FE|BE mux=H2
fcgi : mode=HTTP side=BE mux=FCGI
: mode=HTTP side=FE|BE mux=H1
: mode=TCP side=FE|BE mux=PASS

Available services : none

Available filters :
[SPOE] spoe
[CACHE] cache
[FCGI] fcgi-app
[COMP] compression
[TRACE] trace

before we have…

HA-Proxy version 1.8.10-ec17d7a 2018/06/22
Copyright 2000-2018 Willy Tarreau willy@haproxy.org

Build options :
TARGET = linux2628
CPU = generic
CC = gcc
CFLAGS = -O2 -g -fno-strict-aliasing -Wdeclaration-after-statement -fwrapv -fno-strict-overflow -Wno-unused-label
OPTIONS = USE_LINUX_TPROXY=1 USE_ZLIB=1 USE_REGPARM=1 USE_OPENSSL=1 USE_SYSTEMD=1 USE_PCRE=1 USE_PCRE_JIT=1

Default settings :
maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with OpenSSL version : OpenSSL 1.0.2k-fips 26 Jan 2017
Running on OpenSSL version : OpenSSL 1.0.2k-fips 26 Jan 2017
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : SSLv3 TLSv1.0 TLSv1.1 TLSv1.2
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
Encrypted password support via crypt(3): yes
Built with multi-threading support.
Built with PCRE version : 8.32 2012-11-30
Running on PCRE version : 8.32 2012-11-30
PCRE library supports JIT : yes
Built with zlib version : 1.2.7
Running on zlib version : 1.2.7
Compression algorithms supported : identity(“identity”), deflate(“deflate”), raw-deflate(“deflate”), gzip(“gzip”)
Built with network namespace support.

Available polling systems :
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use epoll.

Available filters :
[SPOE] spoe
[COMP] compression
[TRACE] trace

so… as you can see the TLS 1.0 is active on both.

and the line where we use not changed in the conf. wich is this one.
SSL related

ssl-default-bind-ciphers ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-ECDSA-DES-CBC3-SHA:DES-CBC3-SHA:!DSS
ssl-default-bind-options no-sslv3 no-tls-tickets
ssl-default-server-options no-sslv3 no-tls-tickets

On windows 7, when we try to access to our internal site with 1.8.10 works fine… but with 2.3.1 didn’t work… so we have to back again

on windows we see an error app can’t access to get data when we use 2.3.1
but only win windows 7, with windows 10 works fine…

on haproxy logs we see this lines

Nov 24 08:26:18 localhost.localdomain haproxy[28394]: ip.from.client:26249 [24/Nov/2020:08:26:18.521] main_fe/3: Connection closed during SSL handshake

  • the way to get 2.3.1 is getting from tarball and make install to compile the new binary and quick we got haproxy -v the new version.

On your haproxy VM, try connecting locally with TLSv1.0 and the two ciphers from your list that would be supported in this scenario:

openssl s_client -tls1 -cipher 'DES-CBC3-SHA' -servername www.example.org -connect www.example.org:443
openssl s_client -tls1 -cipher 'ECDHE-RSA-AES128-SHA' -servername www.example.org -connect www.example.org:443

Replace www.example.org twice in each of the above commands to make openssl connect in TLSv1.0 to the haproxy SSL endpoint.

Hi Lukas

sorry by delay…
here the 1st example using cipher: ‘DES-CBC3-SHA’
CONNECTED(00000003)
139795557173136:error:1409442E:SSL routines:ssl3_read_bytes:tlsv1 alert protocol version:s3_pkt.c:1493:SSL alert number 70
139795557173136:error:1409E0E5:SSL routines:ssl3_write_bytes:ssl handshake failure:s3_pkt.c:659:

no peer certificate available

No client certificate CA names sent

SSL handshake has read 7 bytes and written 0 bytes

New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1
Cipher : 0000
Session-ID:
Session-ID-ctx:
Master-Key:
Key-Arg : None
Krb5 Principal: None
PSK identity: None
PSK identity hint: None
Start Time: 1606724201
Timeout : 7200 (sec)
Verify return code: 0 (ok)

second cipher test: ‘ECDHE-RSA-AES128-SHA’
CONNECTED(00000003)
140547371059088:error:1409442E:SSL routines:ssl3_read_bytes:tlsv1 alert protocol version:s3_pkt.c:1493:SSL alert number 70
140547371059088:error:1409E0E5:SSL routines:ssl3_write_bytes:ssl handshake failure:s3_pkt.c:659:

no peer certificate available

No client certificate CA names sent

SSL handshake has read 7 bytes and written 0 bytes

New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1
Cipher : 0000
Session-ID:
Session-ID-ctx:
Master-Key:
Key-Arg : None
Krb5 Principal: None
PSK identity: None
PSK identity hint: None
Start Time: 1606724211
Timeout : 7200 (sec)
Verify return code: 0 (ok)

Could you also provide the output without the specific TLS and cipher selection, so that we see a working connection:

openssl s_client -servername www.example.org -connect www.example.org:443

CONNECTED(00000003)
depth=3 C = US, O = “The Go Daddy Group, Inc.”, OU = Go Daddy Class 2 Certification Authority
verify return:1
depth=2 C = US, ST = Arizona, L = Scottsdale, O = “GoDaddy.com, Inc.”, CN = Go Daddy Root Certificate Authority - G2
verify return:1
depth=1 C = US, ST = Arizona, L = Scottsdale, O = “GoDaddy.com, Inc.”, OU = http://certs.godaddy.com/repository/, CN = Go Daddy Secure Certificate Authority - G2
verify return:1
depth=0 OU = Domain Control Validated, CN = *.ourdomain.net
verify return:1

Certificate chain
0 s:/OU=Domain Control Validated/CN=*.ourdomain.net
i:/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./OU=http://certs.godaddy.com/repository//CN=Go Daddy Secure Certificate Authority - G2
1 s:/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./OU=http://certs.godaddy.com/repository//CN=Go Daddy Secure Certificate Authority - G2
i:/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./CN=Go Daddy Root Certificate Authority - G2
2 s:/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./CN=Go Daddy Root Certificate Authority - G2
i:/C=US/O=The Go Daddy Group, Inc./OU=Go Daddy Class 2 Certification Authority
3 s:/C=US/O=The Go Daddy Group, Inc./OU=Go Daddy Class 2 Certification Authority
i:/C=US/O=The Go Daddy Group, Inc./OU=Go Daddy Class 2 Certification Authority

Server certificate
-----BEGIN CERTIFICATE-----
MIIGsjCCBZqgAwIBAgIJAOUzFbQzCv3NMA0GCSqGSIb3DQEBCwUAMIG0MQswCQYD

4rppqvOiyUKTurhdfNzAl76rhs7YXXLp109YULnJyIdMw4HAYhI=
-----END CERTIFICATE-----
subject=/OU=Domain Control Validated/CN=*.ourdomain.net
issuer=/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./OU=http://certs.godaddy.com/repository//CN=Go Daddy Secure Certificate Authority - G2

No client certificate CA names sent
Peer signing digest: SHA512
Server Temp Key: ECDH, P-256, 256 bits

SSL handshake has read 5660 bytes and written 440 bytes

New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-GCM-SHA256
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES128-GCM-SHA256
Session-ID: 72FB
Session-ID-ctx:
Master-Key: 6DA
Key-Arg : None
Krb5 Principal: None
PSK identity: None
PSK identity hint: None
Start Time: 1606738321
Timeout : 300 (sec)
Verify return code: 0 (ok)

HTTP/1.1 408 Request Time-out
content-length: 110
cache-control: no-cache
content-type: text/html
connection: close

408 Request Time-out

Your browser didn't send a complete request in time. closed

In Haproxy 2.2 the default minimum TLS version was bumped from 1.0 to 1.2.

You have to replace no-sslv3 with ssl-min-ver TLSv1.0 in your configuration, then TLSv1.0 will work again:

ssl-default-bind-options ssl-min-ver TLSv1.0 no-tls-tickets
ssl-default-server-options ssl-min-ver TLSv1.0 no-tls-tickets

it’s works…Thanks so much lukas… for your support…

1 Like