Is 'ssl verify none' work for self-signed certificate in tcp mode helthcheck?

Akihiro · March 6, 2018, 11:28am

I want to configure HAProxy as a tcp pass-through with ssl proxy, but some settings don’t work.

I think ‘ssl verify none’ option at listen directive is work when backend server uses self-signed certificate.
But with ‘ssl verify none’ option with mode tcp, I cannot access backend server with https protocol.
Is it correct behavier?

This config is not work as https frontend, only http access

global
  chroot /var/lib/haproxy
  user haproxy
  group haproxy
  log 127.0.0.1 local2
  pidfile /var/run/haproxy.pid
  maxconn 40000
  daemon
  stats socket /var/lib/haproxy/stats level admin

defaults
  mode tcp
  log global
  option tcplog
  option dontlognull
  option redispatch
  retries 3
  timeout http-request 10s
  timeout queue 1m
  timeout connect 10s
  timeout client 1m
  timeout server 1m
  timeout http-keep-alive 10s
  timeout check 10s
  maxconn 10000

listen https_web
  mode tcp
  bind *:8443
  balance roundrobin
  option log-health-checks
  reqadd X-Forwarded-Proto:\ http
  server sv1 sv1:8443 maxconn 512 check check-ssl inter 60s ssl verify none
  server sv2 sv2:8443 maxconn 512 backup check check-ssl inter 60s ssl verify none
  server sv3 sv3:8443 maxconn 512 backup check check-ssl inter 60s ssl verify none
  option httpchk GET /health
  http-check expect status 200

This config is work as https frontend

global
  chroot /var/lib/haproxy
  user haproxy
  group haproxy
  log 127.0.0.1 local2
  pidfile /var/run/haproxy.pid
  maxconn 40000
  daemon
  ssl-server-verify none
  stats socket /var/lib/haproxy/stats level admin

defaults
  mode tcp
  log global
  option tcplog
  option dontlognull
  option redispatch
  retries 3
  timeout http-request 10s
  timeout queue 1m
  timeout connect 10s
  timeout client 1m
  timeout server 1m
  timeout http-keep-alive 10s
  timeout check 10s
  maxconn 10000

listen https_web
  mode tcp
  bind *:8443
  balance roundrobin
  option log-health-checks
  reqadd X-Forwarded-Proto:\ http
  server sv1 sv1:8443 maxconn 512 check check-ssl inter 60s
  server sv2 sv2:8443 maxconn 512 backup check check-ssl inter 60s
  server sv3 sv3:8443 maxconn 512 backup check check-ssl inter 60s
  option httpchk GET /health
  http-check expect status 200

My haproxy build information is below.

Build options :
TARGET = linux2628
CPU = generic
CC = gcc
CFLAGS = -O2 -g -fno-strict-aliasing -DTCP_USER_TIMEOUT=18
OPTIONS = USE_LINUX_TPROXY=1 USE_ZLIB=1 USE_REGPARM=1 USE_OPENSSL=1 USE_PCRE=1

Default settings :
maxconn = 2000, bufsize = 16384, maxrewrite = 8192, maxpollevents = 200

Encrypted password support via crypt(3): yes
Built with zlib version : 1.2.7
Compression algorithms supported : identity, deflate, gzip
Built with OpenSSL version : OpenSSL 1.0.1e-fips 11 Feb 2013
Running on OpenSSL version : OpenSSL 1.0.1e-fips 11 Feb 2013
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports prefer-server-ciphers : yes
Built with PCRE version : 8.32 2012-11-30
PCRE library supports JIT : no (USE_PCRE_JIT not set)
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND

Available polling systems :
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use epoll.

lukastribus · March 6, 2018, 1:50pm

Then you must NOT use the ssl keyword. You can use check-ssl for SSL health checks, that’s fine, but you don’t use the SSL keyword in the server line, because otherwise you’d be encrypting the already encrypted SSL traffic.

Akihiro · March 7, 2018, 12:14am

lukastribus:

Thank you for your reply.
So, ssl-server-verify none in global directive is the only solution for self-signed ssl health-check ?
Without ‘verify none’ keyword, I cannot get UP status from backend server.

I didn’t understand your answer, and now I understand your answer.
I cannot use ‘ssl’ keyword for ssl health check.
So I only need to delete ssl keyword from server line, and that works, thank you!

Akihiro · March 8, 2018, 5:16am

@lukastribus

Can I ask you more about this?
If I want to verify my self-signed certificate to healthcheck, like condition below.
how can I configure ?

ssl pass-thorough
self-signed certificate
ssl healthcheck with http-check expect keyword like status 200, or string success.

I tried following steps, but it didn’t work.

put self-signed ca-certification at /usr/share/pki/ca-trust-source/anchors/.
run command update-ca-trust extract to import certificate (import ca to /etc/ssl/certs/ca-bundle.trust.crt)
run command ln -s /etc/ssl/certs/ca-bundle.trust.crt /etc/pki/CA/certs/. to place certificate file in th openssl related path

lukastribus · March 8, 2018, 8:22am

You need to specify the ca with the ca-file option.

Akihiro · March 8, 2018, 8:53am

Thank you for your reply,
I tried ca-file, or crt option. but didn’t work.
Anythink incorrect in the config below?

global
 <snip>
defaults
  mode tcp
 <snip>
listen foo
  bind *:443
  balance leastconn
  option log-health-checks
  option httpchk GET /healthcheck HTTP/1.1\r\nHost:\ <FQDN1>
  http-check expect string success
  server bar1 <FQDN1>:443 check inter 30000 rise 1 check-ssl ca-file /etc/pki/CA/certs/ca-bundle.trust.crt

haproxy logged like below:

<snip> failed, reason: Layer6 invalid response, info: "SSL handshake failure", check duration: 3ms, status: 0/1 DOWN.<snip>

lukastribus · March 8, 2018, 9:04am

Does your backend expect request with SNI? In that case you’d have to specify sni str(FQDN1) and check-sni FQDN1.

Akihiro · March 8, 2018, 9:31am

Thank you @lukastribus,

Does HAProxy 1.5.18 support those option?
I couldn’t find sni or check-sni option in the haproxy 1.5.18 configuration manual.

lukastribus · March 8, 2018, 9:49am

Unfortunately you need 1.6 for the sni keyword and 1.8 for the check-sni keyword.

If that is not an option, you probably want to take a look at the backend server and relax this requirement.

Akihiro · March 8, 2018, 9:50am

I see, thank you very much.
I’ll try update haproxy and sni / check-sni option.

Akihiro · March 8, 2018, 11:20am

@lukastribus, our certificate did not use SNI, only Subject Alternative Name and Subject Key I dentifier.
Is there any idea about this error?

lukastribus · March 8, 2018, 11:27am

SNI has nothing to do with the certificate, it has to do with the server HTTPS configuration. What’s the output of:

curl -vk https://fqdn1/healthcheck
openssl s_client -connect example.com:443

Akihiro · March 9, 2018, 4:23am

Thank you for your advice @lukastribus,

$ curl -kv https://FQDN2/healthcheck
* About to connect() to FQDN2 port 443 (#0)
*   Trying 10.101.4.141...
* Connected to FQDN2 (10.101.4.141) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* skipping SSL peer certificate verification
* SSL connection using TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
* Server certificate:
*       subject: CN=*.DOMAIN,O=Example,ST=example,C=example
*       start date: Feb 23 00:59:19 2018 GMT
*       expire date: Feb 16 00:59:19 2048 GMT
*       common name: *.DOMAIN
*       issuer: CN=CA_example,OU=CA,O=Example,L=Example,ST=example,C=example
> GET /healthcheck HTTP/1.1
> User-Agent: curl/7.29.0
> Host: FQDN2
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Fri, 09 Mar 2018 03:57:14 GMT
< Content-Type: application/json
< Content-Length: 20
< Connection: keep-alive
< Host: FQDN2
< X-Forwarded-For: 10.101.4.137
< X-Forwarded-Port: 443
< X-Forwarded-Proto: https
< User-Agent: curl/7.29.0
< Accept: */*
<
* Connection #0 to host FQDN2 left intact
{"status":"success"}

$ openssl s_client -connect FQDN2:443
CONNECTED(00000003)
depth=0 C = example, ST = example, O = Example, CN = *.DOMAIN
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 C = example, ST = example, O = Example, CN = *.DOMAIN
verify error:num=21:unable to verify the first certificate
verify return:1
---
Certificate chain
 0 s:/C=example/ST=example/O=Example/CN=*.DOMAIN
   i:/C=example/ST=example/L=Example/O=Example/OU=CA/CN=CA_example
---
Server certificate
-----BEGIN CERTIFICATE-----
<snip>
-----END CERTIFICATE-----
subject=/C=example/ST=example/O=Example/CN=*.DOMAIN
issuer=/C=example/ST=example/L=Example/O=Example/OU=CA/CN=CA_example
---
No client certificate CA names sent
Peer signing digest: SHA512
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 1712 bytes and written 471 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-SHA384
<snip>

    Start Time: 1520568631
    Timeout   : 300 (sec)
    Verify return code: 21 (unable to verify the first certificate)
---

lukastribus · March 9, 2018, 8:48am

Why FQDN2 now? Please stop replacing all strings, it impossible to help you this way.

Akihiro · March 9, 2018, 9:46am

sorry I replaced our FQDN to FQDN2, and also domain to DOMAIN.
FQDN2 has servername.DOMAIN.
Is real FQDN necessary to solve this problem?

lukastribus · March 9, 2018, 5:29pm

I don’t know whether that information will be the one to solve this issue. The point is that I don’t have enough information here for me to be able to understand why the SSL handshake fails.

And I will have to ask for more informations as long as I don’t have the information that explains this behavior.

Akihiro · March 28, 2018, 8:43am

@lukastribus

Everything became clear.
My colleague gave me wrong cafile.
With collect cafile, it works now.

qq9338745 · December 16, 2020, 2:58pm

@lukastribus can you comment in which condition we should specify ca-file ? and ca-file is for backend server’s ca-file or haproxy server’s ca-file ?

lukastribus · December 16, 2020, 3:53pm

The ca-file is used for the root certificate storage to verify a certificate that is supplied either by the client (SSL client certificate, ca-file on the bind line) or by the server (SSL certificate, ca-file on the server line).

Topic		Replies	Views
HAProxy in `mode tcp` accepts HTTP with HTTPS backend Help!	1	810	March 3, 2020
SSL Passthrough Not Working Help!	2	8590	March 18, 2020
Forwarding SSL traffic through HAProxy with a self signed cert fails, works when pointed directly at the backend (no haproxy) Help!	4	18556	August 25, 2016
Tcp mode and ssl confusion Help!	4	3083	September 20, 2022
SSL requests pass through although check is off Help!	4	388	September 22, 2020

Is 'ssl verify none' work for self-signed certificate in tcp mode helthcheck?

My haproxy build information is below.

Related topics