HTTTP2 with TLS1.2+ pass through... is it possible?

Help!
1

Greetings!

I am using Haproxy 2.05 (upgraded from 1.8 onwards) with HTTP1.1 and SSL pass through configured and working fine on FreeBSD 11.2 with OpenSSL 1.0.x + LetsEncrypt TLS1.2 certs and few backends running Apache and Nginx servers.

Now I am upgrading to a new server running FreeBSD 12.0 and OpenSSL 1.1.1c (OpenSSL has been installed from ports).

I am curious to know if I could enable HTTP2 with SSL pass through. I am not able to find much info on how to do this if it possible with Haproxy 2.05 and all my trial and error have been failing as of now. If it is possible, kindly share how to achieve this.

Or, is SSL termination the only way out to enable HTTP2 with TL1.2+? I have read a bit about CPU spikes in case of SSL termination mode, although I have never personally tried this option. Is that true? Which option would be better to use in production with just a few servers? We haven’t had too much traffic (less than 2k hits daily) but expect it to rise considerably (maybe 20-30k hits daily) in the near future.

I am still learning the tricks of the trade so please do excuse me in case my queries seem too naive.

Thanks in advance.

Warm regards,

Nitz

2

No. SSL pass through means that everything is end-to-end encrypted, which means that haproxy does not have visibility into even higher protocols like HTTP. If you want HTTP2 with SSL pass through, you need to enable HTTP2 on your backend server, Apache and nginx, where you terminate SSL and leave haproxy configuration untouched.

Either:

  • enable HTTP2 on your backend server, leaving haproxy in TCP mode, passing through SSL and all it’s encrypted payloads, or
  • actually terminate SSL on haproxy and enable whatever feature you like, including HTTP2

SSL handshakes are demanding. Currently you have the SSL handshake CPU load on your backend server. If you terminate SSL on the load-balancer, this load will move to haproxy.

3

Thank you Lukas for clarifying.

Yes I am doing that… and I am realizing now that it doesn’t seem to work so smoothly though. Perhaps my configs are not perfect. But in HTTP1.1 the site works perfectly fine.

On enabling HTTP2, I am seeing that pages load and don’t load intermittently showing a 404 error in both Firefox and Chrome.

We use Nginx to deliver static files and Apache for the main site. We are using the same SSL certificate for both the main domain as well as the subdomain that serves the static content.

I was wondering if this is in any ways similar to the issue being discussed here:
https://bugzilla.mozilla.org/show_bug.cgi?id=1427256

or here
https://bugzilla.mozilla.org/show_bug.cgi?id=1222136

I can generate separate certs if required… but maybe my configs are not up to the mark. So I pasted the relevant configs (with IPs and paths hidden) for HA-Proxy, Apache and Nginx on the current server. My apologies in case configs should be attached separately here, but being new I am unable to find a way to do so.

Kindly have a look and let me know if there is anything I am doing wrong or can be improved upon.

Thanks in advance for all your help.

Have a great weekend!

Nitz

[HA-Proxy 2.05]
global
daemon
maxconn 2048
log /var/run/log local0 notice

defaults
mode tcp
option dontlognull
timeout connect 5s
timeout client 10s
timeout server 30s

frontend http
bind *:80
stick-table type ip size 1m expire 10s store gpc0,http_err_rate(10s)
tcp-request connection track-sc1 src
tcp-request connection reject if { src_get_gpc0 gt 0 }
mode http
option http-server-close
option forwardfor
reqadd X-Forwarded-Proto:\ http
acl is_static hdr(host) -i -f /static_urls
use_backend httpstatic if is_static
default_backend httpnodes

frontend https
bind *:443
mode tcp
tcp-request inspect-delay 5s
tcp-request content accept if { req_ssl_hello_type 1 }
option http-server-close
option forwardfor
reqadd X-Forwarded-Proto:\ https
acl is_static req_ssl_sni -i -f /static_urls
use_backend httpsstatic if is_static
default_backend httpsnodes

backend httpnodes
mode http
option forwardfor
server webserver xxx.xxx.xxx.xxx:80 check

backend httpsnodes
mode tcp
tcp-request inspect-delay 5s
tcp-request content accept if { req_ssl_hello_type 1 }
option ssl-hello-chk
server webserver xxx.xxx.xxx.xxx:443 check inter 2000 rise 2 fall 5

backend httpstatic
mode http
option forwardfor
server staticserver xxx.xxx.xxx.xxx:8080 check

backend httpsstatic
mode tcp
tcp-request inspect-delay 5s
tcp-request content accept if { req_ssl_hello_type 1 }
option ssl-hello-chk
server staticserver xxx.xxx.xxx.xxx:4343 check inter 2000 rise 2 fall 5

[Apache ver.2.4.35]

ServerName XYZ.com ServerAlias www.XYZ.com ServerAdmin webmaster@XYZ.com DocumentRoot /XYZ/ ErrorLog /XYZ.com-error_log TransferLog /XYZ.com-access_log Redirect permanent "/" "https://www.XYZ.com/" ServerName www.XYZ.com #ServerAlias www.XYZ.com ServerAdmin webmaster@XYZ.com DocumentRoot /XYZ/ LogLevel debug ErrorLog /XYZ.com-error_log TransferLog /XYZ.com-access_log Protocols h2 h2c http/1.1 Header set Strict-Transport-Security "max-age=63072000; includeSubdomains; preload" Header always set X-Frame-Options DENY SSLEngine on SSLCertificateFile "/XYZ.com.cer" SSLCertificateKeyFile "/XYZ.com.key" SSLCertificateChainFile "/ca.cer" ProxyPassMatch ^/(.*\.php(/.*)?)$ fcgi://127.0.0.1:5000/src/$1 Options Indexes FollowSymLinks MultiViews AllowOverride All DirectoryIndex index.php FallbackResource index.php Require all granted RewriteEngine On RewriteOptions inherit RewriteCond %{REQUEST_FILENAME} (.css|.js) RewriteRule ^(/css|/js)/(.*)\.[0-9]+\.(.*)$ $1/$2.$3 [L,R,NS] #Order Allow,Deny #Allow from all Require all granted Deny from env=bad_bot Header always set Expect-CT "enforce, max-age=300, report-uri='https://www.XYZ.com/'" FallbackResource disabled BrowserMatch "MSIE [2-5]" \ nokeepalive ssl-unclean-shutdown \ downgrade-1.0 force-response-1.0 Header always set Content-Security-Policy: "default-src 'none'; img-src 'self' https://static.XYZ.com; font-src data: https://static.XYZ.com; style-src 'unsaf e-inline' https://static.XYZ.com; script-src 'unsafe-inline' https://static.XYZ.com; object-src 'none'; base-uri 'none';"

[Nginx ver.1.14.0]
server {
add_header Cache-Control “public”;
listen 80;
server_name static.XYZ.com;
return 301 https://static.XYZ.com$uri;
}

server {
listen 443 http2 ssl;
server_name static.XYZ.com;

    ssl_certificate      <path to certs>/fullchain.cer;
    ssl_certificate_key  <path to certs>/XYZ.com.key;

    ssl_session_cache    shared:SSL:10m;
    ssl_session_timeout  5m;

    #ssl_ciphers  HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers  on;
    ssl_ciphers EECDH+CHACHA20:EECDH+AES128:RSA+AES128:EECDH+AES256:RSA+AES256:EECDH+3DES:RSA+3DES:!MD5;
    ssl_stapling off;
    ssl_stapling_verify off;
    location / {
        root   <path to static files>/files;
        index  index.html;
        location ~*  \.(jpg|jpeg|png|svg|svgz|webp|gif|ico|css|js|ttc|ttf|pdf|xml|otf|eot|woff|woff2)$ {

expires 1y;

            add_header Pragma public;
            add_header Cache-Control "public, max-age=31536000, immutable";
            add_header Access-Control-Allow-Origin "https://www.XYZ.com";
            add_header X-Content-Type-Options nosniff;
            rewrite ^/css/[0-9]+/(.*) /css/$1 last;
            rewrite ^/js/[0-9]+/(.*) /js/$1 last;
        }
    }
}
4

Yes, the certificates you are using on the servers of the httpsstatic backend must not overlap with the certificates from the httpsnodes backends servers, otherwise browser will hit the wrong server.

5

Awesome, thank you Lukas!