I’ve been using HAProxy for SSL termination as part of a stack that looks like this:
https http http
Internet <-----> haproxy <----> varnish <----> nginx
Everything works great, but adding HTTP/2 support has slammed me hard into a wall and I can’t figure a way out of it. I don’t want to jettison HAProxy in favor of Hitch, but I think I’m about to unless I can figure out some magical voodoo configuration options to get things working
Problem, the short version: The only way I can figure out how to make HTTP/2 work correctly and in conjunction with HSTS (i.e., using haproxy to redirect all client http attempts to https) is to use mode http for the frontend (necessary because I have about 10 sites each using various http-header settings for various things) and mode tcp for the back end )because I have to communicate to Varnish using the proxy protocol, because varnish in turn needs to communicate to nginx with the proxy protocol).
Problem, the short version, continued: If I switch the frontend to mode tcp, I get beautiful HTTP/2-served web sites with no problem. Everything works great. However, http-request redirect scheme https if http obviously no longer works and I don’t know if there are substitutes for that and all the other http-request commands I’m using for domain redirects and the like. I need that functionality and I need it at the termination layer, not deeper in the stack. Conversely, switching the backend to mode http breaks everything and nothing works—no pages get served and all I get is a ERR_SPDY_PROTOCOL_ERROR when I try to connect to anything.
Problem, the long version: ugh, I don’t know if I have the energy to type all this out, but here are some config snippets at each piece in the stack to show what I’m doing:
- haproxy:
defaults
log global
mode http
option httplog
option dontlognull
option forwardfor
...
frontend my_front
bind :::80 v4v6
bind :::443 v4v6 ssl crt mycert.pem ecdhe secp384r1 alpn h2,http/1.1
acl http ssl_fc,not
acl letsencryptrequest path_beg -i /.well-known/acme-challenge/
acl mastodon hdr(host) beg -i mastodon.bigdinosaur.org
use_backend letsencrypt if letsencryptrequest
use_backend mastodonwtf if mastodon
http-request redirect prefix https://www.bigdinosaur.org code 301 if { hdr(host) -i bigdinosaur.org }
...(imagine lots more 301s here)...
http-request redirect scheme https if http
http-response set-header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload;"
http-response set-header Referrer-Policy "strict-origin-when-cross-origin"
http-response set-header X-Content-Type-Options "nosniff"
http-response set-header X-XSS-Protection "1; mode=block"
use_backend mastodon if mastodon
rsprep ^Set-Cookie:\ (.*) Set-Cookie:\ \1;\ Secure if { ssl_fc }
default_backend tovarnish
...
backend tovarnish
http-request set-header X-Forwarded-Port %[dst_port]
http-request add-header X-Forwarded-Proto https if { ssl_fc }
server local 127.0.0.1:6081 send-proxy-v2
backend mastodon
http-request set-header X-Forwarded-Port %[dst_port]
http-request add-header X-Forwarded-Proto https if { ssl_fc }
http-response set-header Content-Security-Policy redacted for length
http-response set-header Public-Key-Pins redacted for length
server local 127.0.0.1:6081 send-proxy-v2
backend letsencrypt
server letsencrypt 127.0.0.1:54321
- Varnish:
# Backend definition. Set this to point to your content server. Nginx listens on 2 ports,
# one for non-upgraded and non-http2 requests (default), and the other for http2 requests.
backend default {
.host = "127.0.0.1";
.port = "8086";
.first_byte_timeout = 600s;
.between_bytes_timeout = 600s;
.max_connections = 800;
.proxy_header = 1;
}
backend h2 {
.host = "127.0.0.1";
.port = "8088";
.first_byte_timeout = 600s;
.between_bytes_timeout = 600s;
.max_connections = 800;
.proxy_header = 1;
}
...
sub vcl_recv {
# Happens before we check if we have this in cache already.
#
# Typically you clean up the request here, removing cookies you don't need,
# rewriting the request, etc.
if (req.http.protocol ~ "HTTP/2") {
set req.backend_hint = h2;
}
else {
set req.backend_hint = default;
}
}
- Nginx:
server {
server_name www.bigdinosaur.org;
listen 8088 http2 proxy_protocol default_server;
listen 8086 proxy_protocol;
...
(all the rest of the vhost file)
I know based on some quick testing with a virtual machine that I can rip out HAProxy and drop in Hitch and everything Just Works 
, but I like the additional flexibility HAProxy gives me with being able to do redirects and backend voodoo (for example, if I rip out haproxy, I have to rethink my entire LetsEncrypt setup, ugh).
Does anyone have any insight on potential ways forward here that will let me keep HAProxy? I’ve been banging on this for a couple of days on and off and I just can’t seem to reach a solution that works.