HAProxy SSH setup not working

Goal: to use HAProxy to provide port multiplexing including for SSH.
Eventually, HAProxy will need to pass http/https 80/443 to nginx and I’ve gotten that to at least connect to the service it was supposed to.

I installed HAProxy 2.9.9 from the Ubuntu Oracular repo to get all the repo-packaged stuff like systemd files, logging, etc.

I compiled and installed HAProxy version 3.0.4-6cae9b-53 2024/09/10 on top of that.


With respect to SSH, I am following Route SSH Connections with HAProxy (In-Depth Configuration) but I cannot get it to work. Part of the problem is that I only have one backend SSH server so it doesn’t apply perfectly.

$ sshd --version
unknown option -- -
OpenSSH_9.7p1 Ubuntu-7ubuntu3, OpenSSL 3.3.1 4 Jun 2024
$ cat /etc/haproxy/haproxy.conf
global
	log /dev/log	local0 info	
	log /dev/log	local1 notice
	chroot /var/lib/haproxy
	stats socket /run/haproxy/admin.sock mode 660 level admin
	stats timeout 30s
	user haproxy
	group haproxy
	daemon

	# Default SSL material locations
	ca-base /etc/ssl/certs
	crt-base /etc/ssl/private

	# See: https://ssl-config.mozilla.org/#server=haproxy&server-version=2.0.3&config=intermediate
    ssl-default-bind-ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384
    ssl-default-bind-ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256
    ssl-default-bind-options ssl-min-ver TLSv1.2 no-tls-tickets

defaults
	log	global
	mode	http
	option	httplog
#	mode	tcp
#	option	tcplog
	option	dontlognull
        timeout connect 5000
        timeout client  50000
        timeout server  50000
	errorfile 400 /var/log/haproxy/errors/400.http
	errorfile 403 /var/log/haproxy/errors/403.http
	errorfile 408 /var/log/haproxy/errors/408.http
	errorfile 500 /var/log/haproxy/errors/500.http
	errorfile 502 /var/log/haproxy/errors/502.http
	errorfile 503 /var/log/haproxy/errors/503.http
	errorfile 504 /var/log/haproxy/errors/504.http


frontend fe_ssh
   bind *:2222 ssl crt /etc/haproxy/certs/mydomainname.com.pem
#   bind *:2222 ssl crt /etc/haproxy/certs/ssl.pem  #created self-signed 2024.09.19
   mode tcp
   option tcplog
   log-format "%ci:%cp [%t] %ft %b/%s %Tw/%Tc/%Tt %B %ts %ac/%fc/%bc/%sc/%rc %sq/%bq dst:%[var(sess.dst)] "
   tcp-request content set-var(sess.dst) ssl_fc_sni
#   use_backend %[ssl_fc_sni]    #no connection to haproxy.
   use_backend ssh_backend

backend ssh_backend
    mode tcp
#    server ssh1 127.0.0.1:22 check
#    server ssh1 127.0.0.1:22 check ssl verify none # error: "haproxy[165452]: backend ssh_backend has no server available!"
    server ssh1 127.0.0.1:22 ssl verify none

so now when I try to connect to this using something like what the blog example:

ssh -o ProxyCommand="openssl s_client -quiet -connect ssh.mydomainname.com:2222 -servername ssh1" ssh
Connecting to 11.22.33.44
depth=2 C=US, O=Internet Security Research Group, CN=ISRG Root X1
verify return:1
depth=1 C=US, O=Let's Encrypt, CN=E5
verify return:1
depth=0 CN=mydomainname.com
verify return:1
Connection closed by UNKNOWN port 65535

systemctl/sshd reports this:

Sep 12 16:54:48 server sshd[165969]: banner exchange: Connection from 127.0.0.1 port 39668: invalid format

haproxy.log

2024-09-12T16:54:48.990339-04:00 server haproxy[165917]: 192.168.1.1:59638 [12/Sep/2024:16:54:45.933] fe_ssh~ ssh_backend/ssh1 5/-1/3056 0 SC 1/1/0/0/3 0/0 dst:ssh1 

I’m at a loss here to figure out what the blog makes it appear to be simple.

Guidance is appreciated.

You MUST NOT enable SSL for your backend SSH server.

Your backend needs to look like this, no SSL configurations must be present here:

backend ssh_backend
 mode tcp
 server ssh1 127.0.0.1:22