I have a use case where I am expecting two work loads on the same 443 port. One expects client certs to be validated whereas, other doesn’t expect that.
is it possible to have two workloads on the same port like below? And backend is common for both.
frontend https
bind 104.191.20.22:443 ssl crt /opt/haproxy/ssl/host_key_cert.pem
frontend https
bind 35.196.140.91:443 ssl crt /opt/haproxy/ssl/host_key_cert.pem ca-file /opt/haproxy/ssl/ca_cert.pem verify optional
IP address in bold above are the IPs of the load balancers from where the two workloads are coming.
A quick response is appreciated. Thanks.
1 Like
In the bind line you specify the (destination) IP address that you want haproxy to listen to. If you have 2 IP addresses on the box running haproxy and you want to use client cert authentication on one and no client authentication on the other, than you can do it the way you are proposing.
However, you do not specify the source IP “from where the two workloads are coming” in the bind line.
I believe you don’t want the former, but the latter …
So based on the client IP address?
Then no, it is not gonna work like this. I assume you cannot use “verify optional” on both, because causes the browser ask for the client cert in both cases, is that a correct assumption?
In that case, you will have to configure a TCP frontend without SSL termination, and route the traffic based on an IP ACL to specific backends that then reroute the traffic to a SSL terminating frontend, appropriately configured.
Something like this:
frontend port443
bind :443
use_backend recir_noauth if { src 104.191.20.22 }
use_backend recir_default if { src 35.196.140.91 }
#default_backend recir_default
backend recir_noauth
server loopback-for-tls abns@haproxy-noauth send-proxy-v2
backend recir_default
server loopback-for-tls abns@haproxy-default send-proxy-v2
frontend fe-ssl-noauth
mode http
bind abns@haproxy-noauth accept-proxy ssl crt /opt/haproxy/ssl/host_key_cert.pem
# actual frontend configuration
frontend fe-ssl-default
mode http
bind abns@haproxy-default accept-proxy ssl crt /opt/haproxy/ssl/host_key_cert.pem ca-file /opt/haproxy/ssl/ca_cert.pem verify optional
# actual frontend configuration
Thanks for your response. Thats what I thought as well. But when I tested the same request from the two LBs, I saw logs are telling that they are being routed to their respective frontends without any rerouting. So, it seems like, in the bind line, it is honoring the client ip as well. Here is my configuration again with a small test that I performed.
Haproxy configuration
frontend https-1
35.196.239.755.196.140.91:443 ssl crt /opt/haproxy/ssl/host_key_cert.pem ca-file /opt/haproxy/ssl/ca_cert.pem verify optional
#actual fronend configuration
frontend https-2
bind *:443 ssl crt /opt/haproxy/ssl/host_key_cert.pem
#actual frontend configuration
Here are my test request
curl -k https://104.191.20.22/js/api.enlighted.js
curl -k https://35.196.140.91/js/api.enlighted.js
Here is the output from haproxy.log
[08/Dec/2017:19:47:32.389] https-2~ api-server/www-1 231/0/1/3/490 200 298841 - - ---- 1/1/0/1/0 0/0 “GET /js/api.enlighted.js HTTP/1.1”
[08/Dec/2017:19:47:59.005] https-1~ api-server/www-1 195/0/0/3/447 200 298841 - - ---- 1/1/0/1/0 0/0 “GET /js/api.enlighted.js HTTP/1.1”
Do you see any issue here?
No, this is wrong. You may get load-balanced between the 2 sockets and by sheer luck hit the right frontend.
I made changes suggested by you and modified my haproxy.cfg as below but I keep getting this error.
log snippet:
Mar 5 20:11:29 beta1-haproxy1-n01-us-east1-c haproxy[3157]: 40.211.11.12:50362 [05/Mar/2018:20:11:29.397] https-all htt
ps-all/<NOSRV> -1/-1/-1/-1/74 400 188 - - PR-- 0/0/0/0/0 0/0 "<BADREQ>"
haproxy.cfg snippet:
frontend https-all
bind *:443
acl is_gateway_host hdr(host) -i device.mycompany.com
use_backend recir-client-auth if is_gateway_host
default_backend recir-default
backend recir-client-auth
server loopback-for-tls abns@haproxy-client-auth send-proxy-v2
backend recir-default
server loopback-for-tls abns@haproxy-default send-proxy-v2
frontend https-devices
mode http
bind abns@haproxy-client-auth accept-proxy ssl crt /opt/haproxy/ssl/host_key_cert.pem ca-file /opt/haproxy/ssl/ca_cert.pem verify optional
default_backend gateway-server
frontend https-api
mode http
bind abns@haproxy-default accept-proxy ssl crt /opt/haproxy/ssl/host_key_cert.pem
default_backend api-server
Also, what is send-proxy-v2 here?
You are probably setting “mode http” in the default section. Either remove it there, or statically put the first 3 sections from your snippet in “mode tcp”.
This is wrong, you are not in HTTP mode here, you cannot match a HTTP header. Remember, haproxy only sees encrypted SSL traffic here. I proposed the ACL’s matching source IP addresses as per your original requirement in my example. Would you like to match something else?
Please clarify what data you would use to decide between client-authentication or no client-authentication.
It will make sure you are seeing the original client-IP on your backend.
Thanks for the quick response. Originally I was trying to use source IP addresses but then I realized that I could use the hostname that is coming in the header. This way I don’t need to hardcode ip addresses per environment.
The two workloads are coming from two different domains:
device.mycompany.com
and
api.mycompany.com
No, you cannot match the Host header, because this is SSL and that means it is encrypted.
Please try one thing at the time.
So, put your section in the correct mode like I previously suggested, make it work and then go on with improvements you think are useful.
If matching SNI is acceptable for you, that is something that will work as it is unencrypted in the client_hello. But please do one thing at the time here, and make the basics work first.
Ok, I removed the Host header check and mode http as it was there in the default section. Now trying with source IP based routing. But still getting below error in the log.
Mar 6 17:44:51 beta1-haproxy1-n01-us-east1-c haproxy[11318]: 40.211.11.12:63872 [06/Mar/2018:17:44:51.822] https-all https-all/<NOSRV> -1/-1/0 0 SC 0/0/0/0/0 0/0
haproxy.cfg:
defaults
log global
mode http
option httpclose
option forwardfor
option http-server-close
option httplog
option dontlognull
timeout connect 5000
timeout client 50000
timeout server 50000
errorfile 400 /etc/haproxy/errors/400.http
errorfile 403 /etc/haproxy/errors/403.http
errorfile 408 /etc/haproxy/errors/408.http
errorfile 500 /etc/haproxy/errors/500.http
errorfile 502 /etc/haproxy/errors/502.http
errorfile 503 /etc/haproxy/errors/503.http
errorfile 504 /etc/haproxy/errors/504.http
frontend https-all
bind *:443
mode tcp
use_backend recir-client-auth if { src 104.191.20.22 }
use_backend recir-default if { src 35.196.140.91 }
backend recir-client-auth
mode tcp
server loopback-for-tls abns@haproxy-client-auth send-proxy-v2
backend recir-default
mode tcp
server loopback-for-tls abns@haproxy-default send-proxy-v2
frontend https-devices
bind abns@haproxy-client-auth accept-proxy ssl crt /opt/haproxy/ssl/host_key_cert.pem ca-file /opt/haproxy/ssl/ca_cert.pem verify optional
default_backend gateway-server
frontend https-api
bind abns@haproxy-default accept-proxy ssl crt /opt/haproxy/ssl/host_key_cert.pem
default_backend api-server
Ok, that’s a different error message and a different root cause though.
You are coming from the IP 40.211.11.12, however you have only configured haproxy to deal with the IPs 104.191.20.22 and 35.196.140.91.
So either create a default_backend rule, or handle 40.211.11.12.
Looks like things are working for me after making few changes. Thanks for your help here.
Now coming back to SNI in the client_hello, how do I capture it in the tcp mode?
Use req.ssl_sni:
tcp-request inspect-delay 5s
tcp-request content accept if { req.ssl_hello_type 1 }
use_backend recir-client-auth if { req.ssl_sni -i clients.example.org }
use_backend recir-default if { req.ssl_sni -i www.example.org }
Cool … one of my request is still not able to meet this check.
How do I print the value of req.ssl_sni in haproxy logs?
Also, if I have “option httplog” set at default level, can I override it to no-logs in the frontend block where I handle tcp mode? It automatically switches to tcplog there.
Use the custom log format and access the req.ssl_sni variable.
I suggest to remove “mode http” and “option httplog” from you first default section, and create a second default section between the last tcp backend the the first http section, and in that default section you add:
default
mode http
option httplog