Https to external server don't works

Help!
1

Hello.
I have an HAProxy instance running on a pfsense machine, and three IIS 8 servers.

two of this IIS servers are running on the same network of the internal LAN address of the HAProxy. The third IIS server is running on other datacenter.

I have 3 backends configured to all of this servers, with one active server in each and the other servers as backup.

The diference betwin the three servers configuration is that in the two servers of the same network I have the LAN address configuured, and for the third external server I have the public IP (443 port NATed of course).

The 2 servers that are on the same network works fine with HTTPS protocol, but for the server that is on the external datacenter don’t works. I recieve the erros:

“503 Service Unavailable
No server is available to handle this request.”

If I try to connect to the third server directly bypassing the HAProxy works.

I was capturing the traffic with wireshark, and I see that the connection to the twor servers that work on the LAN establishes as TLS protocol, but the connection to the external server establishes as SSL protocol. I guess that is the reason why don’t works.

I will apreciate your help.

2

Share your configuration, the output of haproxy -vv and traces.

I assume your handshake fails, do you have specific TLS settings on haproxy and your backend server?

3

Hi. I attach you a summary of the configuration:

global
maxconn 2000
log 192.168.20.200 local0 info
stats socket /tmp/haproxy.socket level admin
uid 80
gid 80
nbproc 1
chroot /tmp/haproxy_chroot
daemon
tune.ssl.default-dh-param 2048

listen HAProxyLocalStats
bind 127.0.0.1:2200 name localstats
mode http
stats enable
stats refresh 20
stats admin if TRUE
stats uri /haproxy/haproxy_stats.php?haproxystats=1
timeout client 5000
timeout connect 5000
timeout server 5000

frontend FE-Frontend-IIS-merged
bind 91.23.15.107:80 name 91.23.15.107:80
mode http
log global
option dontlog-normal
option http-keep-alive
option forwardfor
acl https ssl_fc
http-request set-header X-Forwarded-Proto http if !https
http-request set-header X-Forwarded-Proto https if https
timeout client 180000
acl host1 hdr(host) -i host1.example.com
acl host2 hdr(host) -i host2.example.com
acl host3 hdr(host) -i host3.example.com
…
use_backend Backend-FIIS_http_ipvANY if host1
use_backend Backend-F2IIS_http_ipvANY if host2
use_backend Backend-F3IIS_http_ipvANY if host3
…

frontend FE-Frontend-IIS-HTTPS-merged
bind 91.23.15.107::443 name 91.23.15.107::443 ssl crt /var/etc/haproxy/FE-Frontend-IIS-HTTPS.pem crt /var/etc/haproxy/FE-Frontend-IIS-HTTPS
mode http
log global
option http-keep-alive
option forwardfor
acl https ssl_fc
http-request set-header X-Forwarded-Proto http if !https
http-request set-header X-Forwarded-Proto https if https
timeout client 30000
acl host1 hdr(host) -i host1.example.com
acl host2 hdr(host) -i host2.example.com
acl host3 hdr(host) -i host3.example.com
…
use_backend Backend-FIIS-SSL_http_ipvANY if host1
use_backend Backend-F2IIS-SSL_http_ipvANY if host2
use_backend Backend-F3IIS-SSL_http_ipvANY if host3
…

backend Backend-FIIS_http_ipvANY
mode http
log global
balance source
timeout connect 180000
timeout server 90000
retries 3
option httpchk OPTIONS /
option httpchk GET /status.html
server server1 192.168.3.21:80 check inter 1000 weight 100 check port 8080
server server2 192.168.3.22:80 check inter 1000 backup weight 1 check port 8080
server server3 135.23.115.157:80 check inter 1000 backup weight 1 check port 8080

backend Backend-F2IIS_http_ipvANY
mode http
log global
balance source
timeout connect 180000
timeout server 90000
retries 3
option httpchk OPTIONS /
option httpchk GET /status.html
server server2 192.168.3.22:80 check inter 1000 weight 100 check port 8080
server server1 192.168.3.21:80 check inter 1000 backup weight 1 check port 8080
server server3 135.23.115.157:80 check inter 1000 backup weight 1 check port 8080

backend Backend-F3IIS_http_ipvANY
mode http
log global
balance source
timeout connect 180000
timeout server 90000
retries 3
option httpchk GET /
option httpchk GET /status.html
server server3 135.23.115.157:80 check inter 1000 weight 100 check port 8080
server server2 192.168.3.22:80 check inter 1000 backup weight 1 check port 8080
server server1 192.168.3.21:80 check inter 1000 backup weight 1 check port 8080

backend Backend-FIIS-SSL_http_ipvANY
mode http
log global
balance source
timeout connect 90000
timeout server 90000
retries 3
option httpchk OPTIONS /
option httpchk GET /status.html
server server1 192.168.3.21:443 ssl check inter 1000 weight 100 verify none check port 8080
server server2 192.168.3.22:443 ssl check inter 1000 backup weight 1 verify none check port 8080
server server3 135.23.115.157:443 ssl check inter 1000 backup weight 1 verify none check port 8080

backend Backend-F2IIS-SSL_http_ipvANY
mode http
log global
balance source
timeout connect 90000
timeout server 90000
retries 3
option httpchk OPTIONS /
option httpchk GET /status.html
server server2 192.168.3.22:443 ssl check inter 1000 weight 100 verify none check port 8080
server server1 192.168.3.21:443 ssl check inter 1000 backup weight 1 verify none check port 8080
server server3 135.23.115.157:443 ssl check inter 1000 backup weight 1 verify none check port 8080

backend Backend-F3IIS-SSL_http_ipvANY
mode http
log global
balance source
timeout connect 90000
timeout server 90000
retries 3
option httpchk OPTIONS /
option httpchk GET /status.html
server server3 135.23.115.157:443 ssl check inter 1000 weight 100 verify none check port 8080
server server2 192.168.3.22:443 ssl check inter 1000 backup weight 1 verify none check port 8080
server server1 192.168.3.21:443 ssl check inter 1000 backup weight 1 verify none check port 8080

The server that fails on the https responses is the “server3”

I attach you the wireshark capture on server3:

No. Time Source Destination Protocol Length Info
3104 3.046924 76.43.23.67 192.168.4.23 TCP 74 29952 → 443 [SYN] Seq=0 Win=65228 Len=0 MSS=1460 WS=128 SACK_PERM=1 TSval=3750355018 TSecr=0

Frame 3104: 74 bytes on wire (592 bits), 74 bytes captured (592 bits) on interface 0
Ethernet II, Src: Vmware_93:02:f9 (00:50:56:93:02:f9), Dst: Vmware_01:21:aa (00:50:56:01:21:aa)
Internet Protocol Version 4, Src: 76.43.23.67, Dst: 192.168.4.23
Transmission Control Protocol, Src Port: 29952, Dst Port: 443, Seq: 0, Len: 0

No. Time Source Destination Protocol Length Info
3105 3.046950 192.168.4.23 76.43.23.67 TCP 74 443 → 29952 [SYN, ACK] Seq=0 Ack=1 Win=8192 Len=0 MSS=1460 WS=256 SACK_PERM=1 TSval=12303884 TSecr=3750355018

Frame 3105: 74 bytes on wire (592 bits), 74 bytes captured (592 bits) on interface 0
Ethernet II, Src: Vmware_01:21:aa (00:50:56:01:21:aa), Dst: Vmware_93:02:f9 (00:50:56:93:02:f9)
Internet Protocol Version 4, Src: 192.168.4.23, Dst: 76.43.23.67
Transmission Control Protocol, Src Port: 443, Dst Port: 29952, Seq: 0, Ack: 1, Len: 0

No. Time Source Destination Protocol Length Info
3140 3.059831 76.43.23.67 192.168.4.23 TCP 66 29952 → 443 [ACK] Seq=1 Ack=1 Win=66560 Len=0 TSval=3750355028 TSecr=12303884

Frame 3140: 66 bytes on wire (528 bits), 66 bytes captured (528 bits) on interface 0
Ethernet II, Src: Vmware_93:02:f9 (00:50:56:93:02:f9), Dst: Vmware_01:21:aa (00:50:56:01:21:aa)
Internet Protocol Version 4, Src: 76.43.23.67, Dst: 192.168.4.23
Transmission Control Protocol, Src Port: 29952, Dst Port: 443, Seq: 1, Ack: 1, Len: 0

No. Time Source Destination Protocol Length Info
3148 3.060179 76.43.23.67 192.168.4.23 SSL 583 Client Hello

Frame 3148: 583 bytes on wire (4664 bits), 583 bytes captured (4664 bits) on interface 0
Ethernet II, Src: Vmware_93:02:f9 (00:50:56:93:02:f9), Dst: Vmware_01:21:aa (00:50:56:01:21:aa)
Internet Protocol Version 4, Src: 76.43.23.67, Dst: 192.168.4.23
Transmission Control Protocol, Src Port: 29952, Dst Port: 443, Seq: 1, Ack: 1, Len: 517
Secure Sockets Layer
SSL Record Layer: Handshake Protocol: Client Hello
Content Type: Handshake (22)
Version: TLS 1.0 (0x0301)
Length: 512
Handshake Protocol: Client Hello
Handshake Type: Client Hello (1)
Length: 508
Version: TLS 1.2 (0x0303)
Random
Session ID Length: 0
Cipher Suites Length: 132
Cipher Suites (66 suites)
Compression Methods Length: 1
Compression Methods (1 method)
Extensions Length: 335
Extension: ec_point_formats
Extension: elliptic_curves
Extension: SessionTicket TLS
Extension: signature_algorithms
Extension: Heartbeat
Extension: Padding

No. Time Source Destination Protocol Length Info
3151 3.060221 192.168.4.23 76.43.23.67 TCP 54 443 → 29952 [RST, ACK] Seq=1 Ack=518 Win=0 Len=0

Frame 3151: 54 bytes on wire (432 bits), 54 bytes captured (432 bits) on interface 0
Ethernet II, Src: Vmware_01:21:aa (00:50:56:01:21:aa), Dst: Vmware_93:02:f9 (00:50:56:93:02:f9)
Internet Protocol Version 4, Src: 192.168.4.23, Dst: 76.43.23.67
Transmission Control Protocol, Src Port: 443, Dst Port: 29952, Seq: 1, Ack: 518, Len: 0

And the haproxy syslog server logs:

Apr 3 20:34:52 76.43.23.67 haproxy[69298]: 218.131.49.68:56858 [03/Apr/2017:20:34:49.219] FE-Frontend-IIS-HTTPS-merged~ Test_http_ipvANY/server3 16/0/-1/-1/3137 503 212 - - SC-- 62/2/0/0/3 0/0 "GET / HTTP/1.1"
Apr 3 20:34:55 76.43.23.67 haproxy[69298]: 218.131.49.68:10293 [03/Apr/2017:20:34:52.738] FE-Frontend-IIS-HTTPS-merged~ Test_http_ipvANY/server3 39/0/-1/-1/3152 503 212 - - SC-- 66/2/0/0/3 0/0 “GET /favicon.ico HTTP/1.1”

The three servers are Windows 2012 server with IIS8 and shared config. They are configured with no changes on the default configuration regards ssl or tls parameters.

Thanks you for advance

4

What’s with the duplicate colons in your entire configuration? A single colon is supposed to be between IP and port, not two or three.

Looks like something rejects the TLS session with a TCP Reset. Check firewalls on all involved hosts and your backend server log.

Also, post the output of:
curl -vvvkI https://135.23.115.157/

from the haproxy server.

5

Hello. I ma sorry, the duplicate colon is an error copying and pasting. The configuration is right. I mended the error on the post.

The response to curl -vvvkI https://135.23.115.157/ from Haproxy server:

  • Rebuilt URL to: https://135.23.115.157/
  • Trying 135.23.115.157…
  • TCP_NODELAY set
  • Connected to 135.23.115.157 (135.23.115.157) port 443 (#0)
  • Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
  • successfully set certificate verify locations:
  • CAfile: /usr/local/share/certs/ca-root-nss.crt
    CApath: none
  • TLSv1.2 (OUT), TLS handshake, Client hello (1):
  • Unknown SSL protocol error in connection to 135.23.115.157:443
  • Curl_http_done: called premature == 1
  • Closing connection 0
    curl: (35) Unknown SSL protocol error in connection to 135.23.115.157:443

I checked the firewall and is all right, If I try to connect to this server from my computer directly bypassing the haproxy works fine.
Thanks you for advance.

6

Your server aborts the HTTPS request even from curl as per the output you just posted. Clearly, haproxy is not the issue here, but your network/backend firewall is.

7

Hello.
Thanks.
I discovered out that the problem is on the server3 IIS certificate confiuration. Resolved and working!

Thanks for your help.

Kind regards.