Http health check issue

Help!
1

I am a newbee to HAProxy world. I am testing HAProxy as Reverse Proxy(RP) as well as Load Balancer(LB) for our internal software environment. Please note that before posting this question, i checked other topics with httpchk health check issue and not found any solution.
I tried using Apache as RP and LB and it was working fine.
Can you please help me in this case? Kindly let me know if you need more information from myside.
#########################################
Information about setup:

  • HAProxy as RP and LB (version 1.5.19)
  • Backend is Tomcat setup with http ports
  • 6 different Application in the environment

HAProxy with httpchk is working fine only for 1 application.
########################################
Here is output of ./haproxy -vv
HA-Proxy version 1.5.19 2016/12/25
Copyright 2000-2016 Willy Tarreau willy@haproxy.org

Build options :
TARGET = linux2628
CPU = generic
CC = gcc
CFLAGS = -O2 -g -fno-strict-aliasing
OPTIONS = USE_OPENSSL=1

Default settings :
maxconn = 2000, bufsize = 16384, maxrewrite = 8192, maxpollevents = 200

Encrypted password support via crypt(3): yes
Built without zlib support (USE_ZLIB not set)
Compression algorithms supported : identity
Built with OpenSSL version : OpenSSL 1.0.1e-fips 11 Feb 2013
Running on OpenSSL version : OpenSSL 1.0.1e-fips 11 Feb 2013
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports prefer-server-ciphers : yes
Built without PCRE support (using libc’s regex instead)
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND

Available polling systems :
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use epoll.
#######################################

##Configuration(working)
frontend TEST1_FE
mode http
bind *:543 ssl crt /home/data/hacert/server1.pem
capture cookie SERVERID len 63

acl pass_node path_sub test1
use_backend TEST1_BE if pass_node
default_backend TEST1_BE

backend TEST1_BE
stats enable
balance roundrobin
mode http
option httpchk OPTIONS /test1/ HTTP/1.0
cookie SERVERID insert indirect nocache secure httponly

server server1 IP_of_machine1:8090 cookie node-sv1 check on-marked-down shutdown-sessions
server server2 IP_of_machine2:8090 cookie node-sv2 check on-marked-down shutdown-sessions

reqadd X-Forwarded-port:\ 543
reqadd X-Forwarded-proto:\ https

http-send-name-header x-dsp-client-node

For other applications httpchk is not working and causing error “backend has no server available!”. I tried various methods in httpchk, but they are not working and giving same error. If i remove httpchk other applications are working fine.

If i hit the URL from browser, it is givin 503 response with following configuration. URL is https like "https://test2-hostname.mydomain.com:545/test2. URL pattern is same for all applications
##Configuration(not working) -
frontend TEST2_FE
mode http
bind *:545 ssl crt /home/data/hacert/server1.pem

capture cookie SERVERID len 63
capture request header origin len 63
http-response add-header Access-Control-Allow-Origin %[capture.req.hdr(0)]

acl test2_node path_sub test2
use_backend TEST2_BE if test2_node

default_backend TEST2_BE

backend TEST2_BE
stats enable
balance roundrobin
mode http
option httpchk OPTIONS /test2/ HTTP/1.0\r\nHOST:\test2-hostname.mydomain.com
cookie SERVERID insert indirect nocache secure httponly
http-send-name-header x-dsp-client-node

server members for load balancing - TOMCAT

server server1 IP_of_machine1:9020 cookie node-sv1 check on-marked-down shutdown-sessions
server server2 IP_of_machine2:9020 cookie node-sv2 check on-marked-down shutdown-sessions

reqadd X-Forwarded-port:\ 545

reqadd X-Forwarded-proto:\ https

Thank you in advance for your help.

2

I’m not sure what you are asking. I understand that the health check doesn’t work for specific applications/backend servers, but we cannot possible know what a working health check looks like, as it really only depends on your backend server. Find a requests that works, and then we can help you configure the health check appriopiatly.

3

Thank you Lukas for your reply.
Here is the response we get in logs:
Health check for test1 http 200 response:
server1_IP - - [17/Feb/2019:23:59:52 +0530] “OPTIONS /test1/ HTTP/1.0” 200 -
server1_IP - - [17/Feb/2019:23:59:54 +0530] “OPTIONS /test1/ HTTP/1.0” 200 -
server1_IP - - [17/Feb/2019:23:59:56 +0530] “OPTIONS /test1/ HTTP/1.0” 200 -
server1_IP - - [17/Feb/2019:23:59:58 +0530] “OPTIONS /test1/ HTTP/1.0” 200 -

Health check for test2 http 500 response:
server1_IP - - [17/Feb/2019:23:59:53 +0530] “HEAD /test2/ HTTP/1.0” 500 -
server1_IP - - [17/Feb/2019:23:59:55 +0530] “HEAD /test2/ HTTP/1.0” 500 -
server1_IP - - [17/Feb/2019:23:59:57 +0530] “HEAD /test2/ HTTP/1.0” 500 -
server1_IP - - [17/Feb/2019:23:59:59 +0530] “HEAD /test2/ HTTP/1.0” 500 -

I tried httpchk method OPTIONS and HEAD. for both methods, i am getting http 500 response.

haproxy logs shows following:
Feb 16 18:33:09 localhost haproxy[2415]: 10.XX.XX.172:59840 [16/Feb/2019:18:33:08.981] TEST2_FE~ TEST2_BE/ 33/-1/-1/-1/+33 503 +212 SERVERID=node-sv1 - SCDN 1/0/0/0/0 0/0 {} “GET /test2/ HTTP/1.1”
Please let me know if this is what you are looking for.

4

You have to figure out why your backend doesn’t like that. There is no way for us to know why your backend rejects this.