Hi Guys,
I have a java web app running behind HAProxy Load balancer, users getting authentication error when connecting (they can’t login), but everything is working fine with incognito mode or if clear browser cache.
This is looks like a sticky session issue. we are running haproxy 2.3 and client requests will be redirected by haproxy using round-robin (our webapp required to have persistent connections) so here is my config. appreciate any feedback. cheers
global
log stdout format raw local0 info
pidfile /run/haproxy.pid
daemon
# Default SSL material locations
ca-base /etc/ssl/certs
crt-base /etc/ssl/private
# Default ciphers to use on SSL-enabled listening sockets.
# For more information, see ciphers(1SSL). This list is from:
# https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/
ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS
ssl-default-bind-options no-sslv3
tune.ssl.default-dh-param 2048
defaults
log global
mode http
option httplog
option dontlognull
timeout connect 60000
timeout client 50000
timeout server 50000
listen stats
bind *:1936
stats enable
stats uri /
stats hide-version
stats auth xxx:xxx
frontend http-frontend
bind *:80
bind *:443 ssl crt /server.pem
redirect scheme https if !{ ssl_fc }
mode http
default_backend http-backend
resolvers localdns
parse-resolv-conf
hold valid 5s
backend http-backend
mode http
balance roundrobin
default-server inter 2s fastinter 2s downinter 2s fall 3 rise 2
cookie JSESSIONID prefix
option forwardfor
http-request set-header X-Forwarded-Port %[dst_port]
http-request add-header X-Forwarded-Proto https if { ssl_fc }
dynamic-cookie-key MYKEY
cookie SRVID insert dynamic
default-server resolvers localdns
option httpchk HEAD /ha/health HTTP/1.0
server-template webapp1 1 ${BACKEND_1} check fall 3 rise 2
server-template webapp2 1 ${BACKEND_2} check fall 3 rise 2
You seem to have conflicting configuration regarding cookies, please remove:
cookie JSESSIONID prefix
from the configuration.
1 Like
@lukastribus thanks for the help, made the change to my config with (removed cookie JSESSIONID prefix as requested)
cookie SRVID insert dynamic nocache
But users are seeing different page details each time they refresh page. but if they remove the cache, then they can see the correct page details. do i need to add any cache expiry ? (most of our users use chrome browser)
Thanks
And when and how do they come in the situation?
You are saying it works if they clear the cache, please explain what happens after that. Does it work for some time and then it stops?
@lukastribus even some people having login issues as well, it worked some times but not stable. sometimes you see different pages after refreshing the page. still we are having login issues, and it worked after cleaning cache.
so i decided to try the following , please let me know your thoughts.
backend http-backend
mode http
balance roundrobin
default-server inter 2s fastinter 2s downinter 2s fall 3 rise 2
option forwardfor
http-request set-header X-Forwarded-Port %[dst_port]
http-request add-header X-Forwarded-Proto https if { ssl_fc }
cookie JSESSIONID prefix
default-server resolvers localdns
option httpchk HEAD /ha/health HTTP/1.0
server-template back1 1 ${BACKEND_NODE1} check cookie back1
server-template back2 1 ${BACKEND_NODE2} check cookie back2
completely removing following and adding only, cookie JSESSIONID prefix
dynamic-cookie-key MYKEY
cookie SRVID insert dynamic nocache
let me know your thoughts.
thanks in advance.
I don’t know the root cause of the issue. Your change means you are no longer using the dynamic cookie mode, but static configuration. If that fixes the issue great, but we don’t really understand the root cause at this point.
Can you share the complete configuration and the output of haproxy -vv?
@lukastribus here is the requested information.
Current running config (having login issues ,page refresh gives different output, clear cache solve the issue, not working in incognito mode- i think chrome will disable third party cookies in that mode so this is acceptable)
global
log stdout format raw local0 info
pidfile /run/haproxy.pid
daemon
# Default SSL material locations
ca-base /etc/ssl/certs
crt-base /etc/ssl/private
# Default ciphers to use on SSL-enabled listening sockets.
# For more information, see ciphers(1SSL). This list is from:
# https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/
ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS
ssl-default-bind-options no-sslv3
tune.ssl.default-dh-param 2048
defaults
log global
mode http
option httplog
option dontlognull
timeout connect 60000
timeout client 50000
timeout server 50000
listen stats
bind *:1936
stats enable
stats uri /
stats hide-version
stats auth admin:admin
frontend http-frontend
bind *:80
bind *:443 ssl crt /server.pem
redirect scheme https if !{ ssl_fc }
mode http
default_backend http-backend
resolvers localdns
parse-resolv-conf
hold valid 5s
backend http-backend
mode http
balance roundrobin
default-server inter 2s fastinter 2s downinter 2s fall 3 rise 2
option forwardfor
http-request set-header X-Forwarded-Port %[dst_port]
http-request add-header X-Forwarded-Proto https if { ssl_fc }
dynamic-cookie-key MYKEY
cookie SRVID insert dynamic nocache
default-server resolvers localdns
option httpchk HEAD /ha/health HTTP/1.0
server-template node1 1 ${BACKEND_NODE1} check fall 3 rise 2
server-template node2 1 ${BACKEND_NODE2} check fall 3 rise 2
haproxy -vv
root@1b70e94b92a8:/# haproxy -vv
HA-Proxy version 2.3.4-10189c9 2021/01/13 - https://haproxy.org/
Status: stable branch - will stop receiving fixes around Q1 2022.
Known bugs: http://www.haproxy.org/bugs/bugs-2.3.4.html
Running on: Linux 5.10.76-linuxkit #1 SMP Mon Nov 8 10:21:19 UTC 2021 x86_64
Build options :
TARGET = linux-glibc
CPU = generic
CC = cc
CFLAGS = -O2 -g -Wall -Wextra -Wdeclaration-after-statement -fwrapv -Wno-unused-label -Wno-sign-compare -Wno-unused-parameter -Wno-clobbered -Wno-missing-field-initializers -Wno-cast-function-type -Wtype-limits -Wshift-negative-value -Wshift-overflow=2 -Wduplicated-cond -Wnull-dereference
OPTIONS = USE_PCRE2=1 USE_PCRE2_JIT=1 USE_GETADDRINFO=1 USE_OPENSSL=1 USE_LUA=1 USE_ZLIB=1
DEBUG =
Feature list : +EPOLL -KQUEUE +NETFILTER -PCRE -PCRE_JIT +PCRE2 +PCRE2_JIT +POLL -PRIVATE_CACHE +THREAD -PTHREAD_PSHARED +BACKTRACE -STATIC_PCRE -STATIC_PCRE2 +TPROXY +LINUX_TPROXY +LINUX_SPLICE +LIBCRYPT +CRYPT_H +GETADDRINFO +OPENSSL +LUA +FUTEX +ACCEPT4 -CLOSEFROM +ZLIB -SLZ +CPU_AFFINITY +TFO +NS +DL +RT -DEVICEATLAS -51DEGREES -WURFL -SYSTEMD -OBSOLETE_LINKER +PRCTL +THREAD_DUMP -EVPORTS
Default settings :
bufsize = 16384, maxrewrite = 1024, maxpollevents = 200
Built with multi-threading support (MAX_THREADS=64, default=4).
Built with OpenSSL version : OpenSSL 1.1.1d 10 Sep 2019
Running on OpenSSL version : OpenSSL 1.1.1d 10 Sep 2019
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
Built with Lua version : Lua 5.3.3
Built with network namespace support.
Built with the Prometheus exporter as a service
Built with zlib version : 1.2.11
Running on zlib version : 1.2.11
Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
Built with PCRE2 version : 10.32 2018-09-10
PCRE2 library supports JIT : yes
Encrypted password support via crypt(3): yes
Built with gcc compiler version 8.3.0
Available polling systems :
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use epoll.
Available multiplexer protocols :
(protocols marked as <default> cannot be specified using 'proto' keyword)
h2 : mode=HTTP side=FE|BE mux=H2
fcgi : mode=HTTP side=BE mux=FCGI
<default> : mode=HTTP side=FE|BE mux=H1
<default> : mode=TCP side=FE|BE mux=PASS
Available services : prometheus-exporter
Available filters :
[SPOE] spoe
[CACHE] cache
[FCGI] fcgi-app
[COMP] compression
[TRACE] trace
Thanks