hi guys!
I want to use haproxy with AWS autoscaling groups. I’m using balancing by ID (hash-consistency). I ran into the following problem:
When autoscaling group UP Haproxy reloaded clear cache (hash tables) and we have a problem with connecting to needed instance. The students pass exam and send request with ID to haproxy -> Haproxy send request to the backend and associate hash with backend -> proctor which should view the exam send the request with same ID and goes to same backend. How i can save hash state when scaling up? Thanks
If you have just one instance you can specify the local hostname, and it will talk to itself (check the documentation from earlier), so a haproxy reload will maintain the hash-table.
Not sure if that answers your question at this point. I don’t know what AWS autoscaling is and how it works, so maybe you can explain what happens at system and userspace level?
I’ve attached my config file in my first message. Now Haproxy is balancing by hash (url_parameter) and it works fine with any number of the backends
The problem is when scaling up and new backends is adding to haproxy conf. My script check autoscaling groups and add new backends to haproxy configuration then haproxy is reloading. And hash table which associate hash with the backend is cleared.
How i can add new backend instances without loses hash data?
Ok, but why do you need to load-balance based on the URL? What is the reason you cannot achieve stickiness with cookies, for example, or with source-IP?
Students passes exams in our system. When exam starts students sent a request with ID to HaProxy which associates ID with backend. Also we have a proctors who should monitor exam. In this way when proctor openned exam he sent request to haproxy with same ID and he can view student session.
global
log /dev/log local0
log /dev/log local1 notice
chroot /var/lib/haproxy
server-state-file global
server-state-base /run/haproxy/server-state/
stats socket /run/haproxy/admin.sock mode 660 level admin
stats timeout 30s
user haproxy
group haproxy
daemon
# Default SSL material locations
ca-base /etc/ssl/certs
crt-base /etc/ssl/private
tune.ssl.default-dh-param 2048
# Default ciphers to use on SSL-enabled listening sockets.
# For more information, see ciphers(1SSL). This list is from:
# https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/
ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS
ssl-default-bind-options no-sslv3
node lb
nbproc 2
maxconn 2000000
defaults
log global
mode http
option forwardfor
option http-server-close
timeout connect 5000
timeout client 50000
timeout server 50000
errorfile 400 /etc/haproxy/errors/400.http
errorfile 403 /etc/haproxy/errors/403.http
errorfile 408 /etc/haproxy/errors/408.http
errorfile 500 /etc/haproxy/errors/500.http
errorfile 502 /etc/haproxy/errors/502.http
errorfile 503 /etc/haproxy/errors/503.http
errorfile 504 /etc/haproxy/errors/504.http
load-server-state-from-file global
frontend www-http
bind *:80
bind-process 1,2
redirect scheme https code 301 if !{ ssl_fc }
default_backend java
maxconn 2000000
frontend www-https
bind-process 1,2
bind *:443 ssl crt /etc/ssl/private/tests.com.pem
acl is_websocket hdr(Upgrade) -i WebSocket
acl is_websocket hdr_beg(Host) -i ws
capture request header origin len 50
default_backend java
maxconn 2000000
peers article
peer ithcy 127.0.0.1:1023
backend java
bind-process 1,2
stick-table type string len 512 size 20k expire 8h peers article
stick on url_param(fid,?)
option forwardfor # This sets X-Forwarded-For
http-request set-header X-Forwarded-Port %[dst_port]
http-request add-header X-Forwarded-Proto https if { ssl_fc }
http-response set-header Access-Control-Allow-Origin %[capture.req.hdr(0)]
option httpchk
default-server inter 3s fall 3 rise 2
option httpchk HEAD /students HTTP/1.0
; autoscaling group instances will be
;dynamically added below
It works fine , when I’ve reloaded haproxy and count of the backend instances doesn’t change. But If I run script which added new backends to config and reloaded haproxy it doen’t work(. Haproxy begin send requests to other instances and proctor doesn’t go to needed student
But if I’ve added new backends manually it works without problem.
I have a template with haproxy config (without backends). My script copys template everytime and added backends it the end of file and run systemctl reload haproxy. Uh, this is very strange
Looks like your automation does something that you don’t expect. Finding the difference between the manual, working sequence of events and the automated, failing sequence is something that I can’t help you with.
First case: I’ve added 5 backend instances and run video stream, haproxy associate each session with backend and when I’ve reloaded haproxy - sticky tables doesn’t clear.
Second case: I’ve run video streams then i add new backend instances to config and reload haproxy. After that sticky tables re-associate and new requests spread randomly.
UP: the problem exist when I added new instances and change backends of order than sticky tables are re-associated , when I just add new backends to end of file all ok.