We have an issue with our specific backend configuration. These backend receive a request from a SERVER_ORIGIN with IP Z.Z.Z.Z The backend has connected two internal servers: SERVER_1 (Y.Y.Y.2)
and SERVER_2 (Y.Y.Y.3) We have a schema for a better understanding:
The problem occurs when SERVER_ORIGIN try to establish three simultaneous connections to the FRONTEND (we connect with telnet X.X.X.X 6447) to the frontend.
The HA-Proxy just open a single session to one of the internal servers, when it should be three sessions.
We have enabled stick-table option, but only through source IP (stick on src) and we need do it through source IP and source port both.
backend backend_1
_ mode tcp_
_ source 0.0.0.0 usesrc clientip_
_ balance leastconn_
_ option abortonclose_
_ option persist _
_ option redispatch _
_ option tcplog_
_ stick-table type ip size 10240k expire 60m peers HAPROXYPEERSPRE_
_ stick on src <----_
_ server server_1 Y.Y.Y.2:6447 weight 1 maxconn 2000 check port 6447 inter 3s rise 2 fall 3 on-marked-down shutdown-sessions_
_ server server_2 Y.Y.Y.3:6447 weight 1 maxconn 2000 check port 6447 inter 3s rise 2 fall 3 on-marked-down shutdown-sessions_
We have looked for in HA-Proxy Documentation (Fetching Samples --> Converters section) but we didn’t found with the correct option to establish that kind of sticky session.
Here we are an example which we have tested but we don’t know how to use correctly the function (there’s no any examples)
backend backend_1
_ mode tcp_
_ source 0.0.0.0 usesrc clientip_
_ balance leastconn_
_ option abortonclose_
_ option persist _
_ option redispatch _
_ option tcplog_
_ stick-table type binary len 32 size 30k expire 60m peers HAPROXYPEERSPRE_
_ stick on and(src, src_port) <----_
_ server server_1 Y.Y.Y.2:6447 weight 1 maxconn 2000 check port 6447 inter 3s rise 2 fall 3 on-marked-down shutdown-sessions_
_ server server_2 Y.Y.Y.3:6447 weight 1 maxconn 2000 check port 6447 inter 3s rise 2 fall 3 on-marked-down shutdown-sessions_
Finally, our question is, ¿what is the correct expression of function?¿We need other option or configuration for a correct working?
What is the reason you have to stick on source port? Please elaborate what problem you are trying to solve.
Every opened session should be balanced to each internal server. When we configurate stick on src option, HAProxy open all TCP sessions to the same internal server, and this is a problem to us because it is not balancing between internal servers. For this reason we need to stick on source port.
The purpose is that, from one original server trying to connect with a internal server through different origin ports, the sessions opened has to be balanced to the differents internal servers.
I understand your answer but, ¿do you want meaning there’s persistence altough you delete stick on src? with your response you are saying that from a origin server with two open sessions through two differents origin ports, HAProxy balancing between both internal servers, it’s true?
Sorry, maybe it’s not clear the previous answer, i wanted to say that if you delete all stick table, it works the persistence for SERVER_ORIGIN? so, when has an unexpected close the HAProxy’s persistence table maintain the Z.Z.Z.Z IP from a new request of that machine, in a specific timeout, of course. It’s correct?
Sorry for last question, maybe it hasn’t been clear. I will try to explain it again our problem.
We think HAProxy should load-balance every new connection between our backend servers from IP origin with balance leastconn option enabled (session to backend server with least connections). The problem we have with this configurarion is that HAProxy consult every new connection from that IP origin in the stick table, which has previous connection from that IP. For this reason, HAProxy always send every new connection at the same backend server instead of balancing between all backend servers we had (option leastconn should work).
For this cause, we want that stick table will be based, not only IP origin, but port origin too. Thus, every new open session could be balanced between backend servers (with least-connection option) but keeping persistence of every session.
So, the question would be, ¿could stick-table being based on IP and port origin both?
Now it’s clear what you saying. Thanks for your patient. I can’t explain better tan the schema because of my poor english, i’m sorry. Naturally, there’s no sense stick table based on source port, because persistence has to be with IP or cookie normally. If it will based on source port, every connection (every source port) would go to every backend server, no matter IP origin and it’s not definition of persistence so it’s clear.
I understand what he was talking about and actually have a use case for it.
If you have devices behind a NAT then you will only see 1 ip address for all the devices. So that if you ‘balance source’ or stick to an IP address then all the devices behind that NAT will go to the same server. Normally it is no big deal when the entire internet is your customer. But when you have a single customer that makes up most or all of your usage then you will not be able to stick connections to more than one server. There will be no initial load balancing, they will all go to one server, the server that the first connector went to.
In my case we have cell devices that will be connecting via data link to our load balancer. So if there are 10k devices, that need persistence during the session, then they will all go to the same server.
But if not then I need to figure out how to key off of incoming IP address AND incoming port. In the world of cellular IOT, this will be something that needs solving, or HAProxy will not work.
Let me know if I am missing something here. I totally could be even though I have been combing the docs, there is a lot to learn, even after using HAProxy for almost 10 years. lol
@davea this not make any sense at all. Your client will either keep the TCP session open, so stickiness is NOT needed, because the connection isn’t even closed. And when your client does not keep the session open, it will come back on a different source port (that is how TCP work with or without NAT), breaking your source port assumption.
So stickiness based on source IP + source port leads to EXACTLY the same behavior as not having stickiness at all.
If you think you have found a use-case, explain why you think the client comes in a new TCP session, with the same TCP source port.
I have exactly the same requirement. I’m looking at switching from nginx to HAproxy, nginx has this tcp src addr & src port persistence option, but I can find how to do this in HAproxy. Why do I need it, because I want to implement a tcp based load balancer that will tunnel TLS/SSL requests to a backend, as that involves a TLS 3 way hand shake I need the all the traffic from a given process on a remote host to go to the same backend server, but I also want to load balance different processes on the same remote server to different backends. So I can’t use roundrobin, as the TLS handshake pkts may go to different backends. I can’t use source as then ALL the processes on a given remote will go to the same backend. If I can have persistence on remote IP and remote TCP port, then all pkts from a given process on a remote host will go to the same backend, and two process on that same remote hosts will go to different backends.
@BrendanDoyle you requirement is not special at all. It is literally the standard use-case for TCP based load-balancing and it does NOT require persistence.
Wrong. A TCP connection terminates on the haproxy box in the frontend, and is mapped 1:1 to a TCP connection on the backend. Those TCP connection stay up and running until one of the TCP sessions dies.
You are thinking way to complicated. Haproxy will pipe one TCP connection on one side to one TCP connection on the other side with a 1:1 mapping, and those TCP connection are just normal TCP connections. There is nothing special about it.
