Stick Table based on IP source and port source both

Hi all,

We have an issue with our specific backend configuration. These backend receive a request from a SERVER_ORIGIN with IP Z.Z.Z.Z The backend has connected two internal servers: SERVER_1 (Y.Y.Y.2)
and SERVER_2 (Y.Y.Y.3) We have a schema for a better understanding:

The problem occurs when SERVER_ORIGIN try to establish three simultaneous connections to the FRONTEND (we connect with telnet X.X.X.X 6447) to the frontend.

The HA-Proxy just open a single session to one of the internal servers, when it should be three sessions.

_echo “show table backend_1” | socat stdio /var/run/haproxy.stat _
#table: backend_1, type: ip, size:10485760, used:3
0x25990e4: key=U.U.U.U use=0 exp=3559075 server_id=1

We have enabled stick-table option, but only through source IP (stick on src) and we need do it through source IP and source port both.

backend backend_1
_ mode tcp_
_ source 0.0.0.0 usesrc clientip_
_ balance leastconn_
_ option abortonclose_
_ option persist _
_ option redispatch _
_ option tcplog_
_ stick-table type ip size 10240k expire 60m peers HAPROXYPEERSPRE_
_ stick on src <----_
_ server server_1 Y.Y.Y.2:6447 weight 1 maxconn 2000 check port 6447 inter 3s rise 2 fall 3 on-marked-down shutdown-sessions_
_ server server_2 Y.Y.Y.3:6447 weight 1 maxconn 2000 check port 6447 inter 3s rise 2 fall 3 on-marked-down shutdown-sessions_

We have looked for in HA-Proxy Documentation (Fetching Samples --> Converters section) but we didn’t found with the correct option to establish that kind of sticky session.
Here we are an example which we have tested but we don’t know how to use correctly the function (there’s no any examples)

backend backend_1
_ mode tcp_
_ source 0.0.0.0 usesrc clientip_
_ balance leastconn_
_ option abortonclose_
_ option persist _
_ option redispatch _
_ option tcplog_
_ stick-table type binary len 32 size 30k expire 60m peers HAPROXYPEERSPRE_
_ stick on and(src, src_port) <----_
_ server server_1 Y.Y.Y.2:6447 weight 1 maxconn 2000 check port 6447 inter 3s rise 2 fall 3 on-marked-down shutdown-sessions_
_ server server_2 Y.Y.Y.3:6447 weight 1 maxconn 2000 check port 6447 inter 3s rise 2 fall 3 on-marked-down shutdown-sessions_

Finally, our question is, ¿what is the correct expression of function?¿We need other option or configuration for a correct working?

No, haproxy opens 3 TCP sessions. The stick table is not equivalent to the number of opened TCP session.

What is the reason you have to stick on source port? Please elaborate what problem you are trying to solve.

What is the reason you have to stick on source port? Please elaborate what problem you are trying to solve.

Every opened session should be balanced to each internal server. When we configurate stick on src option, HAProxy open all TCP sessions to the same internal server, and this is a problem to us because it is not balancing between internal servers. For this reason we need to stick on source port.

The purpose is that, from one original server trying to connect with a internal server through different origin ports, the sessions opened has to be balanced to the differents internal servers.

Ok, I understand, but your conclusion is wrong.

You don’t need any stick table AT ALL.

So just remove all stick table parameters and you will fulfill this exact requirement of yours.

I understand your answer but, ¿do you want meaning there’s persistence altough you delete stick on src? with your response you are saying that from a origin server with two open sessions through two differents origin ports, HAProxy balancing between both internal servers, it’s true?

Delete stick on src and stick table, both lines i want to mean.

Sorry, maybe it’s not clear the previous answer, i wanted to say that if you delete all stick table, it works the persistence for SERVER_ORIGIN? so, when has an unexpected close the HAProxy’s persistence table maintain the Z.Z.Z.Z IP from a new request of that machine, in a specific timeout, of course. It’s correct?

Persistence means that when the client will connect to the server, it connects to the same backend server as last time (as opposed to load-balance).

You specifically requested that you DO NOT WANT that behavior, but to load-balance instead.

So, by disabling the stick table you are disabling persistence, exactly as requested.

Sorry for last question, maybe it hasn’t been clear. I will try to explain it again our problem.

We think HAProxy should load-balance every new connection between our backend servers from IP origin with balance leastconn option enabled (session to backend server with least connections). The problem we have with this configurarion is that HAProxy consult every new connection from that IP origin in the stick table, which has previous connection from that IP. For this reason, HAProxy always send every new connection at the same backend server instead of balancing between all backend servers we had (option leastconn should work).

For this cause, we want that stick table will be based, not only IP origin, but port origin too. Thus, every new open session could be balanced between backend servers (with least-connection option) but keeping persistence of every session.

So, the question would be, ¿could stick-table being based on IP and port origin both?

No, stick-table cannot be based on source port, because this doesn’t make any sense at all.

You are confused about the concept of persistence, lets try to clear that up:

Why exactly do you think you need persistence for your application? Please explain.

Now it’s clear what you saying. Thanks for your patient. I can’t explain better tan the schema because of my poor english, i’m sorry. Naturally, there’s no sense stick table based on source port, because persistence has to be with IP or cookie normally. If it will based on source port, every connection (every source port) would go to every backend server, no matter IP origin and it’s not definition of persistence so it’s clear.

Grettings and thank you again.

Exactly, when the connections drops (for whatever reason), then the client will retry with a new source port.

I understand what he was talking about and actually have a use case for it.

If you have devices behind a NAT then you will only see 1 ip address for all the devices. So that if you ‘balance source’ or stick to an IP address then all the devices behind that NAT will go to the same server. Normally it is no big deal when the entire internet is your customer. But when you have a single customer that makes up most or all of your usage then you will not be able to stick connections to more than one server. There will be no initial load balancing, they will all go to one server, the server that the first connector went to.

In my case we have cell devices that will be connecting via data link to our load balancer. So if there are 10k devices, that need persistence during the session, then they will all go to the same server.

This may help, I need to try it. https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#4.2-option%20prefer-last-server

But if not then I need to figure out how to key off of incoming IP address AND incoming port. In the world of cellular IOT, this will be something that needs solving, or HAProxy will not work.

Let me know if I am missing something here. I totally could be even though I have been combing the docs, there is a lot to learn, even after using HAProxy for almost 10 years. lol

@davea this not make any sense at all. Your client will either keep the TCP session open, so stickiness is NOT needed, because the connection isn’t even closed. And when your client does not keep the session open, it will come back on a different source port (that is how TCP work with or without NAT), breaking your source port assumption.

So stickiness based on source IP + source port leads to EXACTLY the same behavior as not having stickiness at all.

If you think you have found a use-case, explain why you think the client comes in a new TCP session, with the same TCP source port.

I have exactly the same requirement. I’m looking at switching from nginx to HAproxy, nginx has this tcp src addr & src port persistence option, but I can find how to do this in HAproxy. Why do I need it, because I want to implement a tcp based load balancer that will tunnel TLS/SSL requests to a backend, as that involves a TLS 3 way hand shake I need the all the traffic from a given process on a remote host to go to the same backend server, but I also want to load balance different processes on the same remote server to different backends. So I can’t use roundrobin, as the TLS handshake pkts may go to different backends. I can’t use source as then ALL the processes on a given remote will go to the same backend. If I can have persistence on remote IP and remote TCP port, then all pkts from a given process on a remote host will go to the same backend, and two process on that same remote hosts will go to different backends.

@BrendanDoyle you requirement is not special at all. It is literally the standard use-case for TCP based load-balancing and it does NOT require persistence.

Wrong. A TCP connection terminates on the haproxy box in the frontend, and is mapped 1:1 to a TCP connection on the backend. Those TCP connection stay up and running until one of the TCP sessions dies.

You are thinking way to complicated. Haproxy will pipe one TCP connection on one side to one TCP connection on the other side with a 1:1 mapping, and those TCP connection are just normal TCP connections. There is nothing special about it.