Clarification needed on sharing a stick table and server selection

Help!
1

The documentation at the following page https://www.haproxy.com/documentation/hapee/1-8r1/onepage/#4-stick%20on makes this comment:

It is possible to share a table with other backends by
referencing it using the "table" keyword. If another table is referenced,
the server's ID inside the backends are used. By default, all server IDs
start at 1 in each backend, so the server ordering is enough. But in case of
doubt, it is highly recommended to force server IDs using their "id" setting.

This point "If another table is referenced, the server’s ID inside the backends are used." is vague in that what server id will it be using and from where?

Also this point “By default, all server IDs start at 1 in each backend, so the server ordering is enough.”, I’m not sure what it is referring to by server ordering is enough ?

Also, the following example is given in that doc:

2018-10-26_9-43-41
2018-10-26_9-43-41.png1416×1165 82 KB

In my testing when I share a table with another backend and then see what is in the table it only shows one record, for example.

# table: one, type: ip, size:1048576, used:1
0x1a7bb04: key=10.0.0.2 use=0 exp=17718 server_id=1

Since there is one record in the table and it is being shared by another backend and each back having it’s own servers then what is the logic to select a server in each backend?

In the doc note (see above) it states that it is highly recommended to use ids, when I do that for example.

backend one
	stick on src table two
	server s1 10.0.0.5:123 id 100
	
backend two
	stick-table type ip size 200k expire 30m
	stick on src
	server s1 10.0.0.7:222 id 200

When I view table two it still shows only one record:

# table: two, type: ip, size:1048576, used:1
0x1a7bb04: key=10.0.0.2 use=0 exp=17718 server_id=200

So then how would backend one server selection work when there is only one record in the table where the server id = 200 - which is not one of the ids in backend one ?

2

Thats the enterprise edition “HAPEE”. The haproxy documentation is in doc/configuration.txt or online at:
https://cbonte.github.io/haproxy-dconv/1.8/configuration.html

In this case, it’s the same though.

There is only one server ID for each server in a backend, but also for each stick table entry. And the backend is selected by your configuration, so I’m not sure what’s unclear?

If you don’t statically configure the IDs, haproxy will assign them sequentially:

backend bck1
 server s1 10.0.0.1:80 # automatically assigned id = 1 (first server in this backend)
 server s2 10.0.0.2:80 # automatically assigned id = 2
 server s3 10.0.0.3:80 # automatically assigned id = 3
 
backend bck2
 server s1 10.0.0.1:80 # automatically assigned id = 1 (first server in this backend)
 server s2 10.0.0.2:80 # automatically assigned id = 2
 server s3 10.0.0.3:80 # automatically assigned id = 3

But, if you mess up the ordering, the ID’s will not match:

backend bck1
 server s1 10.0.0.1:80 # automatically assigned id = 1 (first server in this backend)
 server s2 10.0.0.2:80 # automatically assigned id = 2
 server s3 10.0.0.3:80 # automatically assigned id = 3
 
backend bck2
 server s1 10.0.0.1:80 # automatically assigned id = 1 (first server in this backend)
 server s3 10.0.0.3:80 # automatically assigned id = 2 (second server so ID = 2, but it is really server3)
 server s2 10.0.0.2:80 # automatically assigned id = 3

That’s why, if you need cross-backend stickiness, you’d better assign static IDs yourself, unless you are sure the ordering will always be the same.

Correct, if 10.0.0.2 was your only client (IP), the you will see only 10.0.0.2 pointing to ID=1.

It shows that your client 10.0.0.2 is assigned to the server with ID 200. I don’t see what’s wrong with that? Did you have clients different than 10.0.0.2 that connected to haproxy and that you expect to show up in the stick table?

Please explain what you are trying to achieve in the first place, in those last 3 threads you never explained what you actually want to do, and I doubt you attempted the correct solutions. Read about the XY problem.

3

Please explain what you are trying to achieve in the first place

In each case the meaning has been pretty simple to derive but in this case it is simply to share a table with multiple backends. Why? Instead of having multiple tables each with the same configuration which can be hard to maintain it is easier to have one table to manage. For example 10 tables with the same config versus 1.

I realized the doc was from enterprise edition but it functions the same as you mentioned so let me continue.

Regarding the “so the server ordering is enough” comment in the doc which was vague, your response regarding the order of the servers being off make sense so thanks for that clarification.

I’m still confused on the other point through, so hopefully you will clear this up further.

There is only one server ID for each server in a backend, but also for each stick table entry. And the backend is selected by your configuration, so I’m not sure what’s unclear?

I understand that each backend has servers defined and each server gets an id, by default starting with 1 and incrementing and that it is possible to assign your own id. However, say for example you have the following backends:

backend one
	stick on src table two
	server s1 10.0.0.5:123 id 100
	
backend two
	stick-table type ip size 200k expire 30m
	stick on src
	server s1 10.0.0.7:222 id 200

Notice, the servers have different IP addresses in each backend however they are sharing the same stick-table and also using the source IP as the key.

Now, consider this scenario:

  • The first request of the user goes to backend one
  • A subsequent request of the user (same IP as the first) goes to backend two

Since both requests are using the same IP but different backends with different servers then how does the stick-table manage that ?

In my testing I’m seeing a single entry in the stick-table as I showed in my previous post.

So how can a single entry in a single stick table be enough information to justify selecting the right server in two backends with different servers with the same source IP ?

4

No, this feature is not intended to save 1 configuration line per backend, it has an entirely different use-case.

The configuration of the tables are not the same. One table belongs to it’s specific backend. If you configure cross-backend tables then it is assumed that you understand your ID’s can collide.

You are miss-using this feature for something it is not intended for.

5

Your being cryptic again. You didn’t even address my question, what happens in the case I described ??

In the case I described does the one entry in the table get overwritten when used with the two backends ?

As stated in my first post I made it clear that I want to share a single table with multiple backends and it clearly stated in documentation which I referenced, in your terminology if that means cross-backend tables then so be it but my IDs are not colliding so I don’t know why your going there.

Provide details don’t just tell me I’m miss understanding, if I have it totally wrong then provide details as to exactly why this feature was created and how it works.

6
  • stick table is empty
  • a user connects from 10.0.0.2, accessing an application on backend one
    • first server is selected via round-robin (as the stick table is empty)
  • stick table stores the 10.0.0.2 => server 1 (10.0.0.5) mapping, meaning it stores 10.0.0.2 => server ID=1
  • a user connects from 10.0.0.2, accessing an application on backend two:
    • source IP of the users is already present in the stick table
    • lookup in the table show mapping to server ID=1
    • in backend two, ID=1 maps to the server with 10.0.0.7
  • because you forced haproxy to use the table for both backends:
    • the mapping makes no sense as there is no server with 10.0.0.5 in backend two
    • it will treat 10.0.0.7 as 10.0.0.5 from a load-balancing perspective (because that’s what ID=1 means here in backend two)
    • if backend two has 30 servers and backend one has 1 server, only the first server in backend two will ever be used (just as an example of the havoc this will cause)

Why would this feature ever be there in the first place?
Because if you specifically WANT to use the same stick table, you can do it. This does make sense, IF you have the same EXACT servers in both backends, but maybe for a different application or protocol, like in the example you posted, where backend https and backend http handle the same application in a different protocol, but the fact that the same stick table is used (along with the same servers in the same order) makes it so that both HTTP and HTTPS traffic use the same stickiness rules, which of course makes sense if the application requires that and the servers are exactly the same (and listed in the same order).

7

Thanks for the detailed answer, this level of details is missing from the documentation so I appreciate you taking the time to clarify.

I understand now.

One more question, you mentioned:

No, this feature is not intended to save 1 configuration line per backend, it has an entirely different use-case.

Up to this point I understand how it works. However, you mentioned it has different use case so can you elaborate on other use case please.

8

The use-case above:

A setup that is passing through both port 80 and port 443 in separate front and backends, but where the desire is to use the same table for both. This may be useful when only parts of the application works with HTTP and the other part works with HTTPS, yet the customer has to hit the same backend server. That is the use-case that this documentation example suggests.

9

Ok, then it doesn’t have an entirely different use case like you mentioned, I was confused by that comment because it was vague and I understand now.

I was trying to consolidate to a single table instead of having to manage many, which now that we are on the same page that is one purpose.

Also, it may even be possible to use different servers in different backends and still use a single table. By concatenating or using a combination of keys for the table. Because currently in our discussion it is only been about using IP as the key for the table - if a combination of keys are used to so that each table is represented unique by the key in the single table then theoretically it is possible - what are your thoughts on that ?

10

Don’t do it, backends will overwrite each others data.

Suppose the config looks like this (fixing ID’s to unique values):

backend one
 server s1 10.0.0.1:80 id=11
 server s2 10.0.0.2:80 id=12
 server s3 10.0.0.3:80 id=13

backend two
 server s4 10.0.0.4:80 id=21
 server s5 10.0.0.5:80 id=22
 server s6 10.0.0.6:80 id=23

Now what could happen is:

  • customer accesses content on backend one, server s1 is assigned, the table will contain customer-IP => 11 (key => value)
  • customer accesses content on backend two, ID=11 is not available, so the request has to be dispatched to a new server, the key “customer-IP” in the table is overwritten by the new ID (let’s say 21). ID=11 is lost.
  • customer accesses backend one again, expecting the ID to be 11, since server s1 is where he just put it’s data, but because we the table was overwritten with what was relevant for backend two, we just broke stickiness for backend one.