How to load balance backend to a shard


I want to associate a set of backends with a user/tenant and load balance those backends by user/tenant.

I want the same user id to load balance to the same set of servers.

I am reading Haproxy sourcecode trying to work out where to put this functionality.

Say I have X tenants or users and Y servers. The users only exist on X/Y servers.

I suspect I can (a) download a user to server mapping list on startup. (B) introduce a header that the backend server replies with the set of servers to be used by that user once logged in.

I need to override the load balancing to only use servers that are mapped by a user id. A user stays on a particular set of servers

I could generate a very large configuration file but I was hoping there was a way I can do this dynamically. I could use Consul and consul template to template the user to server mappings for example.

  mode http
  timeout client 10s
  timeout connect 5s
  timeout server 10s 
  timeout http-request 10s

frontend frontend
  default_backend serverfarm

backend serverfarm
  sharding cookie SERVER  shardserver:5006/users.json
  cookie SERVER insert indirect nocache
  server server1 server1:8000
  server server2 server2:8000
  server server3 server3:8000
  server server4 server4:8000
  server server5 server5:8000
  server server6 server6:8000
  server server7 server7:8000
  server server8 server8:8000
  server server9 server9:8000

I managed to get this working without changing sourcecode but using Lua.

Here’s my github repository. The repository has startup scripts and python code for generating shard configurations.

In short I used the following:

	log /dev/log	local0
	log /dev/log	local1 notice
	chroot /var/lib/haproxy
	stats timeout 30s
	user haproxy
	group haproxy

	lua-load /etc/haproxy/usershard.lua 
	log	global
	mode	http
	option	httplog
	option	dontlognull
        timeout connect 5000
        timeout client  50000
        timeout server  50000
	errorfile 400 /etc/haproxy/errors/400.http
	errorfile 403 /etc/haproxy/errors/403.http
	errorfile 408 /etc/haproxy/errors/408.http
	errorfile 500 /etc/haproxy/errors/500.http
	errorfile 502 /etc/haproxy/errors/502.http
	errorfile 503 /etc/haproxy/errors/503.http
	errorfile 504 /etc/haproxy/errors/504.http
frontend stats
    bind *:8404
    stats enable
    stats uri /stats
    stats refresh 10s
    stats admin if LOCALHOST

frontend frontend
    bind *:7000
    mode http

    # inspect-delay was required or else was seeing timeouts during lua script run
    # tcp-request inspect-delay 1m

    # This line intercepts the incoming tcp request and pipes it through lua function, called "pick backend"
    http-request lua.shard

    # use_backend based off of the "streambackend" response variable we inject via lua script
    use_backend %[var(req.shard)]

backend shard0
    mode http

    server 0_0 check

    server 0_1 check

    server 0_2 check

backend shard1
    mode http

    server 1_0 check

    server 1_1 check

    server 1_2 check

backend shard2
    mode http

    server 2_0 check

    server 2_1 check

    server 2_2 check

backend shard3
    mode http

    server 3_0 check

    server 3_1 check

    server 3_2 check

backend shard4
    mode http

    server 4_0 check

    server 4_1 check

    server 4_2 check

backend shard5
    mode http

    server 5_0 check

    server 5_1 check

    server 5_2 check


local function shard(txn)
    shard = 'shard0'
    local logicalshard = "0"
    local tcp = core.tcp()
    local headers = txn.http:req_get_headers()
    if headers then
		local cookie = headers["cookie"]
		if cookie then
			session = cookie[0]
			if session then
				found_cookie = session:match("username=([a-zA-Z0-9]+)")

				if found_cookie then
					print("User has a shard cookie")
					logicalshard =  string.gsub(found_cookie, "[^a-zA-Z0-9]+", "")
			if tcp:connect("", "5000") then
			  if tcp:send("GET /shards/" .. logicalshard .." HTTP/1.1\r\nconnection: close\r\n\r\n") then
				while true do
					local line, _ = tcp:receive('*l')

					if not line then break end
					if line == '' then break end
				local line, _ = tcp:receive('*a')

				shard = "shard" .. line
			  print('Socket connection to shardserver failed')
			print('Shard is', shard)

	txn:set_var('req.shard', shard)

core.register_action('shard', {'http-req'}, shard)


	"0": 0,
	"1": 1,
	"2": 2,
	"3": 3,
	"4": 4,
	"5": 5,
	"6": 6,
	"7": 7,
	"8": 8

Shard server

import json
from flask import Flask
app = Flask(__name__)

def shard(userid=""):
    users = json.loads(open("usershards.json").read())
    if users.get(userid) != None:
        print("Found user with cookie")
        return str(users.get(userid))
        return ""