Checking multiple conditions does not work

Help!
1

Hi,

I want to check 2 conditions on redis, where it should be master and on other checks, at least 2 of 3 sentinel nodes should provide this redis-instance being master.

Therefore, number of servers found on another backend check, is reused and if number if found server is at least 2, it should be selected.

However, condition with use-server and nbsrv is totally ignored. ge 2 is not working and lt 1 for example, not, too.
Any ideas?
Best
Ronny

backend check_master_redis1
mode tcp
option tcp-check
tcp-check connect
tcp-check send PING\r\n
tcp-check expect string +PONG
tcp-check send SENTINEL\ master\ secucore-redis-001\r\n
tcp-check expect string 10.240.50.1
tcp-check send QUIT\r\n
tcp-check expect string +OK

server sentinel1 10.240.50.1:26379 check inter 2s
server sentinel2 10.240.50.2:26379 check inter 2s
server sentinel3 10.240.50.3:26379 check inter 2s

backend check_master_redis2
mode tcp
option tcp-check
tcp-check connect
tcp-check send PING\r\n
tcp-check expect string +PONG
tcp-check send SENTINEL\ master\ secucore-redis-001\r\n
tcp-check expect string 10.240.50.2
tcp-check send QUIT\r\n
tcp-check expect string +OK

server sentinel1 10.240.50.1:26379 check inter 2s
server sentinel2 10.240.50.2:26379 check inter 2s
server sentinel3 10.240.50.3:26379 check inter 2s

backend check_master_redis3
mode tcp
option tcp-check
tcp-check connect
tcp-check send PING\r\n
tcp-check expect string +PONG
tcp-check send SENTINEL\ master\ secucore-redis-001\r\n
tcp-check expect string 10.240.50.3
tcp-check send QUIT\r\n
tcp-check expect string +OK

server sentinel1 10.240.50.1:26379 check inter 2s
server sentinel2 10.240.50.2:26379 check inter 2s
server sentinel3 10.240.50.3:26379 check inter 2s

frontend ft_redis
bind *:6379
mode tcp
use_backend bk_redis

backend bk_redis
mode tcp

option tcp-check
tcp-check connect
tcp-check send PING\r\n
tcp-check expect string +PONG
tcp-check send info\ replication\r\n
tcp-check expect string role:master
tcp-check send QUIT\r\n
tcp-check expect string +OK

use-server redis1 if { srv_is_up(redis1) } { nbsrv(check_master_redis1) ge 2 }
use-server redis2 if { srv_is_up(redis2) } { nbsrv(check_master_redis2) ge 2 }
use-server redis3 if { srv_is_up(redis3) } { nbsrv(check_master_redis3) ge 2 }

server redis1 10.240.50.1:6379 check inter 1s
server redis2 10.240.50.2:6379 check inter 1s
server redis3 10.240.50.3:6379 check inter 1s

2

You need to set server weight to 0 if you want to content switch like this.

https://cbonte.github.io/haproxy-dconv/2.0/configuration.html#4.2-use-server

The “use-server” statement works both in HTTP and TCP mode. This makes it suitable for use with content-based inspection. For instance, a server could be selected in a farm according to the TLS SNI field. And if these servers have their weight set to zero, they will not be used for other traffic.

3

Thanks for your hint,
adding or removing weight to 0 does not take any effect on the configuration,
the condition is still not checked and ignored :frowning:

4

And what does that actually mean? Is it load-balanced to a random server or not at all?

5

It is loadbalanced, BUT without checking the condition which is in the config like if { srv_is_up(redis1) } { nbsrv(check_master_redis1) ge 2 }

Even if you change it to lt 1 or anything else, it is totally ignored

6

Ok, and when you set weigth to 0, it is not load-balanced anymore, is that correct?

ge 2 will never be true, and lt 1 won’t be either.

You have configured backends that check which server is the master, and I don’t think there will be more than one master active at one time, right?

So ge 2 will never be true, because the number will never be 2 or larger.
And lt 1 will never be true, because the number will never be less than 1.

Please read:
https://cbonte.github.io/haproxy-dconv/2.0/configuration.html#7.1.2

Available operators for integer matching are :

eq : true if the tested value equals at least one value
ge : true if the tested value is greater than or equal to at least one value
gt : true if the tested value is greater than at least one value
le : true if the tested value is less than or equal to at least one value
lt : true if the tested value is less than at least one value

Check your logs and status pages to understand the real number of backend servers available and use the operators appropriately.

7

Well, this is the Problem ;):
It is never true the result, so there should be no server available at the backend, but it always is … because it seems to prefer the tcp check before and to ignore the condition totally in use-server

Have you understood the problem?
If not, we can do a skype or gotomeeting session if you ´re interested and I can explain it to you.
Best
Ronny

8

Right, that’s why I asked:

9

Setting weight to 0 or adding a weight in general, results in a blue selected node, but it still seems to be ignored because I can continue to connect, what should not be correct, because one of the master check results in 3 x available servers at all (means lt 1 to fake it and let them result in false), still does not work.

I did a workaround, using 3 backends now, with then checking with nbsrv, this is working well, so nbsrv seems not to work on use_server, but on use-backend

Maybe we can have a 10 min websession and I present exactly what I mean

10

How are you checking that, EXACTLY? Do you see in the haproxy log, that a connection to a server is established?

11

Lukas,
I am confused.

I now use this config with weight0

frontend ft_redis
bind *:6379
option dontlog-normal
mode tcp
maxconn 6000
tcp-request connection reject if { nbsrv(check_master_redis1) ge 2 } { nbsrv(check_master_redis2) ge 2 } { nbsrv(check_master_redis3) ge 2 }
default_backend bk_redis

backend bk_redis
mode tcp
fullconn 6000
option tcp-check
tcp-check connect
tcp-check send PING\r\n
tcp-check expect string +PONG
tcp-check send info\ replication\r\n
tcp-check expect string role:master
tcp-check send QUIT\r\n
tcp-check expect string +OK
use-server redis1 if { srv_is_up(redis1) } { nbsrv(check_master_redis1) ge 2 }
server redis1 10.240.50.1:6379 check inter 1s maxconn 6000 weight 0
use-server redis2 if { srv_is_up(redis2) } { nbsrv(check_master_redis2) ge 2 }
server redis2 10.240.50.2:6379 check inter 1s maxconn 6000 weight 0
use-server redis3 if { srv_is_up(redis3) } { nbsrv(check_master_redis3) ge 2 }
server redis3 10.240.50.3:6379 check inter 1s maxconn 6000 weight 0

This is now working, I checked it using:

watch ‘redis-cli -h 10.240.40.200 -p 6379 info | grep run_id’

I swore, the last 3 times I did it, it did not work either.

A little confusing is, using “lt 1” instead of “ge 2” to test the conditions, results still in a blue bar in haproxy, on the tcp-checked server which was valid, so “master” server is always marked as blue, because I think, this is a kind of wanted behaviour, that it still is marked as “up” because the tcp-check is valid, and the condition is not a kind of relevance for it?
Best,
Ronny

12

The colors on the stats page are about the health check results and health check result only.

Your use-server directives have nothing to do with that.

13

However, weight 0 is the trick you need, otherwise, the rules are ignored. Thank you very much.

14

The rules are not ignored with the default weight, but normal load-balancing occurs, when none of the use-server directives match.

use-server is an optional configuration, and when the conditions do not match, it won’t do anything as expected.

And with a non-zero weight that means normal load-balancing will occur in that case.

15

If I do not set any weight in the config, it does ignore my check for at least 2 servers on … :wink:
It began to work, if I set it to weight 0 :slight_smile: