No free ports error & nbthreads

Hello everyone,

We recently hit that “no free port” limitation:

Connect() failed for backend backend_mysql_master: no free ports

And worked around it by using the following on a one server backend:
server mysql-main-04-vip1 xxx:3306 check source xx.xx.xx.170:32768-65535 weight 100

  • server mysql-main-04-vip2 xxxx:3306 check source xx.xx.xx.171:32768-65535 weight 100*
  • server mysql-main-04-vip3 xxxx:3306 check source xx.xx.xx.172:32768-65535 weight 100*
  • server mysql-main-04-vip4 xxxx:3306 check source xx.xx.xx.173:32768-65535 weight 100*

But when activating also nbproc & nbthread option giving more resources to the Haproxy 1.8.19 instance, we started having these issues again. We’ve found this thread talking about the incidence of nbproc : Problem: Connect() failed for backend: no free ports.

It talks also about static source port so two questions:

  • Can we use nbproc without static port since we ave a > 4.2 Kernel, like this conf for instance
    server mysql-main-04-vip1 xxx:3306 check source xx.xx.xx.170 weight 100
    server mysql-main-04-vip2 xxxx:3306 check source xx.xx.xx.171 weight 100
    server mysql-main-04-vip3 xxxx:3306 check source xx.xx.xx.172 weight 100
    server mysql-main-04-vip4 xxxx:3306 check source xx.xx.xx.173 weight 100

  • Or it just don’t matter as we only have 1 proc but multithreads and the issue doesn’t affect when having only multiple threads?

    nbproc 1
    nbthread 3
    cpu-map auto:1/1-3 0-2

Thanks for your help :slight_smile:

Have a good day everyone.

Kin

Do not combine nbproc with nbthread, this is strongly discouraged.

That said, using nbproc itself is also discouraged:

USING MULTIPLE PROCESSES IS HARDER TO DEBUG AND IS REALLY DISCOURAGED

Exactly, just don’t use nbproc. It’s use is discouraged, it makes everything more complex and fragile.

But also I’d suggest to drop the source port range anyway, and let the kernel decide. The kernel knows this staff better. If you want to restrict the source port range for whatever reason, you can adjust them via sysctls.

So in summary, I suggest to drop both nbproc and the source port ranges from the config and let the kernel handle this stuff.

Otherwise haproxy will try a specific source port, which the kernel may decline, then haproxy has to try a different port, etc … this can all be avoided by just letting the kernel do it’s job.

Hello Lukas,

You’re right, seeing the doc, even nbproc is discouraged but since we set it at 1, we should be fine.

Removing static source-port helped apparently as we don’t have any errors so far, but we also added some CPU cores so maybe it helped too :slight_smile:

Anyways, thanks for your help!

Kin

Correct, nbproc 1 is equivalent to not setting nbproc at all.

Yes, because the kernel does a much better job of selecting the source ports.

1 Like