Then recompiling nginx should do the job already.
You can confirm that IP_BIND_ADDRESS_NO_PORT is in action by stracing nginx during a request to the backend (strace -ttp<PID>).
Sure, I will check this after installing fresh version on stage server.
Anyway, this doesn’t help much for Haproxy itself. As I saw these errors when problem occured:
haproxy[3070]: [ALERT] 325/132220 (3117) : Cannot bind to source address before connect() for backend backend_redis_write. Aborting
It looks that even separate (from nginx) outgoing address wouldn’t help much in this case.
I checked Haproxy with strace. It uses the same bind() call with sin_port=htons(0). So, Haproxy has to be recompiled with IP_BIND_ADDRESS_NO_PORT too.
Don’t use a different outgoing IP address from haproxy. You don’t need that, so you don’t need IP_BIND_ADDRESS_NO_PORT.
So you suggest only to recompile Nginx, right?
Btw, looking at this thread: Layer4 connection problem, info: “Connection refused at step 1 of tcp-check (connect)” I could say that I have no port defined for tcp-check option. Should I specify it in my configuration?
Yes, recompile nginx so that it picks up IP_BIND_ADDRESS_NO_PORT support which has only been in CentOs libc shortly.
An alternative is to checkout whether you can configure your IP stack on the OS side so that it picks up the source IP you want and drop the proxy_bind keyword from the nginx configuration. This would make it so that the kernel knows what IP address to bind to (for a certain destination IP range), so you don’t have to tell your application (nginx) that.
Please see ip route show src field for a description how you install routes with a specific hint for source IP address selection. If your kernel already knows what source IP address to pick for a specific destination network, you can drop all the source IP bind shenanigans in the application, which will solve all those problems (and you don’t need IP_BIND_ADDRESS_NO_PORT in the application).
Absolutely not. That thread is about a specific bug affecting some 1.7 and 1.8 stable releases. That has nothing to do with your problem. Your are not hitting that bug (which is: health checking fails 100% of the time).
You really need to focus on the specific problems and only apply 1 suggestions at the time, not trying 3 things at the same time.
I have to use proxy_bind to choose different outgoing IPs for some remote IPs. Both in Haproxy and Nginx.
P.S. Btw, why did you say that no need to recompile Haproxy with IP_BIND_ADDRESS_NO_PORT ?
I understand that you need to use certain source IP’s for specific remote IPs. That doesn’t mean you need to handle it in the application (that’s just one option). You can just as well (or better) handle it in the OS stack, by configuring a route with the src keyword as explained above.
If you need to handle it in the application, then you also need IP_BIND_ADDRESS_NO_PORT for reliable behavior.
Because you didn’t say the you need to use a specific source IP in haproxy and the haproxy configuration you shared did not contain that configuration.
If you do, then make sure haproxy support IP_BIND_ADDRESS_NO_PORT.
Everything you need to fix this is here. You have the choice between doing it in the application (nginx/haproxy), but with IP_BIND_ADDRESS_NO_PORT support, or in the kernel by configuring ip routes with the src keyword.
Managing route table when you need to choose outgoing address depending on L4 information (dst port) seems to be harder than specifying source/proxy_bind address in appropriate proxy app.
Thank you for taking your time.
In that case, correct.