HAProxy community

Let HAProxy starts despite servers not yet running in docker


#1

Hello,
I try to setup HAProxy as a reverse proxy and SSL termination for my websites.
After googling for examples I finally managed to have a little setup with HAProxy and two services.
Since the IP addresses of my services could change, I decided to go with using the hostnames of the services (eg. server nginx static:80).

However, ff my service containers are not yet running, the HAProxy container is not starting at all.
In order to get the setup working I should run HAProxy always as the latest.
I found an option (default-server init-addr last,libc,none put in defaults section) that allows the HAProxy container to start even if one of my container are not up.

The problem is that when that container is up, HAProxy does not detects this.
I tried with enabling check interval without success.
Can anyone explain me how I can manage to have my HAProxy container been able to automatically connect (without raising errors when starting) to a backend whenever is up.

Thank you very much.


#2

You need to setup DNS resolution and health checking properly, otherwise it won’t work. I suggest you read through the entire “Server IP address resolution using DNS” section in the manual.

For example, a tiny configuration snippet for DNS resolution would look like this:

resolvers googledns
        nameserver 8888 8.8.8.8:53
        nameserver 8844 8.8.4.4:53
        hold valid 300s

backend bk_example_resolution
        mode http
        server examplesrv www.example.com:80 check inter 30s resolvers googledns resolve-prefer ipv4

Also, don’t use “last” in your init-addr config, unless you really setup server-state-file, it makes no sense. Just libc,none is exactly what you’d want.


#3

Hi,
Thank you for your answer.
I tried your solution, but unfortunately it doesn’t work.

I wonder if isn’t there a “circular referencing” (sorry I don’t know how to name it) since it’s the job of my HAProxy container to ‘route’ the actual domain.

However, by hostnames I mean the docker hostnames which are internal names and normally they are internal address IP replacement and do not need to use a DNS server.
When the containers are already started I’m able to use them instead of the internal IP address of the docker container, something like that server nginx static_site:80 instead of server nginx 172.18.0.2:80


#4

Well, you need to adjust the configuration above to your specific needs of course. Use the nameserver than can actually resolve the hostnames you need, also do specify the FQDN name (without trailing dot).


#5

Yep I changed the values in regards to my configuration.
But I don’t know why it’s not working.

Nevertheless, you said to use the nameserver than can actually resolve the hostnames
But when using the dns of google, the returned address is my external IP, not the privates one that are actually set to my servers.

However, in that sentence do you think about the nameserver of docker itself?
I just realized that docker has maybe such a service that can resolve the hostname (which do not have tld extension as in my examples) to the internal IPs.
But, even if it’s the case, I think that the problem will remain since when not started that hostname will not be resolved yet…

I’m really stucked because of such a simple thing…


#6

This is really very simple:

Do you have a DNS server accessible from your docker container that resolves the “static_site” to 172.18.0.2?

Then you use that nameserver to resolve it - not Google.

If you don’t have any nameserver accessible from docker that will resolve “static_site” to 172.18.0.2, then really I do not know what you are asking here.


#7

Hello,

Before all, thank you for your patience…
In fact I think that this dns “workaround” is not the solution at all… (but of course I’m completely mistaken)
Here is a simple scenario to understand what I mean:

Supposing I have three docker projects, each with its own docker-compose.yml file: an nginx server, a rails server and the haproxy server

  1. I run my rails server using docker-compose -d up in the project directory
  2. I run my nginx server using docker-compose -d up in the project directory
  3. I run my HAProxy server using docker-compose -d up in the project directory
  4. → Everything works fine
  5. Now I stop and remove, say, my nginx server using docker-compose down in the project directory
  6. → Everything else works fine. The browser displays just nothing when visiting that domain.
  7. I restart my nginx server using docker-compose -d up in the project directory
  8. → Everything works fine as before. The nginx server is just responding as if it was never been down.

Something worth mentioning here, is that I do not have any check option (neither option httpchk nor eg. server nginx static_site:80 check inter 30s but only server nginx static_site:80). However the server works just instantly after a restart.

Without all the init-addr, I’m constraint to run steps 1 and 2 before step 3, otherwise HAProxy fails to start.

With those workarounds, I’m not constraint to run steps 1 or 2 before step 3 in order to get HAProxy started. However, in that case the previously non launched service (either step 1 or 2) will never be “connected” afterwards. Example:

  1. I run my rails server using docker-compose -d up in the project directory
  2. I run my HAProxy server using docker-compose -d up in the project directory
  3. → Everything works fine except the missing nginx website (obviously)
  4. Now I start my nginx server using docker-compose down in the project directory
  5. → The nginx website is still unavailable using the domain. (But still accessible in the host using the internal IP address provided by docker)
  6. Stopping and restarting that nginx server does not affect HAProxy

I simply want to be able to have the same behavior of my first scenario without being forced to start my services first.
BTW, I’m sorry for my bad English, but is this behavior really that difficult without complicated configuration?

About the resolving of the hostnames defined in the docker-compose.yml files, I don’t know how its done. I think it’s done internally in docker. Each container can see each others (if in the same docker networks) simply using the service names (eg. a ping static just works).

EDIT:
It seems that actually docker has a dns server internally. I tried using the docker interface docker0 address for the dns address 172.17.0.1:53.
But that does not work too.
HAProxy starts without having all my services started. But when I start the service, HAProxy did not update and the service is still unavailable.


#8

From within the docker container, please provide the ouput /etc/resolv.conf please:

cat /etc/resolv.conf


#9

Here is the output (ran in the host)
# Generated by resolvconf
nameserver 8.8.8.8
nameserver 8.8.4.4

Note: HAProxy as well as my website servers are dockerized…
I also edited my previous answer.


#10

Unless you know how to resolve that hostname, I don’t know if I can be a big help here. I can help you with haproxy, when you know how to resolve names. If you don’t, I suggest you look for support in the docker community.

From other examples on the internet I can see that the resolver is at 127.0.0.11, maybe you should try that:


#11

Thanks for this…
I’ll check it.
I already ask in the docker community for a simpler way to solve my problem. Simply by using static IP, but it seems not possible too…
I’m frustrated because of my needs is (or at least seems) very simple…

EDIT:
:+1:
Using the dns of docker with that address solve the problem and I finally got the behavior I wanted!
Thank you very much. I saw many different resources but I missed that address…

EDIT 2:
Is it possible in HAProxy configuration files to let all the backends use a specified resolvers?
For example I have all my backends (in the docker context) defined something like that:
server service_xyz host_xyz:123 check resolvers dockerdns resolve-prefer ipv4
I wonder about putting the suffix check resolvers dockerdns resolve-prefer ipv4 in a default block for example…
But this is not a big deal, just asking by curiosity…


#12

You should be able to move those keywords to the “default-server” directive where you configure init-addr (just append the rest there).


#13

Great !
That worked!

Thank you very much for all your help…