Managing backend server deployment / restarts at runtime?

Help!
1

Folks;
apologies if this is a very basic question: We use docker containers to provide HTTP services which are fronted by haproxy. Right now, we know there are four “instances” living behind ports known to haproxy, and in course of deployment, we replace one by one of these containers.

This works, but it also always causes a bit of pain for various reasons. So, I am looking for a setup such as this:

  • I want to provide haproxy with a range of ports where backend servers could be listening, given we will never have all of these services actually running.
  • I’m looking for some (straightforward, ideally shell-scriptable) way to send one or multiple backend servers into a mode where they don’t accept new connections so I can bring them down as soon as the last active session on them has been terminated. Editing haproxy config file or restarting haproxy shouldn’t be involved here.
  • Our monitoring environment should have a way to cause an alert if less than backend processes are available to the haproxy.

How do you handle such setups? Any best practises or reading recommendations?
TIA and all the best,
Kristian

2

there is no reason for a configured backend ip:port combo to actually be live when you start your system, so you can have e.g. :80, :81, :82 ports configured in your backend setup, and rotate them as needed.

To dynamically configure ports, you want to use MAINT mode and the UNIX socket. Note that ansible has a module to do this for you, so you can probably skip using the socket directly and just try that. I’m sure there are puppet and salt equivalents.

3

wrt monitoring, you probably already have a preferred system so just google haproxy + that, and you’ll find something already done. I like http://riemann.io/ and https://github.com/riemann/riemann-tools/blob/master/bin/riemann-haproxy as a plugin.