Instllaltion failover

Help!
1

Hi Team,

I want HAproxy with failover(distributed) nodes. could you please give me steps.

2

Could you provide more details in your requirements and the infrastructure you have / want to achieve?

Do you want an active/active or active/passive failover scenario? Are you using HAProxy for HTTP or TCP only?

I’m asking this because actual HAProxy “failover” setups require more than HAProxy configuration, but instead you need to also configure your routing / server environment. It also depends a lot on other factors like cloud provider (if you use one, and if not, do you have some access to the current low-level infrastructure?), operating system (Linux / BSD / etc.), and other such details.

Also it would be important to know how “stringent” the failover requirement is: would a 60 second delay be “good-enough”? Is it OK if the current TCP connections (that are live at the moment of the failover) are dropped?

There is (to my knowledge) no “one-size-fits-all” solution.

For example I would suggest trying to see if a DNS-based failover solution would be enough?

Assuming your clients use DNS to find the IP of your HAProxy — which is certainly 100% the case for web applications, and should be in others — then you could:

  • create an A/AAAA record with low TTL (say 60 seconds), holding multiple values, the IP addresses of your HAProxy instances;
  • monitor your HAProxy instances, and if one of them goes down, then just remove that record from the DNS;

For example this can be easily achieved with AWS Route53 (and perhaps with other more advanced DNS services).

3

hi ciprian,
Thanks for your update.
I have 2 emqx servers and i configured . i configure haproxy with roundrobin its working fine. Now i want to if one node is down immediately the data is transferred to another node. I am sharing nodehaproxy.cfg file as below.

    errorfile 504 /etc/haproxy/errors/504.http

listen stats
bind *:8000
stats enable
stats hide-version
stats uri /stats
stats uri /stats
stats auth admin:admin@123
#frontend firstbalance
bind *:80
option forwardfor
default_backend webservers
backend webservers
balance roundrobin
server webserver1 IP:18083 check
server webserver2 IP:18083 check

4

OK, so it seems that you are using HAProxy in HTTP mode, however could you say whether the clients use Keep-Alive HTTP connections, or each request/reply transaction happens on a new connection. (I ask this, because from https://www.emqx.io/ I see that EMQX seems to be a MQTT broker, which based on your configuration seems to be using HTTP as transport. Therefore this seems an IoT use-case, where I assume you have many clients, that send-back data from time-to-time.)

In any case I would experiment to make sure that clients can handle “broken” Keep-Alive connections. (I.e. if the servers just closes the connection after handling the first request/reply, can the client retry to connect without dropping the second request?)

Now assuming that each client will make each HTTP request over a new connection (or at least if they support broken Keep-Alive HTTP connections), you could just use the DNS-based failover solution mentioned earlier, provided that your clients are able to retry failed requests.

If instead you can’t use the DNS-based failover, you’ll have to resort to a more complex failover mechanism, one that most likely involves configuring either your router or cloud infrastructure, or installing something like keepalived Linux Virtual Server.

Also, again I ask if you are using this in a cloud environment? (Because for example if you are using AWS or another large provider, they already have “load-balancers” that you can put in front of your HAProxy instances to handle the actual failover for you.)

However please take a moment to think thoroughly (and realistically) about the failure scenarios that you want to cover:

  • do you want to cover physical / hardware failure? (if so, you’d better have redundancy at all layers, starting from multiple internet providers with BGP, to high-availability routers and switches; because it’s useless to have two physical servers configured for failover, just to be powered from the same UPS, and connected to the same switch;)
  • do you want to cover just software failures? (i.e. in case “something” happens to HAProxy;)
  • somewhere in between?
5

Hi Ciprian,

**Could you give me steps how to do **DNS-based failover in HAproxy with steps with example.

6

Unfortunately, as said in earlier replies, there is no “one-size-fit-all” solution, including for the DNS failover scenario.

I’ve highlighted the generic solution in my first reply, however the actual steps depend a lot on many factors related to the DNS and monitoring solutions you are using.

For example if using AWS Route53 you can configure alarms, and based on those alarms condition the response of certain records. However if you are using an “self-hosted” DNS solution (like BIND, etc.) you need to update records dynamically.

As said in all my previous replies, because you haven’t provided almost any concrete details regarding the infrastructure and setup, I (and certainly no one) can provide anything more than generic hints about possible solutions.

Ciprian.