Hi good people, could someone answer me the question. I plan the infrastructure and I need advice.
I have two internet providers. One internet goes throught router A with static WAN IP, and other internet goes to the secont internet provider router B with also static WAN IP.
Both routers will be connected to the single WEB server (ports 80, 433). Domain lets say domain.com will be setup on that server.
What I need to achieve is loadbalancing and failover of domain.com on the WEB server on ports 80, 443 using both internets. So I think to setup HAproxy on highly available server which will point to these to routers. Both internet connections will be used at the same time and if one connection goes down, HAproxy will leave only one stable connection. HAproxy will do checks every single second and if both connections are good and in case of error leave only one.
Other solution I think is to have only one primary internet on router A, and if it fails then HAproxy switch suddenly to the other internet on router B. It this case internet on router B will be like in stand by mode all the time exept the error of primary internet. The problem in this solution, that I can’t use the power of both internets all the time as it was written in solution 1. above.
Is this possible to achieve and run both solutions 1 and 2 very stable? Does somenone have practice of such kind of setups and run it smoothly for years? Is realy HAproxy is suit my needs or should I look for the alternative software?
If you run haproxy in a remote-datacenter, sure (you will have to think about how the webserver can use both internet connections, but it is possible).
If you think about running haproxy behind the same two internet connections, then no, because the problem (failover and loadbalancing between the 2 internet connectivities) comes prior to haproxy, so haproxy is not in control of this.
Yes, I plan to run this in remote-datacenter. Could you drop some links to the HAproxy documentation the exact part what I’m talking about?
Webserver will have two nics. One nic will have primary internet IP address and second ethernet adapter should route to the primary NIC, if the trouble would be with primary IP addrees. But I also need to find out how to do it.
Haproxy will failover from one to the other. On the server side you need to make sure that the traffic to NIC1 always leaves at NIC1 (to provider 1) and the traffic that is on NIC2 always goes to provider 2. Otherwise you have a “failover” both on haproxy and on your servers and this will cause issues when haproxy and your servers disagree.
You can use the ip command to configure source based IP routing, it should probably be something like:
ip route add default via <NIC1-gateway> src <NIC1-IP>
ip route add default via <NIC2-gateway> src <NIC2-IP>
Thanks, very informative info.
But let’s say I will do such redirection from the HAproxy to the webserver where website is hosted. Does every function of website will work if the compare this method vs the primary hosting without redirecion of proxy. I mean maybe some kind of functions which has variuos websites and their CMS could not work because of proxy?
There are 2 common problems with reverse-proxying that often arise:
lack of source IP transparency: the solution is to use the proxy protocol or a custom HTTP header like X-Forwarded-For to send this information to the backend server. Consider that your backend server needs to support this and be configured properly
when terminating HTTPS: lack of the information that the customer connected with HTTPS, causing infinite redirects, use X-Forwarded-Proto and configure your backend properly
When you understand those, and configure everything appropriately, your applications should work just fine.
Oh my… I clarify situation. I need to host HAproxy on datacenter and do redirection through 2 ISP providers into a single webserver AND from that server I want also to redirect through HAproxy into local web server. And my task is two host about 3000 domains in that way.
X-Forwarder-For or -Proto must be changed in the code of the website. And if we talk about modern CMS and modules, it could be that a single website CMS will have the dozen places to write this sentence into the PHP. Am I right?
My chain looks like USER BROWSER—>DC HAproxy—> My HAproxy—>local IP web server. Does HTTP method through “double” proxy protocol or X-Forwarder work in this way?
And if talk only about HTTPS, X-Forwarder will do the job good also?
HTTPS will be the biggest problem. Because as I understood if I can’t touch the code of the website and write X-Forwarder-Proto, I can’t get it work. And if one of my clients will request SSL feature, I can’t satisfy the need in my infrastructure.
Redirection is the wrong word. You would like to reverse-proxy/load-balance from Haproxy to that webserver by using the 2 ISPs.
I don’t know what that’s supposed to mean.
No. You do need to configure the webserver appropriately, but you don’t have to change application code. If you don’t change the webserver configuration, then yes, you’d have to touch the application code.
Yes, if appropriately configured. Not sure if you 2 ISP links are between the 2 haproxy instances or between the latter haproxy and the web server.
You have to figure out what the end result should look like. Like, where do you want to terminate SSL actually? In the datacenter or on your end? SSL encryption between the 2 haproxy instances? For traffic that originally was HTTPS or HTTP also?
Hi, thanks I understood that Forwarder part can be change on the server side. It’s very good.
Configuration looks like this. I will have a server in datacenter, where I install HAproxy and put several thousands of domains. Every domain will have 2 IP addresses for loadbalance and failower features. These 2 IP addresses are frome my 2 ISP’s and it point to the my MAIN server with two NICs. And this main server will have also HAproxy installed which point to internal local webservers where excat domain is located.
It sounds good?
HTTPS is important only then when domain owner will ask me to install SSL sertificate. I don’t really know what to do in that situation. I will have only two public IP’s, and my ISP won’t give me more. So I thought to install multidomain Comodo sertificate. It gives option to use ~250 SSL sertificates on the single IP address.
But when I need to do so much redirections/proxies I don’t really know if it’s work if I install it on one of internal local webserver.
Maybe it would be better to have HAproxy only on datacenter server and do redirections thourgh 2 IPs into the MAIN server into my place. But on MAIN server do not install HAproxy or install Apache+MySQL and etc. web server stuff and install multi domain sertificate there. On the MAIN server I would host only SSL website, and for every HTTP website I would do reverse proxie from Apache to the local servers. As I understood Apache is also have reverse proxie feature https://httpd.apache.org/docs/2.4/howto/reverse_proxy.html
What is your opinion? Maybe my setups sounds very unusual, and I will have more trouble in the future.
I’m not sure what the job of the local reverse proxy would be (whether it’s haproxy or apache)? Why not access the webserver directly?
What is the webserver? If it supports the proxy protocol, haproxy in the datacenter could only to simple TCP forwarding for both 80 and 443 traffic, and send the source IP via the proxy protocol to the webserver.
The problem is that every ISP is giving the only one external IP address. And my aim is to have 10-20 servers. I can’t setup external IP on each server, and I’m forced to share one external IP for all servers I have. So the only option I see is to have my external IP on the main server, which will redirect traffic to local servers.
So in datacenter HAproxy VM is installed. It redirects traffic to my main webserver APACHE, which will redirect to local webserver where Apache is also installed with some bunch of domain and websites.
On the main server APACHE I plan to host all SSL websites.
Now I think is it good idea to use APACHE on the main webserver or just install HAproxy instead.
Is it good to have two HAproxy in that chain?
In that case, SSL websites with their certificates will be on one of local webservers.
It’s quite tricky situation, and I don’t even know which approach will give better performance…
Ok, let’s say I will pick up double HAproxy approach:
DC HAproxy (with DC external IP)-> Main HAproxy (my ISP external IP) -> Local apache web server (local IP) where is website located.
So this means I need to install SSL certificate on the local web server as I understood.
But is this connection chain will work in HTTPS and I don’t loose the SSL connection, certificate data?
We don’t need to forget, DNS will point that domain.com is located on DC IP address.
The question is, when you have 20 - 30 webserver, I assume you’d have your local haproxy instance route to specific webserver based on domain name, is that correct?
So you’d need to access the Host header of the HTTP request, to make that routing decision. And in that case, you’d probably want to install the certificate not on the webserver but on your local haproxy instance.
For the haproxy instance in the datacenter on the other hand can just forward both port 80 and port 443 TCP traffic to both ISP’s public ip address.
Yes yes yes, you are absolutely right about the topology. This is exactly what I want to do!
Please clearify, can I install SSL sertificates directly into HAproxy??? Let’s say I need 200 multi domain sertificates? I thougth SSL sertificate can only be installed into web server…
And do you recommend to use proxy protocol between DC HAproxy and local (main server) HAproxy for forwarding 80 and 443 TCP traffic?
Yes, you can install SSL certificates on haproxy, even multiple certificates on the same IP. They will be selected based on the SNI information from the browser.
I’d recommend the following setup:
DC haproxy sends traffic to both your internet connections with load-balancing/fail-over between the two. It should do so transparently (just send port 80 to port 80 and port 443 to port 443, without intercepting SSL in the datacenter). It should also use the proxy protocol to maintain the original source-IP information.
Your local haproxy instance (or maybe just use 2 - one on one internet connection and the other one on the other internet connection so you don’t have to figure out of to handle two internet connections on one server - even if it’s doable, see the default route configuration proposed above).
Depending on your web browser support, either use the proxy protocol or HTTP headers to make your application aware of the original protocol (HTTP or HTTPS) and source IP.
Thanks for advices very much. Your proposal makes sense. The trickest part will be to make networking on local HAproxy server(s) OR local web servers thant single network adapter could understand both gateways. I never did that.
And what about the traffic? If I forward visits from DC HAproxy, does this traffic is counted by DC? For example a single visit to the website uses 5MB (user download 5MB to view a website). So in this situation, does all 5MB is counted by DC, or only just a small fraction is counted for redirection?
I ask that because, if I buy virtual machine in datacenter the plan usualy has limitation for traffic. Let’s say 10TB per month.
I understand that both my ISP will count traffic, but what about a DC?
Like I said either make two haproxy instances, each one pointing to either one or the other ISPs, or use a source-based default gateway configuration as suggested above.
If the users request a 5MB webpage, you datacenter will charge you for 5MB download (from the origin server) and 5MB upload (to the client). This is not a redirect, it’s forwarding traffic both ways.