Site down Alternate Routing

Hi, I have been banging my head on this for a few days and I realize I need a little help.

I am trying to set-up an acl that will route traffic to a site that provides a notice when all servers on a backend are down.

I followed the advice on an article called Failover and Worst Case Management with HAProxy, but it only works when you hit the site’s root. If you hit a specific path it fails. So to ensure that once the condition is met I am setting the path on the frontend to be /.

What I am trying to achieve is similar to the functionality provided in Apache’s proxyPass and proxyPassReverse. I have also followed the advice on this article, but It isn’t quite working. Depending on what I tweak, it either passes the notice back to me w/o any of the css & js that provide the formatting or the browser gets into a request loop.

Here is how things are setup.

  • Notices server (aka backend): notices.example.com/notices/index.html

  • Public Server: public.example.com/somepath

If services in public.example.com/somepath are not available then route traffic to notices.example.com.

Outage/Maintenance ACL

acl outage_state nbsrv(public) le 1
acl url_stats path_beg -i /stats
http-request set-path / if outage_state !url_stats
use_backend notices if outage_state

Backend Configuration

backend notices
mode http

# Emulate ProxyPass Reverse from Apache
acl hdr_set_cookie_path res.hdr(Set-Cookie) -m sub Path=
rspirep ^(Set-Cookie:.*)\ Path=(.*) \1\ Path=/notices if hdr_set_cookie_path

Thanks!

So what are the URLs of the CSS and JS files?

You are resetting them to /, so that’s your main problem. The main site is fine (and actually needs) path /, but you must not rewrite CSS and JS request the same way (otherwise your backend always serves the main site and never serves CSS and JS responses).

How does the backend request URI need to look like to be able to load css and js?

Don’t bother with the cookie rewriting, you don’t need cookies on your notices page, right?

I also tried using regular expressions to the same effect. I replaced the cookie line in my example for this:

reqirep ^([^\ :]*)\ /(.*) \1\ /notices\2

This is the structure on the web server.

/notices contains HTML
/notices/bootstrap/css contains CSS
/notices/assets/js contains JS
/notices/imgs/ contains Images

Thanks!!!

What you say doesn’t add up.

You say this /notices “contains HTML”, but then you also say:

So what is it? What URI does your notice backend expect to deliver the HTML page along with the CSS and JS files? Forget haproxy, use your browser and access the notice backend directly and get an idea of how those requests need to look like.

There is not much point fiddling with regular expressions if we don’t know the basic URIs to call on your backend page.

Let me see if I can unpack my comment.

Without the regular expresion and with only http-request set-path / set, notice.example.com returns content from its / root. This makes sense because the path it is receiving from haproxy is simply /; however if I change http-request set-path /notices No content is returned. Which is odd because bypassing HAproxy and simply using the browser pointed at the backend with the same path, works. On the other hand http-requestt set-path /notices/ returns content, but no js, css or images.

Basically, if the site is down regardless of the URI provided HAproxy should route traffic to the notice.exmaple.com/notices URL. The backend delivers HTML, js & css from /notices, and the end user ideally would only see public.example.com/index.html or something like that.

I did a little more digging. The setting http-request set-path /notices/ is working, but I now realize that for some reason the Content-Type for css, js and imgs is being stripped therefore the page looks blah.

Content-Type is incorrect because it always returns the HTML content, instead of CSS, JS and images.

set-path replaces the path completely, maintaining only the query string. The browser will request CSS, JS and images with different request URIs, but by using set-path your normalize this away so that it requests again /notices/.

You already excluded the stats page via url_stats ACL, you need to do the same for the notices requests and make sure that notices files are reachable (even when perhaps outage_state is no longer true).

acl outage_state nbsrv(public) le 1
acl url_stats path_beg -i /stats
acl notices_paths path_beg -i /notices/bootstrap/ /notices/assets/ /notices/imgs/
http-request set-path / if outage_state !url_stats !notices_paths
use_backend notices if outage_state or notices_paths

Hi! Thank you for helping me out and explaining set-path nuance. I had not fully understood that. I did try to mitigate what I was seeing by creating acl’s on the backend to set the Content-Type, but for some reason or another, that wasn’t working either.

Anyhow, I have a working solution that does exactly what I need. In short, if the site is down and regardless of the URL the user may enter on the address bar, HAProxy responds with a message from the notices backend and obfuscates the path to that message i.e. /notices/. However, going directly to /notices/ does not get re-routed to /. I am sure acl fine-tuning will solve that.

I hope this can help others, I found lots of similar questions about this topic on the web and many were unanswered, so I have a feeling this is a confusing topic for many.

Again, Thank You.

frontend test 
    bind :80
    mode http
    # Load balancer Stats
     acl url_hap_stats path_beg -i /stats
       use_backend hap-stats  if  url_hap_stats
    # Outage/Maintenance ACL
     acl outage_state      nbsrv(test-be) le 1
     acl url_stats         path_beg -i /stats
     acl url_file_ext      path_end -i .css .png .jpg .js
        http-request set-path / if outage_state !url_stats !url_file_ext
        use_backend notices if outage_state or url_file_ext
    # Default backend
     default_backend test-be
backend test-be
    mode http
    option redispatch
    balance roundrobin
    cookie SERVERID insert indirect nocache
    
    server server01 127.0.0.1:8001 check cookie srv1 
#    server server02 127.0.0.1:8002 check cookie srv2  #Commented out to force state vs using UI
#    server server03 127.0.0.1:8003 check cookie srv3  #Commented out to force state vs using UI
backend notices
    mode http
    
	# Obfuscate returning path 
     acl url_root     path_beg -i /
        reqirep ^([^\ :]*)\ /(.*)     \1\ /notices/\2  if url_root 
    
	server notices 127.0.0.1:8080 check 
#---------------------------------------------------------------------
# HA Proxy Stats
#---------------------------------------------------------------------
backend hap-stats
    mode http
    stats enable
    stats hide-version
    stats show-node
    stats uri /stats
    stats auth <USERID:PASSWORD>
    stats admin if TRUE

Careful, this configuration routes all requests ending with .css .png .jpg .js to your notices backend. You don’t have any .css .png .jpg .js files on your main page? Because this will break that.

Also, I don’t know why you still use reqirep.

I could help your more specifically, if you’d be able to finally tell how those js and css request to your notices backend woluld have to look like.

I’m not sure you understood the issue. You can’t fix something an your backend that you already completely broke on the frontend. If you erase the URI information on the frontend, it can’t be magically restored on the backend. And this has nothing to do with Content-type.

Again, if you would explain how the requests (all of them) to your notices backend have to look like exactly, I think we would have an easy solution soon.

I tried to manipulate the Content-Type before I understood that set-path would override everything.

I am still using reqirep because w/o it the routing does not work.

I used path_end with the file extension instead of path_beg with a path because the latter did not work.

let me see if I can explain the what the responses from the back end need to look like.

If the cluster is down, then route all requests to the notices backend regardless of the URL provided ex:
public.example.com/ -> routes request to notices.exmaple.com/notices
public.example.com/xyz -> routes requests to notices.example.com/notices

The user sees on their browser the notice and the URL on the address bar shows public.exmaple.com/

The notice.example.com backend serves:

  • HTML from notice.example.com/notices
  • CSS from notice.example.com/notice/bootstrap/
  • JS from notice.exmaple.com /notices/assets/
  • Images from notice.example.com/notices/imgs/

In my mind & on the browser the requests, bypassing haproxy, look like this.

12:39:35.583 [Show/hide message details.] GET http://127.0.0.1:8080/notices/bootstrap/css/bootstrap.css
12:39:35.585 [Show/hide message details.] GET http://127.0.0.1:8080/notices/style.css
12:39:35.591 [Show/hide message details.] GET http://127.0.0.1:8080/notices/imgs/TheS_Transparent.png
12:39:35.594 [Show/hide message details.] GET http://127.0.0.1:8080/notices/assets/js/jquery.min.js
12:39:35.599 [Show/hide message details.] GET http://127.0.0.1:8080/notices/assets/js/popper.js
12:39:35.602 [Show/hide message details.] GET http://127.0.0.1:8080/notices/bootstrap/js/bootstrap.min.js
12:39:35.605 [Show/hide message details.] GET http://127.0.0.1:8080/notices/assets/js/ie10-viewport-bug-workaround.js

Using the Configuration I posted the requests on the browser looks like

12:49:56.983 [Show/hide message details.] GET http://public.example.com/
12:49:57.060 [Show/hide message details.] GET http://public.example.com/bootstrap/css/bootstrap.css
12:49:57.064 [Show/hide message details.] GET http://public.example.com/style.css
12:49:57.074 [Show/hide message details.] GET http://public.example.com/assets/js/jquery.min.js
12:49:57.075 [Show/hide message details.] GET http://public.example.com/assets/js/popper.js
12:49:57.076 [Show/hide message details.] GET http://public.example.com/bootstrap/js/bootstrap.min.js
12:49:57.084 [Show/hide message details.] GET http://public.example.com/assets/js/ie10-viewport-bug-workaround.js
12:49:57.087 [Show/hide message details.] GET http://public.example.com/imgs/TheS_Transparent.png

Ok, so the links to the css, js, etc files are relative to the request path of the notices site, which is why this is harder than it has to be.

I suggest you rewrite the links in there so that they always work within /notices/, like instead of referencing style.css as-is, you refer to it by /notices/style.css, etc. So that path_beg /notices/ works. Then you can route this easily to the correct backend, without braking your main site.

Just set the path to /notices/ instead of / then. No need to continually shoot yourself in the foot.

acl outage_state nbsrv(public) le 1
acl url_stats path_beg -i /stats
acl notices_paths path_beg -i /notices/
http-request set-path /notices/ if outage_state !url_stats !notices_paths
use_backend notices if outage_state or notices_paths