We had an issue with a group of webservers behind our HAProxy that lost a connection to their DB this morning and those servers could no longer function and wanted to serve to clients a “500 Server Error” that included useful details on why the servers no longer functioned.
HAProxy served instead to clients a “503 Service Unavailable” (as ALL webservers in the group were providing 500 errors so essentially the whole backend was down).
Does HAProxy have an option for actually passing the server provided 500 error to the clients so that HAProxy doesn’t hide from us what’s happening ?
I understand that I could simply change the http-chk to allow 500 as an acceptable HTTP code, but then the HAProxy interface and SNMP systems wouldn’t get alerted to the fact that anything was wrong as the servers would still show as UP and healthy (even though they weren’t).
I’ve looked extensively in the Config manual and some options like “retry-on 500” look like they might function but they don’t. We still see the same “503 Service Unavailable” when used.
Does an option exist that forwards the backend server error to the client ?
You will have to make a new “failure” backend with the same servers but without healthchecks, and divert the traffic to the failure backend when the amount of available servers in the normal backend equals 0.
I don’t think there is another way, because that would defeat the purpose of health checking.
frontend www
acl all_servers_down nbsrv(regularbackend) eq 0
use_backend failurebackend if all_servers_down
default_backend regularbackend
backend regularbackend
server srv1 192.168.5.1:80 check
server srv2 192.168.5.2:80 check
server srv3 192.168.5.3:80 check
server srv4 192.168.5.4:80 check
server srv5 192.168.5.5:80 check
backend failurebackend
server srv1 192.168.5.1:80
server srv2 192.168.5.2:80
server srv3 192.168.5.3:80
server srv4 192.168.5.4:80
server srv5 192.168.5.5:80
Thanks for your reponse Lukas, that’s actually a pretty good workaround. I’m personally of the opinion that sending the actual error to the client isn’t such a good idea as it reveals internal infrastructure information in the error but other colleagues just want a fast way to fix the error hence the want to actually send the original error.
In that case you may also use different conditions to route to the failure backend.
For example, if you now the source IPs from your office/developer/debug stations, you may reroute to the failure backend regardless of the servers being consider up in the main backend.
The big disadvantage is that you stop seeing what the customer is seeing, which could be it’s own problem.