Why does enabling abortonclose result in an uptick of hrsp_5xx?

Help!
1

We’re looking to enable option abortonclose so that our application behind HAProxy can see when a client aborts a connection. Turning the option on worked fine, but we noticed that it resulted in a definite jump in hrsp_5xx counters. From the application’s perspective, the responses still seem to result in 200s. Why is HAProxy logging 5xx responses? Is there something else we can adjust to stop this from happening? We monitor the occurrence of 5xx response as an indicator of backend health, so we can’t simply ignore it. I’ve confirmed that removing the options eliminates the 5xx counter increments.

1 Like
2

For the issue mchesler mentioned, here is a sample log of the 5xx responses:

Oct 10 19:09:53 nikita haproxy[13489]: 112.78.188.152:20049 [10/Oct/2017:19:09:53.881] node_http node/10_0_1_237 0/0/-1/-1/0 503 212 - - CC-- 6193/6097/550/55/0 0/0 "GET /p/rp/239842934823952.png HTTP/1.1"
Oct 10 19:09:53 nikita haproxy[13489]: 112.78.188.152:29793 [10/Oct/2017:19:09:53.884] node_http node/10_0_1_245 0/0/-1/-1/0 503 212 - - CC-- 6190/6094/548/47/0 0/0 "GET /p/rp/348293248248349.png HTTP/1.1"
Oct 10 19:09:53 nikita haproxy[13489]: 112.78.188.152:48497 [10/Oct/2017:19:09:53.892] node_http node/10_0_0_120 0/0/-1/-1/1 503 212 - - CC-- 6185/6089/548/46/0 0/0 "GET /p/rp/435435324324232.png HTTP/1.1"

HAProxy is logging it as a 503, but the connection string is CC, which I believe means that the client disconnected before the backend server was able to establish a connection. But the more curious part was that the total request time is 0ms, which would seem to mean that the backend wasn’t given any time to accept the connection anyway.

If my understanding of it is correct it seems like these errors are safe to ignore, except I can’t reproduce these errors synthetically, only in production. If these are not real errors, is there a way to prevent them from showing up in our stats, or if they are real errors is there a way to track down what the errors actually are?

1 Like
3

After running some traffic through tcpdump, it appears that a client will connect, make an HTTP request, then immediately (~10 microsends later) send a FIN ACK. At this point haproxy sends the client a 503 with No backends were available... message.

This seems to be at odds with abortonclose being intended to help with clients that abort too early. Since the client has sent FIN, why wouldn’t haproxy just send FIN ACK and be done with it? Is there any way to tell the difference between clients trying to disconnect and legitimate client requests where clients are just closing their channel once the request has been made, and are still waiting on a response?

1 Like
4

What exact release are you guys running?

There is none, also with abortonclose haproxy is not HTTP compliant. Take care when using this option:

Also see:

https://cbonte.github.io/haproxy-dconv/1.7/configuration.html#4.2-option%20abortonclose

As there is no way to distinguish between a full STOP and a simple output
close on the client side, HTTP agents should be conservative and consider
that the client might only have closed its output channel while waiting for
the response.

Maybe there is room for improvement though, like return a 4xx instead of a 5xx error.

1 Like