Never mark backend as failed?

Share the output of haproxy -vv and enable logging. Then share the logs (you should see backend server down event and usually you would see the server up event as well).

No idea why this happens though, never heard of such an issue.