Haproxy sends the request to backend server having Disk I/O Error in which OS goes into read only mode and none of the command works becuase hardware storage controller failure.
Since application i.e. Nginx is running on backed server and application port is reachable and IP is accessible, HAproxy doesnt considers that as down and still sends the request on that faulty server.
Application is running so even the httpcheck for sample check.txt file is successful and thus request on that backend server starts failing as application i.e. Nginx running on it is not able to process the new request.
Is there a way to prevent sending the request to faulty server having Disk I/O Error.