Sometimes HAProxy server(load balancer) gives Server 500 error after one of the two webservers(under balancer) go down .
Permanently or temporarily? Did you configure health checks? Does it detect your backend went down?
A lot of this depends on your configuration, which I suggest you share.