503 from haproxy after functioning correctly for a full day

Help!
1

Hey there. I’m using HAProxy in front of a bublic AWS ELB. Once HAProxy is configured, it behaves exactly as I would expect. But after some amount of time (27 hours seems to be the average) HAProxy starts to only respond with a 503 error. I can see that my services are still accessible via the public ELB, but for some reason HAProxy is only returning a 503.

If I include a check in my service definition, the error after ~27 hours will be Layer 4 Timeout in 2002ms and the service will be shown as down. If I omit the check, the same 503 response is shown but the service is still shown as up.

My config looks like:
global

defaults
    log global
    mode http
    option httplog
    # option http-server-close
    timeout connect  5000
    timeout client  10000
    timeout server  10000
    stats enable
     stats auth someuser:somepassword
     stats uri /haproxyStats

frontend http-in
    bind *:80
    default_backend servers
    # acl from_http hdr(X-Forwarded-Proto) -i "http"
    # redirect scheme https code 301 if from_http

backend servers
    server api "${FRONTEND_HOST}:${FRONTEND_PORT}" check
2

i’d guess the ELB changes its IP.
does it work again after restarting haproxy?

3

Hmm yes it does. Does that mean dns resolution is not dynamic?

4

well that was enough of a nudge to get me reading a bit more deeply. modifying my config to

backend servers
    server api "${FRONTEND_HOST}:${FRONTEND_PORT}" resolvers dns check

to see if that was (sounds like it definitely was) the issue.

presumptuous, but thank you for pointing me in the right direction!