HAProxy with HTTP/2 / gRPC fails with persistent connection at client and server is restarted

Help!
1

Issue: When a client is keeping long lived connection for gRPC with HAProxy and backend server restarts (goes down then comes back up again), all the subsequent requests fails with 503 RESET_STREAM even after server comes back up.

Client:

h2load http://<HAProxy-IP>:<Port>/fast -c 1 -n 10 --duration 10000

HAproxy Version: 2.0.14

HAProxy Configuration

global
    daemon
    log 127.0.0.1	local0 info
    maxconn 50000
    ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!MD5:!DSS
    ssl-default-bind-options ssl-min-ver TLSv1.1

defaults
    log global
    maxconn 3000
    mode http
    timeout connect 10s
    timeout client 30s
    timeout server 100s
    option httplog
    option http-use-htx
    #option logasap

frontend fe_proxy
    bind 10.33.251.152:15004 proto h2
    default_backend be_servers

backend be_servers
    balance roundrobin
    server 10.20.30.40 10.20.30.40:8081 weight 255 proto h2

HAProxy Logs for failure cases:

   [14/Dec/2020:20:23:21.422] fe_proxy be_servers/10.20.30.40 0/0/-1/-1/40007 503 0 - - sC-- 1/1/0/0/3 0/0 "GET /fast HTTP/2.0"
   [14/Dec/2020:20:24:01.479] fe_proxy be_servers/10.20.30.40 0/0/-1/-1/40006 503 0 - - sC-- 1/1/0/0/3 0/0 "GET /fast HTTP/2.0"
   [14/Dec/2020:20:24:41.543] fe_proxy be_servers/10.20.30.40 0/0/-1/-1/40007 503 0 - - sC-- 1/1/0/0/3 0/0 "GET /fast HTTP/2.0"

When timeout server 100s is set as 30s, it seems to work fine. Is it mandatory to keep timeout server 30s for http/2 case?

Also, after server restarted successfully, it was noticed that HAProxy sends SYN to server, server sends SYN/ACK back and then HAProxy sends RST packet and throws the above log.

Please help with this issue.