Reload closes partially connected proxies

We are using HAProxy to perform SNI proxying for TCP connections. The frontend matches the SNI hostname to a backend matching that host. The problem we are seeing is that if a connection to the front end hasn’t completed that match to a backend by the time the processes is told to drain then it closes that frontend connection. The net effect is that the client making the connection things the server dropped the connection mid SSL handshake.

Given the configuration below the client opens a socket to haproxy:1234 and starts the SSL handshake. After the SNI hostname is matched the connection to the backend is establish, for example to s1:5678. If the reload signal is received after a connection is established to the frontend but before it is established to the backend the connection on the frontend is immediately closed. Is there a way to allow connections established on the front end to complete their connections through to the backend during a reload? Is there a better way to configure HAProxy that would avoid this issue?

frontend sniproxy
  bind *:1234
  mode tcp
  tcp-request inspect-delay 5s
  tcp-request content accept if { req_ssl_hello_type 1 }
  use_backend s1 if { req.ssl_sni s1 }
  use_backend s2 if { req.ssl_sni s2 }
  use_backend s3 if { req.ssl_sni s3 }
backend s1
  mode tcp
  server host s1:5678
backend s2
  mode tcp
  server host s2:5678
backend s3
  mode tcp
  server host s3:5678