I found this magic option called close-spread-time where is hidden info that header connection: close is added when HAProxy receives SIGUSR1 (soft-stop).
Define a time window during which idle connections and active connections
closing is spread in case of soft-stop. After a SIGUSR1 is received and the
grace period is over (if any), the idle connections will all be closed at
once if this option is not set, and active HTTP or HTTP2 connections will be
ended after the next request is received, either by appending a "Connection:
close" line to the HTTP response, or by sending a GOAWAY frame in case of
HTTP2. When this option is set, connection closing will be spread over this
set <time>.
If the close-spread-time is set to "infinite", active connection closing
during a soft-stop will be disabled. The "Connection: close" header will not
be added to HTTP responses (or GOAWAY for HTTP2) anymore and idle connections
will only be closed once their timeout is reached (based on the various
timeouts set in the configuration).