Botnet password attacks against smtp authentication

Hi
I am using HAProxy in tcp-mode in front of several smtp mail servers that require authentication.
The smtp traffic is TLS port encrypted between the client and the mail servers, so HAProxy will never see smtp L7.
Unfortunately, malicious bots are trying password lists to attack the authentication of the smtp service.

With my configuration, I can’t use fail2ban on the smtp mail servers, because the smtp servers only see one source IP - the one from HAProxy.
On the other hand, I can’t use stick tables on HAProxy, because HAProxy is not able to see unsuccessful authentication, because of TLS port encryption.

Is there a known solution for this dilemma?

If there isn’t, I would like to configure HAProxy to pass on the original source IP to the smtp mail servers, so that I may be able to use fail2ban. How do I do that?

I am using HAProxy 2.4.22
The smtp mail servers don’t support the proxy protocol.

Thanks

Addendum:

  • The mailserver does not support the Proxy Protocol.
  • The mailserver cannot be replaced by another mailserver, that does support the Proxy Protocol, because the server is offering multiple services (e.g. smtp, imap, webmail, etc) and it is the authentication provider for authenticated smtp.
  • HAProxy runs as part of a keepalived cluster and is listening on the Virtual IP.