My server solely serves SSL SNI domains and therefore no default fallback for non sni clients configured. Clients which do not support SNI will get a 503 because of that reason.
However, I’m having some problems with bingbot to work with SNI and after talking with their product team they replied the following:
Basically, how it works is that our bots first offer a TLS handshake to the server withoutSNI info. If the web server refuses this at the TLS layer, Bingbot will know to then offer a TLS handshake withSNI info. Returning a 503 will prevent Bingbot from detecting that it needs to send SNI info. Also, if the handshake is terminated at a deeper layer than the TLS layer, Bingbot will also not take proper action on this.
As I understand it in simpler terms, Bingbot comes to a secret club and asks to come in without saying the password. Because Bingbot didn’t say the password, the bouncer lets Bingbot in but shows him an empty room and then kicks him out the backdoor. Bingbot leaves confused and upset. If the bouncer had refused entry, Bingbot would have remembered to say the password and the bouncer would have lead him to the party.
The Product Team can’t offer anything else besides this explanation on properly terminating the HTTPS handshake so that our bots can make the correct handshake request.
Please configure the server so that HTTPS handshake is terminated if the client “hello” doesn’t contain the expected SNI server name. Basically, if there is an error on TLS layer, the TLS handshake should terminate. Returning info through the higher HTTP layer is too late.
If the web server refuses the TLS handshake when the TLS request doesn’t contain SNI info, Bingbot will be able to determine that SNI might be required and do a retry.
So my question is: how can I configure haproxy to refuse at the TLS layer?
My config is as follows:
frontend www-ssl
mode http
bind 0.0.0.0:443 ssl crt /usr/local/etc/haproxy/certs
use_backend %[ssl_fc_sni,lower,map_end(/usr/local/etc/haproxy2/ssl_domains.map)]
Thanks!