SMTP Load Balancing With Exchange 2013

Help!
1

Hello,

I’ve set up a test environment with Exchange 2013 and Haproxy loadbalancing services at layer 7. I believe I have the exchange urls setup correctly and my Outlook and ActiveSync clients connect ok. My problem is with loadbalancing SMTP connections to the server…

I am unable to connect over port 25 with putty to the exchange servers. The port is open in firewalld, iptables is off, checking what’s listening the haproxy service is listening on port 25. However, when I connect with putty the connection just times out. Likewise, if I try to send an email over smtp I receive the following error…

said: 250 2.1.5 Recipient OK - then a bounce back is sent.

I don’t believe that the issue is with exchange as I can connect with putty directly to the CAS servers and send emails that way. Below is my Haproxy configuration, any insight or corrections would be greatly appreciated!

global
log 127.0.0.1 local0 info
maxconn 10000
daemon quiet
tune.ssl.default-dh-param 2048

defaults
log global
mode http
option httplog
option dontlognull
timeout connect 60000ms
timeout client 30000ms
timeout server 60000ms
timeout check 60000ms
stats enable
stats hide-version
stats show-node
stats auth admin:PASSWORD
stats uri /stats

frontend unsecured 192.168.1.1:80
redirect location https://mail.domain.com/owa

frontend fe_ex2013_smtp
mode tcp
bind *:25 name smtp
log global
option tcplog
option dontlognull
option contstats
timeout client 300s
default_backend bk_exchange_2013_smtp

frontend fe_ex2013
mode http
bind *:443 ssl crt /etc/ssl/certs/exchange_certificate
acl autodiscover url_beg /Autodiscover
acl mapi url_beg /mapi
acl rpc url_beg /rpc
acl owa url_beg /owa
acl eas url_beg /microsoft-server-activesync
acl ecp url_beg /ecp
acl ews url_beg /ews
acl oab url_beg /oab
use_backend be_ex2013_autodiscover if autodiscover
use_backend be_ex2013_mapi if mapi
use_backend be_ex2013_rpc if rpc
use_backend be_ex2013_owa if owa
use_backend be_ex2013_eas if eas
use_backend be_ex2013_ecp if ecp
use_backend be_ex2013_ews if ews
use_backend be_ex2013_oab if oab
default_backend be_ex2013

backend be_ex2013_autodiscover
mode http
balance leastconn
option httpchk GET /autodiscover/healthcheck.htm
option log-health-checks
http-check expect status 200
server Server1 10.1.1.1:443 check ssl inter 15s verify required ca-file /etc/ssl/certs/ca-bundle.crt
server Server2 10.1.1.2:443 check ssl inter 15s verify required ca-file /etc/ssl/certs/ca-bundle.crt

backend be_ex2013_mapi
mode http
balance leastconn
option httpchk GET /mapi/healthcheck.htm
option log-health-checks
http-check expect status 200
server Server1 10.1.1.1:443 check ssl inter 15s verify required ca-file /etc/ssl/certs/ca-bundle.crt
server Server2 10.1.1.2:443 check ssl inter 15s verify required ca-file /etc/ssl/certs/ca-bundle.crt

backend be_ex2013_rpc
mode http
balance leastconn
option httpchk GET /rpc/healthcheck.htm
option log-health-checks
http-check expect status 200
server Server1 10.1.1.1:443 check ssl inter 15s verify required ca-file /etc/ssl/certs/ca-bundle.crt
server Server2 10.1.1.2:443 check ssl inter 15s verify required ca-file /etc/ssl/certs/ca-bundle.crt

backend be_ex2013_owa
mode http
balance leastconn
option httpchk GET /owa/healthcheck.htm
option log-health-checks
http-check expect status 200
server Server1 10.1.1.1:443 check ssl inter 15s verify required ca-file /etc/ssl/certs/ca-bundle.crt
server Server2 10.1.1.2:443 check ssl inter 15s verify required ca-file /etc/ssl/certs/ca-bundle.crt

backend be_ex2013_eas
mode http
balance leastconn
option httpchk GET /microsoft-server-activesync/healthcheck.htm
option log-health-checks
http-check expect status 200
server Server1 10.1.1.1:443 check ssl inter 15s verify required ca-file /etc/ssl/certs/ca-bundle.crt
server Server2 10.1.1.2:443 check ssl inter 15s verify required ca-file /etc/ssl/certs/ca-bundle.crt

backend be_ex2013_ecp
mode http
balance leastconn
option httpchk GET /ecp/healthcheck.htm
option log-health-checks
http-check expect status 200
server Server1 10.1.1.1:443 check ssl inter 15s verify required ca-file /etc/ssl/certs/ca-bundle.crt
server Server2 10.1.1.2:443 check ssl inter 15s verify required ca-file /etc/ssl/certs/ca-bundle.crt

backend be_ex2013_ews
mode http
balance leastconn
option httpchk GET /ews/healthcheck.htm
option log-health-checks
http-check expect status 200
server Server1 10.1.1.1:443 check ssl inter 15s verify required ca-file /etc/ssl/certs/ca-bundle.crt
server Server2 10.1.1.2:443 check ssl inter 15s verify required ca-file /etc/ssl/certs/ca-bundle.crt

backend be_ex2013_oab
mode http
balance leastconn
option httpchk GET /oab/healthcheck.htm
option log-health-checks
http-check expect status 200
server Server1 10.1.1.1:443 check ssl inter 15s verify required ca-file /etc/ssl/certs/ca-bundle.crt
server Server2 10.1.1.2:443 check ssl inter 15s verify required ca-file /etc/ssl/certs/ca-bundle.crt

backend be_ex2013
mode http
balance leastconn
server Server1 10.1.1.1:443 check ssl inter 15s verify required ca-file /etc/ssl/certs/ca-bundle.crt
server Server2 10.1.1.2:443 check ssl inter 15s verify required ca-file /etc/ssl/certs/ca-bundle.crt

backend bk_exchange_2013_smtp
mode tcp
balance leastconn
option tcplog
log global
option redispatch
retries 3
source 0.0.0.0 usesrc clientip
option smtpchk HELO mail.domain.com
server Server1 10.1.1.1:25 send-proxy check
server Server2 10.1.1.2:25 send-proxy check

2

In your backend you appear to be sending proxy protocol on the real server lines:

server Server1 10.1.1.1:25 send-proxy check
server Server2 10.1.1.2:25 send-proxy check

I’m not aware of Exchange 2013 supporting proxy protocol at all so it could well be interfering with the connection.

You also have “source 0.0.0.0 usesrc clientip” which would normally need a 2 arm tproxy setup like this: http://www.loadbalancer.org/eu/blog/configure-haproxy-with-tproxy-kernel-for-full-transparent-proxy

You can almost certainly ignore the bit about compiling it into the Kernel as it’s been in there for ages now in most kernel’s I’ve used. However do you really need these options? Are you trying to make it source IP transparent? If not then remove those options and it will work but non transparently so you may need to lock down access to port 25 on the server running HAproxy rather than at the Exchange server.

3

Thank you for your response! I’ll take a look at the article.

4

This was really helpful. I ran into something similar. Slightly off topic, but i noticed you used leastconn instead of round robin for load balancing. According to your experience and general popularity in SMTP world, what do you recommend using. I am aware of the high level difference between those two, but would like to know any real world feedback.

5

As an update, my final SMTP config looks like the following.

listen smtp *:25

mode tcp

option tcplog

balance leastconn

server Cas1 10.10.10.31:25 check

server Cas2 10.10.10.28:25 check

I have 2 backend exchange servers, each listening on 25 for inbound mail. I removed the “send-proxy” entries and all works; ended up locking down inbound connections on the frewall and in the send connector scope properties of exchange.

As for the performance of leastconn vs roundrobin… I’ve had no issues with leastconn. I’m using it across 3 different load balanced exchange environments. IMO, if one of the exchange servers is queued up or taking it’s time completing smtp transactions, it allows the other servers to pick up the slack, instead of continuing to add new sessions to the server that’s taking longer. This also keeps the number of concurrent transactions on each server lower.

6

Thanks for the Reply Emalacar. That was helpful.

7

What about option smtpchk? It causing some errors in smtpserver log file like: ERROR smtpserver: Socket to Hostname (IP) closed remotely.