Exchange 2016 + HAproxy + MacOS Outlook issues

[KenynMacCormik]

Greetings,

I’m using Exchange 2016 DAG with two servers. Recently I’ve installed pfSense with HAproxy module to ensure web reverse proxy.

My issue is following: MacOS outlook clients are constantly requesting password to connect and RPC over HTTP is not working in my configuration. If I will just forward 443 port to the exchange DAG RPC over HTTP is working fine. If I replace HAproxy with IIS + ARR MacOS clients stops requesting passwords. Does anyone ever faced this issue or something similar?

P.S. Windows Outlook clients working fine in any configuration

The haproxy.cfg file is like following

# Automaticaly generated, dont edit manually.
# Generated on: 2019-01-31 16:22
global
        maxconn                 10000
        log                     syslog1.<my domain>      local0  debug
        stats socket /tmp/haproxy.socket level admin
        uid                     80
        gid                     80
        nbproc                  1
        hard-stop-after         15m
        chroot                          /tmp/haproxy_chroot
        daemon
        tune.ssl.default-dh-param       4096
        log-send-hostname               pfsense
        server-state-file /tmp/haproxy_server_state

listen HAProxyLocalStats
        bind 127.0.0.1:2200 name localstats
        mode http
        stats enable
        stats refresh 10
        stats admin if TRUE
        stats show-legends
        stats uri /haproxy/haproxy_stats.php?haproxystats=1
        timeout client 5000
        timeout connect 5000
        timeout server 5000

frontend http-https-frontend
        bind                    <public ip>:443 name <public ip>:443   ssl crt-list /var/etc/haproxy/http-https-frontend.crt_list
        bind                    <public ip>:80 name <public ip>:80
        mode                    http
        log                     global
        option                  socket-stats
        option                  log-separate-errors
        option                  httplog
        option                  http-keep-alive
        option                  forwardfor
        acl https ssl_fc
        http-request set-header         X-Forwarded-Proto http if !https
        http-request set-header         X-Forwarded-Proto https if https
        maxconn                 10000
        timeout client          30000
        capture request header Host len 32
        capture request header User-Agent len 64
        capture response header Content-Length len 10
        #option httplog
        log-format %ci:%cp\ [%t]\ %ft\ %b/%s\ %Tq/%Tw/%Tc/%Tr/%Tt\ %ST\ %B\ %CC\ %CS\ %tsc\ %ac/%fc/%bc/%sc/%rc\ %sq/%bq\ %hr\ %hs\ {%sslv/%sslc/%[ssl_fc_sni]/%[ssl_fc_session_id]}\ "%[capture.req.method]\ %[capture.req.hdr(0)]%[capture.req.uri]\ HTTP/1.1"
        option  contstats               # Enable continuous traffic statistics updates
        timeout http-keep-alive 30s     # 15 second max for the client to post next request
        timeout http-request 30s        # 15 seconds max for the client to send a request
        acl                     is_ecp  var(txn.txnpath) -m sub -i /ecp/
        acl                     not_https       ssl_fc,not
        acl                     is_portal       var(txn.txnhost) -m str -i portal.<public domain>
        acl                     is_healthcheck  var(txn.txnpath) -m reg -i healthcheck.htm$
        acl                     is_autodiscover var(txn.txnhost) -m str -i autodiscover.<public domain>
        acl                     is_rpc  var(txn.txnpath) -m sub -i /rpc/
        acl                     is_owa  var(txn.txnpath) -m sub -i /OWA/
        acl                     is_ews  var(txn.txnpath) -m sub -i /EWS/
        acl                     is_oab  var(txn.txnpath) -m sub -i /OAB/
        acl                     is_eas  var(txn.txnpath) -m sub -i /EAS/
        acl                     is_mapi var(txn.txnpath) -m sub -i /mapi/
        http-request set-var(txn.txnpath) path
        http-request set-var(txn.txnhost) hdr(host)
        http-response deny  if  is_ecp
        http-response deny  if  is_healthcheck
        http-request redirect scheme https code 301  if  not_https
        use_backend portal-backend_ipvANY  if  is_portal
        use_backend ex-Autodiscover-backend_ipvANY  if  is_autodiscover
        use_backend ex-RPC-backend_ipvANY  if  is_rpc
        use_backend ex-OWA-backend_ipvANY  if  is_owa
        use_backend ex-EWS-backend_ipvANY  if  is_ews
        use_backend ex-OAB-backend_ipvANY  if  is_oab
        use_backend ex-EAS-backend_ipvANY  if  is_eas
        use_backend ex-MAPI-backend_ipvANY  if  is_mapi
        default_backend ex-OWA-backend_ipvANY

frontend smtp-frontend
        bind                    <public ip>:25 name <public ip>:25
        mode                    tcp
        log                     global
        option                  socket-stats
        option                  dontlognull
        option                  dontlog-normal
        maxconn                 10000
        timeout client          300000
        option tcplog
        option contstats
        default_backend ex-smtp-backend_ipvANY

frontend smtptls-frontend
        bind                    <public ip>:587 name <public ip>:587
        mode                    tcp
        log                     global
        option                  dontlognull
        option                  dontlog-normal
        maxconn                 10000
        timeout client          300000
        option tcplog
        option contstats
        default_backend ex-smtptls-backend_ipvANY

frontend smtpssl-frontend
        bind                    <public ip>:465 name <public ip>:465
        mode                    tcp
        log                     global
        option                  dontlognull
        option                  dontlog-normal
        maxconn                 10000
        timeout client          300000
        option tcplog
        option contstats
        default_backend ex-smtpssl-backend_ipvANY

frontend imap-frontend
        bind                    <public ip>:143 name <public ip>:143
        mode                    tcp
        log                     global
        option                  dontlognull
        option                  dontlog-normal
        maxconn                 10000
        timeout client          300000
        option tcplog
        option contstats
        default_backend ex-imap-backend_ipvANY

frontend imaps-frontend
        bind                    <public ip>:993 name <public ip>:993
        mode                    tcp
        log                     global
        option                  dontlognull
        option                  dontlog-normal
        timeout client          300000
        option contstats
        default_backend ex-imaps-backend_ipvANY

frontend pop-frontend
        bind                    <public ip>:110 name <public ip>:110
        mode                    tcp
        log                     global
        option                  dontlognull
        option                  dontlog-normal
        maxconn                 10000
        timeout client          300000
        option tcplog
        option contstats
        default_backend ex-pop-backend_ipvANY

frontend pops-frontend
        bind                    <public ip>:995 name <public ip>:995
        mode                    tcp
        log                     global
        option                  dontlognull
        option                  dontlog-normal
        maxconn                 10000
        timeout client          300000
        option tcplog
        option contstats
        default_backend ex-pops-backend_ipvANY

backend portal-backend_ipvANY
        mode                    http
        id                      103
        log                     global
        stats                   enable
        stats                   uri /haproxy?stats
        stats                   realm .
        timeout connect         30000
        timeout server          30000
        retries                 3
        server                  portal.<my domain> <my local subnet>.11:443 id 104 ssl check inter 1000  verify none

backend ex-Autodiscover-backend_ipvANY
        mode                    http
        id                      105
        log                     global
        stats                   enable
        stats                   uri /haproxy?stats
        stats                   realm .
        balance                 leastconn
        timeout connect         30000
        timeout server          30000
        retries                 3
        option                  httpchk GET /autodiscover/healthcheck.htm
        http-check expect status 200
        option  redispatch              # Try another server in case of connection failure
        server                  ex1 <my local subnet>.3:443 id 101 ssl check inter 3000  verify none
        server                  ex2 <my local subnet>.4:443 id 102 ssl check inter 3000  verify none

backend ex-RPC-backend_ipvANY
        mode                    http
        id                      106
        log                     global
        stats                   enable
        stats                   uri /haproxy?stats
        stats                   realm .
        balance                 leastconn
        timeout connect         30000
        timeout server          30000
        retries                 3
        option                  httpchk GET /RPC/HealthCheck.htm
        http-check expect status 200
        option  redispatch              # Try another server in case of connection failure
        #timeout queue 30s               # 30 seconds max queued on load balancer
        server                  ex1.<my domain> <my local subnet>.3:443 id 101 ssl check inter 1000  verify none
        server                  ex2.<my domain> <my local subnet>.4:443 id 102 ssl check inter 1000  verify none

backend ex-OWA-backend_ipvANY
        mode                    http
        id                      100
        log                     global
        stats                   enable
        stats                   uri /haproxy?stats
        stats                   realm .
        balance                 leastconn
        timeout connect         30000
        timeout server          30000
        retries                 3
        option                  httpchk GET /OWA/HealthCheck.htm
        http-check expect status 200
        option  redispatch              # Try another server in case of connection failure
        server                  ex1.<my domain> <my local subnet>.3:443 id 101 ssl check inter 1000  verify none
        server                  ex2.<my domain> <my local subnet>.4:443 id 102 ssl check inter 1000  verify none

backend ex-EWS-backend_ipvANY
        mode                    http
        id                      107
        log                     global
        stats                   enable
        stats                   uri /haproxy?stats
        stats                   realm .
        balance                 leastconn
        timeout connect         30000
        timeout server          30000
        retries                 3
        option                  httpchk GET /EWS/HealthCheck.htm
        http-check expect status 200
        option  redispatch              # Try another server in case of connection failure
        #timeout queue 30s               # 30 seconds max queued on load balancer
        server                  ex1.<my domain> <my local subnet>.3:443 id 101 ssl check inter 1000  verify none
        server                  ex2.<my domain> <my local subnet>.4:443 id 102 ssl check inter 1000  verify none

backend ex-OAB-backend_ipvANY
        mode                    http
        id                      108
        log                     global
        stats                   enable
        stats                   uri /haproxy?stats
        stats                   realm .
        balance                 leastconn
        timeout connect         30000
        timeout server          30000
        retries                 3
        option                  httpchk GET /OAB/HealthCheck.htm
        http-check expect status 200
        option  redispatch              # Try another server in case of connection failure
        #timeout queue 30s               # 30 seconds max queued on load balancer
        server                  ex1.<my domain> <my local subnet>.3:443 id 101 ssl check inter 1000  verify none
        server                  ex2.<my domain> <my local subnet>.4:443 id 102 ssl check inter 1000  verify none

backend ex-EAS-backend_ipvANY
        mode                    http
        id                      109
        log                     global
        stats                   enable
        stats                   uri /haproxy?stats
        stats                   realm .
        balance                 leastconn
        timeout connect         30000
        timeout server          30000
        retries                 3
        option                  httpchk GET /Microsoft-Server-ActiveSync/HealthCheck.htm
        http-check expect status 200
        option  redispatch              # Try another server in case of connection failure
        #timeout queue 30s               # 30 seconds max queued on load balancer
        server                  ex1.<my domain> <my local subnet>.3:443 id 101 ssl check inter 1000  verify none
        server                  ex2.<my domain> <my local subnet>.4:443 id 102 ssl check inter 1000  verify none

backend ex-MAPI-backend_ipvANY
        mode                    http
        id                      116
        log                     global
        stats                   enable
        stats                   uri /haproxy?stats
        stats                   realm .
        balance                 leastconn
        timeout connect         30000
        timeout server          30000
        retries                 3
        option                  httpchk GET /mapi/HealthCheck.htm
        http-check expect status 200
        option  redispatch              # Try another server in case of connection failure
        #timeout queue 30s               # 30 seconds max queued on load balancer
        server                  ex1.<my domain> <my local subnet>.3:443 id 101 ssl check inter 1000  verify none
        server                  ex2.<my domain> <my local subnet>.4:443 id 102 ssl check inter 1000  verify none

backend ex-smtp-backend_ipvANY
        mode                    tcp
        id                      110
        log                     global
        balance                 leastconn
        timeout connect         5000
        timeout server          30000
        retries                 3
        option redispatch
        option tcp-check
        tcp-check expect string 220
        default-server rise 2 fall 3
        server                  ex1.<my domain> <my local subnet>.3:25 id 111 check inter 3000
        server                  ex2.<my domain> <my local subnet>.4:25 id 112 check inter 3000

backend ex-smtptls-backend_ipvANY
        mode                    tcp
        id                      115
        log                     global
        balance                 leastconn
        timeout connect         5000
        timeout server          30000
        retries                 3
        default-server rise 2 fall 3
        option redispatch
        option tcp-check
        tcp-check expect string 220
        server                  ex1.<my domain> <my local subnet>.3:587 id 111 check inter 3000
        server                  ex2.<my domain> <my local subnet>.4:587 id 112 check inter 3000

backend ex-smtpssl-backend_ipvANY
        mode                    tcp
        id                      121
        log                     global
        balance                 leastconn
        timeout connect         5000
        timeout server          30000
        retries                 3
        option redispatch
        option tcp-check
        tcp-check expect string 220 ssl
        default-server rise 2 fall 3
        server                  ex1.<my domain> <my local subnet>.3:465 id 111 check inter 3000  verify none
        server                  ex2.<my domain> <my local subnet>.4:465 id 112 check inter 3000  verify none

backend ex-imap-backend_ipvANY
        mode                    tcp
        id                      114
        log                     global
        option                  log-health-checks
        balance                 leastconn
        timeout connect         30000
        timeout server          30000
        retries                 3
        option tcp-check
        tcp-check connect
        tcp-check expect string * OK
        server                  ex1.<my domain> <my local subnet>.3:143 id 111 check inter 1000
        server                  ex2.<my domain> <my local subnet>.4:143 id 112 check inter 1000

backend ex-imaps-backend_ipvANY
        mode                    tcp
        id                      117
        log                     global
        balance                 leastconn
        timeout connect         5000
        timeout server          30000
        retries                 3
        option redispatch
        option tcp-check
        tcp-check connect port 993 ssl
        tcp-check expect string * OK
        server                  ex1.<my domain> <my local subnet>.3:993 id 118 check inter 3000  verify none
        server                  ex2.<my domain> <my local subnet>.4:993 id 119 check inter 3000  verify none

backend ex-pop-backend_ipvANY
        mode                    tcp
        id                      120
        log                     global
        option                  log-health-checks
        balance                 leastconn
        timeout connect         5000
        timeout server          30000
        retries                 3
        option tcp-check
        tcp-check connect port 110
        tcp-check expect string +OK
        default-server rise 2 fall 3
        server                  ex1.<my domain> <my local subnet>.3:110 id 111 check inter 5000
        server                  ex2.<my domain> <my local subnet>.4:110 id 112 check inter 5000

backend ex-pops-backend_ipvANY
        mode                    tcp
        id                      113
        log                     global
        option                  log-health-checks
        balance                 leastconn
        timeout connect         5000
        timeout server          30000
        retries                 3
        option tcp-check
        tcp-check connect port 995 ssl
        tcp-check expect string +OK
        default-server rise 2 fall 3
        server                  ex1.<my domain> <my local subnet>.3:995 id 111 check inter 5000  verify none
        server                  ex2.<my domain> <my local subnet>.4:995 id 112 check inter 5000  verify none

When I run “Outlook Connectivity” test from the Microsoft Test Connectivity site I get the following

Here is some more details on the RPC over HTTP error

Here is an exempt from the haproxy logs

Jan 31 16:29:07 pfsense haproxy[65367]: 13.67.59.89:17280 [31/Jan/2019:16:29:07.405] http-https-frontend~ ex-RPC-backend_ipvANY/ex1.<local domain> 478/0/0/2/480 401 262 - - ---- 766/756/0/1/0 0/0 {mail.<public domain>|MSRPC} {0} {TLSv1.2/ECDHE-RSA-AES256-GCM-SHA384/mail.<public domain>/-} RPC_IN_DATA mail.<public domain>/rpc/rpcproxy.dll HTTP/1.1
Jan 31 16:29:08 pfsense haproxy[65367]: 13.67.59.89:17277 [31/Jan/2019:16:28:55.995] http-https-frontend~ ex-RPC-backend_ipvANY/ex2.<local domain> 12115/0/1/1/12117 401 262 - - ---- 768/758/0/1/0 0/0 {mail.<public domain>|MSRPC} {0} {TLSv1.2/ECDHE-RSA-AES256-GCM-SHA384/mail.<public domain>/-} RPC_IN_DATA mail.<public domain>/rpc/rpcproxy.dll?dd0e139f-324e-408a-8b4a-6574977303cd@<public domain>:6002 HTTP/1.1
Jan 31 16:29:08 pfsense haproxy[65367]: 13.67.59.89:17277 [31/Jan/2019:16:29:08.112] http-https-frontend~ ex-RPC-backend_ipvANY/ex2.<local domain> 227/0/0/1/228 401 262 - - ---- 767/757/0/1/0 0/0 {mail.<public domain>|MSRPC} {0} {TLSv1.2/ECDHE-RSA-AES256-GCM-SHA384/mail.<public domain>/-} RPC_IN_DATA mail.<public domain>/Rpc/RpcProxy.dll?dd0e139f-324e-408a-8b4a-6574977303cd@<public domain>:6001 HTTP/1.1
Jan 31 16:29:12 pfsense haproxy[65367]: 13.67.59.89:17281 [31/Jan/2019:16:29:11.818] http-https-frontend~ ex-RPC-backend_ipvANY/ex1.<local domain> 240/0/1/1/242 401 507 - - ---- 756/746/0/1/0 0/0 {mail.<public domain>|MSRPC} {0} {TLSv1.2/ECDHE-RSA-AES256-GCM-SHA384/mail.<public domain>/-} RPC_IN_DATA mail.<public domain>/Rpc/RpcProxy.dll?dd0e139f-324e-408a-8b4a-6574977303cd@<public domain>:6001 HTTP/1.1
Jan 31 16:29:12 pfsense haproxy[65367]: 13.67.59.89:17281 [31/Jan/2019:16:29:12.059] http-https-frontend~ ex-RPC-backend_ipvANY/ex1.<local domain> 234/0/0/20/254 200 377 - - ---- 756/746/0/1/0 0/0 {mail.<public domain>|MSRPC} {} {TLSv1.2/ECDHE-RSA-AES256-GCM-SHA384/mail.<public domain>/-} RPC_IN_DATA mail.<public domain>/Rpc/RpcProxy.dll?dd0e139f-324e-408a-8b4a-6574977303cd@<public domain>:6001 HTTP/1.1
Jan 31 16:29:12 pfsense haproxy[65367]: 13.67.59.89:17280 [31/Jan/2019:16:29:07.885] http-https-frontend~ ex-RPC-backend_ipvANY/ex1.<local domain> 4667/0/0/1/4668 401 507 - - ---- 755/745/0/1/0 0/0 {mail.<public domain>|MSRPC} {0} {TLSv1.2/ECDHE-RSA-AES256-GCM-SHA384/mail.<public domain>/-} RPC_IN_DATA mail.<public domain>/rpc/rpcproxy.dll?dd0e139f-324e-408a-8b4a-6574977303cd@<public domain>:6001 HTTP/1.1
Jan 31 16:29:12 pfsense haproxy[65367]: 13.67.59.89:17280 [31/Jan/2019:16:29:12.552] http-https-frontend~ ex-RPC-backend_ipvANY/ex1.<local domain> 239/0/0/11/250 200 377 - - ---- 754/744/0/1/0 0/0 {mail.<public domain>|MSRPC} {} {TLSv1.2/ECDHE-RSA-AES256-GCM-SHA384/mail.<public domain>/-} RPC_IN_DATA mail.<public domain>/rpc/rpcproxy.dll?dd0e139f-324e-408a-8b4a-6574977303cd@<public domain>:6001 HTTP/1.1
Jan 31 16:29:16 pfsense haproxy[65367]: 13.67.59.89:17302 [31/Jan/2019:16:29:15.616] http-https-frontend~ ex-RPC-backend_ipvANY/ex1.<local domain> 471/0/1/1/473 401 507 - - ---- 753/743/1/1/0 0/0 {mail.<public domain>|MSRPC} {0} {TLSv1.2/ECDHE-RSA-AES256-GCM-SHA384/mail.<public domain>/-} RPC_OUT_DATA mail.<public domain>/rpc/rpcproxy.dll?dd0e139f-324e-408a-8b4a-6574977303cd@<public domain>:6001 HTTP/1.1
Jan 31 16:29:46 pfsense haproxy[65367]: 13.67.59.89:17280 [31/Jan/2019:16:29:12.803] http-https-frontend~ ex-RPC-backend_ipvANY/ex2.<local domain> 2577/0/1/6/33517 401 262 - - sD-- 753/741/1/0/0 0/0 {mail.<public domain>|MSRPC} {0} {TLSv1.2/ECDHE-RSA-AES256-GCM-SHA384/mail.<public domain>/-} RPC_IN_DATA mail.<public domain>/rpc/rpcproxy.dll?dd0e139f-324e-408a-8b4a-6574977303cd@<public domain>:6001 HTTP/1.1
Jan 31 16:29:46 pfsense haproxy[65367]: 13.67.59.89:17302 [31/Jan/2019:16:29:16.089] http-https-frontend~ ex-RPC-backend_ipvANY/ex1.<local domain> 230/0/0/13/30243 200 346 - - sD-- 752/740/0/0/0 0/0 {mail.<public domain>|MSRPC} {} {TLSv1.2/ECDHE-RSA-AES256-GCM-SHA384/mail.<public domain>/-} RPC_OUT_DATA mail.<public domain>/rpc/rpcproxy.dll?dd0e139f-324e-408a-8b4a-6574977303cd@<public domain>:6001 HTTP/1.1
Jan 31 22:09:46 pfsense haproxy[65367]: 46.0.224.14:13159 [31/Jan/2019:22:09:46.265] http-https-frontend~ ex-RPC-backend_ipvANY/ex2.<local domain> 91/0/1/1/93 401 507 - - ---- 236/232/0/1/0 0/0 {mail.<public domain>|MSRPC} {0} {TLSv1/ECDHE-RSA-AES256-SHA/mail.<public domain>/#0357ٞ▒} RPC_IN_DATA mail.<public domain>/rpc/rpcproxy.dll?6627f89e-fa58-4bb4-bbc4-3afb47028417@<public domain>:6002 HTTP/1.1
Jan 31 22:09:46 pfsense haproxy[65367]: 46.0.224.14:13944 [31/Jan/2019:22:09:46.443] http-https-frontend~ ex-RPC-backend_ipvANY/ex1.<local domain> 78/0/0/1/80 401 507 - - ---- 237/233/1/1/0 0/0 {mail.<public domain>|MSRPC} {0} {TLSv1/ECDHE-RSA-AES256-SHA/mail.<public domain>/o▒rY;$▒cҘ▒P#002▒ԩP*#016o▒▒b▒▒f'▒*E▒[} RPC_OUT_DATA mail.<public domain>/rpc/rpcproxy.dll?6627f89e-fa58-4bb4-bbc4-3afb47028417@<public domain>:6002 HTTP/1.1
Jan 31 22:09:46 pfsense haproxy[65367]: 46.0.224.14:13944 [31/Jan/2019:22:09:46.523] http-https-frontend~ ex-RPC-backend_ipvANY/ex1.<local domain> 20/0/0/2/22 401 262 - - ---- 237/233/1/1/0 0/0 {mail.<public domain>|MSRPC} {0} {TLSv1/ECDHE-RSA-AES256-SHA/mail.<public domain>/o▒rY;$▒cҘ▒P#002▒ԩP*#016o▒▒b▒▒f'▒*E▒[} RPC_OUT_DATA mail.<public domain>/rpc/rpcproxy.dll?6627f89e-fa58-4bb4-bbc4-3afb47028417@<public domain>:6002 HTTP/1.1
Jan 31 22:09:46 pfsense haproxy[65367]: 46.0.224.14:13159 [31/Jan/2019:22:09:46.359] http-https-frontend~ ex-RPC-backend_ipvANY/ex2.<local domain> 64/0/0/2/205 401 262 - - CD-- 236/232/0/0/0 0/0 {mail.<public domain>|MSRPC} {0} {TLSv1/ECDHE-RSA-AES256-SHA/mail.<public domain>/#0357ٞ▒} RPC_IN_DATA mail.<public domain>/rpc/rpcproxy.dll?6627f89e-fa58-4bb4-bbc4-3afb47028417@<public domain>:6002 HTTP/1.1
Feb  1 22:09:58 pfsense haproxy[65367]: 46.0.128.3:7409 [01/Feb/2019:22:09:57.881] http-https-frontend~ ex-RPC-backend_ipvANY/ex1.<local domain> 415/0/1/2/418 401 507 - - ---- 159/153/0/1/0 0/0 {mail.<public domain>|MSRPC} {0} {TLSv1/ECDHE-RSA-AES256-SHA/mail.<public domain>/:E▒X▒▒:#024$▒▒▒,#023▒▒▒▒Q▒H▒Q}#022▒Ip▒▒Bp} RPC_IN_DATA mail.<public domain>/rpc/rpcproxy.dll?6627f89e-fa58-4bb4-bbc4-3afb47028417@<public domain>:6002 HTTP/1.1
Feb  1 22:09:58 pfsense haproxy[65367]: 46.0.128.3:7462 [01/Feb/2019:22:09:58.359] http-https-frontend~ ex-RPC-backend_ipvANY/ex2.<local domain> 89/0/1/1/91 401 507 - - ---- 160/154/1/1/0 0/0 {mail.<public domain>|MSRPC} {0} {TLSv1/ECDHE-RSA-AES256-SHA/mail.<public domain>/▒h$▒,#031p▒#025▒뉬▒#025Şl#012M_▒e▒Xe#020#0159#007̑} RPC_OUT_DATA mail.<public domain>/rpc/rpcproxy.dll?6627f89e-fa58-4bb4-bbc4-3afb47028417@<public domain>:6002 HTTP/1.1
Feb  1 22:09:58 pfsense haproxy[65367]: 46.0.128.3:7462 [01/Feb/2019:22:09:58.450] http-https-frontend~ ex-RPC-backend_ipvANY/ex2.<local domain> 19/0/0/2/21 401 262 - - ---- 160/154/1/1/0 0/0 {mail.<public domain>|MSRPC} {0} {TLSv1/ECDHE-RSA-AES256-SHA/mail.<public domain>/▒h$▒,#031p▒#025▒뉬▒#025Şl#012M_▒e▒Xe#020#0159#007̑} RPC_OUT_DATA mail.<public domain>/rpc/rpcproxy.dll?6627f89e-fa58-4bb4-bbc4-3afb47028417@<public domain>:6002 HTTP/1.1
Feb  1 22:09:58 pfsense haproxy[65367]: 46.0.128.3:7409 [01/Feb/2019:22:09:58.298] http-https-frontend~ ex-RPC-backend_ipvANY/ex1.<local domain> 31/0/0/2/194 401 262 - - CD-- 159/153/0/0/0 0/0 {mail.<public domain>|MSRPC} {0} {TLSv1/ECDHE-RSA-AES256-SHA/mail.<public domain>/:E▒X▒▒:#024$▒▒▒,#023▒▒▒▒Q▒H▒Q}#022▒Ip▒▒Bp} RPC_IN_DATA mail.<public domain>/rpc/rpcproxy.dll?6627f89e-fa58-4bb4-bbc4-3afb47028417@<public domain>:6002 HTTP/1.1

[AaronWest]

You had me thinking of this discussion…

Exchange 2013, random disconnects in Outlook 2016 for Mac

Just to add to why I’m fussing about timeout’s a little, I’ve worked with a lot of Exchange and HAProxy deployments and I’ve always found I need longer timeout’s than those seen in configs online. After my last post I did a google search to see if my theory of 45-60m timeout’s are justifiable and this guy certainly thinks so: He even goes further and tunes Exchange to be more aggressive with keep-alive which is something I have yet to need to try. However, he uses lower timeout’s than me so …

As I remember Outlook for Mac was using older auth and security level, if the reg keys don’t help then playing with the SSL options (Ciphers TLS level) in HAProxy might.

Anyway, a starting point maybe…

[KenynMacCormik]

Greetings,

It is definitely not an SSL options. Errors are being thrown with TLSv1.2/ECDHE-RSA-AES256-GCM-SHA384. Same sypher suites are used by Windows Outlook and are successful.

[lukastribus]

The other thread is not about the specific SSL cipher suite. It’s about Insecure Renegotiation settings. I suggest you try it.

[Speedy]

I hit this issue as well…not so much with DAG groups, but with Exchange with Outlook on a Mac was a pain!! Oulook on Mac doesn’t use RPC, but uses EWS…

To resolve the issue, I made the following changes. Since then, I’ve had no issues.
on the frontend for exchange:
remove “timeout http-keep-alive” and “timeout http-request”

on your backend,
change the “timeout server” to 30m

hope this helps

[KenynMacCormik]

The only option that helped me is to configure custom cookie for EWS and enable session affinity based on that cookie. See example below

defaults
        log     global
        mode    http
        option  httplog

        option  redispatch
        option  http-keep-alive
        option  forwardfor
        option  log-health-checks

        no option httpclose

        retries 3
        backlog 10000
        balance leastconn

        timeout connect         30s
        timeout http-keep-alive 15s
        timeout http-request    15s
        timeout queue           30s
        timeout tarpit          1m

        timeout client          30s
        timeout server          30s

        default-server inter 5s rise 2 fall 3

backend ews_bck
        option httpchk GET /EWS/HealthCheck.htm
        http-check expect status 200
        cookie SERVERID insert nocache

        server ex1 <edited>:443 ssl check verify none id 101 cookie unit1
        server ex2 <edited>:443 ssl check verify none id 102 cookie unit2
backend rpc_bck
        option httpchk GET /RPC/HealthCheck.htm
        http-check expect status 200
        #cookie SERVERID insert nocache

        server ex1 <edited>:443 ssl check verify none #id 101 cookie unit1
        server ex2 <edited>:443 ssl check verify none #id 102 cookie unit2

[andrewheberle]

Are you using the exact same certificate on each Exchange 2016 server?

Exchange 2016, removed the requirement for session affinity however this requires the exact same SSL certificate to be o each Exchange server and assigned to the IIS service.

If a client hits one server for the initial authentication but then hits a different server for subsequent requests (this will often happen when accessing ECP as when you hit ECP there is a redirect to OWA for auth then back to ECP), this will fail if the certificates are not identicial.

This is covered here:

Load balancing in Exchange Server

With some relevant excerpts (emphasis is mine):

With the HTTP protocol in use, all native clients connect using HTTP and HTTPs in Exchange Server. This standard protocol removes the need for affinity, which was previously required to avoid a new prompting for user credentials whenever load balancing redirected the connection to a different server.

Beginning with Exchange 2016, all native Exchange clients use the HTTP protocol to connect to a designated service, with HTTP cookies provided to the user at log in which are encrypted using the Client Access services SSL certificate. A logged in user can resume the session on a different Mailbox server running Client Access services without reauthenticating. Servers using the same SSL certificate can decrypt the client authentication cookie.

[KenynMacCormik]

Greetings,

I’m 120% sure certificates are same. Issue only hits mac users. Windows users are fine. But as I’ve stated above, session affinity fixes issue.