SD termination states with MySQL proxy when doing SSL

Help!
1

I have a haproxy infront of Percona MySQL cluster. HAproxy is TCP proxying connection towards Percona with send-proxy-v2.

In the log lines, I’m seeing around 40% of SD connection termination states which haproxy logs as an error. If I disable SSL login, then the connections terminates OK and there is no problem. I have search the web for similar issues but couldn’t find a solution. The closest was Intermittent "SD" termination state - #3 by uliromahn, but we are not using nolinger option.

Error:

Aug 9 16:13:49 mysql-lb1 haproxy[346659]: 2001:yyyy:xxxx:44::4:51724 [09/Aug/2024:16:13:49.260] mysql-db1X mysql-db1X/mysql-db1a 1/0/149 39165 SD 44/42/41/41/0 0/0

HAproxy, client and DB servers are all in the same L2 network and I tried disabling firewall on the servers just to be sure firewall is not messing things up. No win:)

tcpdump of a request that returns SD state (cja host is the client):

Screenshot_2024-08-09_at_16.19.55
Screenshot_2024-08-09_at_16.19.552190×356 159 KB

When I disable SSL, the last RST package does not happen and seems like the only difference in the tcpdump.

Haproxy config:

global
  chroot  /var/lib/haproxy
  daemon
  group  haproxy
  maxconn  2048
  pidfile  /var/run/haproxy.pid
  user  haproxy

defaults
  log  global
  maxconn  8000
  option  redispatch
  retries  3
  timeout  http-request 10s
  timeout  queue 1m
  timeout  connect 10s
  timeout  client 1m
  timeout  server 1m
  timeout  check 10s

listen mysql-pdc2-db1X
  bind xxxx:3306
  bind yyy:3306
  mode tcp
  balance leastconn
  option httpchk
  option clitcpka
  option srvtcpka
  option tcplog
  timeout queue 1m
  timeout connect 10s
  timeout client 8h
  timeout server 8h
  server mysql-pdc2-db1a x.y.v.z:3306 check send-proxy-v2 port 9200 inter 2s rise 3 fall 2
  server mysql-pdc2-db1b x.y.v.z1:3306 check send-proxy-v2 backup port 9200 inter 2s rise 3 fall 2
  server mysql-pdc2-db1c x.y.v.z:3306 check send-proxy-v2 backup port 9200 inter 2s rise 3 fall 2

The application (client) works OK as I always get the whole response from the MySQL server back, but haproxy is incrementing error counters which is not ideal:)

We are using haproxy 2.4.22 (bundled with RHEL 9).

Any idea / help would be appreciated, because I have no idea where to search for solution.

Thanks.

2

Can anyone shed some light to this issue, I have tried everything I can think off, but no success:/

3

After a lot of testing and debugging, I got a little further.

I am pretty sure this is a PHP issue. I tried running same code with Python and Go and it works flawlessly without a single SD termination state even if I remove the connection close functions. But switching back to PHP, I get random SD states.

Looking at tcpdump, Go and Python tear down connection with FIN, FIN, ACK, but PHP does it with FIN,FIN and then some RST pkgs arrive from DB side. I search the web for some explanation, but couldn’t find anything until now.

Will close this for now.