The proxy instance continues to send 31 bytes of TLS encrypted alert towards that backend, which is already correctly acknowledged. Wireshark shows this as Spurious Retransmission which is correct.
I think your local firewall may be dropping the TCP ACK signal because of the close_wait timeout.
Can you bump net.netfilter.nf_conntrack_tcp_timeout_close_wait from 60 to something way beyond 100, actually make it 300 just for testing.