I have set up a HAproxy to handel about 100K rps and then proxied that to the 4 nginx
but suddenly all the connection from HAproxy to the nginxes become
connection wating (red line) and the connection established (blue line) become minimum and all the cpu’s that allocated to the haproxy become 100%
this is the HAproxy configuration :
global nbproc 1 nbthread 8 cpu-map auto:1/1-8 0-7 maxconn 500000 ... defaults timeout connect 7s timeout client 14s timeout server 14s ... frontend https_in maxconn 500000 mode tcp ...
and here is the kernel configuration:
net.ipv4.ip_forward=1 net.ipv4.ip_nonlocal_bind = 1 fs.file-max = 10000000 fs.nr_open = 10000000 net.ipv4.ip_local_port_range = 1024 65535 net.ipv4.tcp_max_syn_backlog = 65536 net.core.somaxconn = 65535 net.core.netdev_max_backlog = 65536 net.ipv4.tcp_max_orphans = 262144
keepalive timeout in nginxes are 30s
HAproxy Vevrsion : HA-Proxy version 2.3.10-1ppa1~focal 2021/04/24
OS Version : Ubuntu 20.04.2 LTS
Kernel Version : 5.4.0-74-generic
what could be the problem ?