Rate limitting on real IP instead of previous hop in Kubernetes

Help!
1

Hello

Setup

I want to use HAproxy to rate limit incoming requests my ideal setup woud be:

HAProxy -> Traefik -> Nginx -> SVC

where HAProxy does the rate limitting forwards http on port 80 and tcp on port 443
Traefik does SSL terminaltion with Let’s Encrypt
Nginx does specific header validation, add required headers, and file serving
SVC a service written in Go or Python

The problem

The problem I have is that HAProxy counts abuses on the IP of previous hop which is a node in Kubernetes so the IPv4 is always the same (well my cluster has 10 nodes so the IP is one of those 10) Which means that whenever I block someone during 30sec no one is able to access from different public IP’s.

What is the ideal config to rate limit on real ips or is there an alternative approach?

HAProxy Config

this is my HAProxy configuration:

    global
      maxconn 20000
      log stdout local0 notice

    defaults
        maxconn 20000
        option dontlognull
        retries 3
        timeout connect 10s
        timeout client 300s
        timeout server 300
        log global
        balance source
        option forwardfor
        option http-server-close

    frontend proxy_http_mode_http
        bind *:80
        mode http

        # ACL
        acl abuse src_http_req_rate(Abuse) ge 300
        acl flag_abuser src_inc_gpc0(Abuse) ge 10
        acl scanner src_http_err_rate(Abuse) ge 10

        # track
        http-request track-sc0 src table Abuse

        # Returns a 403 to the abuser and flags for tcp-reject next time
        http-request deny deny_status 429 if abuse flag_abuser scanner

        use_backend backend_http

    frontend metrics
      bind *:9090
      no log
      stats uri /
      stats enable

    frontend proxy_https_mode_tcp
        bind *:443
        mode tcp
        option tcplog

        # ACL function declarations
        acl is_abuse src_http_req_rate(Abuse) ge 1000
        acl inc_abuse_cnt src_inc_gpc0(Abuse) gt 30
        acl abuse_cnt src_get_gpc0(Abuse) gt 30

        # Rules
        tcp-request connection track-sc0 src table Abuse
        tcp-request connection reject if abuse_cnt

        use_backend backend_https

    backend backend_http
        balance roundrobin
        server traefik localhost:800
        mode http

    backend backend_https
        balance roundrobin
        server traefik localhost:4430
        mode tcp

    backend Abuse
        stick-table type ip size 1m expire 30s store conn_rate(3s),conn_cur,gpc0,http_req_rate(5s),http_err_rate(20s)