TCP mode - forward request to different backends based on substring (without delay)

Help!
1

What I’d like:

  • one frontend section (e.g. bound to port 80)
  • two backends
    • the first one is “static files” served over HTTP
    • the second one is a TCP response generation service that let’s users generate responses based on HTTP request, e.g. ANYTHING /response/{base64-encoded response} - mostly used for testing HTTP responses, but can respond with any arbitrary bytes not conforming to HTTP (so a raw TCP stream) to test out clients handling faulty “HTTP” responses

Problem:

  • how to proxy requests based on the magic string /response/ - if it is encountered in e.g. the first 50 bytes of the TCP stream, instantly use the response generation backend; otherwise, serve static files from the first backend
  • the overall request (especially for the service) may be shorter than 50 bytes
  • it should be as fast as possible, but also deal with somewhat slower clients or lost packets (e.g. TCP handshake made, but some time to get the first data packets flowing in)

Current test set-up - I can get the needed condition working with either an arbitrary delay (not wanted! do it as fast as possible, but have a timeout) or “reject/accept” instantly, but then no choice can be made between backends.

haproxy.cfg:

global
    log stdout format raw local0

defaults
    mode tcp
    timeout connect 28s
    timeout client 29s
    timeout server 30s

frontend fe_main
    log global
    option tcplog
    
    bind :80

    tcp-request inspect-delay 3s

    # OPTION 1: it works, but delay is necessary (otherwise acl not enforced)
    tcp-request content accept if WAIT_END
    acl contains_bbb req.payload(0,0) -m sub BBB
    use_backend be_data if contains_bbb

    # (Almost an) OPTION 2: it works instantly, but can't choose backends this way
    # tcp-request content reject if { req.payload(0,0) -m sub AAA }

    default_backend be_static

backend be_data
    server server1 data_backend:9001

backend be_static
    mode http
    server server1 static_backend:9002

docker-compose.yml:

services:
  haproxy:
    image: haproxy:3.0
    volumes:
      - ./haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro
    ports:
      - "80:80"
    depends_on:
      - data_backend
      - static_backend

  data_backend:
    image: alpine/socat
    command: "tcp-l:9001,fork,reuseaddr exec:/bin/cat"

  static_backend:
    image: alpine/socat
    command: "tcp-l:9002,fork,reuseaddr exec:'/bin/echo -e HTTP/1.1 200 OK\\r\\n\\r\\nbody'"

Build:

$ docker compose up --build --force-recreate

Test:

$ printf "BBB" | nc localhost 80
$ printf "BBB" | nc localhost 80  # when have chosen the 2nd option
2

accept if it matches your rule, you so don’t have to wait:

# OPTION 1: it works, but delay is necessary (otherwise acl not enforced)
tcp-request content accept if WAIT_END
acl contains_bbb req.payload(0,0) -m sub BBB
+tcp-request content accept if contains_bbb
use_backend be_data if contains_bbb

For the static case, similarly, match HTTP traffic to cut the inspect-delay short (of course after the bbb rules):

tcp-request content accept if HTTP
3

Now it’s the other way around, the static case has a delay
(also note that requests to both may trigger HTTP to be true)

frontend fe_main
    log global
    option tcplog
    
    bind :80

    tcp-request inspect-delay 5s

    #tcp-request content accept if WAIT_END  # commented this out, otherwise ALWAYS full inspect-delay
    
    # Works now without any delay
    acl contains_bbb req.payload(0,0) -m sub BBB
    tcp-request content accept if contains_bbb
    use_backend be_data if contains_bbb
    
    # ... but now this has delay
    tcp-request content accept if HTTP
    use_backend be_static if HTTP

Testing out

# delays for 3 seconds and then responds ("static content service", demo response):
$ time printf "GET /AAA HTTP/1.1\r\n\r\n" | nc localhost 80
HTTP/1.1 200 OK
connection: close

body

real	0m3,016s

# instant reply ("data service"), mirrors request for demo purposes
$ printf "GET /BBB HTTP/1.1\r\n\r\n" | nc localhost 80
GET /BBB HTTP/1.1

# also instant reply if the last B reaches the stream; be sure to type pretty fast
$ nc localhost 80
BB<Ctrl + D>B<Ctrl + D>
4

Use a single accept statement with a or condition:

tcp-request content accept if contains_bbb || HTTP
5

Thank you very much! This works as intended.

Posting the full demo config here for future readers.
haproxy.cfg:

global
    log stdout format raw local0

defaults
    mode tcp
    timeout connect 28s
    timeout client 29s
    timeout server 30s

frontend fe_main
    log global
    option tcplog
    
    bind :80

    tcp-request inspect-delay 5s

    acl is_data_request req.payload(0,0) -m sub /data/
    tcp-request content accept if is_data_request || HTTP
    use_backend be_data if is_data_request
    default_backend be_static 

backend be_data
    server server1 data_backend:9001

backend be_static
    mode http
    server server1 static_backend:9002