Expensive JWT ACL only for some backends

Help!
1

I have a common frontend that accepts connections from our external gateway and routes them to backends:

frontend https-rev-auth
    mode http
    bind [::]:443 crt /etc/haproxy/ssl/fauth/ alpn h2,http/1.1

    # Additional variables used later
    http-request set-var-fmt(req._someVar) ....

    # Direct to error page if source doesn't match
    use_backend _forbidden unless { src -f /etc/haproxy/cloudflare.lst }

    # Route request
    default_backend _deny-no-match
    use_backend foo-app   if { hdr(host) -i foo.example.com }
    use_backend bar-app   if { hdr(host) -i bar.example.com }
    use_backend baz-app   if { hdr(host) -i baz.example.com }

backend foo-app
    mode http
    server web1 foo-app-srv:1234 check inter 10s

Now I need to add authentication where client would either have to arrive with Authorization: Basic ... or a valid JWT token for a given application. The backend will then receive a custom header with authenticated username, regardless of whether the user used JWT or HTTP auth.
However, just the JWT validation is very heavy, and thus I want to only do it for a few endpoints where I expect JWT at all.

I know I can do something stupid like below, but this sends the request on a long and expensive journey, and duplicates my host list:

frontend https-rev-auth
    # ...
    use_backend baz-app     if { hdr(host) -i baz.example.com }   # non-JWT app
    use_backend _force-jwt  if { hdr(host) -i abc.example.com }   # direct to "shin" JWT backend
    use_backend _force-jwt  if { hdr(host) -i def.example.com }

backend _force-jwt
    mode http
    server jwt 127.0.0.1:99999

frontend jwt
    bind 127.0.0.1:9999

    http-request deny content-type 'text/html' string 'Missing Authorization HTTP header' unless { req.hdr(authorization) -m found }
    #...
    http-request deny content-type 'text/html' string 'JWT has expired' if { var(txn.exp),sub(txn.now) -m int lt 0 }

    # Direct to a real app server backend
    use_backend abc-app   if { hdr(host) -i abc.example.com }
    use_backend def-app   if { hdr(host) -i def.example.com }

Is there a smarter way to, I guess, generically apply a set of expensive ACLs to only a subset of hosts that arrive through a frontend? I tried searching with various keywords and I wasn’t able to find a smart solution.