Hello there,
frontend local
bind 127.0.0.1:8118
mode http
default_backend main
backend main
balance leastconn
http-reuse always
http-request set-header Proxy-Authorization Basic\ <base64 user:pass>
http-request set-header Proxy-Connection Keep-Alive
server 1 <ip:port> check fall 3 rise 3 inter 15s observe layer7 error-limit 10 on-error mark-down
The rest of the config is default from Debian 12 package.
This worked until yesterday but has broken all of a sudden:
HTTP/1.0 407 Proxy Authentication Required
proxy-authenticate: Basic realm="proxy"
< content-type: text/html; charset=utf-8
<
* CONNECT tunnel failed, response 407
I made sure via curl that all proxy servers are still working, i.e.:
curl -I -v -x <ip:port> --proxy-user <user:pass> http://www.google.com
which returns HTTP/1.1 200 OK.
But HAProxy is just not working. Not anymore. Nothing changed. I am at a loss…
govind
September 23, 2024, 4:14pm
2
Weidenwiesel:
server 1 <ip:port>
can you check if your server1 is active and you are able to connect to it from haproxy server/pod ?
I actually have 5 servers in total, 3 of them are active, 2 of them are marked as backup, all of them are up. However, when I try to curl the local frontend, i.e.:
curl -v -x 127.0.0.1:8118 https://www.google.com
I get:
* Trying 127.0.0.1:8118...
* Connected to 127.0.0.1 (127.0.0.1) port 8118 (#0)
* allocate connect buffer
* Establish HTTP proxy tunnel to www.google.com:443
> CONNECT www.google.com:443 HTTP/1.1
> Host: www.google.com:443
> User-Agent: curl/7.88.1
> Proxy-Connection: Keep-Alive
>
< HTTP/1.0 407 Proxy Authentication Required
< proxy-authenticate: Basic realm="proxy"
< content-type: text/html; charset=utf-8
<
* CONNECT tunnel failed, response 407
* Closing connection 0
curl: (56) CONNECT tunnel failed, response 407
which is weird since my HAProxy specifically sets the Proxy Authentication http header and it has worked for the past 2 months without any problems.
When I use curl with one of the proxy servers directly, i.e.:
curl -v -x <server_1_ip:port> --proxy-user <user:pass> http://www.google.com
it works perfectly.
So for some reason, setting the http-request header in HAProxy, i.e.:
http-request set-header Proxy-Authorization Basic\ <base64 user:pass>
has stopped working.
I dunno what I can do to make HAProxy work with external proxies that require that auth again…
It’s getting weirder.
HAProxy log for one of the successfull requests:
local main/1 0/0/30/33/331 200 28862 - - ---- 1/1/0/0/0 0/0 "CONNECT www.google.com:443 HTTP/1.1"
So IT IS WORKING. but why is is so unbelievably unreliable?
When I cURL the proxies directly, none fail.
Any ideas anyone?
Found this which seems related to my problem:
opened 10:49AM - 10 Mar 15 UTC
closed 02:36PM - 10 Mar 15 UTC
Propose Close
I'm having a problem I traced down to requests as the equivalent curl command wo… rks. I am running HAProxy on `localhost:5555` to load balance requests to a pool of authenticated backend http proxies as explained [here](http://serverfault.com/questions/386431/authenticated-proxies-on-haproxy-load-balancer). It works fine for fetching urls without redirection, as shown below using [httpie](https://github.com/jakubroztocil/httpie) (same behaviour observed using requests directly from the shell):
```
$ http 'http://httpbin.org/get' --proxy http:http://localhost:5555 --headers --debug
HTTPie 0.9.2
HTTPie data: /home/gsakkis/.httpie
Requests 2.5.3
Pygments 2.0.2
Python 2.7.6 (default, Mar 22 2014, 22:59:56)
[GCC 4.8.2] linux2
>>> requests.request({'allow_redirects': False,
'auth': None,
'cert': None,
'data': OrderedDict(),
'files': DataDict(),
'headers': {'User-Agent': 'HTTPie/0.9.2'},
'method': 'get',
'params': ParamsDict(),
'proxies': {u'http': u'http://localhost:5555'},
'stream': True,
'timeout': 30,
'url': u'http://httpbin.org/get',
'verify': True})
HTTP/1.0 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Connection: keep-alive
Content-Length: 269
Content-Type: application/json
Date: Tue, 10 Mar 2015 10:38:37 GMT
Server: nginx
Via: 1.0 fl291 (squid/3.1.10)
X-Cache: MISS from fl291
X-Cache-Lookup: MISS from fl291:80
```
If I fetch a url that redirects without actually redirecting it still works, returning 302:
```
$ http 'http://httpbin.org/redirect-to?url=http://example.com/' --proxy http:http://localhost:5555 --headers --debug
HTTPie 0.9.2
HTTPie data: /home/gsakkis/.httpie
Requests 2.5.3
Pygments 2.0.2
Python 2.7.6 (default, Mar 22 2014, 22:59:56)
[GCC 4.8.2] linux2
>>> requests.request({'allow_redirects': False,
'auth': None,
'cert': None,
'data': OrderedDict(),
'files': DataDict(),
'headers': {'User-Agent': 'HTTPie/0.9.2'},
'method': 'get',
'params': ParamsDict(),
'proxies': {u'http': u'http://localhost:5555'},
'stream': True,
'timeout': 30,
'url': u'http://httpbin.org/redirect-to?url=http://example.com/',
'verify': True})
HTTP/1.0 302 Moved Temporarily
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Connection: keep-alive
Content-Length: 0
Content-Type: text/html; charset=utf-8
Date: Tue, 10 Mar 2015 10:41:00 GMT
Location: http://example.com/
Server: nginx
Via: 1.0 mf5 (squid/3.1.10)
X-Cache: MISS from mf5
X-Cache-Lookup: MISS from mf5:80
```
However attempting to follow the redirect fails with 407:
```
$ http 'http://httpbin.org/redirect-to?url=http://example.com/' --proxy http:http://localhost:5555 --headers --follow --debug
HTTPie 0.9.2
HTTPie data: /home/gsakkis/.httpie
Requests 2.5.3
Pygments 2.0.2
Python 2.7.6 (default, Mar 22 2014, 22:59:56)
[GCC 4.8.2] linux2
>>> requests.request({'allow_redirects': True,
'auth': None,
'cert': None,
'data': OrderedDict(),
'files': DataDict(),
'headers': {'User-Agent': 'HTTPie/0.9.2'},
'method': 'get',
'params': ParamsDict(),
'proxies': {u'http': u'http://localhost:5555'},
'stream': True,
'timeout': 30,
'url': u'http://httpbin.org/redirect-to?url=http://example.com/',
'verify': True})
HTTP/1.0 407 Proxy Authentication Required
Connection: keep-alive
Content-Length: 4
Content-Type: text/html
Date: Tue, 10 Mar 2015 10:42:02 GMT
Mime-Version: 1.0
Proxy-Authenticate: Basic realm="login"
Server: squid/3.1.10
Via: 1.0 mf5 (squid/3.1.10)
X-Cache: MISS from mf5
X-Cache-Lookup: NONE from mf5:80
X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0
```
The equivalent curl command seems to work fine though:
```
$ curl 'http://httpbin.org/redirect-to?url=http://example.com/' -s -L -x localhost:5555 -v -o /dev/null
* Hostname was NOT found in DNS cache
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 5555 (#0)
> GET http://httpbin.org/redirect-to?url=http://example.com/ HTTP/1.1
> User-Agent: curl/7.35.0
> Host: httpbin.org
> Accept: */*
> Proxy-Connection: Keep-Alive
>
* HTTP 1.0, assume close after body
< HTTP/1.0 302 Moved Temporarily
< Server: nginx
< Date: Tue, 10 Mar 2015 10:44:56 GMT
< Content-Type: text/html; charset=utf-8
< Content-Length: 0
< Location: http://example.com/
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Credentials: true
< X-Cache: MISS from fl291
< X-Cache-Lookup: MISS from fl291:80
< Via: 1.0 fl291 (squid/3.1.10)
* HTTP/1.0 connection set to keep alive!
< Connection: keep-alive
<
* Connection #0 to host localhost left intact
* Issue another request to this URL: 'http://example.com/'
* Hostname was found in DNS cache
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 5555 (#1)
> GET http://example.com/ HTTP/1.0
> User-Agent: curl/7.35.0
> Host: example.com
> Accept: */*
> Proxy-Connection: Keep-Alive
>
* HTTP 1.0, assume close after body
< HTTP/1.0 200 OK
< Accept-Ranges: bytes
< Cache-Control: max-age=604800
< Content-Type: text/html
< Date: Tue, 10 Mar 2015 10:44:57 GMT
< ETag: "359670651"
< Expires: Tue, 17 Mar 2015 10:44:57 GMT
< Last-Modified: Fri, 09 Aug 2013 23:54:35 GMT
< Server: ECS (cpm/F9D5)
< X-Cache: HIT
< x-ec-custom-error: 1
< Content-Length: 1270
< X-Cache: MISS from bv400
< X-Cache-Lookup: MISS from bv400:80
< Via: 1.0 bv400 (squid/3.1.10)
* HTTP/1.0 connection set to keep alive!
< Connection: keep-alive
<
{ [data not shown]
* Connection #1 to host localhost left intact
```