Http Pipelining Config (HA Proxy + Spray)

Help!
1

In our existing system, haproxy is used for load balancing requests from clients (spray based) to server (spray-can based). Config is following:

API Client
host-connector {
max-connections = 100
max-retries = 0
max-redirects = 0
pipelining = on
}

HA Proxy
defaults
mode http
option http-server-close
option forwardfor except 127.0.0.0/8
option redispatch
retries 3
timeout http-request 10s
timeout queue 1m
timeout connect 10s
timeout client 1m
timeout server 1m
timeout http-keep-alive 10s
timeout check 10s
maxconn 3000

API Server
server{
server-header=""
pipelining-limit=0
}

I am getting premature connection closed in this setting. I think this is coming because on the same connection HA Proxy sends more requests and my spray server has pipelining-limit 0.

Should i remove http-server-close from ha proxy because i dont want http-close on server side (This will put HA Proxy in tunneling mode where i dont see any disadvantage). Also, I will add increase pipelining-limit to 1024 in spray server.

2

On which part of the stack do you get what exact message, and please provide the log of them all.

pipelining-limit can be “must be > 0 and <= 128.” according to:

Do mean you have “pipelining-limit 1”?

The effect of removing http-server-close from the configuration depends on the haproxy version. Please provide the output of haproxy -vv so we actually know which haproxy default your setup will turn to in this configuration.

Only in old haproxy releases that is the case (<= 1.4). The disadvantage is that haproxy will no longer put the client IP in the x-forwarded-for header. Do you need the client IP on your backend? Then you cannot use tunnel mode.

I would recommend a recent haproxy stable release with a sane http-keep-alive configuration.

3

My answers to your questions are as following:

  1. I am getting this error on server side. Exception is:
    spray.can.Http$ConnectionException: Premature connection close (the server doesn’t appear to support request pipelining)

  2. Although config asks for pipeline-limit value to be greater than 0. In code following is used:
    require(0 <= pipeliningLimit && pipeliningLimit <= 128, “pipelining-limit must be >= 0 and <= 128”) ( http://spray.io/documentation/1.2.4/spray-can/configuration/)

  3. My version is 1.5.18

  4. The client id is used in the code base, I need to understand from the team the impact of being able to remove it.

So what do you recommend about not using http-server-close in ha proxy in 1.5.18, any disadvantage other than loosing client ips. The advantage is performance boost because now i dont need to close connection with the server after every request.

4

In 1.5 http-keep-alive is the default, not tunnel mode. http-keep-alive will be fine, but I doubt it will solve your problem. It probably has not really anything to do with haproxy at all, but with the webserver spray is running on top.