# Extremely Large HTTP Responses with Chunked Encoding Blocked

**URL:** <https://discourse.haproxy.org/t/extremely-large-http-responses-with-chunked-encoding-blocked/6175>\
**Category:** Help!\
**Created:** [January 22, 2021, 10:06pm UTC](https://discourse.haproxy.org/t/extremely-large-http-responses-with-chunked-encoding-blocked/6175 "2021-01-22T22:06:52Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![MysticRyuujin](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.haproxy.org/mysticryuujin/32/1358_2.png) [@MysticRyuujin](https://discourse.haproxy.org/u/MysticRyuujin)\
**Post date:** [January 22, 2021, 10:06pm UTC](https://discourse.haproxy.org/t/extremely-large-http-responses-with-chunked-encoding-blocked/6175/1 "2021-01-22T22:06:52Z")

</div>

I’m trying to figure out why haproxy is blocking server responses with the “PH” flag.

The responses are VERY large, we’re talking 3-5GB responses. This particular response is 3633514102 bytes (3.4GB).

I have tried a number of things such as setting `tune.bufsize` to 5GB in size. I’ve tried turning on `option accept-invalid-http-response` however, I’m kind of just fumbling around in the dark trying to guess what the problem might be.

I’m not sure what combination of settings I need to support such large responses, or if something else is going on here?

I’m on Version `2.2.6-1~bpo10+1`

I can see that it’s complaining about the error at position 130, which is the hex representation of the chunk size (3633514102). But I can’t for the life of me figure out why it’s complaining at that location.

The “show error” output is here:

```auto
Total events captured on [22/Jan/2021:18:41:25.300] : 1
 
[22/Jan/2021:18:41:20.864] backend turbogeth (#11): invalid response
  frontend http-https-in (#2), server archive04 (#2), event #0, src <redacted>:39386
  buffer starts at 0 (including 0 out), 14936 free,
  len 1448, wraps at 16336, error at position 130
  H1 connection flags 0x00000000, H1 stream flags 0x00004014
  H1 msg state MSG_CHUNK_SIZE(26), H1 msg flags 0x00001716
  H1 chunk len 0 bytes, H1 body len 0 bytes :
  
  00000 HTTP/1.1 200 OK\r\n
  00017 Content-Type: application/json\r\n
  00049 Vary: Origin\r\n
  00063 Date: Fri, 22 Jan 2021 18:41:20 GMT\r\n
  00100 Transfer-Encoding: chunked\r\n
  00128 \r\n
  00130 d8930676\r\n
  00140 {"jsonrpc":"2.0","id":1,"result":{"gas":4300872,"failed":false,"return
  00210+ Value":"","structLogs":[{"pc":0,"op":"CALLDATASIZE","gas":5810070,"gas
  00280+ Cost":2,"depth":1,"stack":[],"memory":[],"storage":{}},{"pc":1,"op":"R
  00350+ ETURNDATASIZE","gas":5810068,"gasCost":2,"depth":1,"stack":["000000000
  00420+ 0000000000000000000000000000000000000000000000000000b24"],"memory":[],
  00490+ "storage":{}},{"pc":2,"op":"RETURNDATASIZE","gas":5810066,"gasCost":2,
  00560+ "depth":1,"stack":["00000000000000000000000000000000000000000000000000
  00630+ 00000000000b24","00000000000000000000000000000000000000000000000000000
  00700+ 00000000000"],"memory":[],"storage":{}},{"pc":3,"op":"CALLDATACOPY","g
  00770+ as":5810064,"gasCost":558,"depth":1,"stack":["000000000000000000000000
  00840+ 0000000000000000000000000000000000000b24","000000000000000000000000000
  00910+ 0000000000000000000000000000000000000","000000000000000000000000000000
  00980+ 0000000000000000000000000000000000"],"memory":["0000000000000000000000
  01050+ 000000000000000000000000000000000000000000","0000000000000000000000000
  01120+ 000000000000000000000000000000000000000","0000000000000000000000000000
  01190+ 000000000000000000000000000000000000","0000000000000000000000000000000
  01260+ 000000000000000000000000000000000","0000000000000000000000000000000000
  01330+ 000000000000000000000000000000","0000000000000000000000000000000000000
  01400+ 000000000000000000000000000","000000000000000000

```

---

<div class="post-metadata">

**Author:** ![lukastribus](https://avatars.discourse-cdn.com/v4/letter/l/7ea924/32.png) [@lukastribus](https://discourse.haproxy.org/u/lukastribus)\
**Post date:** [January 22, 2021, 10:24pm UTC](https://discourse.haproxy.org/t/extremely-large-http-responses-with-chunked-encoding-blocked/6175/2 "2021-01-22T22:24:05Z")

</div>

[https://cbonte.github.io/haproxy-dconv/2.2/configuration.html#8.5](https://cbonte.github.io/haproxy-dconv/2.2/configuration.html#8.5)

> PD The proxy blocked an incorrectly formatted chunked encoded message in  
> a request or a response, after the server has emitted its headers. In  
> most cases, this will indicate an invalid message from the server to  
> the client. **HAProxy supports chunk sizes of up to 2GB - 1 (2147483647**  
> **bytes). Any larger size will be considered as an error.**

---

<div class="post-metadata">

**Author:** ![MysticRyuujin](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.haproxy.org/mysticryuujin/32/1358_2.png) [@MysticRyuujin](https://discourse.haproxy.org/u/MysticRyuujin)\
**Post date:** [January 22, 2021, 11:07pm UTC](https://discourse.haproxy.org/t/extremely-large-http-responses-with-chunked-encoding-blocked/6175/3 "2021-01-22T23:07:40Z")

</div>

This is interesting as the logs indicate that the block was due to “PH” so I didn’t look at “PD”

I take it this is not configurable?

---

<div class="post-metadata">

**Author:** ![lukastribus](https://avatars.discourse-cdn.com/v4/letter/l/7ea924/32.png) [@lukastribus](https://discourse.haproxy.org/u/lukastribus)\
**Post date:** [January 22, 2021, 11:36pm UTC](https://discourse.haproxy.org/t/extremely-large-http-responses-with-chunked-encoding-blocked/6175/4 "2021-01-22T23:36:40Z")

</div>

No, I don’t think this is configurable, I don’t think anybody ever anticipated the need for such huge chunks. If you need this, you may want to file a feature request on github. This limitation dates back quite some time, and large parts of haproxy have been rewritten, so it’s possible we don’t really need this limitation anymore.

That said, if this shows up as PH instead of PD then it’s a bug either in the code or in the documentation, an should be filed as such:

> **[haproxy/haproxy](https://github.com/haproxy/haproxy/issues)**
>
> s

---

<div class="post-metadata">

**Author:** ![MysticRyuujin](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.haproxy.org/mysticryuujin/32/1358_2.png) [@MysticRyuujin](https://discourse.haproxy.org/u/MysticRyuujin)\
**Post date:** [January 22, 2021, 11:57pm UTC](https://discourse.haproxy.org/t/extremely-large-http-responses-with-chunked-encoding-blocked/6175/5 "2021-01-22T23:57:48Z")

</div>

I’ll file both. Thank you, sir.