Haproxy 1.8.3 HTTP/2 HPACK Compression Controls?

eva2000 · February 7, 2018, 8:18pm

Hi I just installed Haproxy 1.8.3 via source compile to compare against Nginx 1.13.8 also source compiled in proxy config with and without caching.

I setup HTTP/2 HTTPS to see how they fair for nghttp2’s h2load HTTP/2 HTTPS load tester and noticed differences in header compression ratio and compression size of requests. What settings control Haproxy’s HTTP/2 compression ?

System is CentOS 7.4 64bit based with haproxy enabled multithreading with 6 threads on Xeon E3-1270v6 4C/8T server Vultr Baremetal with 10Gbps network. With 2x Vultr baremetal Xeon E3-1270v5 64GB ram Nginx backends. Full tests at https://community.centminmod.com/posts/59266/

haproxy -vv
HA-Proxy version 1.8.3-205f675 2017/12/30
Copyright 2000-2017 Willy Tarreau <willy@haproxy.org>

Build options :
  TARGET  = linux2628
  CPU     = native
  CC      = gcc
  CFLAGS  = -march=native -m64 -march=x86-64 -O2 -g
  OPTIONS = USE_LINUX_SPLICE=1 USE_LINUX_TPROXY=1 USE_ZLIB=1 USE_REGPARM=1 USE_THREAD=1 USE_OPENSSL=1 USE_LUA=1 USE_PCRE=1 USE_PCRE_JIT=1

Default settings :
  maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with OpenSSL version : OpenSSL 1.1.0g  2 Nov 2017
Running on OpenSSL version : OpenSSL 1.1.0g  2 Nov 2017
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2
Built with Lua version : Lua 5.3.4
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
Encrypted password support via crypt(3): yes
Built with multi-threading support.
Built with PCRE version : 8.41 2017-07-05
Running on PCRE version : 8.41 2017-07-05
PCRE library supports JIT : yes
Built with zlib version : 1.2.8
Running on zlib version : 1.2.8
Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with network namespace support.

Available polling systems :
      epoll : pref=300,  test result OK
       poll : pref=200,  test result OK
     select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

Available filters :
        [SPOE] spoe
        [COMP] compression
        [TRACE] trace

Haproxy 1.8.3 HTTP/2 h2load test 7x runs

Haproxy 1.8.3 HTTP/2 h2load stress test
4192.64 req/s 100% completed  status codes: 50000 2xx, 0 3xx, 0 4xx, 0 5xx  traffic: 102.51MB (107492141) total, 20.44MB (21434340) headers (space savings 8.92%), 80.73MB (84650000) data
43055.35 req/s 100% completed  status codes: 50000 2xx, 0 3xx, 0 4xx, 0 5xx  traffic: 102.51MB (107492162) total, 20.44MB (21434379) headers (space savings 8.92%), 80.73MB (84650000) data
43342.96 req/s 100% completed  status codes: 50000 2xx, 0 3xx, 0 4xx, 0 5xx  traffic: 102.51MB (107492323) total, 20.44MB (21434369) headers (space savings 8.92%), 80.73MB (84650000) data
4225.61 req/s 100% completed  status codes: 50000 2xx, 0 3xx, 0 4xx, 0 5xx  traffic: 102.51MB (107492273) total, 20.44MB (21434355) headers (space savings 8.92%), 80.73MB (84650000) data
4462.79 req/s 100% completed  status codes: 50000 2xx, 0 3xx, 0 4xx, 0 5xx  traffic: 102.51MB (107492048) total, 20.44MB (21434355) headers (space savings 8.92%), 80.73MB (84650000) data
4238.36 req/s 100% completed  status codes: 50000 2xx, 0 3xx, 0 4xx, 0 5xx  traffic: 102.51MB (107492414) total, 20.44MB (21434388) headers (space savings 8.92%), 80.73MB (84650000) data
4452.79 req/s 100% completed  status codes: 50000 2xx, 0 3xx, 0 4xx, 0 5xx  traffic: 102.51MB (107491991) total, 20.44MB (21434379) headers (space savings 8.92%), 80.73MB (84650000) data

Centmin Mod Nginx 1.13.8 HTTP/2 h2load test 7 runs with proxy_cache

with proxy_cache CentminMod.com Nginx 1.13.8 HTTP/2 h2load stress test
41114.53 req/s 100% completed  status codes: 50000 2xx, 0 3xx, 0 4xx, 0 5xx  traffic: 99.02MB (103825021) total, 17.41MB (18250521) headers (space savings 22.34%), 80.73MB (84650000) data
41366.07 req/s 100% completed  status codes: 50000 2xx, 0 3xx, 0 4xx, 0 5xx  traffic: 99.17MB (103985124) total, 17.56MB (18410624) headers (space savings 22.32%), 80.73MB (84650000) data
44522.57 req/s 100% completed  status codes: 50000 2xx, 0 3xx, 0 4xx, 0 5xx  traffic: 99.15MB (103970551) total, 17.54MB (18396051) headers (space savings 22.32%), 80.73MB (84650000) data
43232.74 req/s 100% completed  status codes: 50000 2xx, 0 3xx, 0 4xx, 0 5xx  traffic: 99.12MB (103938667) total, 17.51MB (18364167) headers (space savings 22.32%), 80.73MB (84650000) data
42330.59 req/s 100% completed  status codes: 50000 2xx, 0 3xx, 0 4xx, 0 5xx  traffic: 99.16MB (103973060) total, 17.55MB (18398560) headers (space savings 22.32%), 80.73MB (84650000) data
44525.14 req/s 100% completed  status codes: 50000 2xx, 0 3xx, 0 4xx, 0 5xx  traffic: 99.18MB (103998511) total, 17.57MB (18424011) headers (space savings 22.32%), 80.73MB (84650000) data
42880.43 req/s 100% completed  status codes: 50000 2xx, 0 3xx, 0 4xx, 0 5xx  traffic: 99.13MB (103949287) total, 17.52MB (18374787) headers (space savings 22.32%), 80.73MB (84650000) data

Centmin Mod Nginx 1.13.8 HTTP/2 h2load test 7 runs without proxy_cache

no proxy_cache CentminMod.com Nginx 1.13.8 HTTP/2 h2load stress test
3255.46 req/s 100% completed  status codes: 50000 2xx, 0 3xx, 0 4xx, 0 5xx  traffic: 99.16MB (103975000) total, 17.55MB (18400500) headers (space savings 22.20%), 80.73MB (84650000) data
3270.14 req/s 100% completed  status codes: 50000 2xx, 0 3xx, 0 4xx, 0 5xx  traffic: 99.16MB (103975000) total, 17.55MB (18400500) headers (space savings 22.20%), 80.73MB (84650000) data
3206.48 req/s 100% completed  status codes: 50000 2xx, 0 3xx, 0 4xx, 0 5xx  traffic: 99.16MB (103975000) total, 17.55MB (18400500) headers (space savings 22.20%), 80.73MB (84650000) data
3256.87 req/s 100% completed  status codes: 50000 2xx, 0 3xx, 0 4xx, 0 5xx  traffic: 99.16MB (103975000) total, 17.55MB (18400500) headers (space savings 22.20%), 80.73MB (84650000) data
3236.25 req/s 100% completed  status codes: 50000 2xx, 0 3xx, 0 4xx, 0 5xx  traffic: 99.16MB (103975000) total, 17.55MB (18400500) headers (space savings 22.20%), 80.73MB (84650000) data
3304.80 req/s 100% completed  status codes: 50000 2xx, 0 3xx, 0 4xx, 0 5xx  traffic: 99.16MB (103975000) total, 17.55MB (18400500) headers (space savings 22.20%), 80.73MB (84650000) data
3270.14 req/s 100% completed  status codes: 50000 2xx, 0 3xx, 0 4xx, 0 5xx  traffic: 99.16MB (103975000) total, 17.55MB (18400500) headers (space savings 22.20%), 80.73MB (84650000) data

h2load header space savings differences for Haproxy = 8.92% vs Nginx = 22+%

lukastribus · February 7, 2018, 11:26pm

Try increasing tune.h2.header-table-size.

eva2000 · February 8, 2018, 12:10am

thanks @lukastribus tried setting it at 8192 and 65536 bytes but no change in header space savings still 8.92%. Nginx defaults to http2_max_header_size = 16KB http://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_max_header_size, While Centmin Mod Nginx uses the following:

  http2_max_field_size 16k;
  http2_max_header_size 32k;

Using nghttp2 HTTP/2 tools compare HTTP/2 settings for

Centmin Mod Nginx 1.13.8 HTTP/2

nghttp -nv https://baremetal.domain.com:443/ | grep SETTINGS 
[  0.002] recv SETTINGS frame <length=18, flags=0x00, stream_id=0>
          [SETTINGS_MAX_CONCURRENT_STREAMS(0x03):128]
          [SETTINGS_INITIAL_WINDOW_SIZE(0x04):65536]
          [SETTINGS_MAX_FRAME_SIZE(0x05):16777215]
[  0.002] send SETTINGS frame <length=12, flags=0x00, stream_id=0>
          [SETTINGS_MAX_CONCURRENT_STREAMS(0x03):100]
          [SETTINGS_INITIAL_WINDOW_SIZE(0x04):65535]
[  0.002] send SETTINGS frame <length=0, flags=0x01, stream_id=0>
[  0.002] recv SETTINGS frame <length=0, flags=0x01, stream_id=0>

Haproxy 1.8.3 HTTP/2 default tune.h2.header-table-size = 4096 bytes

nghttp -nv https://baremetal.domain.com:444/ | grep SETTINGS
[  0.007] send SETTINGS frame <length=12, flags=0x00, stream_id=0>
          [SETTINGS_MAX_CONCURRENT_STREAMS(0x03):100]
          [SETTINGS_INITIAL_WINDOW_SIZE(0x04):65535]
[  0.007] recv SETTINGS frame <length=6, flags=0x00, stream_id=0>
          [SETTINGS_MAX_CONCURRENT_STREAMS(0x03):100]
[  0.007] recv SETTINGS frame <length=0, flags=0x01, stream_id=0>
[  0.007] send SETTINGS frame <length=0, flags=0x01, stream_id=0>

Haproxy 1.8.3 HTTP/2 adjusted tune.h2.header-table-size = 8192 bytes

nghttp -nv https://baremetal.domain.com:444/ | grep SETTINGS
[  0.002] send SETTINGS frame <length=12, flags=0x00, stream_id=0>
          [SETTINGS_MAX_CONCURRENT_STREAMS(0x03):100]
          [SETTINGS_INITIAL_WINDOW_SIZE(0x04):65535]
[  0.002] recv SETTINGS frame <length=12, flags=0x00, stream_id=0>
          [SETTINGS_HEADER_TABLE_SIZE(0x01):8192]
          [SETTINGS_MAX_CONCURRENT_STREAMS(0x03):100]
[  0.002] recv SETTINGS frame <length=0, flags=0x01, stream_id=0>
[  0.002] send SETTINGS frame <length=0, flags=0x01, stream_id=0>

no change in header space saved at 8.92%

/usr/local/bin/h2load -t4 -c5000 -m100 -n100000 -H 'Accept-Encoding: gzip' https://baremetal.domain.com:444/   
starting benchmark...
spawning thread #0: 1250 total client(s). 25000 total requests
spawning thread #1: 1250 total client(s). 25000 total requests
spawning thread #2: 1250 total client(s). 25000 total requests
spawning thread #3: 1250 total client(s). 25000 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-RSA-AES128-GCM-SHA256
Server Temp Key: ECDH P-256 256 bits
Application protocol: h2
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 14.82s, 6745.48 req/s, 13.84MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 205.12MB (215083384) total, 40.88MB (42868755) headers (space savings 8.92%), 161.46MB (169300000) data
                     min         max         mean         sd        +/- sd
time for request:    66.93ms      13.13s       1.91s    590.01ms    72.83%
time for connect:   139.66ms       2.03s       1.04s    496.19ms    58.04%
time to 1st byte:   390.78ms       4.56s       2.94s    948.24ms    60.60%
req/s           :       1.35       50.65        7.84        4.20    92.38%

Haproxy 1.8.3 HTTP/2 adjusted tune.h2.header-table-size = 65536 bytes maximum

nghttp -nv https://baremetal.domain.com:444/ | grep SETTINGS                                                 
[  0.003] send SETTINGS frame <length=12, flags=0x00, stream_id=0>
          [SETTINGS_MAX_CONCURRENT_STREAMS(0x03):100]
          [SETTINGS_INITIAL_WINDOW_SIZE(0x04):65535]
[  0.003] recv SETTINGS frame <length=12, flags=0x00, stream_id=0>
          [SETTINGS_HEADER_TABLE_SIZE(0x01):65536]
          [SETTINGS_MAX_CONCURRENT_STREAMS(0x03):100]
[  0.003] recv SETTINGS frame <length=0, flags=0x01, stream_id=0>
[  0.003] send SETTINGS frame <length=0, flags=0x01, stream_id=0>

Still no cahnge header space saved at 8.92%

/usr/local/bin/h2load -t4 -c5000 -m100 -n100000 -H 'Accept-Encoding: gzip' https://baremetal.domain.com:444/
starting benchmark...
spawning thread #0: 1250 total client(s). 25000 total requests
spawning thread #1: 1250 total client(s). 25000 total requests
spawning thread #2: 1250 total client(s). 25000 total requests
spawning thread #3: 1250 total client(s). 25000 total requests
TLS Protocol: TLSv1.2
Cipher: ECDHE-RSA-AES128-GCM-SHA256
Server Temp Key: ECDH P-256 256 bits
Application protocol: h2
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 12.44s, 8040.71 req/s, 16.49MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 205.12MB (215083226) total, 40.88MB (42868651) headers (space savings 8.92%), 161.46MB (169300000) data
                     min         max         mean         sd        +/- sd
time for request:   120.10ms      11.96s       1.93s    690.10ms    63.94%
time for connect:   105.87ms       1.80s    996.26ms    437.36ms    55.24%
time to 1st byte:   392.25ms       4.62s       2.91s       1.04s    59.30%
req/s           :       1.61       50.97        8.00        4.17    91.78%

Topic		Replies	Views
Haproxy 1.8 vs 2.7.3 Help!	11	872	February 28, 2023
Haproxy 1.8.9 CPU usage increased significantly Help!	7	1577	June 5, 2018
Haproxy 1.7.x 1.8.x 1.9.x tcp:receive crash Help!	1	573	August 2, 2018
HAProxy 1.8.18/19 occasional crashes with multi-threading enabled Help!	5	1283	March 8, 2019
Send-proxy not modifying some traffic with proxy ip/port details Help!	13	1710	July 8, 2019

Haproxy 1.8.3 HTTP/2 HPACK Compression Controls?

Related topics