I am having a chance to dive into it just now. I can log now the SSL version and cipher (thanks). It is important also to turn off global config “dontlog-normal”. If this is enabled, client cert checks are sometimes logged, sometimes NOT.
The new token (mentioned above) is working fine. When the old token is used page download is sometimes stuck for 10-15sec. See the logs. There are a tremendous “SSL handshake failure” messages. When connection is slowed down, there are a higher number of them.
Feb 9 12:36:26 localhost haproxy[5820]: 100.64.37.142:53063 [09/Feb/2021:12:36:26.509] admin_https_in/2: SSL handshake failure
Feb 9 12:36:27 localhost haproxy[5820]: 100.64.37.142:53054 [09/Feb/2021:12:36:22.944] admin_https_in~ frontend_admin/vik-t-vpfe01b 4429/0/1/1/4466 200 88443 - - ---- 6/6/0/0/0 0/0 {0,"/C=../L=.../organizationIdentifier=...","/C=../L=.../O=..."} "GET /js/jquery.js HTTP/1.1" ECDHE-RSA-AES128-SHA TLSv1.2
Feb 9 12:36:27 localhost haproxy[5820]: 100.64.37.142:53065 [09/Feb/2021:12:36:27.462] admin_https_in/2: SSL handshake failure
Feb 9 12:36:28 localhost haproxy[5820]: 100.64.37.142:53058 [09/Feb/2021:12:36:23.846] admin_https_in~ frontend_admin/vik-t-vpfe01b 4722/0/2/0/4724 200 5748 - - ---- 6/6/0/0/0 0/0 {0,"/C=../L=.../organizationIdentifier=...","/C=../L=.../O=..."} "GET /js/micromodal/micromodal.min.js HTTP/1.1" ECDHE-RSA-AES128-SHA TLSv1.2
Feb 9 12:36:28 localhost haproxy[5820]: 100.64.37.142:53067 [09/Feb/2021:12:36:28.590] admin_https_in/2: SSL handshake failure
Feb 9 12:36:29 localhost haproxy[5820]: 100.64.37.142:53057 [09/Feb/2021:12:36:23.845] admin_https_in~ frontend_admin/vik-t-vpfe01b 5317/0/1/1/5319 200 1267 - - ---- 6/6/0/0/0 0/0 {0,"/C=../L=.../organizationIdentifier=...","/C=../L=.../O=..."} "GET /js/tooltips/tooltips.css HTTP/1.1" ECDHE-RSA-AES128-SHA TLSv1.2
Feb 9 12:36:30 localhost haproxy[5820]: 100.64.37.142:53060 [09/Feb/2021:12:36:25.072] admin_https_in~ frontend_admin/vik-t-vpfe01b 4990/0/2/1/5052 200 140153 - - ---- 5/5/0/0/0 0/0 {0,"/C=../L=.../organizationIdentifier=...","/C=../L=.../O=..."} "GET /js/scriptaculous/lib/prototype.js HTTP/1.1" ECDHE-RSA-AES128-SHA TLSv1.2
Feb 9 12:36:30 localhost haproxy[5820]: 100.64.37.142:53069 [09/Feb/2021:12:36:30.162] admin_https_in/2: SSL handshake failure
Feb 9 12:36:30 localhost haproxy[5820]: 100.64.37.142:53062 [09/Feb/2021:12:36:25.782] admin_https_in~ frontend_admin/vik-t-vpfe01b 5188/0/2/1/5191 200 3156 - - ---- 5/5/0/0/0 0/0 {0,"/C=../L=.../organizationIdentifier=...","/C=../L=.../O=..."} "GET /js/scriptaculous/src/scriptaculous.js HTTP/1.1" ECDHE-RSA-AES128-SHA TLSv1.2
Feb 9 12:36:31 localhost haproxy[5820]: 100.64.37.142:53071 [09/Feb/2021:12:36:30.998] admin_https_in/2: SSL handshake failure
Feb 9 12:36:31 localhost haproxy[5820]: 100.64.37.142:53072 [09/Feb/2021:12:36:31.004] admin_https_in/2: SSL handshake failure
Feb 9 12:36:31 localhost haproxy[5820]: 100.64.37.142:53073 [09/Feb/2021:12:36:31.020] admin_https_in/2: SSL handshake failure
Feb 9 12:36:31 localhost haproxy[5820]: 100.64.37.142:53074 [09/Feb/2021:12:36:31.020] admin_https_in/2: SSL handshake failure
Feb 9 12:36:31 localhost haproxy[5820]: 100.64.37.142:53075 [09/Feb/2021:12:36:31.038] admin_https_in/2: SSL handshake failure
Feb 9 12:36:31 localhost haproxy[5820]: 100.64.37.142:53076 [09/Feb/2021:12:36:31.039] admin_https_in/2: SSL handshake failure
Feb 9 12:36:31 localhost haproxy[5820]: 100.64.37.142:53077 [09/Feb/2021:12:36:31.055] admin_https_in/2: SSL handshake failure
Feb 9 12:36:31 localhost haproxy[5820]: 100.64.37.142:53064 [09/Feb/2021:12:36:26.525] admin_https_in~ frontend_admin/vik-t-vpfe01b 5322/0/1/1/5326 200 38699 - - ---- 6/6/0/0/0 0/0 {0,"/C=../L=.../organizationIdentifier=...","/C=../L=.../O=..."} "GET /js/scriptaculous/src/effects.js HTTP/1.1" ECDHE-RSA-AES128-SHA TLSv1.2
SSL handshake failures also happen with the new hw token, but it works fine. This log is made with the new hw token, I can see no relevant difference, even the TLS version and ciphers are the same.
Feb 9 12:48:33 localhost haproxy[5820]: 100.64.37.142:53367 [09/Feb/2021:12:48:33.360] admin_https_in/2: SSL handshake failure
Feb 9 12:48:33 localhost haproxy[5820]: 100.64.37.142:53366 [09/Feb/2021:12:48:33.360] admin_https_in/2: SSL handshake failure
Feb 9 12:48:33 localhost haproxy[5820]: 100.64.37.142:53368 [09/Feb/2021:12:48:33.386] admin_https_in/2: SSL handshake failure
Feb 9 12:48:41 localhost haproxy[5820]: 100.64.37.142:53371 [09/Feb/2021:12:48:36.615] admin_https_in~ frontend_admin/vik-t-vpfe01a 4735/0/2/4/4741 302 940 - - ---- 1/1/0/0/0 0/0 {0,"/C=...","/C=..."} "GET / HTTP/1.1" ECDHE-RSA-AES128-SHA TLSv1.2
Feb 9 12:48:41 localhost haproxy[5820]: 100.64.37.142:53372 [09/Feb/2021:12:48:41.372] admin_https_in/2: SSL handshake failure
Feb 9 12:48:41 localhost haproxy[5820]: 100.64.37.142:53373 [09/Feb/2021:12:48:41.388] admin_https_in~ frontend_admin/vik-t-vpfe01b 389/0/2/8/400 200 3373 - - ---- 1/1/0/0/0 0/0 {0,"/C=...","/C=..."} "GET /login?forward=%2F HTTP/1.1" ECDHE-RSA-AES128-SHA TLSv1.2
Feb 9 12:48:41 localhost haproxy[5820]: 100.64.37.142:53375 [09/Feb/2021:12:48:41.881] admin_https_in/2: SSL handshake failure
Feb 9 12:48:41 localhost haproxy[5820]: 100.64.37.142:53374 [09/Feb/2021:12:48:41.881] admin_https_in/2: SSL handshake failure
Feb 9 12:48:41 localhost haproxy[5820]: 100.64.37.142:53376 [09/Feb/2021:12:48:41.887] admin_https_in/2: SSL handshake failure
Feb 9 12:48:41 localhost haproxy[5820]: 100.64.37.142:53378 [09/Feb/2021:12:48:41.887] admin_https_in/2: SSL handshake failure
Feb 9 12:48:41 localhost haproxy[5820]: 100.64.37.142:53377 [09/Feb/2021:12:48:41.888] admin_https_in/2: SSL handshake failure
Feb 9 12:48:41 localhost haproxy[5820]: 100.64.37.142:53379 [09/Feb/2021:12:48:41.888] admin_https_in/2: SSL handshake failure
Feb 9 12:48:41 localhost haproxy[5820]: 100.64.37.142:53383 [09/Feb/2021:12:48:41.913] admin_https_in/2: SSL handshake failure
Feb 9 12:48:41 localhost haproxy[5820]: 100.64.37.142:53381 [09/Feb/2021:12:48:41.912] admin_https_in/2: SSL handshake failure
Feb 9 12:48:41 localhost haproxy[5820]: 100.64.37.142:53380 [09/Feb/2021:12:48:41.912] admin_https_in/2: SSL handshake failure
Feb 9 12:48:41 localhost haproxy[5820]: 100.64.37.142:53382 [09/Feb/2021:12:48:41.912] admin_https_in/2: SSL handshake failure
Feb 9 12:48:42 localhost haproxy[5820]: 100.64.37.142:53384 [09/Feb/2021:12:48:41.913] admin_https_in~ frontend_admin/vik-t-vpfe01b 439/0/1/1/441 200 2924 - - ---- 6/6/0/0/0 0/0 {0,"/C=...","/C=..."} "GET /js/analytics/dygraph.css HTTP/1.1" ECDHE-RSA-AES128-SHA TLSv1.2
Feb 9 12:48:42 localhost haproxy[5820]: 100.64.37.142:53390 [09/Feb/2021:12:48:42.372] admin_https_in~ frontend_admin/vik-t-vpfe01b 9/0/1/1/11 200 2960 - - ---- 6/6/0/0/0 0/0 {0,"/C=...","/C=..."} "GET /js/micromodal/micromodal.css HTTP/1.1" ECDHE-RSA-AES128-SHA TLSv1.2
Feb 9 12:48:42 localhost haproxy[5820]: 100.64.37.142:53391 [09/Feb/2021:12:48:42.400] admin_https_in~ frontend_admin/vik-t-vpfe01b 7/0/1/1/10 200 3596 - - ---- 6/6/0/0/0 0/0 {0,"/C=...","/C=..."} "GET /js/croppie/croppie.min.css HTTP/1.1" ECDHE-RSA-AES128-SHA TLSv1.2
Feb 9 12:48:42 localhost haproxy[5820]: 100.64.37.142:53392 [09/Feb/2021:12:48:42.425] admin_https_in~ frontend_admin/vik-t-vpfe01b 8/0/2/2/12 200 3551 - - ---- 6/6/0/0/0 0/0 {0,"/C=...","/C=..."} "GET /contents/admincss?_v20210125 HTTP/1.1" ECDHE-RSA-AES128-SHA TLSv1.2
Feb 9 12:48:42 localhost haproxy[5820]: 100.64.37.142:53393 [09/Feb/2021:12:48:42.453] admin_https_in~ frontend_admin/vik-t-vpfe01b 7/0/1/1/9 200 1267 - - ---- 6/6/0/0/0 0/0 {0,"/C=...","/C=..."} "GET /js/tooltips/tooltips.css HTTP/1.1" ECDHE-RSA-AES128-SHA TLSv1.2
Feb 9 12:48:42 localhost haproxy[5820]: 100.64.37.142:53394 [09/Feb/2021:12:48:42.479] admin_https_in/2: SSL handshake failure
Feb 9 12:48:42 localhost haproxy[5820]: 100.64.37.142:53395 [09/Feb/2021:12:48:42.492] admin_https_in/2: SSL handshake failure
Feb 9 12:48:42 localhost haproxy[5820]: 100.64.37.142:53396 [09/Feb/2021:12:48:42.509] admin_https_in/2: SSL handshake failure
Feb 9 12:48:42 localhost haproxy[5820]: 100.64.37.142:53397 [09/Feb/2021:12:48:42.528] admin_https_in/2: SSL handshake failure
Feb 9 12:48:42 localhost haproxy[5820]: 100.64.37.142:53398 [09/Feb/2021:12:48:42.544] admin_https_in/2: SSL handshake failure
Feb 9 12:48:42 localhost haproxy[5820]: 100.64.37.142:53399 [09/Feb/2021:12:48:42.561] admin_https_in/2: SSL handshake failure
Feb 9 12:48:42 localhost haproxy[5820]: 100.64.37.142:53400 [09/Feb/2021:12:48:42.578] admin_https_in/2: SSL handshake failure
Feb 9 12:48:42 localhost haproxy[5820]: 100.64.37.142:53401 [09/Feb/2021:12:48:42.594] admin_https_in/2: SSL handshake failure
Feb 9 12:48:42 localhost haproxy[5820]: 100.64.37.142:53402 [09/Feb/2021:12:48:42.612] admin_https_in/2: SSL handshake failure
Feb 9 12:48:42 localhost haproxy[5820]: 100.64.37.142:53385 [09/Feb/2021:12:48:41.927] admin_https_in~ frontend_admin/vik-t-vpfe01b 821/0/1/0/852 200 88443 - - ---- 6/6/0/0/0 0/0 {0,"/C=...","/C=..."} "GET /js/jquery.js HTTP/1.1" ECDHE-RSA-AES128-SHA TLSv1.2
Feb 9 12:48:42 localhost haproxy[5820]: 100.64.37.142:53404 [09/Feb/2021:12:48:42.812] admin_https_in~ frontend_admin/vik-t-vpfe01b 12/0/1/1/14 200 5748 - - ---- 6/6/0/0/0 0/0 {0,"/C=...","/C=..."} "GET /js/micromodal/micromodal.min.js HTTP/1.1" ECDHE-RSA-AES128-SHA TLSv1.2
Feb 9 12:48:42 localhost haproxy[5820]: 100.64.37.142:53405 [09/Feb/2021:12:48:42.842] admin_https_in~ frontend_admin/vik-t-vpfe01b 9/0/1/1/77 200 140153 - - ---- 6/6/0/0/0 0/0 {0,"/C=...","/C=..."} "GET /js/scriptaculous/lib/prototype.js HTTP/1.1" ECDHE-RSA-AES128-SHA TLSv1.2
Feb 9 12:48:42 localhost haproxy[5820]: 100.64.37.142:53406 [09/Feb/2021:12:48:42.951] admin_https_in~ frontend_admin/vik-t-vpfe01b 6/0/1/1/8 200 3156 - - ---- 6/6/0/0/0 0/0 {0,"/C=...","/C=..."} "GET /js/scriptaculous/src/scriptaculous.js HTTP/1.1" ECDHE-RSA-AES128-SHA TLSv1.2
Feb 9 12:48:42 localhost haproxy[5820]: 100.64.37.142:53407 [09/Feb/2021:12:48:42.987] admin_https_in/2: SSL handshake failure
Feb 9 12:48:43 localhost haproxy[5820]: 100.64.37.142:53387 [09/Feb/2021:12:48:41.943] admin_https_in~ frontend_admin/vik-t-vpfe01b 1216/0/1/1/1218 200 5489 - - ---- 6/6/0/0/0 0/0 {0,"/C=...","/C=..."} "GET /js/tooltips/tooltips.js HTTP/1.1" ECDHE-RSA-AES128-SHA TLSv1.2
Feb 9 12:48:43 localhost haproxy[5820]: 100.64.37.142:53409 [09/Feb/2021:12:48:43.211] admin_https_in~ frontend_admin/vik-t-vpfe01b 7/0/1/1/10 200 23998 - - ---- 6/6/0/0/0 0/0 {0,"/C=...","/C=..."} "GET /js/croppie/croppie.min.js HTTP/1.1" ECDHE-RSA-AES128-SHA TLSv1.2
Feb 9 12:48:43 localhost haproxy[5820]: 100.64.37.142:53410 [09/Feb/2021:12:48:43.247] admin_https_in~ frontend_admin/vik-t-vpfe01b 6/0/2/0/10 200 46233 - - ---- 6/6/0/0/0 0/0 {0,"/C=...","/C=..."} "GET /js/springboard.js?_v20210125 HTTP/1.1" ECDHE-RSA-AES128-SHA TLSv1.2
Feb 9 12:48:43 localhost haproxy[5820]: 100.64.37.142:53411 [09/Feb/2021:12:48:43.302] admin_https_in~ frontend_admin/vik-t-vpfe01b 7/0/2/1/10 200 5056 - - ---- 6/6/0/0/0 0/0 {0,"/C=...","/C=..."} "GET /js/custom.js?_v20210125 HTTP/1.1" ECDHE-RSA-AES128-SHA TLSv1.2
Feb 9 12:48:43 localhost haproxy[5820]: 100.64.37.142:53412 [09/Feb/2021:12:48:43.327] admin_https_in/2: SSL handshake failure
Feb 9 12:48:43 localhost haproxy[5820]: 100.64.37.142:53413 [09/Feb/2021:12:48:43.342] admin_https_in/2: SSL handshake failure
Feb 9 12:48:43 localhost haproxy[5820]: 100.64.37.142:53414 [09/Feb/2021:12:48:43.364] admin_https_in/2: SSL handshake failure
Feb 9 12:48:43 localhost haproxy[5820]: 100.64.37.142:53415 [09/Feb/2021:12:48:43.385] admin_https_in/2: SSL handshake failure
Feb 9 12:48:43 localhost haproxy[5820]: 100.64.37.142:53416 [09/Feb/2021:12:48:43.411] admin_https_in/2: SSL handshake failure
Feb 9 12:48:43 localhost haproxy[5820]: 100.64.37.142:53417 [09/Feb/2021:12:48:43.433] admin_https_in/2: SSL handshake failure
Feb 9 12:48:43 localhost haproxy[5820]: 100.64.37.142:53386 [09/Feb/2021:12:48:41.942] admin_https_in~ frontend_admin/vik-t-vpfe01b 1628/0/1/1/1630 200 1558 - - ---- 6/6/0/0/0 0/0 {0,"/C=...","/C=..."} "GET /js/side-bar.js HTTP/1.1" ECDHE-RSA-AES128-SHA TLSv1.2
Feb 9 12:48:43 localhost haproxy[5820]: 100.64.37.142:53419 [09/Feb/2021:12:48:43.589] admin_https_in~ frontend_admin/vik-t-vpfe01b 8/0/1/1/10 200 4971 - - ---- 6/6/0/0/0 0/0 {0,"/C=...","/C=..."} "GET /js/scriptaculous/src/builder.js HTTP/1.1" ECDHE-RSA-AES128-SHA TLSv1.2
Feb 9 12:48:43 localhost haproxy[5820]: 100.64.37.142:53388 [09/Feb/2021:12:48:41.949] admin_https_in~ frontend_admin/vik-t-vpfe01b 2021/0/2/1/2026 200 38699 - - ---- 5/5/0/0/0 0/0 {0,"/C=...","/C=..."} "GET /js/scriptaculous/src/effects.js HTTP/1.1" ECDHE-RSA-AES128-SHA TLSv1.2
Feb 9 12:48:44 localhost haproxy[5820]: 100.64.37.142:53389 [09/Feb/2021:12:48:41.949] admin_https_in~ frontend_admin/vik-t-vpfe01b 2393/0/1/1/2396 200 31283 - - ---- 4/4/0/0/0 0/0 {0,"/C=...","/C=..."} "GET /js/scriptaculous/src/dragdrop.js HTTP/1.1" ECDHE-RSA-AES128-SHA TLSv1.2
Feb 9 12:48:44 localhost haproxy[5820]: 100.64.37.142:53403 [09/Feb/2021:12:48:42.630] admin_https_in~ frontend_admin/vik-t-vpfe01b 2106/0/1/1/2109 200 35014 - - ---- 3/3/0/0/0 0/0 {0,"/C=...","/C=..."} "GET /js/scriptaculous/src/controls.js HTTP/1.1" ECDHE-RSA-AES128-SHA TLSv1.2
Feb 9 12:48:45 localhost haproxy[5820]: 100.64.37.142:53408 [09/Feb/2021:12:48:43.000] admin_https_in~ frontend_admin/vik-t-vpfe01b 2130/0/1/1/2133 200 10391 - - ---- 2/2/0/0/0 0/0 {0,"/C=...","/C=..."} "GET /js/scriptaculous/src/slider.js HTTP/1.1" ECDHE-RSA-AES128-SHA TLSv1.2
Feb 9 12:48:45 localhost haproxy[5820]: 100.64.37.142:53420 [09/Feb/2021:12:48:45.157] admin_https_in/2: SSL handshake failure
Feb 9 12:48:45 localhost haproxy[5820]: 100.64.37.142:53418 [09/Feb/2021:12:48:43.460] admin_https_in~ frontend_admin/vik-t-vpfe01b 2046/0/2/1/2050 200 2684 - - ---- 2/2/0/0/0 0/0 {0,"/C=...","/C=..."} "GET /js/scriptaculous/src/sound.js HTTP/1.1" ECDHE-RSA-AES128-SHA TLSv1.2
Feb 9 12:48:45 localhost haproxy[5820]: 100.64.37.142:53422 [09/Feb/2021:12:48:45.533] admin_https_in/2: SSL handshake failure
Feb 9 12:48:45 localhost haproxy[5820]: 100.64.37.142:53421 [09/Feb/2021:12:48:45.171] admin_https_in~ frontend_admin/vik-t-vpfe01b 749/0/1/1/752 200 30471 - - ---- 2/2/0/0/0 0/0 {0,"/C=...","/C=..."} "GET /js/clonefish.js HTTP/1.1" ECDHE-RSA-AES128-SHA TLSv1.2
Feb 9 12:48:45 localhost haproxy[5820]: 100.64.37.142:53424 [09/Feb/2021:12:48:45.961] admin_https_in/2: SSL handshake failure
Feb 9 12:48:46 localhost haproxy[5820]: 100.64.37.142:53423 [09/Feb/2021:12:48:45.551] admin_https_in~ frontend_admin/vik-t-vpfe01b 755/0/2/6/763 302 814 - - ---- 2/2/0/0/0 0/0 {0,"/C=...","/C=..."} "GET /index/ping HTTP/1.1" ECDHE-RSA-AES128-SHA TLSv1.2
Feb 9 12:48:46 localhost haproxy[5820]: 100.64.37.142:53426 [09/Feb/2021:12:48:46.329] admin_https_in/2: SSL handshake failure
@lukastribus would be grateful if any additional hints… thanks, Lukas.