Remove the trailing slash

Help!
1

Hello,

We would like to remove the trailing slash from the URLs. E.g. www.example.com/stuff/ redirects to www.example.com/stuff

Have tried using reqrep ^(.*?)[/]$ \1 (and variations of it) but that seems to cause an infinite redirect.

Any help is appreciated.

Thanks!

2

Hi,

I think the problem is that the web browser will keep adding the trailing slash back in because, in reality, you are requesting “www.example.com/stuff/sompage.html”.

It’s just showing you whatever default document is in that directory when you request “www.example.com/stuff/”

3

Hi,

Thanks for the response, but I can request the URL either with or without the slash, so it doesn’t seem to be a default document thing. I can successfully rewrite the URL on the IIS backend, but not all our backends are IIS.

Thanks

4

I might be slightly wrong but can you repeat a test for me, If I setup a simple webserver and haproxy configuration and apply a rule like:

acl rule_1 path_end /
redirect location http://172.16.202.111/stuff code 301 if rule_1

I’d get a redirect loop. If I remove the rule, press F12 when using my browser and check what’s getting requested for “http://172.16.202.111/stuff” I’d see something like

HTTP/1.1 301 Moved Permanently
Content-Length: 236
Content-Type: text/html; charset=iso-8859-1
Date: Fri, 30 Mar 2018 15:44:37 GMT
Location: http://172.16.202.111/stuff/

Now I don’t have any redirects setup to do that, I think it’s just that the file “stuff” doesn’t exist so it’s sending me to the folder “stuff/” using a redirect.

Maybe when you do it on your webserver it disables this redirect behavior… Do you see a redirect when examining the request in the same way with no redirects configured as I do?

5

Thanks for looking into it.

If I have that rule disabled, I just get a 200 response (whether the path ends with / or not). With the redirect loop, it seems the redirect is keeping the trailing slash, and I can’t seem to find a way to get the reqrep to work properly to remove it.

6

reqrep is rewriting the HTTP request. It does not redirect at all.

Check what happens when you request directly at your backend (without haproxy involved).

curl -vv www.example.com/stuff

Does it redirect you to the URI with the trailing slash www.example.com/stuff/? If haproxy redirects to remove the trailing slash and your backend redirects to add a trailing slash, endless loops is exactly what will happen.

Like I said reqrep does not redirect, it rewrites the request. Use the redirect directive instead.

7

Ah, so what you say about the rewrite makes sense, thanks for that.

Looking further in the documentation at the redirect options, I see there is an option to “append-slash” but not one to drop it. Perhaps there is an option to use a regex in the redirect rules that I’m missing?

Thanks again

8

So I’ve been trying to use regsub with the http-request redirect command in the config:

http-request redirect code 301 location https://www.example.com%[capture.req.uri,regsub(/$,)] if { path_end / }
But it is still causing the redirect loop.

9

Because you also redirect /, which can’t be redirected as there is no shorter path than /.

Only do this when path_len is greater than 1:

http-request redirect code 301 location https://www.example.com%[capture.req.uri,regsub(/$,)] if { path_end / } { path_len gt 1 }
10

Thanks again Lukas. That fixes the redirect loop.

We’re ending up back at www.example.com though, instead of www.example.com/stuff, I’m going to keep digging but I think we’re on the right track.

11

I think haproxy does exactly what you want. The question is whether the backend works against this (maybe the backend sees /stuff - doesn’t recognize it and redirects to /

12

Thanks Lukas,

I’ve tried this rule against a couple backends, and the pages will load whether they are requested with the trailing slash or not, so it doesn’t appear to be a matter of the backend not recognizing it.

13

Well the redirection comes from somewhere. Use curl -v to find out what happens exactly.

14

It does a 301 to the host and drops the path:

GET /stuff/ HTTP/1.1
Host: www.example.com
User-Agent: curl/7.45.0
Accept: /

< HTTP/1.1 301 Moved Permanently
< Content-length: 0
< Location: http://www.example.com
< Connection: close
<

  • Closing connection 0
15

Can you show me the entire configuration you are using and the output of haproxy -vv please?

16

Thanks for your help so far, I’ve setup the config on a new system so just this host is configured for testing:

$ haproxy -vv
HA-Proxy version 1.7.8 2017/07/07
Copyright 2000-2017 Willy Tarreau willy@haproxy.org

Build options :
TARGET = linux2628
CPU = generic
CC = gcc
CFLAGS = -O2 -g -fno-strict-aliasing -Wdeclaration-after-statement -fwrapv
OPTIONS =

Default settings :
maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Encrypted password support via crypt(3): yes
Built without compression support (neither USE_ZLIB nor USE_SLZ are set)
Compression algorithms supported : identity(“identity”)
Built without OpenSSL support (USE_OPENSSL not set)
Built without PCRE support (using libc’s regex instead)
Built without Lua support
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND

Available polling systems :
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use epoll.

Available filters :
[COMP] compression
[TRACE] trace
[SPOE] spoe

The config file:
#---------------------------------------------------------------------

Global settings

#---------------------------------------------------------------------
global
# to have these messages end up in /var/log/haproxy.log you will
# need to:
#
# 1) configure syslog to accept network log events. This is done
# by adding the ‘-r’ option to the SYSLOGD_OPTIONS in
# /etc/sysconfig/syslog
#
# 2) configure local2 events to go to the /var/log/haproxy.log
# file. A line like the following can be added to
# /etc/sysconfig/syslog
#
# local2.* /var/log/haproxy.log
#
log 127.0.0.1 local2

chroot      /var/lib/haproxy
pidfile     /var/run/haproxy.pid
maxconn     4000

user haproxy

group haproxy

daemon
tune.ssl.default-dh-param 2048

# turn on stats unix socket
stats socket /var/lib/haproxy/stats

#---------------------------------------------------------------------

common defaults that all the ‘listen’ and ‘backend’ sections will

use if not designated in their block

#---------------------------------------------------------------------
defaults
mode http
log global
option httplog
option dontlognull
option http-server-close
option forwardfor except 127.0.0.0/8
option redispatch
retries 3
timeout http-request 30s
timeout queue 1m
timeout connect 30s
timeout client 1m
timeout server 1m
timeout http-keep-alive 30s
timeout check 30s
maxconn 3000

frontend localnodes
bind *:80
mode http
option forwardfor except 127.0.0.0/8
capture request header host len 35
capture request header X-Forwarded-For len 50
capture request header Front-End-HTTPS len 50
capture request header User-agent len 225

http-request redirect code 301 location http://%[hdr(host)]%[url,regsub(/$,)] if { hdr(host) -i www.example.com } { path_end / } { path_len gt 1 }

acl examplecom hdr(host) -i www.example.com

use_backend examplecom if examplecom

backend examplecom
balance leastconn
option httpclose
option forwardfor
http-request add-header X-CLIENT-IP %[src]
cookie JSESSIOND prefix
server e0 192.168.16.207:6560 cookie A check
server e1 192.168.16.217:6560 cookie A check

Here are the log entries:

With the redirect commented out I can request a page with the slash or without:
Apr 9 12:46:04 localhost haproxy[1309]: 192.168.16.60:57762 [09/Apr/2018:12:46:03.902] localnodes examplecom/e0 11/0/0/301/312 200 19804 - - --NN 0/0/0/0/0 0/0 {www.example.com|||Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:59.0) Gecko/20100101 Firefox/59.0} “GET /privacy-policy HTTP/1.1”
Apr 9 12:46:10 localhost haproxy[1309]: 192.168.16.60:57772 [09/Apr/2018:12:46:09.875] localnodes examplecom/e1 12/0/0/271/283 200 19804 - - --NN 0/0/0/0/0 0/0 {www.example.com|||Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:59.0) Gecko/20100101 Firefox/59.0} “GET /privacy-policy/ HTTP/1.1”

with the redirect active we get a 301 to ‘/’:
Apr 9 12:49:03 localhost haproxy[1358]: 192.168.16.60:57908 [09/Apr/2018:12:49:03.583] localnodes localnodes/ 22/-1/-1/-1/22 301 106 - - LR-- 0/0/0/0/0 0/0 {www.example.com|||Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:59.0) Gecko/20100101 Firefox/59.0} “GET /privacy-policy/ HTTP/1.1”
Apr 9 12:49:04 localhost haproxy[1358]: 192.168.16.60:57909 [09/Apr/2018:12:49:03.626] localnodes examplecom/e0 21/0/1/783/807 200 58964 - - --NN 0/0/0/0/0 0/0 {www.example.com|||Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:59.0) Gecko/20100101 Firefox/59.0} “GET / HTTP/1.1”

17

Your configuration works perfectly for me:

C:\Users\lukas>curl -v --resolve www.example.com:80:10.0.0.33 www.example.com/privacy-policy/
* Added www.example.com:80:10.0.0.33 to DNS cache
* Hostname www.example.com was found in DNS cache
*   Trying 10.0.0.33...
* Connected to www.example.com (10.0.0.33) port 80 (#0)
> GET /privacy-policy/ HTTP/1.1
> Host: www.example.com
> User-Agent: curl/7.48.0
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
< Content-length: 0
< Location: http://www.example.com/privacy-policy
<
* Connection #0 to host www.example.com left intact

Kill all haproxy processes manually and restart it. I think you may have an old haproxy instances answering requests with an old configuration.

Also, try using noreuseport in the global section if in doubt, and see if it reloads successfully.

18

Thanks Lukas, did you use a copy and paste of our config?

I still seem to have the same problem even after restarting. Are you also on 1.7?

19

Yes I compiled 1.7.8 and used your config for this.

But wait, something is not right here.

In your configuration, you have the option tune.ssl.default-dh-param 2048, however you have not compiled haproxy with OpenSSL support (Built without OpenSSL support (USE_OPENSSL not set)) as seen in haproxy -vv.

However, with this configuration and without OpenSSL support, haproxy 1.7.8 would reject the configuration with:

[ALERT] 098/234840 (1005) : parsing [../cert/removetrailingslash.cfg:10] : unknown keyword 'tune.ssl.default-dh-param' in 'global' section
[ALERT] 098/234840 (1005) : Error(s) found in configuration file : ../cert/removetrailingslash.cfg
[ALERT] 098/234840 (1005) : Fatal errors found in configuration.

Can you confirm you are using the configuration posted above, with the haproxy executable you used to generate the -vv output?

Furthermore, can you:

  • stop haproxy completely
  • double check that no haproxy process are running anymore
  • run a config check with haproxy -f /path/to/haproxy.cfg
  • run haproxy manually in debug mode (-d) and without reuseport (-dR): haproxy -f /path/to/haproxy.cfg -d -dR
  • try the request again

Of course, do it in a closed environment, not with production traffic on it.

20

Well you’ve saved the day:

C:\Windows\System32>curl -v http://www.example.com/privacy-policy/

  • Trying 192.168.16.21…
  • Connected to www.example.com (192.168.16.21) port 80 (#0)

GET /privacy-policy/ HTTP/1.1
Host: www.example.com
User-Agent: curl/7.45.0
Accept: /

< HTTP/1.1 301 Moved Permanently
< Content-length: 0
< Location: http://www.example.com/privacy-policy
<

So this worked after a fresh install with OpenSSL support. I’m not sure if I can add OpenSSL support to an existing install (that’s a different problem I can look into).

Thanks for all your help!