# IP routing with haproxy tcp for ftp

**URL:** <https://discourse.haproxy.org/t/ip-routing-with-haproxy-tcp-for-ftp/643>\
**Category:** Help!\
**Created:** [September 8, 2016, 12:27pm UTC](https://discourse.haproxy.org/t/ip-routing-with-haproxy-tcp-for-ftp/643 "2016-09-08T12:27:55Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![ebourgui](https://avatars.discourse-cdn.com/v4/letter/e/b3f665/32.png) [@ebourgui](https://discourse.haproxy.org/u/ebourgui)\
**Post date:** [September 8, 2016, 12:27pm UTC](https://discourse.haproxy.org/t/ip-routing-with-haproxy-tcp-for-ftp/643/1 "2016-09-08T12:27:55Z")

</div>

Hi guys,

My configuration:

server1  
ha proxy 1.5.14 - centos 7  
public ip eth0 : 1.2.3.4  
public ip eth0:0 : 5.6.7.8 (ovh ip failover which can point to another haproxy server when failover occured)  
||  
server3  
vsftp server 3.0.2 - centos7  
public ip eth0: 9.10.11.12

Only public addresses. server1 and server3 have backups in a pacemaker cluster (active / passive) : server2 (haproxy backup) and server4 (ftp backup)

When I connect in passive mode to server3 : no problem.  
When I connect in passive mode to server1 (on 5.6.7.8):

```
ftp *****.com
Connected to *****.com (5.6.7.8).
220 Welcome to FTP service.
Name ( ******.com:root): toto
331 Please specify the password.
Password:
230 Login successful.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> ls
227 Entering Passive Mode (9,10,11,12,198,124).
ftp: connect: Connexion terminée par expiration du délai d'attente
=> in english : timeout

```

I take a look at /var/log/messages on the client and see  
`Sep 8 12:26:47 *****kernel: [11686946.541337] Firewall: *TCP_OUT Blocked* IN= OUT=eth2 SRC=******* DST=9.10.11.12 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=45793 DF PROTO=TCP SPT=52258 DPT=50812 WINDOW=29200 RES=0x00 SYN URGP=0 UID=0 GID=0`

The client try to connect directly to server3 and not to server1. I think that the packets are blocked because packets should be iptables RELATED to server1 and not to server3 to pass the firewall.  
I don’t want to change iptables output rules because it concerns a lot of clients.  
Is there a way to respond to server1 and not to server3 (or another way to work properly)?

server1 : haproxy.cfg  
…  
listen ftplb 5.6.7.8:21, 5.6.7.8:50000-50999  
mode tcp  
option tcplog  
server server3 9.10.11.12 check port 21  
server server4 13.14.15.16 check port 21

server1 et server3 : iptables  
`-A INPUT -p tcp -i eth0 -m multiport --dport 21,50000:50999 -m conntrack --ctstate NEW,ESTABLISHED,RELATED -j ACCEPT`

server3 vsftpd.conf

```
anonymous_enable=NO
local_enable=YES
write_enable=YES
local_umask=022
dirmessage_enable=YES
xferlog_enable=YES
xferlog_std_format=YES
chroot_local_user=YES
listen=YES
pam_service_name=vsftpd
userlist_enable=YES
tcp_wrappers=YES
allow_writeable_chroot=YES
ftpd_banner=Welcome to FTP service.
pasv_enable=YES
pasv_promiscuous=YES
pasv_min_port=50000
pasv_max_port=50999
dual_log_enable=YES
log_ftp_protocol=YES
port_enable=YES
port_promiscuous=YES
connect_from_port_20=NO
seccomp_sandbox=NO

```

Any idea?

Regards,  
Eric

---

<div class="post-metadata">

**Author:** ![lukastribus](https://avatars.discourse-cdn.com/v4/letter/l/7ea924/32.png) [@lukastribus](https://discourse.haproxy.org/u/lukastribus)\
**Post date:** [September 9, 2016, 7:24am UTC](https://discourse.haproxy.org/t/ip-routing-with-haproxy-tcp-for-ftp/643/2 "2016-09-09T07:24:43Z")

</div>

The proper solution is to configure a dedicated range of ports on server3 and open that range on iptables.

If thats not what you want, for whatever reason, then you need to look at a proxy server that intercepts FTP message and redirects data traffic as well. Haproxy does not do that (as it is not a FTP proxy, but a TCP proxy in this case).

Quick sidenote: iptables would also fail to open passive ports when you would use FTP with SSL. Once you enable SSL, iptables FTP helper are no longer able to intercept the passive ports, meaning it would be closed. Relying on those FTP helpers in iptables is therefor almost always a bad idea.

---

<div class="post-metadata">

**Author:** ![ebourgui](https://avatars.discourse-cdn.com/v4/letter/e/b3f665/32.png) [@ebourgui](https://discourse.haproxy.org/u/ebourgui)\
**Post date:** [September 9, 2016, 8:49am UTC](https://discourse.haproxy.org/t/ip-routing-with-haproxy-tcp-for-ftp/643/3 "2016-09-09T08:49:28Z")

</div>

Thank you for your response.

> [@lukastribus](#):
>
> The proper solution is to configure a dedicated range of ports on server3 and open that range on iptables.

Already done.

> [@lukastribus](#):
>
> If thats not what you want, for whatever reason, then you need to look at a proxy server that intercepts FTP message and redirects data traffic as well. Haproxy does not do that (as it is not a FTP proxy, but a TCP proxy in this case).

Do you know ftp proxies? ftp-proxy seems to be outdated…

> [@lukastribus](#):
>
> Quick sidenote: iptables would also fail to open passive ports when you would use FTP with SSL. Once you enable SSL, iptables FTP helper are no longer able to intercept the passive ports, meaning it would be closed. Relying on those FTP helpers in iptables is therefor almost always a bad idea.

I won’t use FTP with SSL. I use SFTP with haproxy and It works well.

Eric

---

<div class="post-metadata">

**Author:** ![lukastribus](https://avatars.discourse-cdn.com/v4/letter/l/7ea924/32.png) [@lukastribus](https://discourse.haproxy.org/u/lukastribus)\
**Post date:** [September 9, 2016, 9:40am UTC](https://discourse.haproxy.org/t/ip-routing-with-haproxy-tcp-for-ftp/643/4 "2016-09-09T09:40:34Z")

</div>

No, I don’t. You could also forward the passive port range from haproxy to your backend FTP server. That would probably be the easiest way.

You would force the FTP server to announce the IP of haproxy in the control channel, and forward the pasv ports as well through haproxy.

This is what’s explained here and is actually the simplest approach:

> **[ludflu/haproxy-vsftp](https://github.com/ludflu/haproxy-vsftp)**
>
> Contribute to ludflu/haproxy-vsftp development by creating an account on GitHub.

---

<div class="post-metadata">

**Author:** ![ebourgui](https://avatars.discourse-cdn.com/v4/letter/e/b3f665/32.png) [@ebourgui](https://discourse.haproxy.org/u/ebourgui)\
**Post date:** [September 9, 2016, 12:23pm UTC](https://discourse.haproxy.org/t/ip-routing-with-haproxy-tcp-for-ftp/643/5 "2016-09-09T12:23:41Z")

</div>

I’ve ever seen this link and it helped me. But if I add in /etc/vsftp/vsftp.conf `pasv_address=5.6.7.8`:

- client firewall doesn’t block output packets : OK
- but I get  
ftp failover fqdn  
Connected to failover fqdn (5.6.7.8).  
220 Welcome to FTP service.  
Name (failover fqdn:root): toto  
331 Please specify the password.  
Password:  
230 Login successful.  
Remote system type is UNIX.  
Using binary mode to transfer files.  
ftp\> ls  
227 Entering Passive Mode (5,6,7,8,196,94).  
ftp: connect: Connexion refusée  
=\> connection refused

---

<div class="post-metadata">

**Author:** ![lukastribus](https://avatars.discourse-cdn.com/v4/letter/l/7ea924/32.png) [@lukastribus](https://discourse.haproxy.org/u/lukastribus)\
**Post date:** [September 9, 2016, 1:17pm UTC](https://discourse.haproxy.org/t/ip-routing-with-haproxy-tcp-for-ftp/643/6 "2016-09-09T13:17:17Z")

</div>

Did you configure haproxy correctly? How does the configuration look like?

---

<div class="post-metadata">

**Author:** ![ebourgui](https://avatars.discourse-cdn.com/v4/letter/e/b3f665/32.png) [@ebourgui](https://discourse.haproxy.org/u/ebourgui)\
**Post date:** [September 9, 2016, 1:39pm UTC](https://discourse.haproxy.org/t/ip-routing-with-haproxy-tcp-for-ftp/643/7 "2016-09-09T13:39:31Z")

</div>

```
global
    log 127.0.0.1 local2

    chroot /var/lib/haproxy
    pidfile /var/run/haproxy.pid
    maxconn 1000
    #user haproxy
    #group haproxy
    daemon

    # turn on stats unix socket
    stats socket /var/lib/haproxy/stats

    tune.ssl.default-dh-param 2048

defaults
    mode http
    log global
    option httplog
    option dontlognull
    option http-server-close
    option forwardfor except 127.0.0.0/8
    option redispatch
    retries 3
    timeout http-request 10s

    timeout queue 1m
    timeout connect 10s
    timeout client 1m
    timeout server 1m
    timeout http-keep-alive 10s
    timeout check 10s

frontend stats
        bind 5.6.7.8:8080
        stats enable
        stats uri / ******
        stats realm Haproxy\ Statistics
        stats auth ****:******

(....)

listen ftplb2 5.6.7.8:21, 5.6.7.8:50000-50999
    mode tcp
    option tcplog
    server server3 9.10.11.12 check port 21 source interface eth0:0
    server server4 13.14.15.16 check port 21 source interface eth0:0

```

where eth0:0=5.6.7.8

---

<div class="post-metadata">

**Author:** ![lukastribus](https://avatars.discourse-cdn.com/v4/letter/l/7ea924/32.png) [@lukastribus](https://discourse.haproxy.org/u/lukastribus)\
**Post date:** [September 9, 2016, 2:21pm UTC](https://discourse.haproxy.org/t/ip-routing-with-haproxy-tcp-for-ftp/643/8 "2016-09-09T14:21:19Z")

</div>

Does haproxy actually listen on the port range?

You would be better up configuring a proper frontend/backend section, instead of using this obsolete approach with the listen section. And don’t use “source interface”, specify the source IP.

```
frontend ftplb2
 bind 5.6.7.8:21
 bind 5.6.7.8:50000-50999
 mode tcp
 option tcplog
 default_backend ftpservers

backend ftpservers
 mode tcp
 server server3 9.10.11.12 check port 21 source 5.6.7.8
 server server4 13.14.15.16 check port 21 source 5.6.7.8
```

---

<div class="post-metadata">

**Author:** ![ebourgui](https://avatars.discourse-cdn.com/v4/letter/e/b3f665/32.png) [@ebourgui](https://discourse.haproxy.org/u/ebourgui)\
**Post date:** [September 9, 2016, 2:56pm UTC](https://discourse.haproxy.org/t/ip-routing-with-haproxy-tcp-for-ftp/643/9 "2016-09-09T14:56:35Z")

</div>

Yes!!! It works in passive mode. THANK YOU!!!

Just a problem in active mode:  
200 PORT command successful. Consider using PASV.  
421 Service not available, remote server has closed connection

It’s week end time for me. I will retry on monday.  
Thank you again.

---

<div class="post-metadata">

**Author:** ![ebourgui](https://avatars.discourse-cdn.com/v4/letter/e/b3f665/32.png) [@ebourgui](https://discourse.haproxy.org/u/ebourgui)\
**Post date:** [September 13, 2016, 7:42am UTC](https://discourse.haproxy.org/t/ip-routing-with-haproxy-tcp-for-ftp/643/10 "2016-09-13T07:42:57Z")

</div>

From client (which accept all output traffic in firewall):

- I can connect in active mode directly to ftp server (server3)

- I can’t connect in active mode to ftp server by haproxy

In this situation no packets are blocked by firewall on server3 and server1.

---

<div class="post-metadata">

**Author:** ![lukastribus](https://avatars.discourse-cdn.com/v4/letter/l/7ea924/32.png) [@lukastribus](https://discourse.haproxy.org/u/lukastribus)\
**Post date:** [September 13, 2016, 12:21pm UTC](https://discourse.haproxy.org/t/ip-routing-with-haproxy-tcp-for-ftp/643/11 "2016-09-13T12:21:38Z")

</div>

For active mode to work you will have to NAT your active traffic through the haproxy box, otherwise the client will see the active connection attempt from a different IP (the IP of server3 instead of the IP of the haproxy box) and will close the connection.

---

<div class="post-metadata">

**Author:** ![ebourgui](https://avatars.discourse-cdn.com/v4/letter/e/b3f665/32.png) [@ebourgui](https://discourse.haproxy.org/u/ebourgui)\
**Post date:** [September 14, 2016, 7:31am UTC](https://discourse.haproxy.org/t/ip-routing-with-haproxy-tcp-for-ftp/643/12 "2016-09-14T07:31:27Z")

</div>

Thanks for your help.

I add this rule on ftp server  
`-A POSTROUTING -p tcp -o eth0 -j SNAT --to 5.6.7.8`  
It doesn’t work. tcpdump says:  
`08:53:34.562777 IP 5.6.7.8.35191 > client's private ip.50301: Flags [S], seq 113635828, win 29200, options [mss 1460,sackOK,TS val 1309390568 ecr 0,nop,wscale 7], length 0`  
Without the rule it was  
`08:54:19.567917 IP ftp server ip.41179 > client's private ip.51138: Flags [S], seq 1040902272, win 29200, options [mss 1460,sackOK,TS val 1309435573 ecr 0,nop,wscale 7], length 0`

So the iptables rule is OK : the packet is from 5.6.7.8 (haproxy). But ftp server replies to the client’s private ip (and not the client’s public ip). But why?

---

<div class="post-metadata">

**Author:** ![lukastribus](https://avatars.discourse-cdn.com/v4/letter/l/7ea924/32.png) [@lukastribus](https://discourse.haproxy.org/u/lukastribus)\
**Post date:** [September 14, 2016, 2:10pm UTC](https://discourse.haproxy.org/t/ip-routing-with-haproxy-tcp-for-ftp/643/13 "2016-09-14T14:10:47Z")

</div>

Because the client is behind NAT, incorrectly configured and the NAT gateway doesn’t have an “ftp helper”.

You have to use a properly configured FTP client.

edit: I don’t think the iptables configuration suffices though; you would have to tunnel to the haproxy box and NAT it there, as your FTP doesn’t have see traffic desalinated to 5.6.7.8. You will also have to restrict the tunnel+NAT rule to only the active traffic. This goes beyond what we can support here.

---

<div class="post-metadata">

**Author:** ![karsten](https://avatars.discourse-cdn.com/v4/letter/k/e47c2d/32.png) [@karsten](https://discourse.haproxy.org/u/karsten)\
**Post date:** [September 12, 2018, 9:40am UTC](https://discourse.haproxy.org/t/ip-routing-with-haproxy-tcp-for-ftp/643/14 "2018-09-12T09:40:30Z")

</div>

When you are using PASV the server will tell the client which IP and port to connect to. - so you’ll have to set up the server to tell the client to connect to your Proxy-server. - look for something like ‘firewall support’  
Remember also that you must configure individual portranges for PASV on the servers, and setup the rules for that in haproxy.

So setup one rule for port 21 that does roundrobin (or leastconn) - remember to set ‘timeout tunnel’, as the server will shut down the session if this conn disappears.  
Then setup one rule for each of the servers to listen for the PASV portrange eg:  
listen ftp  
bind \*:21  
mode tcp  
option tcplog  
balance leastconn  
timeout tunnel 300s  
server ftp1 x.y.x.i check port 21  
server ftp2 x.y.z.ii check port 21

listen ftp\_pasv1  
bind \*:63535-64534  
mode tcp  
option tcplog  
server ftp1 x.y.z.i check port 21

listen ftp\_pasv2  
bind \*:64535-65535  
mode tcp  
option tcplog  
server ftp2 x.y.z.ii check port 21