HAProxy community

Haproxy in front of containers....more!


#21

Perfect. I’m taking a look now.


#22

What is the IP address of the following:

  • the server where HAProxy runs; (I’m assuming this is on the containers “host”; i.e. not within a container;)
  • the container IP; (i.e. LPC1.lxd);

Are you using some kind of NAT? (I’m asking because although you have the www_http prontend listening on 443, curl says that the connection is refused…)


#23

I am using LXD containers.

So the haproxy is in a container. IP=10.106.37.15
LPC1 = 10.106.37.94

Also, port forwarding as such:
sudo iptables -t nat -I PREROUTING -i eth0 -p TCP -d 23.239.31.177/32 --dport 443 -j DNAT --to-destination 10.106.37.15:443

sudo iptables -t nat -I PREROUTING -i eth0 -p TCP -d 23.239.31.177/32 --dport 80 -j DNAT --to-destination 10.106.37.15:80

sudo iptables -t nat -I PREROUTING -i eth0 -p TCP -d 23.239.31.177/32 --dport 8888 -j DNAT --to-destination 10.106.37.15:8888

sudo iptables -t nat -I PREROUTING -i eth0 -p TCP -d lpc1.streamingworld.us/32 --dport 3000 -j DNAT --to-destination 10.106.37.15:3000


#24

Exactly what I thought…

In order to have it work also from within the host itself you’ll have to add the same rule, but this time on the OUTPUT chain.


#25

It would be a good idea to actually use an IP not a name, as this is resolved only when the command is executed.


#26

So what would be the rule for the OUTPUT chain?


#27

Exactly as for the PREROUTING but for the OUTPUT chain. Namely:

sudo iptables -t nat -I OUTPUT -i eth0 -p TCP -d 23.239.31.177/32 --dport 443 -j DNAT --to-destination 10.106.37.15:443

#28

can’t use -i with OUTPUT?


#29

You are right. Just remove -i eth0. (Assuming you have only one network card, I would suggest dropping that also from the other rules…)


#30

Stilll have the same issue after adding the port forwarding.


#31

Please also provide an output for iptables -t nat -L -n -v -x. (And other tables if you know you’ve configured something…)


#32
root@localhost:~# iptables -t nat -L -n -v -x
Chain PREROUTING (policy ACCEPT 212 packets, 12738 bytes)
    pkts      bytes target     prot opt in     out     source               destination
     124     6424 DNAT       tcp  --  eth0   *       0.0.0.0/0            23.239.31.177        tcp dpt:3000 to:10.106.37.15:3000
       9      400 DNAT       tcp  --  eth0   *       0.0.0.0/0            23.239.31.177        tcp dpt:8888 to:10.106.37.15:8888
    1551    81352 DNAT       tcp  --  eth0   *       0.0.0.0/0            23.239.31.177        tcp dpt:80 to:10.106.37.15:80
     727    42276 DNAT       tcp  --  eth0   *       0.0.0.0/0            23.239.31.177        tcp dpt:443 to:10.106.37.15:443

Chain INPUT (policy ACCEPT 8 packets, 742 bytes)
    pkts      bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 7 packets, 500 bytes)
    pkts      bytes target     prot opt in     out     source               destination
       0        0 DNAT       tcp  --  *      *       0.0.0.0/0            23.239.31.177        tcp dpt:443 to:10.106.37.15:443

Chain POSTROUTING (policy ACCEPT 217 packets, 12908 bytes)
    pkts      bytes target     prot opt in     out     source               destination
     140     8400 MASQUERADE  all  --  *      *       10.106.37.0/24      !10.106.37.0/24       /* generated for LXD network lxdbr0 */


#33

[First of all thanks for reformatting the output. It saved me from a copy-paste to an editor…] :slight_smile:

Are you sure you’re trying to curl https://23.239.31.177/?


#34

curl https://23.239.31.177
curl: (51) SSL: certificate subject name (lpc1.streamingworld.us) does not match target host name ‘23.239.31.177’


#35

OK. Now it seems it works. Just try now with the proper domain.


#36

But this will not get to my container! I need to be able to use https://lpc1.streamingworld.us


#37

Please try to explain what exactly are you trying to achieve? (Do you want to bypass HAProxy or?)

Because now the requests should go to HAProxy.


#38

Yes I wan this request to forward to haproxy. The haproxy will pass it to the correct container.
(right now I am only concerned with one container. Later I will have lpc2.streamingworld.us, lpc3, etc.)

My domain name is www.streamingworld.us IP=23.239.31.177
I have an LXD container (LPC1) configured with Ubuntu 16.04/nginx
Also, there is a REACT/node application on port 3000. This is a video player.

The index.html file in LPC1 has an iFrame inside with src=https://lpc1.streamingworld.us:3000/embed/0x5…

Prior to making the port forwarding change, I was at least getting to the index.html page in LPC1.
I have lpc1.streamingworld.us defined as a subdomain.

I think at this point it does not matter what the app does as I can’t get to it?


#39

OK. Let’s first try to verify each item in isolation.

  • First of all remove all your iptables DNAT rules.

  • Update /etc/hosts and set 10.106.37.15 www.streamingworld.us lpc1.streamingworld.us (and other domains you want to try.)

  • Now try to curl the domains on both 80, 443 and 3000.

  • (by now everything should work OK; if not try to solve it so that it works;)

  • then add the PREROUTING DNAT rules and check that it works (from another host, not the server or its containers;)

  • check to see that the counters for your rules are actually incremented; else it means they don’t actually work;

  • then add the OUTPUT DNAT rules and check that it works (from the host); (the same, check the counters;)

  • then remove the line from /etc/hosts;


#40

So I’m setting the haproxy IP (10.106.37.15) to www.streamingworld.us lpc1.streamingworld.us

curl http://www.streamingworld.us

<html><body><h1>503 Service Unavailable</h1>
No server is available to handle this request.
</body></html>

curl https://lpc1.streamingworld.us

<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8"/>
<title>Welcome to Nginx on LPC1!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to Nginx!</h1>
<h1>You are in LPC1!</h1>
<p>If you see this page, the nginx  web server is successfully installed and
working. Further configuration may be required.</p>

<p><em>Thank you for using Nginx.</em></p>

<!-- <iframe  width="640" height="360" src="LPC1.lxd:3000/Embed/0x35e01199f8137077346fdd476191f1967a52bdb5" frameborder="0"></iframe> -->

<!-- <iframe  width="640" height="360" src="http://23.239.31.177:3000/Embed/0x35e01199f8137077346fdd476191f1967a52bdb5" frameborder="0" sandbox="allow-same-origin"></iframe> -->

<!-- <iframe  width="640" height="360" src="/livepeerjs/packages/player/public/index.html" name="myFrame"></iframe>
     <a href="http://LPC1.lxd:3000" target="myFrame"></a> -->

<iframe  width="640" height="360" src="https://lpc1.streamingworld.us:3000/embed/0x35e01199f8137077346fdd476191f1967a52bdb5" frameborder="1"></iframe>

<!-- <iframe width="640" height="360" src="http://lpc1.streamingworld.us:3000/Embed/0x35e01199f8137077346fdd476191f1967a52bdb5" sandbox="allow-same-origin allow-scripts" frameborder="1"></iframe> -->

<!-- <object data="https://lpc1.streamingworld.us:3000/embed/0x35e01199f8137077346fdd476191f1967a52bdb5" width="400" height="300" type="video/MP2T">
    Alternative Content
</object> -->

</body>
</html>

root@localhost:/etc# curl https://lpc1.streamingworld.us:3000

curl: (35) gnutls_handshake() failed: An unexpected TLS packet was received.

root@localhost:/etc#