Haproxy in front of containers....more!

Help!
21

Perfect. I’m taking a look now.

22

What is the IP address of the following:

  • the server where HAProxy runs; (I’m assuming this is on the containers “host”; i.e. not within a container;)
  • the container IP; (i.e. LPC1.lxd);

Are you using some kind of NAT? (I’m asking because although you have the www_http prontend listening on 443, curl says that the connection is refused…)

23

I am using LXD containers.

So the haproxy is in a container. IP=10.106.37.15
LPC1 = 10.106.37.94

Also, port forwarding as such:
sudo iptables -t nat -I PREROUTING -i eth0 -p TCP -d 23.239.31.177/32 --dport 443 -j DNAT --to-destination 10.106.37.15:443

sudo iptables -t nat -I PREROUTING -i eth0 -p TCP -d 23.239.31.177/32 --dport 80 -j DNAT --to-destination 10.106.37.15:80

sudo iptables -t nat -I PREROUTING -i eth0 -p TCP -d 23.239.31.177/32 --dport 8888 -j DNAT --to-destination 10.106.37.15:8888

sudo iptables -t nat -I PREROUTING -i eth0 -p TCP -d lpc1.streamingworld.us/32 --dport 3000 -j DNAT --to-destination 10.106.37.15:3000

24

Exactly what I thought…

In order to have it work also from within the host itself you’ll have to add the same rule, but this time on the OUTPUT chain.

25

It would be a good idea to actually use an IP not a name, as this is resolved only when the command is executed.

26

So what would be the rule for the OUTPUT chain?

27

Exactly as for the PREROUTING but for the OUTPUT chain. Namely:

sudo iptables -t nat -I OUTPUT -i eth0 -p TCP -d 23.239.31.177/32 --dport 443 -j DNAT --to-destination 10.106.37.15:443
28

can’t use -i with OUTPUT?

29

You are right. Just remove -i eth0. (Assuming you have only one network card, I would suggest dropping that also from the other rules…)

30

Stilll have the same issue after adding the port forwarding.

31

Please also provide an output for iptables -t nat -L -n -v -x. (And other tables if you know you’ve configured something…)

32 
root@localhost:~# iptables -t nat -L -n -v -x
Chain PREROUTING (policy ACCEPT 212 packets, 12738 bytes)
    pkts      bytes target     prot opt in     out     source               destination
     124     6424 DNAT       tcp  --  eth0   *       0.0.0.0/0            23.239.31.177        tcp dpt:3000 to:10.106.37.15:3000
       9      400 DNAT       tcp  --  eth0   *       0.0.0.0/0            23.239.31.177        tcp dpt:8888 to:10.106.37.15:8888
    1551    81352 DNAT       tcp  --  eth0   *       0.0.0.0/0            23.239.31.177        tcp dpt:80 to:10.106.37.15:80
     727    42276 DNAT       tcp  --  eth0   *       0.0.0.0/0            23.239.31.177        tcp dpt:443 to:10.106.37.15:443

Chain INPUT (policy ACCEPT 8 packets, 742 bytes)
    pkts      bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 7 packets, 500 bytes)
    pkts      bytes target     prot opt in     out     source               destination
       0        0 DNAT       tcp  --  *      *       0.0.0.0/0            23.239.31.177        tcp dpt:443 to:10.106.37.15:443

Chain POSTROUTING (policy ACCEPT 217 packets, 12908 bytes)
    pkts      bytes target     prot opt in     out     source               destination
     140     8400 MASQUERADE  all  --  *      *       10.106.37.0/24      !10.106.37.0/24       /* generated for LXD network lxdbr0 */
33

[First of all thanks for reformatting the output. It saved me from a copy-paste to an editor…] :slight_smile:

Are you sure you’re trying to curl https://23.239.31.177/?

34

curl https://23.239.31.177
curl: (51) SSL: certificate subject name (lpc1.streamingworld.us) does not match target host name ‘23.239.31.177’

35

OK. Now it seems it works. Just try now with the proper domain.

36

But this will not get to my container! I need to be able to use https://lpc1.streamingworld.us

37

Please try to explain what exactly are you trying to achieve? (Do you want to bypass HAProxy or?)

Because now the requests should go to HAProxy.

38

Yes I wan this request to forward to haproxy. The haproxy will pass it to the correct container.
(right now I am only concerned with one container. Later I will have lpc2.streamingworld.us, lpc3, etc.)

My domain name is www.streamingworld.us IP=23.239.31.177
I have an LXD container (LPC1) configured with Ubuntu 16.04/nginx
Also, there is a REACT/node application on port 3000. This is a video player.

The index.html file in LPC1 has an iFrame inside with src=https://lpc1.streamingworld.us:3000/embed/0x5…

Prior to making the port forwarding change, I was at least getting to the index.html page in LPC1.
I have lpc1.streamingworld.us defined as a subdomain.

I think at this point it does not matter what the app does as I can’t get to it?

39

OK. Let’s first try to verify each item in isolation.

  • First of all remove all your iptables DNAT rules.

  • Update /etc/hosts and set 10.106.37.15 www.streamingworld.us lpc1.streamingworld.us (and other domains you want to try.)

  • Now try to curl the domains on both 80, 443 and 3000.

  • (by now everything should work OK; if not try to solve it so that it works;)

  • then add the PREROUTING DNAT rules and check that it works (from another host, not the server or its containers;)

  • check to see that the counters for your rules are actually incremented; else it means they don’t actually work;

  • then add the OUTPUT DNAT rules and check that it works (from the host); (the same, check the counters;)

  • then remove the line from /etc/hosts;

40

So I’m setting the haproxy IP (10.106.37.15) to www.streamingworld.us lpc1.streamingworld.us

curl http://www.streamingworld.us

<html><body><h1>503 Service Unavailable</h1>
No server is available to handle this request.
</body></html>

curl https://lpc1.streamingworld.us

<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8"/>
<title>Welcome to Nginx on LPC1!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to Nginx!</h1>
<h1>You are in LPC1!</h1>
<p>If you see this page, the nginx  web server is successfully installed and
working. Further configuration may be required.</p>

<p><em>Thank you for using Nginx.</em></p>

<!-- <iframe  width="640" height="360" src="LPC1.lxd:3000/Embed/0x35e01199f8137077346fdd476191f1967a52bdb5" frameborder="0"></iframe> -->

<!-- <iframe  width="640" height="360" src="http://23.239.31.177:3000/Embed/0x35e01199f8137077346fdd476191f1967a52bdb5" frameborder="0" sandbox="allow-same-origin"></iframe> -->

<!-- <iframe  width="640" height="360" src="/livepeerjs/packages/player/public/index.html" name="myFrame"></iframe>
     <a href="http://LPC1.lxd:3000" target="myFrame"></a> -->

<iframe  width="640" height="360" src="https://lpc1.streamingworld.us:3000/embed/0x35e01199f8137077346fdd476191f1967a52bdb5" frameborder="1"></iframe>

<!-- <iframe width="640" height="360" src="http://lpc1.streamingworld.us:3000/Embed/0x35e01199f8137077346fdd476191f1967a52bdb5" sandbox="allow-same-origin allow-scripts" frameborder="1"></iframe> -->

<!-- <object data="https://lpc1.streamingworld.us:3000/embed/0x35e01199f8137077346fdd476191f1967a52bdb5" width="400" height="300" type="video/MP2T">
    Alternative Content
</object> -->

</body>
</html>

root@localhost:/etc# curl https://lpc1.streamingworld.us:3000

curl: (35) gnutls_handshake() failed: An unexpected TLS packet was received.

root@localhost:/etc#