Haproxy in front of containers....more!

Help!
1

So, I have progressed a little with my former issue although things still not working.

To review…I am using port forwarding for various ports to an haproxy server installed on an LXD container.
I also have a container LPC1 with nginx serving pages.

Then I have the following simple frontend:

frontend www_http

    bind *:80 #bind to port 80        
 # it matches if the http host: field mentions any of the hostnames (after the '-i').
    acl host_LPC1 hdr(host) -i lpc1.streamingworld.us
 
# Redirect the connection to the proper server container, depending on the match.
    use_backend cont_LPC1  if host_LPC1

backend cont_LPC1
balance leastconn
# We set the X-Client-IP HTTP header. This is useful if we want the web server to know the real client IP.
http-request set-header X-Client-IP %[src]
# This backend, named here “LPC1”, directs to container “LPC1.lxd”.
server LPC1 LPC1.lxd:80 check

This appears to work as I can display the index.html. Now inside the index.html, I have an iFrame with the src= port 3000 of the same container (LPC1.lxd), so here is what I thought would work for the haproxy:

frontend play
bind *:3000
acl url_static hdr(host) -i lpc1.streamingworld.us/embed/0x35e01199f8137077346fdd476191f1967a52bdb5
use_backend static if url_static

backend static
balance leastconn
#http-request set-header X-Client-IP %[src]
server LPC1play LPC1.lxd:3000 check

I am not sure of either front or backend configurations?

When I run sudo haproxy -f /etc/haproxy/haproxy.cfg -c
Configuration file is valid

But when I restart haproxy, I get:

Mar 9 15:28:44 HAPROXY haproxy[13210]: [WARNING] 067/144947 (13210) : Exiting Master process…
Mar 9 15:28:44 HAPROXY haproxy[13210]: [ALERT] 067/144947 (13210) : Current worker 13211 exited with code 143
Mar 9 15:28:44 HAPROXY haproxy[13210]: [WARNING] 067/144947 (13210) : All workers exited. Exiting… (143)
Mar 9 15:28:44 HAPROXY haproxy[13237]: Proxy www_http started.
Mar 9 15:28:44 HAPROXY haproxy[13237]: Proxy www_http started.
Mar 9 15:28:44 HAPROXY haproxy[13237]: Proxy www_https started.
Mar 9 14:49:47 HAPROXY haproxy[13211]: backend static has no server available!
Mar 9 15:28:44 HAPROXY haproxy[13238]: Server static/LPC1play is DOWN, reason: Layer4 connection problem, info: “Connection refused”, check duration: 0ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
Mar 9 15:28:44 HAPROXY haproxy[13238]: Server static/LPC1play is DOWN, reason: Layer4 connection problem, info: “Connection refused”, check duration: 0ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
Mar 9 15:28:44 HAPROXY haproxy[13238]: backend static has no server available!
Mar 9 15:28:44 HAPROXY haproxy[13237]: Proxy www_https started.
Mar 9 15:28:44 HAPROXY haproxy[13237]: Proxy play started.
Mar 9 15:28:44 HAPROXY haproxy[13237]: Proxy play started.
Mar 9 15:28:44 HAPROXY haproxy[13237]: Proxy static started.
Mar 9 15:28:44 HAPROXY haproxy[13237]: Proxy static started.
Mar 9 15:28:44 HAPROXY haproxy[13237]: Proxy cont_LPC1 started.
Mar 9 15:28:44 HAPROXY haproxy[13237]: Proxy cont_LPC1 started.
Mar 9 15:28:44 HAPROXY haproxy[13237]: Proxy subdomain1 started.

First I’m not sure why I am seeing double outputs?
But worse, why the connection refused?

Your help us much appreciated!

Ray

2

I think your solution is to actually fix the application running inside the container to reply with proper URL’s.

Getting back to your issue, I think the port 3000 isn’t actually exposed from the container and that is why the backend server checks fail.

Try to curl http://LPC1.lxd:3000/ and see if it works.

3

Where should I run the curl http from?
From the haproxy container I’m seeing an html template come back in response to curl http://LPC1.lxd:3000

Then I try this: curl https://LPC1.lxd:3000
curl: (35) gnutls_handshake() failed: An unexpected TLS packet was received.

4

You should run curl from both outside the host (i.e. targeting HAProxy, by using HAProxy’s IP and port) to see what it replies, but also from within the HAProxy container (by using the container’s IP and port) to see what HAProxy “sees”.

Is the server on port 3000 using SSL?

If so you tell HAProxy that the server uses HTTPS by using the ssl and check-ssl options on the server line.

5

I try to set ssl and check-ssl on the server line and got this:

haproxy -c -V -f /etc/haproxy/haproxy.cfg
[ALERT] 068/214046 (1222) : Proxy ‘static’, server ‘LPC1play’ [/etc/haproxy/haproxy.cfg:65] verify is enabled by default but no CA file specified. If you’re running on a LAN where you’re certain to trust the server’s certificate, please set an explicit ‘verify none’ statement on the ‘server’ l ine, or use ‘ssl-server-verify none’ in the global section to disable server-side verifications by default.
[ALERT] 068/214046 (1222) : Fatal errors found in configuration.

6

Well, do as the error says: set verify none on the server line. :slight_smile:

7

Added ‘verify none’ to the server line but problem still exists. Can you look at my haproxy.cfg?

frontend www_http

    bind *:80
    bind *:443 ssl crt /etc/ssl/lpc1.streamingworld.us/lpc1.streamingworld.us.pem

    acl letsencrypt-acl path_beg /.well-known/acme-challenge/
    use_backend letsencrypt-backend if letsencrypt-acl

    acl host_LPC1 hdr(host) -i lpc1.streamingworld.us
    use_backend cont_LPC1  if host_LPC1

frontend play
bind *:3000
acl url_static hdr(host) -i /embed/0x35e01199f8137077346fdd476191f1967a52bdb5
use_backend static if url_static

backend static
#server LPC1play lpc1.streamingworld.us:3000/embed/0x35e01199f8137077346fdd476191f1967a52bdb5
server LPC1play LPC1.lxd:3000/embed/0x35e01199f8137077346fdd476191f1967a52bdb5 verify none

backend cont_LPC1
redirect scheme https if !{ ssl_fc }
balance leastconn
#http-request set-header X-Client-IP %[src]
#server LPC1 LPC1.lxd:80 check
server LPC1 10.106.37.94:80 check

backend letsencrypt-backend
server letsencrypt 127.0.0.1:8888

8

This line seems to include the URL on the server line… I think it’s not OK.

What does the haproxy -c -f /etc/haproxy/haproxy.cfg (or wherever the configuration file lies) say?

(BTW, please try to use MarkDown frormatting and include your code snippets between ``` on separate lines, as Discourse tries to interpret the configuration file as MarkDown and makes it look horrible…)

9

root@HAPROXY:~# haproxy -c -f /etc/haproxy/haproxy.cfg
Configuration file is valid

10

Please try to remove the /embedded/... path and see if it works. (Always check with haproxy -c -f ...).

11

Removed the /embed/…

root@HAPROXY:/etc/haproxy# haproxy -c -f /etc/haproxy/haproxy.cfg
Configuration file is valid

Still have the problem: image

Ray

12

The previous screen grab was for Chrome. Here is what I get in FireFox:

image
image519×564 18.3 KB

13

Now it seems you just have to come-up with proper TLS certificates so that the browsers trust your domains. (Try Let’s Encrypt. I would suggest DNS-based issuing if you use one of the supported DNS plugins for Let’s Encrypt.)

14

I am using letsencrypt for the haproxy.

15

Then it means that the certificate doesn’t match the domains you’re using.

Please use https://www.ssllabs.com/ssltest to test the certfificate. (If the endpoint is public. Else use curl and see what it complains about.)

16

lpc1.streamingworld.us is graded “A”.

17

BTW, try to use Let’s Encrypt on the domain / endpoint / URL that doesn’t actually work.

OK. Let’s start from the beginning. (As through the various replies I think I’ve lost the “synchronization” of the exact situation.)

Please provide again the following:

  • the current HAproxy configuration;
  • what URL are you trying to access (i.e. curl -v -s -o /dev/null https://...) and what that command signals as error; (also if it’s an TLS error, try to use openssl s_client to diagnosticate further;)
  • what errors are logged in HAProxy’s log;
  • what exact error does Firefox / Chrome complain about;

Also the public certificate (as installed in HAProxy) would be helpful. (Please do not provide also the private key.)

18

Here ya go:

haproxy.cfg

frontend www_http

        bind *:80
        bind *:443 ssl crt /etc/ssl/lpc1.streamingworld.us/lpc1.streamingworld.us.pem

        acl letsencrypt-acl path_beg /.well-known/acme-challenge/
        use_backend letsencrypt-backend if letsencrypt-acl

        acl host_LPC1 hdr(host) -i lpc1.streamingworld.us
        use_backend cont_LPC1  if host_LPC1

frontend play
        bind *:3000
        acl url_static hdr(host) -i /embed/0x35e01199f8137077346fdd476191f1967a52bdb5
        use_backend static if url_static

backend static
        #server LPC1play lpc1.streamingworld.us:3000/embed/0x35e01199f8137077346fdd476191f1967a52bdb5
        server LPC1play LPC1.lxd:3000/embed/0x35e01199f8137077346fdd476191f1967a52bdb5

backend cont_LPC1
        redirect scheme https if !{ ssl_fc }
        balance leastconn
        http-request set-header X-Client-IP %[src]
        #server LPC1 LPC1.lxd:80 check
        server LPC1 10.106.37.94:80 check


backend letsencrypt-backend
     server letsencrypt 127.0.0.1:8888

LXD Container LPC1 – nginx version: nginx/1.10.3 (Ubuntu)
LXD Container LPC1 – REACT/node application on port 3000

  1. root@localhost:~# curl -v -s -o /dev/null https://lpc1.streamingworld.us 
* Rebuilt URL to: https://lpc1.streamingworld.us/
*   Trying 23.239.31.177...
* connect to 23.239.31.177 port 443 failed: Connection refused
* Failed to connect to lpc1.streamingworld.us port 443: Connection refused
* Closing connection 0

root@localhost:~# curl -v -s -o /dev/null https://lpc1.streamingworld.us:3000
* Rebuilt URL to: https://lpc1.streamingworld.us:3000/
*   Trying 23.239.31.177...
* connect to 23.239.31.177 port 3000 failed: Connection refused
* Failed to connect to lpc1.streamingworld.us port 3000: Connection refused
* Closing connection 0

root@localhost:~# curl -v -s -o /dev/null http://lpc1.streamingworld.us

* Rebuilt URL to: http://lpc1.streamingworld.us/
*   Trying 23.239.31.177...
* Connected to lpc1.streamingworld.us (23.239.31.177) port 80 (#0)
> GET / HTTP/1.1
> Host: lpc1.streamingworld.us
> User-Agent: curl/7.47.0
> Accept: */*
>
< HTTP/1.1 302 Found
< Date: Mon, 11 Mar 2019 13:37:55 GMT
< Server: Apache/2.4.18 (Ubuntu)
< X-FRAME-OPTIONS: SAMEORIGIN
< Set-Cookie: PHPSESSID=90bul202v0ff5441i2fofa03b7; path=/; HttpOnly
< Expires: Thu, 19 Nov 1981 08:52:00 GMT
< Cache-Control: no-store, no-cache, must-revalidate
< Pragma: no-cache
< Set-Cookie: ad-con=a%3A2%3A%7Bs%3A4%3A%26quot%3Bdate%26quot%3B%3Bs%3A10%3A%26quot%3B2019-03-11%26quot%3B%3Bs%3A3%3A%26quot%3Bads%26quot%3B%3Ba%3A0%3A%7B%7D%7D; expires=Thu, 08-Mar-2029 13:37:55 GMT; Max-Age=315360000
< Set-Cookie: ad-con=a%3A2%3A%7Bs%3A4%3A%26quot%3Bdate%26quot%3B%3Bs%3A10%3A%26quot%3B2019-03-11%26quot%3B%3Bs%3A3%3A%26quot%3Bads%26quot%3B%3Ba%3A0%3A%7B%7D%7D; expires=Thu, 08-Mar-2029 13:37:55 GMT; Max-Age=315360000
< Set-Cookie: _us=1552397875; expires=Thu, 08-Mar-2029 13:37:55 GMT; Max-Age=315360000
< Set-Cookie: _us=1552397875; expires=Thu, 08-Mar-2029 13:37:55 GMT; Max-Age=315360000
< Location: http://streamingworld.us
< Content-Length: 0
< Content-Type: text/html; charset=UTF-8
<
* Connection #0 to host lpc1.streamingworld.us left intact

I am not seeing any errors in the haproxy.log file?

root@localhost:~# openssl s_client -showcerts -servername lpc1.streamingworld.us:3000 -connect gnupg.org:443 2>/dev/null | openssl x509 -inform pem -noout -text

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            04:67:a0:ae:ef:dc:96:07:d0:7f:9a:07:c8:8c:71:44:94:74
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: C=US, O=Let's Encrypt, CN=Let's Encrypt Authority X3
        Validity
            Not Before: Feb 16 00:55:11 2019 GMT
            Not After : May 17 00:55:11 2019 GMT
        Subject: CN=gnupg.org
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:
                    00:b6:5a:e3:8d:48:a8:2d:54:d5:98:48:7f:73:54:
                    b2:fe:68:54:09:e7:37:15:93:a8:b9:55:89:a5:ef:
                    88:e9:21:76:0a:4f:dc:ab:59:a1:79:83:99:ae:73:
                    57:26:7d:74:ac:42:f0:60:e4:9d:04:61:95:4e:4f:
                    71:74:72:41:a6:ca:bc:98:b9:68:a3:5c:8c:90:de:
                    3b:0b:9b:65:cf:84:3b:6f:b7:9c:4b:e7:fa:19:58:
                    bb:0c:26:19:09:7d:e2:59:52:1d:d9:96:a6:6d:ef:
                    1a:71:48:ff:d5:10:37:d9:9f:ce:39:d0:c2:33:6a:
                    ac:c7:a7:b8:ae:47:9a:7c:fb:3a:58:cb:7b:53:c8:
                    69:32:67:31:56:b7:8d:03:f9:4b:d5:13:06:39:2c:
                    89:23:6c:00:6a:8b:0b:91:5d:f0:34:e3:c5:1f:c4:
                    ce:82:e1:f4:fa:21:41:e0:38:64:8f:0a:c1:a7:af:
                    a9:2a:2b:07:5a:61:7b:4b:82:d9:2b:d6:76:01:d9:
                    b4:50:23:1a:0d:1e:d5:3d:79:9d:de:0e:36:55:a5:
                    a8:6e:bd:bd:b9:15:3b:bb:df:2c:90:05:55:a9:6e:
                    fb:d9:4d:3b:ea:91:e5:52:19:d5:16:06:d3:bd:a2:
                    52:a8:d3:db:f0:18:78:a1:a0:da:61:54:fc:9d:2b:
                    58:49
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment
            X509v3 Extended Key Usage:
                TLS Web Server Authentication, TLS Web Client Authentication
            X509v3 Basic Constraints: critical
                CA:FALSE
            X509v3 Subject Key Identifier:
                33:6A:E0:20:3C:A5:02:1E:FF:1A:42:A1:9C:1A:17:F3:2D:82:0B:D0
            X509v3 Authority Key Identifier:
                keyid:A8:4A:6A:63:04:7D:DD:BA:E6:D1:39:B7:A6:45:65:EF:F3:A8:EC:A1

            Authority Information Access:
                OCSP - URI:http://ocsp.int-x3.letsencrypt.org
                CA Issuers - URI:http://cert.int-x3.letsencrypt.org/

            X509v3 Subject Alternative Name:
                DNS:gnupg.org, DNS:www.gnupg.org
            X509v3 Certificate Policies:
                Policy: 2.23.140.1.2.1
                Policy: 1.3.6.1.4.1.44947.1.1.1
                  CPS: http://cps.letsencrypt.org

            CT Precertificate SCTs:
                Signed Certificate Timestamp:
                    Version   : v1(0)
                    Log ID    : 74:7E:DA:83:31:AD:33:10:91:21:9C:CE:25:4F:42:70:
                                C2:BF:FD:5E:42:20:08:C6:37:35:79:E6:10:7B:CC:56
                    Timestamp : Feb 16 01:55:11.732 2019 GMT
                    Extensions: none
                    Signature : ecdsa-with-SHA256
                                30:46:02:21:00:C8:2D:77:E4:EC:EE:AA:E3:7C:39:79:
                                0F:B9:E5:B2:E7:D4:59:C0:95:76:F6:1E:DC:19:ED:D1:
                                DC:70:A8:07:DD:02:21:00:9A:78:1F:C8:BB:45:CC:C9:
                                48:D2:5D:E2:E2:06:6A:14:D6:B8:F1:59:D4:40:98:83:
                                A5:EC:1F:B7:42:A3:A6:1C
                Signed Certificate Timestamp:
                    Version   : v1(0)
                    Log ID    : 63:F2:DB:CD:E8:3B:CC:2C:CF:0B:72:84:27:57:6B:33:
                                A4:8D:61:77:8F:BD:75:A6:38:B1:C7:68:54:4B:D8:8D
                    Timestamp : Feb 16 01:55:11.741 2019 GMT
                    Extensions: none
                    Signature : ecdsa-with-SHA256
                                30:45:02:21:00:E7:49:61:1A:AD:C0:24:03:AC:A2:BD:
                                4F:49:FF:6C:52:64:E9:43:C9:D6:62:98:D2:CF:01:0F:
                                A9:C7:8A:D4:6B:02:20:56:03:1C:7A:55:9C:BE:7D:B7:
                                28:98:2F:2D:AE:38:78:A2:21:FE:AB:E8:3B:23:DD:E2:
                                45:1C:11:4E:A6:9C:81
    Signature Algorithm: sha256WithRSAEncryption
         79:d6:bb:e6:94:62:c9:65:5a:7e:cd:d2:78:b3:83:55:2e:fd:
         a7:dc:7c:f1:88:84:2b:0c:95:14:d3:5e:4e:70:a4:f3:9b:c7:
         9b:69:2c:c5:2a:45:b9:37:bc:70:b5:32:62:8d:6c:6e:81:ef:
         07:0d:0c:e3:b5:2a:41:79:1c:72:50:86:5a:f8:72:a7:36:8c:
         d2:ed:a6:a4:7e:7c:bb:4b:9a:13:30:2e:78:c2:ee:43:ac:93:
         92:d0:1b:3c:37:aa:88:78:7d:04:99:4b:2a:7b:d7:01:5e:6c:
         3e:9f:3f:1a:e6:36:d8:b3:2e:c9:24:98:64:ad:6c:b7:a4:25:
         7b:ab:17:59:e7:87:5d:03:26:2b:a3:b6:6c:87:a7:45:5f:c1:
         62:9a:52:b9:b0:54:02:f0:a7:3d:0a:e4:af:c1:39:e9:03:c3:
         2b:d1:e2:0a:c5:6d:da:ec:b7:2e:9c:85:6a:3f:fa:11:05:fe:
         d8:eb:16:85:5b:d9:4b:59:f4:6c:da:32:e9:19:f8:67:f0:f8:
         64:41:fa:99:3d:50:8e:69:2a:5d:88:be:ed:ec:c9:fe:9e:b9:
         ef:07:f6:8a:82:9a:32:1a:ff:2b:f9:79:68:d4:55:a9:47:e0:
         64:37:5f:02:0c:1e:ed:44:28:94:00:6e:3a:74:15:fb:82:75:
         03:84:64:4d
19

[Minor note. You’ve formatted perfectly the HAProxy configuration as Markdown “block” snippets. Please do the same for the other command outputs. As it stands now it’s hard to “see”…]

[It’s not necessarily your fault, but Discourse’s stubbornness to format everything as Markdown…]

20

Is that better? I edited it.