I’m using HA-Proxy LB (Independent server - Outside of my cluster) and Kubernetes v1.24.9 cluster (One master and three workers) bare metal based.
HAProxy Environment:
OS: Ubuntu 18.04.6 LTS
HAProxy version 2.4.20-1ppa1~bionic 2022/12/09
My Application access workflow:-
Users → HAproxy(Front End → Back End) → Kubernetes Cluster(Nginx Ingress → Service → Jenkins POD)
I have deployed Jenkins application as a pod inside my cluster. When i access my Jenkins URL - http://jenkins.company.com/jenkins on browser i should see my Jenkins page. But on browser it shows below errors.
This site can’t be reached
jenkins.company.com refused to connect.
Try:
Checking the connection Checking the proxy and the firewall
ERR_CONNECTION_REFUSED
However curl command onMaster, Worker and HAProxy(LB) terminal it is working. Could see the log in kubernetes nginx-ingress controller as well.
$ curl -L -D- http://jenkins.company.com/jenkins
HTTP/1.1 302 Found
server: nginx/1.23.3
date: Fri, 27 Jan 2023 03:09:16 GMT
transfer-encoding: chunked
location: http://jenkins.company.com/jenkins/
HTTP/1.1 403 Forbidden
server: nginx/1.23.3
date: Fri, 27 Jan 2023 03:09:16 GMT
content-type: text/html;charset=utf-8
content-length: 577
x-content-type-options: nosniff
set-cookie: JSESSIONID.4c905946=node01rf6pgi17m6zrmfr1lt0e2sjf11.node0; Path=/jenkins; HttpOnly
expires: Thu, 01 Jan 1970 00:00:00 GMT
x-hudson: 1.395
x-jenkins: 2.361.2
x-jenkins-session: 3e8dd8d9
<html><head><meta http-equiv='refresh' content='1;url=/jenkins/login?from=%2Fjenkins%2F'/><script>window.location.replace('/jenkins/login?from=%2Fjenkins%2F');</script></head><body style='background-color:white; color:white;'>
Authentication required
Whereas when i tried to access URL from browser couldn’t see the log information in nginx-ingress controller pod. Have tried with Firebox, Chrome and Edge browsers.
My HAproxy /etc/haproxy/haproxy.cfg configurations as follows,
$ sudo cat /etc/haproxy/haproxy.cfg
global
log /dev/log local0
log /dev/log local1 notice
chroot /var/lib/haproxy
stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd listeners
stats timeout 30s
user haproxy
group haproxy
daemon
# Default SSL material locations
ca-base /etc/ssl/certs
crt-base /etc/ssl/private
# See: https://ssl-config.mozilla.org/#server=haproxy&server-version=2.0.3&config=intermediate
ssl-default-bind-ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384
ssl-default-bind-ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256
ssl-default-bind-options ssl-min-ver TLSv1.2 no-tls-tickets
defaults
log global
mode http
option httplog
option dontlognull
timeout connect 5000
timeout client 50000
timeout server 50000
errorfile 400 /etc/haproxy/errors/400.http
errorfile 403 /etc/haproxy/errors/403.http
errorfile 408 /etc/haproxy/errors/408.http
errorfile 500 /etc/haproxy/errors/500.http
errorfile 502 /etc/haproxy/errors/502.http
errorfile 503 /etc/haproxy/errors/503.http
errorfile 504 /etc/haproxy/errors/504.http
frontend http_front
bind *:80
mode http
default_backend http_back
backend http_back
balance roundrobin
mode http
server Kmaster 163.12.13.21:80 check
server Kworker-1 163.12.13.22:80 check
server Kworker-2 163.12.13.23:80 check
server Kworker-3 163.12.13.24:80 check
$ ps -ef | grep haproxy
root 16205 1 0 09:51 ? 00:00:00 /usr/sbin/haproxy -Ws -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -S /run/haproxy-master.sock
haproxy 16207 16205 0 09:51 ? 00:00:00 /usr/sbin/haproxy -Ws -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -S /run/haproxy-master.sock
On my HAProxy(LB) server port 80 is listening
$ sudo netstat -pnltu | grep 80
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 7529/haproxy
$ sudo ss -4 -tlnp | grep 80
LISTEN 0 2005 0.0.0.0:80 0.0.0.0:* users:(("haproxy",pid=13517,fd=8))
My ingress-resource.yaml file content as follows
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
name: jenkins-ingress
namespace: jenkins
spec:
ingressClassName: nginx
rules:
- host: jenkins.company.com
http:
paths:
- path: /jenkins
pathType: Prefix
backend:
service:
name: jenkins-svc
port:
number: 80
$ kubectl get all -n nginx-ingress
NAME READY STATUS RESTARTS AGE
pod/nginx-ingress-kv88r 1/1 Running 1 (45h ago) 2d20h
pod/nginx-ingress-ns5ff 1/1 Running 1 (45h ago) 2d20h
pod/nginx-ingress-pzkt4 1/1 Running 1 (45h ago) 2d20h
pod/nginx-ingress-v7cgx 1/1 Running 1 (45h ago) 2d20h
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/nginx-ingress 4 4 4 4 4 <none> 2d20h
$ kubectl get svc --all-namespaces
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 17d
jenkins jenkins-svc ClusterIP 10.99.106.220 <none> 80/TCP 3d15h
kube-system kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 17d
Please let me know why i can’t access the url in browser?
My request not processed and forwarded to Kubernetes nginx-contoller,
Whats wrong with my configuration side?
Any guidance to solve this issue would be helpful.
Thank you!