Chroot, log, /dev/log, sendmsg logger failed, and a rabbit hole

Help!
1

Hi all,

I went down this rabbit hole, so here ya go.

HAProxy 1.8, RHEL 8. I wanted to use a unix socket for the logging messages.

I created a new chroot for haproxy using systemd-tmpfiles.

d /run/haproxy-chroot 750 root haproxy

Throw that in a config file under /etc/tmpfiles.d/

I then had rsyslog forward the messages it gets at a new unix socket inside the jail, cribbed from How To Configure HAProxy Logging with Rsyslog on CentOS 8 [Quickstart] | DigitalOcean

$AddUnixListenSocket /run/haproxy-chroot/log.socket

# Send HAProxy messages to a dedicated logfile
:programname, startswith, "haproxy" {
  /var/log/haproxy.log
  stop
}

In the haproxy config file:

    chroot      /run/haproxy-chroot
    log         /log.socket local0

Looks good, right? After the chroot, the socket /log.socket was available, logging was working.

But during service start, I was getting this message:

[ALERT] 054/184714 (---SUPERVISOR PROCESS PID HERE---) : sendmsg logger #1 failed: No such file or directory (errno=2)

Here’s what I think is going on: the log directive is processed TWICE. Once before chroot, once after. Once by the ‘master’ process, once by the ‘worker’ process.

So the logging socket has to be available INSIDE and OUTSIDE the chroot. It needs to be available at both /the/absolute/path and /inside/chroot/the/absolute/path.

My fix:

d /run/haproxy-chroot 750 root haproxy
d /run/haproxy-chroot/dev 750 root haproxy

create two temp dirs

$AddUnixListenSocket /run/haproxy-chroot/dev/log

# Send HAProxy messages to a dedicated logfile
:programname, startswith, "haproxy" {
  /var/log/haproxy.log
  stop
}

Haproxy config:

    chroot      /run/haproxy-chroot
    log         /dev/log local0

When the master process is running, it can see the ‘real’ /dev/log, which is piped to the systemd journal in RHEL 8, as far as I can figure.

After the chroot, the worker process sees the socket created by rsyslog in the choot, at /run/haproxy-chroot/dev/log.

If you’re using master/worker, to avoid errors, make sure the log ‘destination’ is available before AND after the chroot. Or just use TCP, like the sample config suggests. :laughing:

Hope that helps someone else.