I assume that halog would be able to parse the log, if the first field was written by haproxy.
What do I need to do, to get a logfile, that becomes parsable to halog?
> haproxy -v
HA-Proxy version 2.2.8-7bf78d7 2021/01/13 - https://haproxy.org/
Status: long-term supported branch - will stop receiving fixes around Q2 2025.
Known bugs: http://www.haproxy.org/bugs/bugs-2.2.8.html
Running on: Linux 3.10.0-1160.11.1.el7.x86_64 #1 SMP Fri Dec 18 16:34:56 UTC 2020 x86_64
Excerpts from haproxy.cfg:
global
chroot /var/lib/haproxy
log /dev/log local0 info
pidfile /var/run/haproxy.pid
user haproxy
group haproxy
daemon
defaults
mode http
log global
option httplog
option dontlognull
/etc/rsyslog.d/49-haproxy.conf :
$AddUnixListenSocket /var/lib/haproxy/dev/log
$template HAProxy,"%TIMESTAMP% %syslogseverity-text:::UPPERCASE%: %msg:::drop-last-lf%\n"
$template HAProxyAccess,"%msg%\n"
if $programname startswith 'haproxy' then {
if $syslogseverity == 6 then {
action(type="omfile" file="/data/log/haproxy/access.log" template="HAProxyAccess")
stop
}
if $syslogseverity <= 3 then {
action(type="omfile" file="/data/log/haproxy/error.log" template="HAProxy")
stop
}
if $syslogseverity <= 5 then {
action(type="omfile" file="/data/log/haproxy/status.log" template="HAProxy")
stop
}
}
Indeed, the documentation is not clear enough for someone who doesn’t know how halog is parsing the logs.
Now, that you explained, I understand that halog is ignoring the first 5 columns (and in my case 4), and is then parsing the rest of the line.
Secondly, it is not obvious, that the number, that is passed alongside with -s also has to be prepended with a dash (i.e. -s -4 and not -s 4 or -s=4).
I am posting my halog --help only for further reference here:
# halog --help
Usage: halog [-h|--help] for long help
halog [-q] [-c] [-m <lines>]
{-cc|-gt|-pct|-st|-tc|-srv|-u|-uc|-ue|-ua|-ut|-uao|-uto|-uba|-ubt|-ic}
[-s <skip>] [-e|-E] [-H] [-rt|-RT <time>] [-ad <delay>] [-ac <count>]
[-v] [-Q|-QS] [-tcn|-TCN <termcode>] [ -hs|-HS [min][:[max]] ] [ -time [min][:[max]] ] < log
Input filters (several filters may be combined) :
-H only match lines containing HTTP logs (ignore TCP)
-E only match lines without any error (no 5xx status)
-e only match lines with errors (status 5xx or negative)
-rt|-RT <time> only match response times larger|smaller than <time>
-Q|-QS only match queued requests (any queue|server queue)
-tcn|-TCN <code> only match requests with/without termination code <code>
-hs|-HS <[min][:][max]> only match requests with HTTP status codes within/not
within min..max. Any of them may be omitted. Exact
code is checked for if no ':' is specified.
-time <[min][:max]> only match requests recorded between timestamps.
Any of them may be omitted.
Modifiers
-v invert the input filtering condition
-q don't report errors/warnings
-m <lines> limit output to the first <lines> lines
-s <skip_n_fields> skip n fields from the beginning of a line (default 5)
you can also use -n to start from earlier then field 5
Output filters - only one may be used at a time
-c only report the number of lines that would have been printed
-pct output connect and response times percentiles
-st output number of requests per HTTP status code
-cc output number of requests per cookie code (2 chars)
-tc output number of requests per termination code (2 chars)
-srv output statistics per server (time, requests, errors)
-ic output statistics per ip count (time, requests, errors)
-u* output statistics per URL (time, requests, errors)
Additional characters indicate the output sorting key :
-u : by URL, -uc : request count, -ue : error count
-ua : average response time, -ut : average total time
-uao, -uto: average times computed on valid ('OK') requests
-uba, -ubt: average bytes returned, total bytes returned