Halog not able to parse logfile

Hi
I noticed that halog is not able to parse my logfile.
I did not specify a custom log format.

> cat access.log | halog -u -H -q
#req err ttot tavg oktot okavg bavg btot src
>

It seems that the first field is not written to the log. (Notice the leading space)

> cat access.log | head -1
 91.1.2.3:52057 [17/Feb/2021:03:09:01.597] EXT_WEB~ be/srv2:443 0/0/0/11/11 401 1737 - - --NI 469/346/156/268009/0 0/0 "GET / HTTP/1.1"

I assume that halog would be able to parse the log, if the first field was written by haproxy.
What do I need to do, to get a logfile, that becomes parsable to halog?

> haproxy -v
HA-Proxy version 2.2.8-7bf78d7 2021/01/13 - https://haproxy.org/
Status: long-term supported branch - will stop receiving fixes around Q2 2025.
Known bugs: http://www.haproxy.org/bugs/bugs-2.2.8.html
Running on: Linux 3.10.0-1160.11.1.el7.x86_64 #1 SMP Fri Dec 18 16:34:56 UTC 2020 x86_64

Excerpts from haproxy.cfg:

global
    chroot      /var/lib/haproxy
    log         /dev/log local0 info
    pidfile     /var/run/haproxy.pid
    user        haproxy
    group       haproxy
    daemon
defaults
    mode                    http
    log                     global
    option                  httplog
    option                  dontlognull

/etc/rsyslog.d/49-haproxy.conf :

$AddUnixListenSocket /var/lib/haproxy/dev/log
$template HAProxy,"%TIMESTAMP% %syslogseverity-text:::UPPERCASE%: %msg:::drop-last-lf%\n"
$template HAProxyAccess,"%msg%\n"

if $programname startswith 'haproxy' then {
  if $syslogseverity == 6 then {
      action(type="omfile" file="/data/log/haproxy/access.log" template="HAProxyAccess")
      stop
  }
  if $syslogseverity <= 3 then {
      action(type="omfile" file="/data/log/haproxy/error.log" template="HAProxy")
      stop
  }
  if $syslogseverity <= 5 then {
      action(type="omfile" file="/data/log/haproxy/status.log" template="HAProxy")
      stop
  }
}

Don’t use -q when you troubleshoot this, because it will hide the summary which you need to know:

1 lines in, 0 lines out, 1 parsing errors

halog expects additional garbage in front of the log by default. If your log looks like you this, you need to add -s -4:

lukas@dev:~/haproxy/contrib/halog$ echo ' 91.1.2.3:52057 [17/Feb/2021:03:09:01.597] EXT_WEB~ be/srv2:443 0/0/0/11/11 401 1737 - - --NI 469/346/156/268009/0 0/0 "GET / HTTP/1.1"' | \
> ./halog -H -u
#req err ttot tavg oktot okavg bavg btot src
1 lines in, 0 lines out, 1 parsing errors
lukas@dev:~/haproxy/contrib/halog$
lukas@dev:~/haproxy/contrib/halog$
lukas@dev:~/haproxy/contrib/halog$ echo ' 91.1.2.3:52057 [17/Feb/2021:03:09:01.597] EXT_WEB~ be/srv2:443 0/0/0/11/11 401 1737 - - --NI 469/346/156/268009/0 0/0 "GET / HTTP/1.1"' | \
> ./halog -s -4 -H -u
#req err ttot tavg oktot okavg bavg btot src
1 0 11 11 11 11 1737 1737 /
1 lines in, 1 lines out, 0 parsing errors
lukas@dev:~/haproxy/contrib/halog$
lukas@dev:~/haproxy/contrib/halog$

I’m not claiming this is documented, let alone self-explanatory …

1 Like

Many thanks @lukastribus this resolved my issue.

Indeed, the documentation is not clear enough for someone who doesn’t know how halog is parsing the logs.
Now, that you explained, I understand that halog is ignoring the first 5 columns (and in my case 4), and is then parsing the rest of the line.
Secondly, it is not obvious, that the number, that is passed alongside with -s also has to be prepended with a dash (i.e. -s -4 and not -s 4 or -s=4).

I am posting my halog --help only for further reference here:

# halog --help
Usage: halog [-h|--help] for long help
       halog [-q] [-c] [-m <lines>]
       {-cc|-gt|-pct|-st|-tc|-srv|-u|-uc|-ue|-ua|-ut|-uao|-uto|-uba|-ubt|-ic}
       [-s <skip>] [-e|-E] [-H] [-rt|-RT <time>] [-ad <delay>] [-ac <count>]
       [-v] [-Q|-QS] [-tcn|-TCN <termcode>] [ -hs|-HS [min][:[max]] ] [ -time [min][:[max]] ] < log

Input filters (several filters may be combined) :
 -H                      only match lines containing HTTP logs (ignore TCP)
 -E                      only match lines without any error (no 5xx status)
 -e                      only match lines with errors (status 5xx or negative)
 -rt|-RT <time>          only match response times larger|smaller than <time>
 -Q|-QS                  only match queued requests (any queue|server queue)
 -tcn|-TCN <code>        only match requests with/without termination code <code>
 -hs|-HS <[min][:][max]> only match requests with HTTP status codes within/not
                         within min..max. Any of them may be omitted. Exact
                         code is checked for if no ':' is specified.
 -time <[min][:max]>     only match requests recorded between timestamps.
                         Any of them may be omitted.
Modifiers
 -v                      invert the input filtering condition
 -q                      don't report errors/warnings
 -m <lines>              limit output to the first <lines> lines
 -s <skip_n_fields>      skip n fields from the beginning of a line (default 5)
                         you can also use -n to start from earlier then field 5

Output filters - only one may be used at a time
 -c    only report the number of lines that would have been printed
 -pct  output connect and response times percentiles
 -st   output number of requests per HTTP status code
 -cc   output number of requests per cookie code (2 chars)
 -tc   output number of requests per termination code (2 chars)
 -srv  output statistics per server (time, requests, errors)
 -ic   output statistics per ip count (time, requests, errors)
 -u*   output statistics per URL (time, requests, errors)
       Additional characters indicate the output sorting key :
       -u : by URL, -uc : request count, -ue : error count
       -ua : average response time, -ut : average total time
       -uao, -uto: average times computed on valid ('OK') requests
       -uba, -ubt: average bytes returned, total bytes returned

Yeah … halog has been written as a tool for the developers initially and it shows. Helptext output would have to be clarified.

1 Like