Segmentation Fault on raspberry pi

Help!
1

Hello,

I have been running HA-Proxy version 1.7.5-2 2017/05/27 from the raspbian repo for several months without issue.

I’d like to now run a more up to date version on Centos for ARM so I’ve compiled both 1.9.0 and 2.0-dev4 on a fresh raspberry pi with CentOS 7 installed. However for both versions i get an immediate segmentation fault.

I dont really know what I’m doing with gdb but I tried running it with gdb and this is the output below; can anyone give me any insights as to what may be happening here?

TIA.
John.

# gdb /usr/local/sbin/haproxy
GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-114.el7
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "armv7hl-redhat-linux-gnueabi".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/local/sbin/haproxy...done.
(gdb) set pagination 0
(gdb) run -f /etc/haproxy/haproxy.cfg -d
Starting program: /usr/local/sbin/haproxy -f /etc/haproxy/haproxy.cfg -d
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/libthread_db.so.1".
warning: Probes-based dynamic linker interface failed.
Reverting to original interface.

Cannot access memory at address 0xc2400000
Missing separate debuginfos, use: debuginfo-install bzip2-libs-1.0.6-13.el7.armv7hl elfutils-libelf-0.172-2.el7.armv7hl elfutils-libs-0.172-2.el7.armv7hl glibc-2.17-260.el7_6.5.armv7hl keyutils-libs-1.5.8-3.el7.armv7hl krb5-libs-1.15.1-37.el7_6.armv7hl libattr-2.4.46-13.el7.armv7hl libcap-2.22-9.el7.armv7hl libcom_err-1.42.9-13.el7.armv7hl libgcc-4.8.5-36.el7_6.2.armv7hl libgcrypt-1.5.3-14.el7.armv7hl libgpg-error-1.12-3.el7.armv7hl libselinux-2.5-14.1.el7.armv7hl lz4-1.7.5-2.el7.armv7hl nss-softokn-freebl-3.36.0-5.el7_5.armv7hl openssl-libs-1.0.2k-16.el7_6.1.armv7hl pcre-8.32-17.el7.armv7hl systemd-libs-219-62.el7_6.6.armv7hl xz-libs-5.2.2-1.el7.armv7hl zlib-1.2.7-18.el7.armv7hl
(gdb) bt
#0  0x76fe46f4 in dl_open_worker () from /lib/ld-linux-armhf.so.3
#1  0x76fdf970 in _dl_catch_error () from /lib/ld-linux-armhf.so.3
#2  0x76fe3c68 in _dl_open () from /lib/ld-linux-armhf.so.3
#3  0x76b6c9c8 in do_dlopen () from /lib/libc.so.6
#4  0x76fdf970 in _dl_catch_error () from /lib/ld-linux-armhf.so.3
#5  0x76b6cad4 in __libc_dlopen_mode () from /lib/libc.so.6
#6  0x76b401a4 in nss_load_library () from /lib/libc.so.6
#7  0x76b40d38 in __nss_lookup_function () from /lib/libc.so.6
#8  0x76b40e90 in __nss_lookup () from /lib/libc.so.6
#9  0x76aee4dc in getpwnam_r@@GLIBC_2.4 () from /lib/libc.so.6
#10 0x76aedc14 in getpwnam () from /lib/libc.so.6
#11 0x00146d78 in cfg_parse_global (file=0x2c7178 "/etc/haproxy/haproxy.cfg", linenum=7, args=0x7efff02c, kwm=<optimized out>) at src/cfgparse-global.c:475
#12 0x000dd7f4 in readcfgfile (file=0x2a9528 "/etc/haproxy/haproxy.cfg") at src/cfgparse.c:2147
#13 0x000d6e64 in init (argc=<optimized out>, argc@entry=4, argv=<optimized out>, argv@entry=0x76a556b0) at src/haproxy.c:1649
#14 0x0002bc90 in main (argc=4, argv=0x76a556b0) at src/haproxy.c:2683
2

Can you provide the output of haproxy -vv (unless that crashes also)?

3

here you go:

# /usr/local/sbin/haproxy -vv
HA-Proxy version 2.0-dev4 2019/05/22 - https://haproxy.org/
Build options :
  TARGET  = linux2628
  CPU     = generic
  CC      = gcc
  CFLAGS  = -O2 -g -fno-strict-aliasing -Wdeclaration-after-statement -fwrapv -Wno-unused-label -Wno-sign-compare -Wno-unused-parameter -Wno-old-style-declaration -Wno-ignored-qualifiers -Wno-clobbered -Wno-missing-field-initializers -Wtype-limits
  OPTIONS = USE_PCRE=1 USE_OPENSSL=1 USE_LUA=1 USE_ZLIB=1 USE_TFO=1 USE_NS=1 USE_SYSTEMD=1

Feature list : +EPOLL -KQUEUE -MY_EPOLL -MY_SPLICE +NETFILTER +PCRE -PCRE_JIT -PCRE2 -PCRE2_JIT +POLL -PRIVATE_CACHE +THREAD -PTHREAD_PSHARED -REGPARM -STATIC_PCRE -STATIC_PCRE2 +TPROXY +LINUX_TPROXY +LINUX_SPLICE +LIBCRYPT +CRYPT_H -VSYSCALL -GETADDRINFO +OPENSSL +LUA +FUTEX +ACCEPT4 -MY_ACCEPT4 +ZLIB -SLZ +CPU_AFFINITY +TFO +NS +DL +RT -DEVICEATLAS -51DEGREES -WURFL +SYSTEMD -OBSOLETE_LINKER +PRCTL +THREAD_DUMP -EVPORTS

Default settings :
  bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with multi-threading support (MAX_THREADS=32, default=4).
Built with OpenSSL version : OpenSSL 1.0.2k-fips  26 Jan 2017
Running on OpenSSL version : OpenSSL 1.0.2k-fips  26 Jan 2017
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : SSLv3 TLSv1.0 TLSv1.1 TLSv1.2
Built with Lua version : Lua 5.3.4
Built with network namespace support.
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
Built with zlib version : 1.2.7
Running on zlib version : 1.2.7
Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with PCRE version : 8.32 2012-11-30
Running on PCRE version : 8.32 2012-11-30
PCRE library supports JIT : no (USE_PCRE_JIT not set)
Encrypted password support via crypt(3): yes

Available polling systems :
      epoll : pref=300,  test result OK
       poll : pref=200,  test result OK
     select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

Available multiplexer protocols :
(protocols marked as <default> cannot be specified using 'proto' keyword)
              h2 : mode=HTX        side=FE|BE     mux=H2
              h2 : mode=HTTP       side=FE        mux=H2
       <default> : mode=HTX        side=FE|BE     mux=H1
       <default> : mode=TCP|HTTP   side=FE|BE     mux=PASS

Available services : none

Available filters :
	[SPOE] spoe
	[COMP] compression
	[CACHE] cache
	[TRACE] trace
4

Can you try without the user and group directives in the configuration (keep using root)?

5

Running as root and no user/group in the cofig I still get a Segmentation Fault but there’s a diffferent output from gdb:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x75e5bbf0 (LWP 1001)]
fd_rm_from_fd_list (list=0x2997d8 <update_list>, fd=fd@entry=8, off=off@entry=24) at src/fd.c:275
275		    unlikely(!_HA_ATOMIC_DWCAS(((void **)(void *)&_GET_NEXT(fd, off)), ((void **)(void *)&cur_list), (*(void **)(void *)&next_list))))
6

I’m investigating, it’s about threading.

Workaround for now is to disable threading by adding nbthread 1 to your global section.

7

You probably meant 1.9.8 because the bug was only introduced in that release (1.9.7 and older work fine).

I’ve filed a bug:

8

A fix has been committed to -dev and queued up for 1.9 (will be in the 1.9.9 release):

9

Thanks guys, built this version and its no longer crashing:

HA-Proxy version 2.0-dev4-c3b595-33 2019/05/27 - https://haproxy.org/
10

May you share your build steps for armv7 ?

I am trying to use the min version 1.9 too on Raspberry Pi

is it just:
make TARGET=custom

11

this worked:
make TARGET=custom
sudo make install

pi@RPi01:~/haproxy-2.0-dev6 $ haproxy -vv
HA-Proxy version 2.0-dev6 2019/06/07 - https://haproxy.org/
Build options :
TARGET = custom
CPU = generic
CC = gcc
CFLAGS = -O2 -g -fno-strict-aliasing -Wdeclaration-after-statement -fwrapv -Wno-unused-label -Wno-sign-compare -Wno-unused-parameter -Wno-old-style-declaration -Wno-ignored-qualifiers -Wno-clobbered -Wno-missing-field-initializers -Wtype-limits -Wshift-negative-value -Wshift-overflow=2 -Wduplicated-cond -Wnull-dereference
OPTIONS =

Feature list : -EPOLL -KQUEUE -MY_EPOLL -MY_SPLICE -NETFILTER -PCRE -PCRE_JIT -PCRE2 -PCRE2_JIT +POLL -PRIVATE_CACHE -THREAD -PTHREAD_PSHARED -REGPARM -STATIC_PCRE -STATIC_PCRE2 -TPROXY -LINUX_TPROXY -LINUX_SPLICE -LIBCRYPT -CRYPT_H -VSYSCALL -GETADDRINFO -OPENSSL -LUA -FUTEX -ACCEPT4 -MY_ACCEPT4 -ZLIB -SLZ -CPU_AFFINITY -TFO -NS -DL -RT -DEVICEATLAS -51DEGREES -WURFL -SYSTEMD -OBSOLETE_LINKER -PRCTL -THREAD_DUMP -EVPORTS

Default settings :
bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with transparent proxy support using: IP_TRANSPARENT IP_FREEBIND
Built without compression support (neither USE_ZLIB nor USE_SLZ are set).
Compression algorithms supported : identity(“identity”)
Built without PCRE or PCRE2 support (using libc’s regex instead)
Encrypted password support via crypt(3): yes
Built without multi-threading support (USE_THREAD not set).

Available polling systems :
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 2 (2 usable), will use poll.

Available multiplexer protocols :
(protocols marked as cannot be specified using ‘proto’ keyword)
h2 : mode=HTX side=FE|BE mux=H2
h2 : mode=HTTP side=FE mux=H2
: mode=HTX side=FE|BE mux=H1
: mode=TCP|HTTP side=FE|BE mux=PASS

Available services : none

Available filters :
[SPOE] spoe
[COMP] compression
[CACHE] cache
[TRACE] trace

12

Please don’t use make TARGET=custom unless you really know what you are doing.

Use make TARGET=linux2628 if you are an Linux (which I assume you are).

CPU=generic is fine.