[OpenSIPS-Users] segfault at 7f4c471852 ip 00007f4dc4a0ccc0 sp 00007ffc74776350 error 4 in libc-2.23.so[7f4dc49be000+1c0000]

Fri Oct 2 01:03:14 EST 2020

Although I have not sparked any interest in this so far, I was able to get more information with DEBUG turned on.
This looks like it is an issue connecting to mysql, the mysql logs are turned on with no additional information shown pertaining to this.  The query log shows normal dispatcher updates right up to the crash

Oct  1 20:53:18 [7833] DBG:core:do_workers_auto_scaling: group 1 (with 5 procs) has average load of 100
Oct  1 20:53:18 [7833] NOTICE:core:do_workers_auto_scaling: score 5/5 -> forking new proc in group 1 (with 5 procs)
Oct  1 20:53:18 [7833] DBG:core:internal_fork: forking new process "UDP receiver" on slot 13
Oct  1 20:53:18 [8083] DBG:core:io_watch_add: [UDP_worker] io_watch_add op (5 on 71) (0xa5b9e0, 5, 1, (nil),1), fd_no=0/1024
Oct  1 20:53:18 [8083] DBG:core:io_watch_add: [UDP_worker] io_watch_add op (38 on 71) (0xa5b9e0, 38, 4, (nil),1), fd_no=1/1024
Oct  1 20:53:18 [8083] DBG:core:io_watch_add: [UDP_worker] io_watch_add op (7 on 71) (0xa5b9e0, 7, 4, (nil),1), fd_no=2/1024
Oct  1 20:53:18 [8083] DBG:core:io_watch_add: [UDP_worker] io_watch_add op (68 on 71) (0xa5b9e0, 68, 17, 0x7f1d8c848f78,1), fd_no=3/1024
Oct  1 20:53:18 [8083] DBG:core:init_mod_child: type=CHILD, rank=10000, module=event_jsonrpc
Oct  1 20:53:18 [8083] DBG:core:init_mod_child: type=CHILD, rank=10000, module=tm
Oct  1 20:53:18 [8083] DBG:tm:child_init_callid: callid: '089b59c7452bb9fd-8083 at xxx.xxx.xxx.xxx'
Oct  1 20:53:18 [8083] DBG:core:init_mod_child: type=CHILD, rank=10000, module=domain
Oct  1 20:53:18 [8083] DBG:core:init_mod_child: type=CHILD, rank=10000, module=dispatcher
Oct  1 20:53:18 [8083] CRITICAL:dispatcher:ds_connect_db: BUG - db connection found already open
Oct  1 20:53:18 [8083] ERROR:dispatcher:ds_child_init: failed to do DB connect
Oct  1 20:53:18 [8083] ERROR:core:init_mod_child: failed to initializing module dispatcher, rank 10000
Oct  1 20:53:18 [8083] DBG:core:send_status_code: sent code -1 (1 byte)
Oct  1 20:53:18 [8083] ERROR:core:fork_dynamic_udp_process: Initializing new process failed, exiting with error
Oct  1 20:53:18 [7833] DBG:core:wait_status_code: read code -1 (1 byte)
Oct  1 20:53:18 [7833] ERROR:core:do_workers_auto_scaling: failed to fork new process for group 1 (current 5 procs)
Oct  1 20:53:18 [7833] DBG:core:handle_sigs: OpenSIPS exit status = 139
Oct  1 20:53:18 [7833] INFO:core:handle_sigs: child process 7847 exited by a signal 11
Oct  1 20:53:18 [7833] INFO:core:handle_sigs: core was generated
Oct  1 20:53:18 [7833] INFO:core:handle_sigs: terminating due to SIGCHLD

From: Users <users-bounces at lists.opensips.org> On Behalf Of CARTWRIGHT, CORY C
Sent: Monday, September 21, 2020 10:53
To: users at lists.opensips.org
Subject: [OpenSIPS-Users] segfault at 7f4c471852 ip 00007f4dc4a0ccc0 sp 00007ffc74776350 error 4 in libc-2.23.so[7f4dc49be000+1c0000]

***Security Advisory: This Message Originated Outside of AT&T ***
Reference http://cso.att.com/EmailSecurity/IDSP.html for more information.
Hello,   Intermittently opensips is crashing  "segfault at 7f4c471852 ip 00007f4dc4a0ccc0 sp 00007ffc74776350 error 4 in libc-2.23.so[7f4dc49be000+1c0000]" showing in the syslog.  I have been unable to duplicate in LAB or determine any reason for the crash.

Searched the  archives and was unable to find anything related to libc-2.23.so, google has a many hits but all seem to eventually point back to the application.
Any help would be greatly appreciated

version: opensips 3.0.2 (x86_64/linux)
flags: STATS: On, DISABLE_NAGLE, USE_MCAST, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, HP_MALLOC, DBG_MALLOC, FAST_LOCK-ADAPTIVE_WAIT
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535
poll method support: poll, epoll, sigio_rt, select.
git revision: 7e8167853
main.c compiled on 15:39:14 Jun  2 2020 with gcc 5.4.0

# gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/5/lto-wrapper
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 5.4.0-6ubuntu1~16.04.12' --with-bugurl=file:///usr/share/doc/gcc-5/README.Bugs --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-5 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-libmpx --enable-plugin --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-5-amd64/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-5-amd64 --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-5-amd64 --with-arch-directory=amd64 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-objc-gc --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.12)

Thanks
Cory

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opensips.org/pipermail/users/attachments/20201002/bfeaba46/attachment-0001.html>