[OpenSIPS-Users] opensips crash when doing load test
Bogdan-Andrei Iancu
bogdan at opensips.org
Tue Oct 4 17:52:13 CEST 2016
Could you also print r_buf and*r_buf ?
Regards,
Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
http://www.opensips-solutions.com
On 04.10.2016 16:08, Ramachandran, Agalya (Contractor) wrote:
>
> Hi Bogdan,
>
> I have printed the “t” variable value. It says “$1 = (struct cell *)
> 0x0”.
>
> Is there any link available to upload/share the core files? Because
> size of the core file 45KB and couldn’t attach in e-mail.
>
> (gdb) frame 0
>
> #0 final_response_handler (fr_tl=0x7fded2484110) at timer.c:383
>
> 383 if (is_invite(t) && should_cancel_branch(t,
> r_buf->branch) ) {
>
> (gdb) list +
>
> 378 current_processing_ctx = my_ctx;
>
> 379 /* set the T context too */
>
> 380 set_t( t );
>
> 381
>
> 382 /* out-of-lock do the cancel I/O */
>
> 383 if (is_invite(t) && should_cancel_branch(t,
> r_buf->branch) ) {
>
> 384 set_cancel_extra_hdrs( CANCEL_REASON_SIP_480,
> sizeof(CANCEL_REASON_SIP_480)-1);
>
> 385 cancel_branch(t, r_buf->branch );
>
> 386 set_cancel_extra_hdrs( NULL, 0);
>
> 387 }
>
> (gdb) p t
>
> $1 = (struct cell *) 0x0
>
> Regards,
>
> Agalya
>
> *From:*Bogdan-Andrei Iancu [mailto:bogdan at opensips.org]
> *Sent:* Tuesday, October 04, 2016 6:44 AM
> *To:* Ramachandran, Agalya (Contractor)
> <Agalya_Ramachandran at comcast.com>; OpenSIPS users mailling list
> <users at lists.opensips.org>
> *Subject:* Re: opensips crash when doing load test
>
> Hello Agalya,
>
> In frame 0, could you print the "t" variable ? BTW, is there any way
> to get access to the core file ?
>
> Best regards,
>
> Bogdan-Andrei Iancu
> OpenSIPS Founder and Developer
> http://www.opensips-solutions.com
>
> On 30.09.2016 16:48, Ramachandran, Agalya (Contractor) wrote:
>
> Hi Bogdan,
>
> One more dump
>
> #0 final_response_handler (fr_tl=0x7fded2484110) at timer.c:383
>
> 383 if (is_invite(t) && should_cancel_branch(t, r_buf->branch) ) {
>
> Missing separate debuginfos, use: debuginfo-install
> cyrus-sasl-lib-2.1.26-20.el7_2.x86_64
> glibc-2.17-106.el7_2.4.x86_64 keyutils-libs-1.5.8-3.el7.x86_64
> krb5-libs-1.13.2-12.el7_2.x86_64 libcom_err-1.42.9-7.el7.x86_64
> libcurl-7.29.0-25.el7.centos.x86_64 libidn-1.28-4.el7.x86_64
> libselinux-2.2.2-6.el7.x86_64 libssh2-1.4.3-10.el7_2.1.x86_64
> nspr-4.10.8-2.el7_1.x86_64 nss-3.19.1-19.el7_2.x86_64
> nss-softokn-freebl-3.16.2.3-13.el7_1.x86_64
> nss-util-3.19.1-9.el7_2.x86_64 openldap-2.4.40-9.el7_2.x86_64
> openssl-libs-1.0.1e-51.el7_2.4.x86_64 pcre-8.32-15.el7.x86_64
> xz-libs-5.1.2-12alpha.el7.x86_64 zlib-1.2.7-15.el7.x86_64
>
> (gdb) bt
>
> #0 final_response_handler (fr_tl=0x7fded2484110) at timer.c:383
>
> #1 timer_routine (ticks=<optimized out>, set=<optimized out>) at
> timer.c:1066
>
> #2 0x000000000049d3cd in handle_timer_job () at timer.c:597
>
> #3 0x000000000059223e in handle_io (fm=<optimized out>,
> fm=<optimized out>, fm=<optimized out>, idx=<optimized out>,
> event_type=2)
>
> at net/net_udp.c:265
>
> #4 io_wait_loop_epoll (h=<optimized out>, t=<optimized out>,
> repeat=<optimized out>) at net/../io_wait_loop.h:225
>
> #5 udp_rcv_loop (si=si at entry=0x7fded841dcb0) at net/net_udp.c:308
>
> #6 0x0000000000593e05 in udp_start_processes
> (chd_rank=chd_rank at entry=0x84c570 <chd_rank.10940>,
> startup_done=startup_done at entry=0x0)
>
> at net/net_udp.c:448
>
> #7 0x0000000000419b73 in main_loop () at main.c:736
>
> #8 main (argc=<optimized out>, argv=<optimized out>) at main.c:1282
>
> *From:* Ramachandran, Agalya (Contractor)
> *Sent:* Friday, September 30, 2016 9:44 AM
> *To:* 'Bogdan-Andrei Iancu' <bogdan at opensips.org>
> <mailto:bogdan at opensips.org>; OpenSIPS users mailling list
> <users at lists.opensips.org> <mailto:users at lists.opensips.org>
> *Subject:* opensips crash when doing load test
>
> Hi Bogdan,
>
> I have taken 2.1.4 code base, added the below fixes on top of it
>
> 1.Fix potential starvation for the timer tasks.
> 66c337cc89a2b5c3e1bbd78c289371efcda21886
>
> 2.rest_client: Improve polling logic during TCP connect -
> c40f2b057209b8078520abc95e5d1bf5e5a28db6
>
> 3.rest_client: Add "connect_poll_interval" module parameter -
> f5abfb3dee649a14ae39ad74d7f04454f815748f
>
> 4.rest_client: Fix bogus condition
> -3ba7be6692c6987faecb9c4c904035d5e7c0f916
>
> When I try to perform load test on that code base, randomly am
> seeing OpenSIPS is crashing.
>
> Here is the full trace of the core file. Let me know if you need
> any additional information or logs.
>
> Am seeing crash is occurring at start_timer_extra_processes().
>
> [root at poc-siplb-cmc-e-004 corefiles]# gdb /usr/local/sbin/opensips
> core.opensips.sig11.3977
>
> GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-80.el7
>
> Copyright (C) 2013 Free Software Foundation, Inc.
>
> License GPLv3+: GNU GPL version 3 or later
> <http://gnu.org/licenses/gpl.html>
>
> This is free software: you are free to change and redistribute it.
>
> There is NO WARRANTY, to the extent permitted by law. Type "show
> copying"
>
> and "show warranty" for details.
>
> This GDB was configured as "x86_64-redhat-linux-gnu".
>
> For bug reporting instructions, please see:
>
> <http://www.gnu.org/software/gdb/bugs/>...
>
> Reading symbols from /usr/local/sbin/opensips...done.
>
> [New LWP 3977]
>
> [Thread debugging using libthread_db enabled]
>
> Using host libthread_db library "/lib64/libthread_db.so.1".
>
> Core was generated by `/usr/local/sbin/opensips -P
> /var/run/opensips.pid'.
>
> Program terminated with signal 11, Segmentation fault.
>
> #0 final_response_handler (fr_tl=0x7f55fcd82948) at timer.c:383
>
> 383 if (is_invite(t) && should_cancel_branch(t, r_buf->branch) ) {
>
> Missing separate debuginfos, use: debuginfo-install
> cyrus-sasl-lib-2.1.26-20.el7_2.x86_64
> glibc-2.17-106.el7_2.4.x86_64 keyutils-libs-1.5.8-3.el7.x86_64
> krb5-libs-1.13.2-12.el7_2.x86_64 libcom_err-1.42.9-7.el7.x86_64
> libcurl-7.29.0-25.el7.centos.x86_64 libidn-1.28-4.el7.x86_64
> libselinux-2.2.2-6.el7.x86_64 libssh2-1.4.3-10.el7_2.1.x86_64
> nspr-4.10.8-2.el7_1.x86_64 nss-3.19.1-19.el7_2.x86_64
> nss-softokn-freebl-3.16.2.3-13.el7_1.x86_64
> nss-util-3.19.1-9.el7_2.x86_64 openldap-2.4.40-9.el7_2.x86_64
> openssl-libs-1.0.1e-51.el7_2.4.x86_64 pcre-8.32-15.el7.x86_64
> xz-libs-5.1.2-12alpha.el7.x86_64 zlib-1.2.7-15.el7.x86_64
>
> (gdb) bt
>
> #0 final_response_handler (fr_tl=0x7f55fcd82948) at timer.c:383
>
> #1 timer_routine (ticks=<optimized out>, set=<optimized out>) at
> timer.c:1066
>
> #2 0x000000000049d03a in handle_timer_job () at timer.c:597
>
> #3 start_timer_extra_processes (chd_rank=chd_rank at entry=0x84c570
> <chd_rank.10940>) at timer.c:557
>
> #4 0x0000000000419b9a in main_loop () at main.c:748
>
> #5 main (argc=<optimized out>, argv=<optimized out>) at main.c:1282
>
> Regards,
>
> Agalya
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opensips.org/pipermail/users/attachments/20161004/0f094db1/attachment-0001.htm>
More information about the Users
mailing list