[OpenSIPS-Devel] OpenSIPS Crash
Bogdan-Andrei Iancu
bogdan at opensips.org
Thu Nov 1 13:19:01 EDT 2018
Hi Ben,
According to the backtrace, the memory debugger was not activated. Do an
"opensips -V" to check the resulting compile flags - do you see the
DBG_MALLOC and QM_MALLOC ?
Regards,
Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
http://www.opensips-solutions.com
OpenSIPS Bootcamp 2018
http://opensips.org/training/OpenSIPS_Bootcamp_2018/
On 10/31/2018 05:04 PM, Ben Newlin wrote:
>
> Bogdan,
>
> I was able to compile with those options and the crash has occurred
> again. Backtrace is here: https://pastebin.com/dezi9xUU.
>
> Even though I had `memdump=1` set in my script, there was no extra
> memory debugging information in the logs prior to or at the time of
> the crash. I’m not sure if that is expected or not.
>
> Ben Newlin
>
> *From: *Bogdan-Andrei Iancu <bogdan at opensips.org>
> *Date: *Monday, October 29, 2018 at 8:11 AM
> *To: *Ben Newlin <Ben.Newlin at genesys.com>, OpenSIPS devel mailling
> list <devel at lists.opensips.org>
> *Subject: *Re: [OpenSIPS-Devel] OpenSIPS Crash
>
> Hi Ben,
>
> You can change the compile flags via the Makefile.conf file - the
> menuconfig is also updating that file. So during your build you can
> simply push a pre-modified Makefile.conf file with the options needed
> for memory debugging.
>
> Regards,
>
> Bogdan-Andrei Iancu
> OpenSIPS Founder and Developer
> http://www.opensips-solutions.com
> OpenSIPS Bootcamp 2018
> http://opensips.org/training/OpenSIPS_Bootcamp_2018/
>
> On 10/26/2018 05:14 PM, Ben Newlin wrote:
>
> Bogdan,
>
> Unfortunately, we have run into a similar issue before. Our build
> system is completely automated and there is no way to inject the
> `make menuconfig` interactive step into that process. If I were to
> be testing this locally I might be able to work something out, but
> I could never get such a build into our testing environment which
> is where the crashes are occurring.
>
> Do you have instructions for enabling memory debugging that do not
> require using the interactive TUI tool? What does the menuconfig
> program do when these options are selected? Are there some defines
> or other settings we can change ourselves and bypass menuconfig?
>
> Ben Newlin
>
> *From: *Bogdan-Andrei Iancu <bogdan at opensips.org>
> <mailto:bogdan at opensips.org>
> *Date: *Friday, October 26, 2018 at 4:59 AM
> *To: *OpenSIPS devel mailling list <devel at lists.opensips.org>
> <mailto:devel at lists.opensips.org>, Ben Newlin
> <Ben.Newlin at genesys.com> <mailto:Ben.Newlin at genesys.com>
> *Subject: *Re: [OpenSIPS-Devel] OpenSIPS Crash
>
> Hi Ben,
>
> all the BT's points to crashes while doing memory ops. I suspect a
> memory corruption that randomly triggers crashes in different
> parts of the code.
>
> Could you try to re-compile with memory debugging support ? See
> http://www.opensips.org/Documentation/TroubleShooting-OutOfMem,
> the "How to handle it" section.
>
> Regards,
>
>
> Bogdan-Andrei Iancu
>
>
>
> OpenSIPS Founder and Developer
>
> http://www.opensips-solutions.com
>
> OpenSIPS Bootcamp 2018
>
> http://opensips.org/training/OpenSIPS_Bootcamp_2018/
>
> On 10/24/2018 04:28 AM, Ben Newlin wrote:
>
> We have had 2 more crashes today.
>
> Crash 2: https://pastebin.com/rMruBQcZ
> <https://pastebin.com/rMruBQcZ>
>
> This crash appears to have occurred while processing an
> initial INVITE request. I could not see anything unusual about
> the request. I cannot tell if this crash is related to the others.
>
> Crash 3: https://pastebin.com/Gmk1m4NT
> <https://pastebin.com/Gmk1m4NT>
>
> This crash follows the pattern of the original crash I reported.
>
> Ben Newlin
>
> *From: *Devel <devel-bounces at lists.opensips.org>
> <mailto:devel-bounces at lists.opensips.org> on behalf of Ben
> Newlin <Ben.Newlin at genesys.com> <mailto:Ben.Newlin at genesys.com>
> *Reply-To: *OpenSIPS devel mailling list
> <devel at lists.opensips.org> <mailto:devel at lists.opensips.org>
> *Date: *Monday, October 22, 2018 at 4:45 PM
> *To: *OpenSIPS devel mailling list <devel at lists.opensips.org>
> <mailto:devel at lists.opensips.org>
> *Subject: *Re: [OpenSIPS-Devel] OpenSIPS Crash
>
> Here is a better trace of the call:
> https://pastebin.com/gWpQR8E7 <https://pastebin.com/gWpQR8E7>
>
> Ben Newlin
>
> *From: *Ben Newlin <Ben.Newlin at genesys.com>
> <mailto:Ben.Newlin at genesys.com>
> *Date: *Monday, October 22, 2018 at 4:34 PM
> *To: *OpenSIPS devel mailling list <devel at lists.opensips.org>
> <mailto:devel at lists.opensips.org>
> *Subject: *OpenSIPS Crash
>
> Hello,
>
> We have been having sporadic crashes and I was recently able
> to recover a core dump for one. I have uploaded it here:
> https://pastebin.com/ABktcYcH <https://pastebin.com/ABktcYcH>.
>
> I picked out a Call-ID from the crash data and took a look in
> our tracing. I have uploaded it here:
> https://pastebin.com/ZEzUUKZ5 <https://pastebin.com/ZEzUUKZ5>.
>
> It appears that a downstream server was extremely lagged and
> failed to respond to an INVITE. We sent the INVITE to another
> server and the call was connected, but then eventually the
> original server “caught up” and sent a burst of 200 OK
> responses. The crash seems to have occurred processing the ACK
> to one of these responses.
>
> Ben Newlin
>
>
>
>
>
> _______________________________________________
>
> Devel mailing list
>
> Devel at lists.opensips.org <mailto:Devel at lists.opensips.org>
>
> http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
>
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opensips.org/pipermail/devel/attachments/20181101/e986582d/attachment-0001.html>
More information about the Devel
mailing list