<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<tt>Hi Ben,<br>
<br>
all the BT's points to crashes while doing memory ops. I suspect a
memory corruption that randomly triggers crashes in different
parts of the code.<br>
<br>
Could you try to re-compile with memory debugging support ? See
<a class="moz-txt-link-freetext" href="http://www.opensips.org/Documentation/TroubleShooting-OutOfMem">http://www.opensips.org/Documentation/TroubleShooting-OutOfMem</a>,
the "How to handle it" section.<br>
<br>
Regards,<br>
</tt>
<pre class="moz-signature" cols="72">Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
<a class="moz-txt-link-freetext" href="http://www.opensips-solutions.com">http://www.opensips-solutions.com</a>
OpenSIPS Bootcamp 2018
<a class="moz-txt-link-freetext" href="http://opensips.org/training/OpenSIPS_Bootcamp_2018/">http://opensips.org/training/OpenSIPS_Bootcamp_2018/</a>
</pre>
<div class="moz-cite-prefix">On 10/24/2018 04:28 AM, Ben Newlin
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:0564CECC-E8C6-4B7D-AF5C-D0FA0D4010EE@genesys.com">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered
medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Calibri",sans-serif;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle19
{mso-style-type:personal;
font-family:"Calibri",sans-serif;}
span.EmailStyle20
{mso-style-type:personal;
font-family:"Calibri",sans-serif;}
span.EmailStyle21
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt">We have had
2 more crashes today.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Crash 2: <a
href="https://pastebin.com/rMruBQcZ"
moz-do-not-send="true">
https://pastebin.com/rMruBQcZ</a><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">This crash
appears to have occurred while processing an initial INVITE
request. I could not see anything unusual about the request.
I cannot tell if this crash is related to the others.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Crash 3: <a
href="https://pastebin.com/Gmk1m4NT"
moz-do-not-send="true">
https://pastebin.com/Gmk1m4NT</a><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">This crash
follows the pattern of the original crash I reported.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:black">Ben
Newlin </span>
<span style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF
1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="color:black">From: </span></b><span
style="color:black">Devel
<a class="moz-txt-link-rfc2396E" href="mailto:devel-bounces@lists.opensips.org"><devel-bounces@lists.opensips.org></a> on behalf of Ben
Newlin <a class="moz-txt-link-rfc2396E" href="mailto:Ben.Newlin@genesys.com"><Ben.Newlin@genesys.com></a><br>
<b>Reply-To: </b>OpenSIPS devel mailling list
<a class="moz-txt-link-rfc2396E" href="mailto:devel@lists.opensips.org"><devel@lists.opensips.org></a><br>
<b>Date: </b>Monday, October 22, 2018 at 4:45 PM<br>
<b>To: </b>OpenSIPS devel mailling list
<a class="moz-txt-link-rfc2396E" href="mailto:devel@lists.opensips.org"><devel@lists.opensips.org></a><br>
<b>Subject: </b>Re: [OpenSIPS-Devel] OpenSIPS Crash<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:11.0pt">Here is a
better trace of the call:
<a href="https://pastebin.com/gWpQR8E7"
moz-do-not-send="true">https://pastebin.com/gWpQR8E7</a></span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:black">Ben
Newlin </span>
<o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<div style="border:none;border-top:solid #B5C4DF
1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="color:black">From: </span></b><span
style="color:black">Ben Newlin
<a class="moz-txt-link-rfc2396E" href="mailto:Ben.Newlin@genesys.com"><Ben.Newlin@genesys.com></a><br>
<b>Date: </b>Monday, October 22, 2018 at 4:34 PM<br>
<b>To: </b>OpenSIPS devel mailling list
<a class="moz-txt-link-rfc2396E" href="mailto:devel@lists.opensips.org"><devel@lists.opensips.org></a><br>
<b>Subject: </b>OpenSIPS Crash</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
</div>
<p class="MsoNormal"><span style="font-size:11.0pt">Hello,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt">We have been
having sporadic crashes and I was recently able to recover a
core dump for one. I have uploaded it here:
<a href="https://pastebin.com/ABktcYcH"
moz-do-not-send="true">https://pastebin.com/ABktcYcH</a>.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt">I picked out
a Call-ID from the crash data and took a look in our
tracing. I have uploaded it here:
<a href="https://pastebin.com/ZEzUUKZ5"
moz-do-not-send="true">https://pastebin.com/ZEzUUKZ5</a>.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt">It appears
that a downstream server was extremely lagged and failed to
respond to an INVITE. We sent the INVITE to another server
and the call was connected, but then eventually the original
server “caught up” and sent a burst of 200 OK responses. The
crash seems to have occurred processing the ACK to one of
these responses.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:black">Ben
Newlin </span>
<o:p></o:p></p>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Devel mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Devel@lists.opensips.org">Devel@lists.opensips.org</a>
<a class="moz-txt-link-freetext" href="http://lists.opensips.org/cgi-bin/mailman/listinfo/devel">http://lists.opensips.org/cgi-bin/mailman/listinfo/devel</a>
</pre>
</blockquote>
<br>
</body>
</html>