<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<tt>Zupperrr...I remember I saw this backtrace before, but without
the ability to reproduce it.<br>
<br>
Do you have OpenSIPS locally compiled or installed from packages ?<br>
<br>
Best regards,<br>
</tt>
<pre class="moz-signature" cols="72">Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
<a class="moz-txt-link-freetext" href="http://www.opensips-solutions.com">http://www.opensips-solutions.com</a></pre>
<div class="moz-cite-prefix">On 20.09.2016 20:22, Richard Robson
wrote:<br>
</div>
<blockquote
cite="mid:86258e55-14ac-6982-8f29-0629dcf44073@greenlightcrm.com"
type="cite">
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
<div class="moz-cite-prefix">just made one for you.<br>
<br>
<a moz-do-not-send="true" class="moz-txt-link-freetext"
href="http://pastebin.com/HJkwuTDc">http://pastebin.com/HJkwuTDc</a><br>
<br>
I'd ramped this up to 800cps and the recieving asterisk was
struggling crashed 5 times and autorestarted before i could ramp
down the sipp<br>
<br>
<br>
<br>
<br>
On 20/09/2016 08:51, Bogdan-Andrei Iancu wrote:<br>
</div>
<blockquote
cite="mid:154f2dd0-9879-4a1c-6573-8c674048c6fe@opensips.org"
type="cite">
<meta content="text/html; charset=utf-8"
http-equiv="Content-Type">
<tt>Hi Richard,<br>
<br>
If not too much for you, I would really like to see the
backtrace - even under OS stress conditions, OpenSIPS should
NOT crash. <br>
<br>
Best regards,<br>
</tt>
<pre class="moz-signature" cols="72">Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://www.opensips-solutions.com">http://www.opensips-solutions.com</a></pre>
<div class="moz-cite-prefix">On 16.09.2016 13:15, Richard Robson
wrote:<br>
</div>
<blockquote
cite="mid:5b38f1ec-b7d5-1fef-dfee-c1a1d48c0274@greenlightcrm.com"
type="cite">
<meta content="text/html; charset=utf-8"
http-equiv="Content-Type">
<div class="moz-cite-prefix">Hi Bogdan,<br>
<br>
It looks like it was the extra logging we were putting in
using xlog. We'd added them while we were developing the
script and left them in. this was causing systemd-journald
to max out the box and would segfault the opensips, which
restarted. journald was using 90% of the box and opensips
10%<br>
<br>
I've taken out the xlogs ( there was about 20 per iteration
of the script) and using sipp i've tested throughput to
250cps and 2000 concurrent calls. (enought to kill the
asterisk processing the calls the other side)<br>
<br>
Unfortunately i deleted all the core files as the filled the
disk up. If you still want one I'll be able to reproduce it
as I've still got the script. I seem to remember though that
it was the transaction module that was the culprit<br>
<br>
Regards,<br>
<br>
Richard<br>
<br>
<br>
<br>
On 14/09/2016 15:28, Bogdan-Andrei Iancu wrote:<br>
</div>
<blockquote
cite="mid:2abee136-64f0-4b79-dd98-981c1dfa1add@opensips.org"
type="cite">
<meta content="text/html; charset=utf-8"
http-equiv="Content-Type">
<tt>Hi Richard,<br>
<br>
Have you managed to get a corefile and extract a backtrace
?<br>
<br>
Best regards,<br>
</tt>
<pre class="moz-signature" cols="72">Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://www.opensips-solutions.com">http://www.opensips-solutions.com</a></pre>
<div class="moz-cite-prefix">On 06.09.2016 17:07, Richard
Robson wrote:<br>
</div>
<blockquote
cite="mid:c8fa4fda-fc27-80cb-b046-c942a4e1ff3a@greenlightcrm.com"
type="cite">
<meta http-equiv="content-type" content="text/html;
charset=utf-8">
<p>If its any help, I can see packets coming in
particularly BYEs that are not being processes at high
call rates and are not getting to the point in the
script where the rtpengine_delete is being triggered.
this then causes the number of concurrent calls on the
RTPengine to grow and fill its allocation of ports. This
then stops calls being made. and the opensips then
crashes.<br>
</p>
<div class="moz-forward-container"><br>
<br>
-------- Forwarded Message --------
<table class="moz-email-headers-table" cellpadding="0"
cellspacing="0" border="0">
<tbody>
<tr>
<th align="RIGHT" valign="BASELINE"
nowrap="nowrap">Subject: </th>
<td>2.2.1 crashing</td>
</tr>
<tr>
<th align="RIGHT" valign="BASELINE"
nowrap="nowrap">Date: </th>
<td>Tue, 6 Sep 2016 13:45:11 +0100</td>
</tr>
<tr>
<th align="RIGHT" valign="BASELINE"
nowrap="nowrap">From: </th>
<td>Richard Robson <a moz-do-not-send="true"
class="moz-txt-link-rfc2396E"
href="mailto:rrobson@greenlightcrm.com"><rrobson@greenlightcrm.com></a></td>
</tr>
<tr>
<th align="RIGHT" valign="BASELINE"
nowrap="nowrap">Organisation: </th>
<td>Greenlight Innovation</td>
</tr>
<tr>
<th align="RIGHT" valign="BASELINE"
nowrap="nowrap">To: </th>
<td>OpenSIPS users mailling list <a
moz-do-not-send="true"
class="moz-txt-link-rfc2396E"
href="mailto:users@lists.opensips.org"><users@lists.opensips.org></a></td>
</tr>
</tbody>
</table>
<br>
<br>
<pre>During testing we are ramping up sipp to give ~100 CPS between internal
servers and opensips crashes with a seg fault
sipp -sn uac 192.168.36.141:5060 -s 441382250180 -r 1 -rp 1000
-recv_timeout 7500 -send_timeout 7500 -d 5000
the sipp calls are being routed to an asterisk server, which is playing
audio
Calls are going via an RTPengine on a different box
everything is on CENTOS 7.2
I'm using the latest git of 2.2.1 on a virtual host 8 cores and 8GB
its OK up to around 60 CPS (350 calls) but ramping up to 100 CPS (750
concurrent calls) causes the segfault.
we are seeing the systemd-journal being the heaviest CPU user, but not
sure if this is unrelated
here is the backtrace: <a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://pastebin.com/SjaSJx7w">http://pastebin.com/SjaSJx7w</a>
--
Richard Robson
Greenlight Support
01382 843843
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:support@greenlightcrm.com">support@greenlightcrm.com</a>
</pre>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Users mailing list
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:Users@lists.opensips.org">Users@lists.opensips.org</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users">http://lists.opensips.org/cgi-bin/mailman/listinfo/users</a>
</pre>
</blockquote>
<br>
</blockquote>
<br>
<p><br>
</p>
<pre class="moz-signature" cols="72">--
Richard Robson
Greenlight Support
01382 843843
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:support@greenlightcrm.com">support@greenlightcrm.com</a></pre>
</blockquote>
<br>
</blockquote>
<br>
<p><br>
</p>
<pre class="moz-signature" cols="72">--
Richard Robson
Greenlight Support
01382 843843
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:support@greenlightcrm.com">support@greenlightcrm.com</a></pre>
</blockquote>
<br>
</body>
</html>