<html>
  <head>
    <meta content="text/html; charset=utf-8" http-equiv="Content-Type">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    <tt>Zupperrr...I remember I saw this backtrace before, but without
      the ability to reproduce it.<br>
      <br>
      Do you have OpenSIPS locally compiled or installed from packages ?<br>
      <br>
      Best regards,<br>
    </tt>
    <pre class="moz-signature" cols="72">Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
<a class="moz-txt-link-freetext" href="http://www.opensips-solutions.com">http://www.opensips-solutions.com</a></pre>
    <div class="moz-cite-prefix">On 20.09.2016 20:22, Richard Robson
      wrote:<br>
    </div>
    <blockquote
      cite="mid:86258e55-14ac-6982-8f29-0629dcf44073@greenlightcrm.com"
      type="cite">
      <meta content="text/html; charset=utf-8" http-equiv="Content-Type">
      <div class="moz-cite-prefix">just made one for you.<br>
        <br>
        <a moz-do-not-send="true" class="moz-txt-link-freetext"
          href="http://pastebin.com/HJkwuTDc">http://pastebin.com/HJkwuTDc</a><br>
        <br>
        I'd ramped this up to 800cps and the recieving asterisk was
        struggling crashed 5 times and autorestarted before i could ramp
        down the sipp<br>
        <br>
        <br>
        <br>
        <br>
        On 20/09/2016 08:51, Bogdan-Andrei Iancu wrote:<br>
      </div>
      <blockquote
        cite="mid:154f2dd0-9879-4a1c-6573-8c674048c6fe@opensips.org"
        type="cite">
        <meta content="text/html; charset=utf-8"
          http-equiv="Content-Type">
        <tt>Hi Richard,<br>
          <br>
          If not too much for you, I would really like to see the
          backtrace - even under OS stress conditions, OpenSIPS should
          NOT crash. <br>
          <br>
          Best regards,<br>
        </tt>
        <pre class="moz-signature" cols="72">Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://www.opensips-solutions.com">http://www.opensips-solutions.com</a></pre>
        <div class="moz-cite-prefix">On 16.09.2016 13:15, Richard Robson
          wrote:<br>
        </div>
        <blockquote
          cite="mid:5b38f1ec-b7d5-1fef-dfee-c1a1d48c0274@greenlightcrm.com"
          type="cite">
          <meta content="text/html; charset=utf-8"
            http-equiv="Content-Type">
          <div class="moz-cite-prefix">Hi Bogdan,<br>
            <br>
            It looks like it was the extra logging we were putting in
            using xlog. We'd added them while we were developing the
            script and left them in. this was causing systemd-journald
            to max out the box and would segfault the opensips, which
            restarted. journald was using 90% of the box and opensips
            10%<br>
            <br>
            I've taken out the xlogs ( there was about 20 per iteration
            of the script) and using sipp i've tested throughput to
            250cps and 2000 concurrent calls. (enought to kill the
            asterisk processing the calls the other side)<br>
            <br>
            Unfortunately i deleted all the core files as the filled the
            disk up. If you still want one I'll be able to reproduce it
            as I've still got the script. I seem to remember though that
            it was the transaction module that was the culprit<br>
            <br>
            Regards,<br>
            <br>
            Richard<br>
             <br>
            <br>
            <br>
            On 14/09/2016 15:28, Bogdan-Andrei Iancu wrote:<br>
          </div>
          <blockquote
            cite="mid:2abee136-64f0-4b79-dd98-981c1dfa1add@opensips.org"
            type="cite">
            <meta content="text/html; charset=utf-8"
              http-equiv="Content-Type">
            <tt>Hi Richard,<br>
              <br>
              Have you managed to get a corefile and extract a backtrace
              ?<br>
              <br>
              Best regards,<br>
            </tt>
            <pre class="moz-signature" cols="72">Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://www.opensips-solutions.com">http://www.opensips-solutions.com</a></pre>
            <div class="moz-cite-prefix">On 06.09.2016 17:07, Richard
              Robson wrote:<br>
            </div>
            <blockquote
              cite="mid:c8fa4fda-fc27-80cb-b046-c942a4e1ff3a@greenlightcrm.com"
              type="cite">
              <meta http-equiv="content-type" content="text/html;
                charset=utf-8">
              <p>If its any help, I can see packets coming in
                particularly BYEs that are not being processes at high
                call rates and are not getting to the point in the
                script where the rtpengine_delete is being triggered.
                this then causes the number of concurrent calls on the
                RTPengine to grow and fill its allocation of ports. This
                then stops calls being made. and the opensips then
                crashes.<br>
              </p>
              <div class="moz-forward-container"><br>
                <br>
                -------- Forwarded Message --------
                <table class="moz-email-headers-table" cellpadding="0"
                  cellspacing="0" border="0">
                  <tbody>
                    <tr>
                      <th align="RIGHT" valign="BASELINE"
                        nowrap="nowrap">Subject: </th>
                      <td>2.2.1 crashing</td>
                    </tr>
                    <tr>
                      <th align="RIGHT" valign="BASELINE"
                        nowrap="nowrap">Date: </th>
                      <td>Tue, 6 Sep 2016 13:45:11 +0100</td>
                    </tr>
                    <tr>
                      <th align="RIGHT" valign="BASELINE"
                        nowrap="nowrap">From: </th>
                      <td>Richard Robson <a moz-do-not-send="true"
                          class="moz-txt-link-rfc2396E"
                          href="mailto:rrobson@greenlightcrm.com">&lt;rrobson@greenlightcrm.com&gt;</a></td>
                    </tr>
                    <tr>
                      <th align="RIGHT" valign="BASELINE"
                        nowrap="nowrap">Organisation: </th>
                      <td>Greenlight Innovation</td>
                    </tr>
                    <tr>
                      <th align="RIGHT" valign="BASELINE"
                        nowrap="nowrap">To: </th>
                      <td>OpenSIPS users mailling list <a
                          moz-do-not-send="true"
                          class="moz-txt-link-rfc2396E"
                          href="mailto:users@lists.opensips.org">&lt;users@lists.opensips.org&gt;</a></td>
                    </tr>
                  </tbody>
                </table>
                <br>
                <br>
                <pre>During testing we are ramping up sipp to give ~100 CPS between internal 
servers and opensips crashes with a seg fault

sipp  -sn uac 192.168.36.141:5060 -s 441382250180  -r 1 -rp 1000 
-recv_timeout 7500 -send_timeout 7500 -d 5000

the sipp calls are being routed to an asterisk server, which is playing 
audio
Calls are going via an RTPengine on a different box
everything is on CENTOS 7.2
I'm using the latest git of 2.2.1 on a virtual host 8 cores and 8GB
its OK up to around 60 CPS (350 calls) but ramping up to 100 CPS (750 
concurrent calls) causes the segfault.
we are seeing the systemd-journal being the heaviest CPU user, but not 
sure if this is unrelated



here is the backtrace: <a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://pastebin.com/SjaSJx7w">http://pastebin.com/SjaSJx7w</a>

-- 
Richard Robson
Greenlight Support
01382 843843
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:support@greenlightcrm.com">support@greenlightcrm.com</a>

</pre>
              </div>
              <br>
              <fieldset class="mimeAttachmentHeader"></fieldset>
              <br>
              <pre wrap="">_______________________________________________
Users mailing list
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:Users@lists.opensips.org">Users@lists.opensips.org</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users">http://lists.opensips.org/cgi-bin/mailman/listinfo/users</a>
</pre>
            </blockquote>
            <br>
          </blockquote>
          <br>
          <p><br>
          </p>
          <pre class="moz-signature" cols="72">-- 
Richard Robson
Greenlight Support
01382 843843
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:support@greenlightcrm.com">support@greenlightcrm.com</a></pre>
        </blockquote>
        <br>
      </blockquote>
      <br>
      <p><br>
      </p>
      <pre class="moz-signature" cols="72">-- 
Richard Robson
Greenlight Support
01382 843843
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:support@greenlightcrm.com">support@greenlightcrm.com</a></pre>
    </blockquote>
    <br>
  </body>
</html>