<div dir="ltr">OK..May i send you my script privately?</div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Mar 9, 2017 at 6:13 PM, Răzvan Crainea <span dir="ltr"><<a href="mailto:razvan@opensips.org" target="_blank">razvan@opensips.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
  
    
  
  <div bgcolor="#FFFFFF" text="#000000">
    <tt>Hi, John!<br>
      <br>
      No, I nothing is suspicious. Definitely not from the drouting
      module.<br>
      Try to make two captures: one after 10 calls, another one after 20
      calls.<br>
      <br>
      Best regards,<br>
    </tt><span class="">
    <pre class="m_-6234719319706105107moz-signature" cols="72">Răzvan Crainea
OpenSIPS Solutions
<a class="m_-6234719319706105107moz-txt-link-abbreviated" href="http://www.opensips-solutions.com" target="_blank">www.opensips-solutions.com</a></pre>
    </span><div><div class="h5"><div class="m_-6234719319706105107moz-cite-prefix">On 03/09/2017 01:50 PM, John Nash
      wrote:<br>
    </div>
    <blockquote type="cite">
      <div dir="ltr">Do you see anything suspicious in the latest mem
        dump? </div>
      <div class="gmail_extra"><br>
        <div class="gmail_quote">On Wed, Mar 8, 2017 at 7:20 PM, John
          Nash <span dir="ltr"><<a href="mailto:john.nash778@gmail.com" target="_blank">john.nash778@gmail.com</a>></span>
          wrote:<br>
          <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
            <div dir="ltr">One more useful info. I disabled drouting
              functions and just rewrote RURI to hardcoded address
              keeping rest of the functions same and I do not see drop
              in private memory of that process.</div>
            <div class="m_-6234719319706105107HOEnZb">
              <div class="m_-6234719319706105107h5">
                <div class="gmail_extra"><br>
                  <div class="gmail_quote">On Wed, Mar 8, 2017 at 4:40
                    PM, John Nash <span dir="ltr"><<a href="mailto:john.nash778@gmail.com" target="_blank">john.nash778@gmail.com</a>></span>
                    wrote:<br>
                    <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
                      <div dir="ltr">OK Here is the dump
                        <div><a href="https://drive.google.com/open?id=0BxJKNwFalcRMX0xDUlRIa2VUdG8" target="_blank">https://drive.google.com/open?<wbr>id=0BxJKNwFalcRMX0xDUlRIa2VUdG<wbr>8</a><br>
                        </div>
                        <div><br>
                        </div>
                        <div><br>
                        </div>
                        <div>I increased syslog message rate to 500000,
                          Made around 10 call attempts. Waited for some
                          time and made sure no call is on server and
                          then sent signal to dump memory to the process
                          ID i suspect.</div>
                      </div>
                      <div class="m_-6234719319706105107m_-6775075754875813829HOEnZb">
                        <div class="m_-6234719319706105107m_-6775075754875813829h5">
                          <div class="gmail_extra"><br>
                            <div class="gmail_quote">On Wed, Mar 8, 2017
                              at 4:07 PM, Răzvan Crainea <span dir="ltr"><<a href="mailto:razvan@opensips.org" target="_blank">razvan@opensips.org</a>></span>
                              wrote:<br>
                              <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
                                <div bgcolor="#FFFFFF" text="#000000"> <tt>No,
                                    you should not kill any process.
                                    Simply send a SIGUSR1 to the process
                                    you suspect.<br>
                                  </tt><span>
                                    <pre class="m_-6234719319706105107m_-6775075754875813829m_2381737702605022529m_1855888088469774506moz-signature" cols="72">Răzvan Crainea
OpenSIPS Solutions
<a class="m_-6234719319706105107m_-6775075754875813829m_2381737702605022529m_1855888088469774506moz-txt-link-abbreviated" href="http://www.opensips-solutions.com" target="_blank">www.opensips-solutions.com</a></pre>
                                  </span>
                                  <div>
                                    <div class="m_-6234719319706105107m_-6775075754875813829m_2381737702605022529h5">
                                      <div class="m_-6234719319706105107m_-6775075754875813829m_2381737702605022529m_1855888088469774506moz-cite-prefix">On
                                        03/08/2017 12:28 PM, John Nash
                                        wrote:<br>
                                      </div>
                                      <blockquote type="cite">
                                        <div dir="ltr">Sorry...Should I
                                          kill only the process where i
                                          see memory leak?</div>
                                        <div class="gmail_extra"><br>
                                          <div class="gmail_quote">On
                                            Wed, Mar 8, 2017 at 3:41 PM,
                                            Răzvan Crainea <span dir="ltr"><<a href="mailto:razvan@opensips.org" target="_blank">razvan@opensips.org</a>></span>
                                            wrote:<br>
                                            <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
                                              <div bgcolor="#FFFFFF" text="#000000"> <tt>use
                                                  only memdump set to 1.</tt><span><br>
                                                  <pre class="m_-6234719319706105107m_-6775075754875813829m_2381737702605022529m_1855888088469774506m_-3851260453542495443moz-signature" cols="72">Răzvan Crainea
OpenSIPS Solutions
<a class="m_-6234719319706105107m_-6775075754875813829m_2381737702605022529m_1855888088469774506m_-3851260453542495443moz-txt-link-abbreviated" href="http://www.opensips-solutions.com" target="_blank">www.opensips-solutions.com</a></pre>
                                                </span>
                                                <div>
                                                  <div class="m_-6234719319706105107m_-6775075754875813829m_2381737702605022529m_1855888088469774506h5">
                                                    <div class="m_-6234719319706105107m_-6775075754875813829m_2381737702605022529m_1855888088469774506m_-3851260453542495443moz-cite-prefix">On
                                                      03/08/2017 12:11
                                                      PM, John Nash
                                                      wrote:<br>
                                                    </div>
                                                    <blockquote type="cite">
                                                      <div dir="ltr">Ok
                                                        i will give
                                                        another try what
                                                        should be the
                                                        values of
                                                        memdump and
                                                        memlog</div>
                                                      <div class="gmail_extra"><br>
                                                        <div class="gmail_quote">On
                                                          Wed, Mar 8,
                                                          2017 at 3:13
                                                          PM, Răzvan
                                                          Crainea <span dir="ltr"><<a href="mailto:razvan@opensips.org" target="_blank">razvan@opensips.org</a>></span>
                                                          wrote:<br>
                                                          <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
                                                          <div bgcolor="#FFFFFF" text="#000000">
                                                          <tt>Hi, John!<br>
                                                          </tt><br>
                                                          <tt>The traces
                                                          you showed me
                                                          are
                                                          incomplete:
                                                          they do not
                                                          have all the
                                                          memory chunks
                                                          allocated,
                                                          thus I can't
                                                          say wether
                                                          something is
                                                          wrong or not.<br>
                                                          As I said
                                                          earlier, it is
                                                          normal for
                                                          opensips to
                                                          use extra
                                                          memory every
                                                          call. But
                                                          after a while,
                                                          this should
                                                          stabilize.
                                                          After a while
                                                          might mean
                                                          more than
                                                          1000k calls.
                                                          As long as you
                                                          never reach
                                                          the upper
                                                          limit of the
                                                          memory, you
                                                          can't conclude
                                                          that there is
                                                          a memory leak.
                                                          Even then,
                                                          you're limit
                                                          might be too
                                                          low for the
                                                          kind of
                                                          traffic you
                                                          are doing, so
                                                          it still might
                                                          not be a
                                                          memory leak.
                                                          But only then
                                                          it is worth to
                                                          investigate.<br>
                                                          When we
                                                          investigate,
                                                          we need all
                                                          the data (i.e.
                                                          the entire
                                                          trace of the
                                                          memory dump).<br>
                                                          So please try
                                                          to send as
                                                          many calls as
                                                          possilble, and
                                                          if this issue
                                                          still
                                                          persists, make
                                                          a pkg memory
                                                          dump when the
                                                          server is in
                                                          idle mode and
                                                          send it over.<br>
                                                          <br>
                                                          Best regards,<br>
                                                          </tt><span>
                                                          <pre class="m_-6234719319706105107m_-6775075754875813829m_2381737702605022529m_1855888088469774506m_-3851260453542495443m_-7777753411478014366moz-signature" cols="72">Răzvan Crainea
OpenSIPS Solutions
<a class="m_-6234719319706105107m_-6775075754875813829m_2381737702605022529m_1855888088469774506m_-3851260453542495443m_-7777753411478014366moz-txt-link-abbreviated" href="http://www.opensips-solutions.com" target="_blank">www.opensips-solutions.com</a></pre>
                                                          </span><span>
                                                          <div class="m_-6234719319706105107m_-6775075754875813829m_2381737702605022529m_1855888088469774506m_-3851260453542495443m_-7777753411478014366moz-cite-prefix">On
                                                          03/08/2017
                                                          11:26 AM, John
                                                          Nash wrote:<br>
                                                          </div>
                                                          </span>
                                                          <blockquote type="cite"><span>
                                                          <div dir="ltr">any
                                                          suggestion for
                                                          me?..should i
                                                          try to crash
                                                          opensips by
                                                          sending many
                                                          calls?</div>
                                                          </span>
                                                          <div class="gmail_extra"><br>
                                                          <div class="gmail_quote"><span>On
                                                          Tue, Mar 7,
                                                          2017 at 4:54
                                                          PM, John Nash
                                                          <span dir="ltr"><<a href="mailto:john.nash778@gmail.com" target="_blank">john.nash778@gmail.com</a>></span>
                                                          wrote:<br>
                                                          </span>
                                                          <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span>
                                                          <div dir="ltr">
                                                          <div>version:
                                                          opensips 2.1.5
                                                          (x86_64/linux)</div>
                                                          <div>flags:
                                                          STATS: On,
                                                          DISABLE_NAGLE,
                                                          USE_MCAST,
                                                          SHM_MMAP,
                                                          PKG_MALLOC,
                                                          DBG_QM_MALLOC,
FAST_LOCK-ADAPTIVE_WAIT</div>
                                                          <div>ADAPTIVE_WAIT_LOOPS=1024,
MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE
                                                          65535</div>
                                                          <div>poll
                                                          method
                                                          support: poll,
                                                          epoll_lt,
                                                          epoll_et,
                                                          sigio_rt,
                                                          select.</div>
                                                          <div>git
                                                          revision:
                                                          39b19dd</div>
                                                          <div>main.c
                                                          compiled on
                                                          19:27:59 Mar
                                                           5 2017 with
                                                          gcc 4.4.7</div>
                                                          <div><br>
                                                          </div>
                                                          </div>
                                                          </span><span>
                                                          <div class="m_-6234719319706105107m_-6775075754875813829m_2381737702605022529m_1855888088469774506m_-3851260453542495443m_-7777753411478014366HOEnZb">
                                                          <div class="m_-6234719319706105107m_-6775075754875813829m_2381737702605022529m_1855888088469774506m_-3851260453542495443m_-7777753411478014366h5">
                                                          <div class="gmail_extra"><tt>memory
                                                          stabilizing in
                                                          time? Or it is
                                                          continously
                                                          decreasing?<br>
                                                          Yes, that's
                                                          how you should
                                                          make the dump.<br>
                                                          <br>
                                                          Best regards,<br>
                                                          </tt><span>
                                                          <pre class="m_-6234719319706105107m_-6775075754875813829m_2381737702605022529m_1855888088469774506m_-3851260453542495443m_-7777753411478014366m_-9062359371326778634m_2619109260263126880m_-9028378031016354439m_6794548611486144135m_-5328551591246680141m_-3972373151872949598m_-1935339535774106372m_-8335070280687897772m_5518641707690872261moz-signature" cols="72">Răzvan Crainea
OpenSIPS Solutions
<a class="m_-6234719319706105107m_-6775075754875813829m_2381737702605022529m_1855888088469774506m_-3851260453542495443m_-7777753411478014366m_-9062359371326778634m_2619109260263126880m_-9028378031016354439m_6794548611486144135m_-5328551591246680141m_-3972373151872949598m_-1935339535774106372m_-8335070280687897772m_5518641707690872261moz-txt-link-abbreviated" href="http://www.opensips-solutions.com" target="_blank">www.opensips-solutions.com</a>
</pre>
                                                          </span><br>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </span></blockquote>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          <br>
______________________________<wbr>_________________<br>
                                                          Users mailing
                                                          list<br>
                                                          <a href="mailto:Users@lists.opensips.org" target="_blank">Users@lists.opensips.org</a><br>
                                                          <a href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.opensips.org/cgi-<wbr>bin/mailman/listinfo/users</a><br>
                                                          <br>
                                                          </blockquote>
                                                        </div>
                                                        <br>
                                                      </div>
                                                      <br>
                                                      <fieldset class="m_-6234719319706105107m_-6775075754875813829m_2381737702605022529m_1855888088469774506m_-3851260453542495443mimeAttachmentHeader"></fieldset>
                                                      <br>
                                                      <pre>______________________________<wbr>_________________
Users mailing list
<a class="m_-6234719319706105107m_-6775075754875813829m_2381737702605022529m_1855888088469774506m_-3851260453542495443moz-txt-link-abbreviated" href="mailto:Users@lists.opensips.org" target="_blank">Users@lists.opensips.org</a>
<a class="m_-6234719319706105107m_-6775075754875813829m_2381737702605022529m_1855888088469774506m_-3851260453542495443moz-txt-link-freetext" href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users" target="_blank">http://lists.opensips.org/cgi-<wbr>bin/mailman/listinfo/users</a>
</pre>
    </blockquote>
    

  </div></div></div>


______________________________<wbr>_________________

Users mailing list

<a href="mailto:Users@lists.opensips.org" target="_blank">Users@lists.opensips.org</a>

<a href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.opensips.org/cgi-<wbr>bin/mailman/listinfo/users</a>


</blockquote></div>
</div>


<fieldset class="m_-6234719319706105107m_-6775075754875813829m_2381737702605022529m_1855888088469774506mimeAttachmentHeader"></fieldset>
<pre>______________________________<wbr>_________________
Users mailing list
<a class="m_-6234719319706105107m_-6775075754875813829m_2381737702605022529m_1855888088469774506moz-txt-link-abbreviated" href="mailto:Users@lists.opensips.org" target="_blank">Users@lists.opensips.org</a>
<a class="m_-6234719319706105107m_-6775075754875813829m_2381737702605022529m_1855888088469774506moz-txt-link-freetext" href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users" target="_blank">http://lists.opensips.org/cgi-<wbr>bin/mailman/listinfo/users</a>
</pre>

</blockquote>
</div></div></div>
______________________________<wbr>_________________

Users mailing list

<a href="mailto:Users@lists.opensips.org" target="_blank">Users@lists.opensips.org</a>

<a href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.opensips.org/cgi-<wbr>bin/mailman/listinfo/users</a>


</blockquote></div>
</div>
</div></div></blockquote></div>
</div>
</div></div></blockquote></div>
</div>


<fieldset class="m_-6234719319706105107mimeAttachmentHeader"></fieldset>
<pre>______________________________<wbr>_________________
Users mailing list
<a class="m_-6234719319706105107moz-txt-link-abbreviated" href="mailto:Users@lists.opensips.org" target="_blank">Users@lists.opensips.org</a>
<a class="m_-6234719319706105107moz-txt-link-freetext" href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users" target="_blank">http://lists.opensips.org/cgi-<wbr>bin/mailman/listinfo/users</a>
</pre>

</blockquote>
</div></div></div><br>______________________________<wbr>_________________<br>
Users mailing list<br>
<a href="mailto:Users@lists.opensips.org">Users@lists.opensips.org</a><br>
<a href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.opensips.org/cgi-<wbr>bin/mailman/listinfo/users</a><br>
<br></blockquote></div><br></div>