<div dir="ltr">with every call attempt it decreases. I tried some changes by rejecting invite before drouting call (That means after auth , dispatcher) and found memory is stable but when drouting sends Invite to external gateway and external gateway rejects it. Then this issue happens.<div><br></div><div>Inuse transactions and active dialogs also 0. Somthing wrong happening in handling of failure replies. But apart from <span style="background-color:rgb(254,254,238);color:rgb(0,0,0)">use_next_gw and setting some avps for CDR not much going on there.</span></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Mar 6, 2017 at 5:54 PM, Răzvan Crainea <span dir="ltr"><<a href="mailto:razvan@opensips.org" target="_blank">razvan@opensips.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
  
    
  
  <div bgcolor="#FFFFFF" text="#000000">
    <tt>Ok, so it is the first listener for the private IP that leaks.
      Next, is the memory stabilizing in time? Or it is continously
      decreasing?<br>
      Yes, that's how you should make the dump.<br>
      <br>
      Best regards,<br>
    </tt><span class="">
    <pre class="m_5518641707690872261moz-signature" cols="72">Răzvan Crainea
OpenSIPS Solutions
<a class="m_5518641707690872261moz-txt-link-abbreviated" href="http://www.opensips-solutions.com" target="_blank">www.opensips-solutions.com</a></pre>
    </span><div><div class="h5"><div class="m_5518641707690872261moz-cite-prefix">On 03/06/2017 10:57 AM, John Nash
      wrote:<br>
    </div>
    <blockquote type="cite">
      <div dir="ltr">Dear Razvan,
        <div><br>
        </div>
        <div>Below is the info on my processes</div>
        <div>
          <div>Process::  ID=0 PID=17351 Type=attendant</div>
          <div>Process::  ID=1 PID=17352 Type=MI FIFO</div>
          <div>Process::  ID=2 PID=17353 Type=MI Datagram</div>
          <div>Process::  ID=3 PID=17354 Type=time_keeper</div>
          <div>Process::  ID=4 PID=17355 Type=timer</div>
          <div>Process::  ID=5 PID=17356 Type=SIP receiver udp:<a href="http://1.1.1.1:9094" target="_blank">1.1.1.1:9094</a> </div>
          <div>Process::  ID=6 PID=17357 Type=SIP receiver udp:<a href="http://1.1.1.1:5060" target="_blank">1.1.1.1:5060</a> </div>
          <div>Process::  ID=7 PID=17358 Type=SIP receiver udp:<a href="http://192.168.45.5:5064" target="_blank">192.168.45.5:5064</a> </div>
          <div>Process::  ID=8 PID=17359 Type=Timer handler</div>
        </div>
        <div><br>
        </div>
        <div>1.1.1.1 is public IP (I changed). The decrease in memory I
          see is for Process::  ID=7 PID=17358 mainly. My call flow is
          as following </div>
        <div><br>
        </div>
        <div>- New Invite hits the opensips on <a href="http://1.1.1.1:9094" target="_blank">1.1.1.1:9094</a></div>
        <div>- Apart from message validity checks I query DB to check if
          its a valid user (Using local cache also there)</div>
        <div>- Create dialog, Topology_hiding functions are called along
          with some avp population</div>
        <div>- Using dispatcher ds_select_domain Call sent to udp:<a href="http://192.168.45.2:7060" target="_blank">192.168.45.2:7060</a>
          (using force socket). This <a href="http://192.168.45.2:7060" target="_blank">192.168.45.2:7060</a> is
          actually freeswitch </div>
        <div>- Call again comes back to opensips on udp:<a href="http://192.168.45.5:5064" target="_blank">192.168.45.5:5064</a></div>
        <div>- New dialog is created and topology_hiding is called</div>
        <div>- Drouting function do_routing is called which tries one
          gateway and fails<br>
        </div>
        <div><br>
        </div>
        <div><br>
        </div>
        <div>Dump i need to create with memlog=4 memdump=1 right?</div>
        <div><br>
        </div>
        <div><br>
        </div>
        <div><br>
        </div>
        <div><br>
        </div>
        <div><br>
        </div>
        <div><br>
        </div>
        <div><br>
        </div>
        <div><br>
        </div>
      </div>
      <div class="gmail_extra"><br>
        <div class="gmail_quote">On Mon, Mar 6, 2017 at 2:05 PM, Răzvan
          Crainea <span dir="ltr"><<a href="mailto:razvan@opensips.org" target="_blank">razvan@opensips.org</a>></span>
          wrote:<br>
          <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
            <div bgcolor="#FFFFFF" text="#000000"> <tt>Hi, John!<br>
                <br>
                Transactions are stored in shared memory, not in the
                private one. So the possible leak you are facing its not
                related to transactions.<br>
                During runtime, OpenSIPS might resize some internal
                structures, which may lead to increase memory usage.
                However, after a while, these allocations should
                stabilize</tt><tt> .<br>
                Can you post the output of the kill -SIGUSR1 on pastebin
                so we can take a look? Also, what type of process is the
                one you are seeing the leak into? You can find out using
                the 'opensipsctl ps' command.<br>
                <br>
                Best regards,<br>
              </tt>
              <pre class="m_5518641707690872261m_2485402866099392609moz-signature" cols="72">Răzvan Crainea
OpenSIPS Solutions
<a class="m_5518641707690872261m_2485402866099392609moz-txt-link-abbreviated" href="http://www.opensips-solutions.com" target="_blank">www.opensips-solutions.com</a></pre>
              <div>
                <div class="m_5518641707690872261h5">
                  <div class="m_5518641707690872261m_2485402866099392609moz-cite-prefix">On
                    03/06/2017 09:55 AM, John Nash wrote:<br>
                  </div>
                </div>
              </div>
              <blockquote type="cite">
                <div>
                  <div class="m_5518641707690872261h5">
                    <div dir="ltr">I am using OpenSIPS (2.1.5
                      (x86_64/linux)) in production. I observed private
                      memory is decreasing constantly for one process
                      mainly and ultimately leading to memory errors and
                      crash.
                      <div><br>
                      </div>
                      <div>To debug this issue I prepared a test server
                        and compiled opensips as per <a href="https://www.opensips.org/Documentation/TroubleShooting-OutOfMem" target="_blank">https://www.opensips.org/D<wbr>ocumentation/TroubleShooting-O<wbr>utOfMem</a></div>
                      <div><br>
                      </div>
                      <div>I made only one single call (which was
                        rejected by opensips as it was not authorized
                        user) and I saw private free memory decreased. I
                        was hoping since transaction is done ideally it
                        should release memory and should show me same
                        memory as startup but it did not. I verified
                        this with many call attempts and i see free
                        memory is always decreasing slowly.</div>
                      <div><br>
                      </div>
                      <div>I used kill -SIGUSR1 <lowest pid> to
                        create memory dump. But i am unable to make
                        sense of it. It shows log like ...</div>
                      <div><br>
                      </div>
                      <div>r  6 07:29:19 Server3021 opensips[13276]:
                        Memory status (pkg):</div>
                      <div>Mar  6 07:29:19 Server3021 opensips[13276]:
                        qm_status (0x7f5b8ebba010):</div>
                      <div>Mar  6 07:29:19 Server3021 opensips[13276]:
                         heap size= 4194304</div>
                      <div>Mar  6 07:29:19 Server3021 opensips[13276]:
                         used= 346768, used+overhead=848792,
                        free=3345512</div>
                      <div>Mar  6 07:29:19 Server3021 opensips[13276]:
                         max used (+overhead)= 931920</div>
                      <div>Mar  6 07:29:19 Server3021 opensips[13276]:
                        dumping all alloc'ed. fragments:</div>
                      <div>Mar  6 07:29:19 Server3021 opensips[13276]:  
                            0. N  address=0x7f5b8ebef528
                        frag=0x7f5b8ebef4f8 size=40 used=1</div>
                      <div>Mar  6 07:29:19 Server3021 opensips[13276]:  
                                  alloc'd from script_cb.c:
                        add_callback(60)</div>
                      <div>Mar  6 07:29:19 Server3021 opensips[13276]:  
                              start check=f0f0f0f0f0f0f0f0, end check=
                        c0c0c0c0c0c0c0c0, abcdefedabcdefed</div>
                      <div>Mar  6 07:29:19 Server3021 opensips[13276]:  
                            1. N  address=0x7f5b8ebef5b0 <br>
                      </div>
                      <div><br>
                      </div>
                      <div>I pasted only few lines in this mail. What
                        should be my next step?...How can i really trace
                        what is wrong in my script or any other memory
                        leak?</div>
                      <div><br>
                      </div>
                    </div>
                    <br>
                    <fieldset class="m_5518641707690872261m_2485402866099392609mimeAttachmentHeader"></fieldset>
                    <br>
                  </div>
                </div>
                <pre>______________________________<wbr>_________________
Users mailing list
<a class="m_5518641707690872261m_2485402866099392609moz-txt-link-abbreviated" href="mailto:Users@lists.opensips.org" target="_blank">Users@lists.opensips.org</a>
<a class="m_5518641707690872261m_2485402866099392609moz-txt-link-freetext" href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users" target="_blank">http://lists.opensips.org/cgi-<wbr>bin/mailman/listinfo/users</a>
</pre>
    </blockquote>
    

  </div>


______________________________<wbr>_________________

Users mailing list

<a href="mailto:Users@lists.opensips.org" target="_blank">Users@lists.opensips.org</a>

<a href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.opensips.org/cgi-<wbr>bin/mailman/listinfo/users</a>


</blockquote></div>
</div>


<fieldset class="m_5518641707690872261mimeAttachmentHeader"></fieldset>
<pre>______________________________<wbr>_________________
Users mailing list
<a class="m_5518641707690872261moz-txt-link-abbreviated" href="mailto:Users@lists.opensips.org" target="_blank">Users@lists.opensips.org</a>
<a class="m_5518641707690872261moz-txt-link-freetext" href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users" target="_blank">http://lists.opensips.org/cgi-<wbr>bin/mailman/listinfo/users</a>
</pre>

</blockquote>
</div></div></div><br>______________________________<wbr>_________________<br>
Users mailing list<br>
<a href="mailto:Users@lists.opensips.org">Users@lists.opensips.org</a><br>
<a href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.opensips.org/cgi-<wbr>bin/mailman/listinfo/users</a><br>
<br></blockquote></div><br></div>