<div dir="ltr">with every call attempt it decreases. I tried some changes by rejecting invite before drouting call (That means after auth , dispatcher) and found memory is stable but when drouting sends Invite to external gateway and external gateway rejects it. Then this issue happens.<div><br></div><div>Inuse transactions and active dialogs also 0. Somthing wrong happening in handling of failure replies. But apart from <span style="background-color:rgb(254,254,238);color:rgb(0,0,0)">use_next_gw and setting some avps for CDR not much going on there.</span></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Mar 6, 2017 at 5:54 PM, Răzvan Crainea <span dir="ltr"><<a href="mailto:razvan@opensips.org" target="_blank">razvan@opensips.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<tt>Ok, so it is the first listener for the private IP that leaks.
Next, is the memory stabilizing in time? Or it is continously
decreasing?<br>
Yes, that's how you should make the dump.<br>
<br>
Best regards,<br>
</tt><span class="">
<pre class="m_5518641707690872261moz-signature" cols="72">Răzvan Crainea
OpenSIPS Solutions
<a class="m_5518641707690872261moz-txt-link-abbreviated" href="http://www.opensips-solutions.com" target="_blank">www.opensips-solutions.com</a></pre>
</span><div><div class="h5"><div class="m_5518641707690872261moz-cite-prefix">On 03/06/2017 10:57 AM, John Nash
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Dear Razvan,
<div><br>
</div>
<div>Below is the info on my processes</div>
<div>
<div>Process:: ID=0 PID=17351 Type=attendant</div>
<div>Process:: ID=1 PID=17352 Type=MI FIFO</div>
<div>Process:: ID=2 PID=17353 Type=MI Datagram</div>
<div>Process:: ID=3 PID=17354 Type=time_keeper</div>
<div>Process:: ID=4 PID=17355 Type=timer</div>
<div>Process:: ID=5 PID=17356 Type=SIP receiver udp:<a href="http://1.1.1.1:9094" target="_blank">1.1.1.1:9094</a> </div>
<div>Process:: ID=6 PID=17357 Type=SIP receiver udp:<a href="http://1.1.1.1:5060" target="_blank">1.1.1.1:5060</a> </div>
<div>Process:: ID=7 PID=17358 Type=SIP receiver udp:<a href="http://192.168.45.5:5064" target="_blank">192.168.45.5:5064</a> </div>
<div>Process:: ID=8 PID=17359 Type=Timer handler</div>
</div>
<div><br>
</div>
<div>1.1.1.1 is public IP (I changed). The decrease in memory I
see is for Process:: ID=7 PID=17358 mainly. My call flow is
as following </div>
<div><br>
</div>
<div>- New Invite hits the opensips on <a href="http://1.1.1.1:9094" target="_blank">1.1.1.1:9094</a></div>
<div>- Apart from message validity checks I query DB to check if
its a valid user (Using local cache also there)</div>
<div>- Create dialog, Topology_hiding functions are called along
with some avp population</div>
<div>- Using dispatcher ds_select_domain Call sent to udp:<a href="http://192.168.45.2:7060" target="_blank">192.168.45.2:7060</a>
(using force socket). This <a href="http://192.168.45.2:7060" target="_blank">192.168.45.2:7060</a> is
actually freeswitch </div>
<div>- Call again comes back to opensips on udp:<a href="http://192.168.45.5:5064" target="_blank">192.168.45.5:5064</a></div>
<div>- New dialog is created and topology_hiding is called</div>
<div>- Drouting function do_routing is called which tries one
gateway and fails<br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>Dump i need to create with memlog=4 memdump=1 right?</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Mon, Mar 6, 2017 at 2:05 PM, Răzvan
Crainea <span dir="ltr"><<a href="mailto:razvan@opensips.org" target="_blank">razvan@opensips.org</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"> <tt>Hi, John!<br>
<br>
Transactions are stored in shared memory, not in the
private one. So the possible leak you are facing its not
related to transactions.<br>
During runtime, OpenSIPS might resize some internal
structures, which may lead to increase memory usage.
However, after a while, these allocations should
stabilize</tt><tt> .<br>
Can you post the output of the kill -SIGUSR1 on pastebin
so we can take a look? Also, what type of process is the
one you are seeing the leak into? You can find out using
the 'opensipsctl ps' command.<br>
<br>
Best regards,<br>
</tt>
<pre class="m_5518641707690872261m_2485402866099392609moz-signature" cols="72">Răzvan Crainea
OpenSIPS Solutions
<a class="m_5518641707690872261m_2485402866099392609moz-txt-link-abbreviated" href="http://www.opensips-solutions.com" target="_blank">www.opensips-solutions.com</a></pre>
<div>
<div class="m_5518641707690872261h5">
<div class="m_5518641707690872261m_2485402866099392609moz-cite-prefix">On
03/06/2017 09:55 AM, John Nash wrote:<br>
</div>
</div>
</div>
<blockquote type="cite">
<div>
<div class="m_5518641707690872261h5">
<div dir="ltr">I am using OpenSIPS (2.1.5
(x86_64/linux)) in production. I observed private
memory is decreasing constantly for one process
mainly and ultimately leading to memory errors and
crash.
<div><br>
</div>
<div>To debug this issue I prepared a test server
and compiled opensips as per <a href="https://www.opensips.org/Documentation/TroubleShooting-OutOfMem" target="_blank">https://www.opensips.org/D<wbr>ocumentation/TroubleShooting-O<wbr>utOfMem</a></div>
<div><br>
</div>
<div>I made only one single call (which was
rejected by opensips as it was not authorized
user) and I saw private free memory decreased. I
was hoping since transaction is done ideally it
should release memory and should show me same
memory as startup but it did not. I verified
this with many call attempts and i see free
memory is always decreasing slowly.</div>
<div><br>
</div>
<div>I used kill -SIGUSR1 <lowest pid> to
create memory dump. But i am unable to make
sense of it. It shows log like ...</div>
<div><br>
</div>
<div>r 6 07:29:19 Server3021 opensips[13276]:
Memory status (pkg):</div>
<div>Mar 6 07:29:19 Server3021 opensips[13276]:
qm_status (0x7f5b8ebba010):</div>
<div>Mar 6 07:29:19 Server3021 opensips[13276]:
heap size= 4194304</div>
<div>Mar 6 07:29:19 Server3021 opensips[13276]:
used= 346768, used+overhead=848792,
free=3345512</div>
<div>Mar 6 07:29:19 Server3021 opensips[13276]:
max used (+overhead)= 931920</div>
<div>Mar 6 07:29:19 Server3021 opensips[13276]:
dumping all alloc'ed. fragments:</div>
<div>Mar 6 07:29:19 Server3021 opensips[13276]:
0. N address=0x7f5b8ebef528
frag=0x7f5b8ebef4f8 size=40 used=1</div>
<div>Mar 6 07:29:19 Server3021 opensips[13276]:
alloc'd from script_cb.c:
add_callback(60)</div>
<div>Mar 6 07:29:19 Server3021 opensips[13276]:
start check=f0f0f0f0f0f0f0f0, end check=
c0c0c0c0c0c0c0c0, abcdefedabcdefed</div>
<div>Mar 6 07:29:19 Server3021 opensips[13276]:
1. N address=0x7f5b8ebef5b0 <br>
</div>
<div><br>
</div>
<div>I pasted only few lines in this mail. What
should be my next step?...How can i really trace
what is wrong in my script or any other memory
leak?</div>
<div><br>
</div>
</div>
<br>
<fieldset class="m_5518641707690872261m_2485402866099392609mimeAttachmentHeader"></fieldset>
<br>
</div>
</div>
<pre>______________________________<wbr>_________________
Users mailing list
<a class="m_5518641707690872261m_2485402866099392609moz-txt-link-abbreviated" href="mailto:Users@lists.opensips.org" target="_blank">Users@lists.opensips.org</a>
<a class="m_5518641707690872261m_2485402866099392609moz-txt-link-freetext" href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users" target="_blank">http://lists.opensips.org/cgi-<wbr>bin/mailman/listinfo/users</a>
</pre>
</blockquote>
</div>
______________________________<wbr>_________________
Users mailing list
<a href="mailto:Users@lists.opensips.org" target="_blank">Users@lists.opensips.org</a>
<a href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.opensips.org/cgi-<wbr>bin/mailman/listinfo/users</a>
</blockquote></div>
</div>
<fieldset class="m_5518641707690872261mimeAttachmentHeader"></fieldset>
<pre>______________________________<wbr>_________________
Users mailing list
<a class="m_5518641707690872261moz-txt-link-abbreviated" href="mailto:Users@lists.opensips.org" target="_blank">Users@lists.opensips.org</a>
<a class="m_5518641707690872261moz-txt-link-freetext" href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users" target="_blank">http://lists.opensips.org/cgi-<wbr>bin/mailman/listinfo/users</a>
</pre>
</blockquote>
</div></div></div><br>______________________________<wbr>_________________<br>
Users mailing list<br>
<a href="mailto:Users@lists.opensips.org">Users@lists.opensips.org</a><br>
<a href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.opensips.org/cgi-<wbr>bin/mailman/listinfo/users</a><br>
<br></blockquote></div><br></div>