[OpenSIPS-Users] Quest to find memory leak

John Nash john.nash778 at gmail.com
Mon Mar 6 08:02:16 EST 2017 

Here is the dump
http://pastebin.com/DTEHF5Vc

On Mon, Mar 6, 2017 at 6:20 PM, Răzvan Crainea <razvan at opensips.org> wrote:

> None of the "actions" you are talking about have big impact on private
> memory, but the shared one. Better do the dump and send it over to point
> out what is "eating" memory.
>
> Best regards,
>
> Răzvan Crainea
> OpenSIPS Solutionswww.opensips-solutions.com
>
> On 03/06/2017 02:39 PM, John Nash wrote:
>
> with every call attempt it decreases. I tried some changes by rejecting
> invite before drouting call (That means after auth , dispatcher) and found
> memory is stable but when drouting sends Invite to external gateway and
> external gateway rejects it. Then this issue happens.
>
> Inuse transactions and active dialogs also 0. Somthing wrong happening in
> handling of failure replies. But apart from use_next_gw and setting some
> avps for CDR not much going on there.
>
> On Mon, Mar 6, 2017 at 5:54 PM, Răzvan Crainea <razvan at opensips.org>
> wrote:
>
>> Ok, so it is the first listener for the private IP that leaks. Next, is
>> the memory stabilizing in time? Or it is continously decreasing?
>> Yes, that's how you should make the dump.
>>
>> Best regards,
>>
>> Răzvan Crainea
>> OpenSIPS Solutionswww.opensips-solutions.com
>>
>> On 03/06/2017 10:57 AM, John Nash wrote:
>>
>> Dear Razvan,
>>
>> Below is the info on my processes
>> Process::  ID=0 PID=17351 Type=attendant
>> Process::  ID=1 PID=17352 Type=MI FIFO
>> Process::  ID=2 PID=17353 Type=MI Datagram
>> Process::  ID=3 PID=17354 Type=time_keeper
>> Process::  ID=4 PID=17355 Type=timer
>> Process::  ID=5 PID=17356 Type=SIP receiver udp:1.1.1.1:9094
>> Process::  ID=6 PID=17357 Type=SIP receiver udp:1.1.1.1:5060
>> Process::  ID=7 PID=17358 Type=SIP receiver udp:192.168.45.5:5064
>> Process::  ID=8 PID=17359 Type=Timer handler
>>
>> 1.1.1.1 is public IP (I changed). The decrease in memory I see is for
>> Process::  ID=7 PID=17358 mainly. My call flow is as following
>>
>> - New Invite hits the opensips on 1.1.1.1:9094
>> - Apart from message validity checks I query DB to check if its a valid
>> user (Using local cache also there)
>> - Create dialog, Topology_hiding functions are called along with some avp
>> population
>> - Using dispatcher ds_select_domain Call sent to udp:192.168.45.2:7060
>> (using force socket). This 192.168.45.2:7060 is actually freeswitch
>> - Call again comes back to opensips on udp:192.168.45.5:5064
>> - New dialog is created and topology_hiding is called
>> - Drouting function do_routing is called which tries one gateway and fails
>>
>>
>> Dump i need to create with memlog=4 memdump=1 right?
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> On Mon, Mar 6, 2017 at 2:05 PM, Răzvan Crainea <razvan at opensips.org>
>> wrote:
>>
>>> Hi, John!
>>>
>>> Transactions are stored in shared memory, not in the private one. So the
>>> possible leak you are facing its not related to transactions.
>>> During runtime, OpenSIPS might resize some internal structures, which
>>> may lead to increase memory usage. However, after a while, these
>>> allocations should stabilize .
>>> Can you post the output of the kill -SIGUSR1 on pastebin so we can take
>>> a look? Also, what type of process is the one you are seeing the leak into?
>>> You can find out using the 'opensipsctl ps' command.
>>>
>>> Best regards,
>>>
>>> Răzvan Crainea
>>> OpenSIPS Solutionswww.opensips-solutions.com
>>>
>>> On 03/06/2017 09:55 AM, John Nash wrote:
>>>
>>> I am using OpenSIPS (2.1.5 (x86_64/linux)) in production. I observed
>>> private memory is decreasing constantly for one process mainly and
>>> ultimately leading to memory errors and crash.
>>>
>>> To debug this issue I prepared a test server and compiled opensips as
>>> per https://www.opensips.org/Documentation/TroubleShooting-OutOfMem
>>>
>>> I made only one single call (which was rejected by opensips as it was
>>> not authorized user) and I saw private free memory decreased. I was hoping
>>> since transaction is done ideally it should release memory and should show
>>> me same memory as startup but it did not. I verified this with many call
>>> attempts and i see free memory is always decreasing slowly.
>>>
>>> I used kill -SIGUSR1 <lowest pid> to create memory dump. But i am unable
>>> to make sense of it. It shows log like ...
>>>
>>> r  6 07:29:19 Server3021 opensips[13276]: Memory status (pkg):
>>> Mar  6 07:29:19 Server3021 opensips[13276]: qm_status (0x7f5b8ebba010):
>>> Mar  6 07:29:19 Server3021 opensips[13276]:  heap size= 4194304
>>> Mar  6 07:29:19 Server3021 opensips[13276]:  used= 346768,
>>> used+overhead=848792, free=3345512
>>> Mar  6 07:29:19 Server3021 opensips[13276]:  max used (+overhead)= 931920
>>> Mar  6 07:29:19 Server3021 opensips[13276]: dumping all alloc'ed.
>>> fragments:
>>> Mar  6 07:29:19 Server3021 opensips[13276]:       0. N
>>>  address=0x7f5b8ebef528 frag=0x7f5b8ebef4f8 size=40 used=1
>>> Mar  6 07:29:19 Server3021 opensips[13276]:             alloc'd from
>>> script_cb.c: add_callback(60)
>>> Mar  6 07:29:19 Server3021 opensips[13276]:         start
>>> check=f0f0f0f0f0f0f0f0, end check= c0c0c0c0c0c0c0c0, abcdefedabcdefed
>>> Mar  6 07:29:19 Server3021 opensips[13276]:       1. N
>>>  address=0x7f5b8ebef5b0
>>>
>>> I pasted only few lines in this mail. What should be my next step?...How
>>> can i really trace what is wrong in my script or any other memory leak?
>>>
>>>
>>>
>>> _______________________________________________
>>> Users mailing listUsers at lists.opensips.orghttp://lists.opensips.org/cgi-bin/mailman/listinfo/users
>>>
>>> _______________________________________________ Users mailing list
>>> Users at lists.opensips.org http://lists.opensips.org/cgi-
>>> bin/mailman/listinfo/users
>>
>> _______________________________________________
>> Users mailing listUsers at lists.opensips.orghttp://lists.opensips.org/cgi-bin/mailman/listinfo/users
>>
>> _______________________________________________ Users mailing list
>> Users at lists.opensips.org http://lists.opensips.org/cgi-
>> bin/mailman/listinfo/users
>
> _______________________________________________
> Users mailing listUsers at lists.opensips.orghttp://lists.opensips.org/cgi-bin/mailman/listinfo/users
>
>
> _______________________________________________
> Users mailing list
> Users at lists.opensips.org
> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opensips.org/pipermail/users/attachments/20170306/5467a102/attachment-0001.html>

More information about the Users mailing list