[OpenSIPS-Users] Debugging memory leaks

Fabian Gast fgast+opensips at only640k.net
Wed Mar 11 10:21:35 EST 2020 

Ok, i think i know why the numbers look strange:  
The output was at the shutdown, but a few seconds before i send the SIGUSR1...


this is the correct output:

Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: Memory status (shm):
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: qm_status (0x7ff2182ea000):
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:  heap size= 4294967296
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:  used= 88432160, used+overhead=95192920, free=4199774376
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:  max used (+overhead)= 119231640
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:  dumping summary of all alloc'ed. fragments:
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: ----------------------------------------------------
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: total_bytes | num_allocations x [file: func, line]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: ----------------------------------------------------
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:         256 : 32 x [net/net_tcp.c: tcp_init, line 1728]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:          32 : 1 x [dlist.c: new_dlist, line 901]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:       14464 : 1 x [timer.c: tm_init_timers, line 533]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:          16 : 1 x [dlist.c: new_dlist, line 909]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:       38528 : 826 x [../../ut.h: shm_nt_str_dup, line 716]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:         536 : 52 x [statistics.c: register_stat2, line 399]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:       19840 : 1107 x [statistics.c: build_stat_name, line 122]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:          32 : 1 x [dlg_timer.c: init_dlg_reinvite_ping_timer, line 185]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:         256 : 1 x [lock.c: lock_initialize, line 88]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:      131104 : 1 x [dlg_hash.c: init_dlg_table, line 136]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:     3645840 : 31527 x [usr_avp.c: new_avp, line 117]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:       17392 : 826 x [urecord.c: new_urecord, line 85]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:         480 : 1 x [statistics.c: __add_stat_module, line 166]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:         408 : 48 x [mi/mi.c: register_mi_cmd, line 174]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:           8 : 1 x [timer.c: init_timer, line 81]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:     5170344 : 3893 x [sip_msg.c: update_cloned_msg_from_msg, line 1190]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:          32 : 1 x [dlg_timer.c: init_dlg_timer, line 55]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:       98696 : 3893 x [t_fwd.c: add_phony_uac, line 507]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:          16 : 1 x [t_hooks.c: init_tmcb_lists, line 64]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:     1048576 : 1 x [hash.c: lcache_htable_init, line 50]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:         208 : 12 x [statistics.c: __add_stat_module, line 182]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:           8 : 1 x [dlg_timer.c: init_dlg_timer, line 64]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:         128 : 2 x [ebr_data.c: add_ebr_event, line 79]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:         168 : 14 x [map.c: map_get, line 150]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:       85552 : 2165 x [map.c: map_create, line 79]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:        5904 : 1 x [core_stats.c: init_pkg_stats, line 173]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:         128 : 16 x [../../rw_locking.h: lock_init_rw, line 45]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:      271632 : 826 x [ucontact.c: new_ucontact, line 105]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:           8 : 1 x [timer.c: init_timer, line 82]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:           8 : 1 x [usr_avp.c: init_extra_avps, line 83]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:         480 : 6 x [statistics.c: register_stat2, line 385]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:           8 : 1 x [net/net_tcp.c: tcp_init, line 1718]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:      955024 : 3489 x [hash.c: lcache_htable_insert, line 126]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:          24 : 1 x [rw_locking.h: lock_init_rw, line 40]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:          32 : 1 x [dlg_timer.c: init_dlg_ping_timer, line 155]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:           8 : 1 x [mem/shm_mem.c: shm_mem_init_mallocs, line 390]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:       16384 : 1 x [udomain.c: new_udomain, line 88]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:          80 : 10 x [evi/event_interface.c: evi_publish_event, line 75]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:        3368 : 38 x [timer.c: new_os_timer, line 146]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:          40 : 1 x [cachedb_local.c: parse_collections, line 608]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:         136 : 1 x [event_route.c: fixup_scriptroute_fetch, line 564]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:           8 : 1 x [usr_avp.c: init_extra_avps, line 74]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:         984 : 1 x [core_stats.c: init_pkg_stats, line 174]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:        4120 : 1 x [statistics.c: init_stats_collector, line 223]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:         696 : 19 x [ucontact.c: mem_update_ucontact, line 250]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:      565800 : 1 x [pt.c: init_multi_proc_support, line 70]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:           8 : 1 x [timer.c: init_timer, line 83]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:         400 : 1 x [evi/event_interface.c: evi_publish_event, line 61]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:        8208 : 1 x [../../evi/../lock_alloc.h: lock_set_alloc, line 66]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:       79464 : 826 x [urecord.c: new_urecord, line 70]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:         464 : 2 x [event_routing.c: ebr_parse, line 380]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:         384 : 16 x [../../rw_locking.h: lock_init_rw, line 40]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:          32 : 2 x [evi/evi_transport.c: register_event_mod, line 84]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:           8 : 1 x [dlg_timer.c: init_dlg_ping_timer, line 162]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:      262144 : 32 x [net/net_tcp.c: tcp_init, line 1741]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:          24 : 1 x [ul_callback.c: init_ulcb_list, line 44]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:          56 : 1 x [udomain.c: new_udomain, line 81]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:       81792 : 1167 x [statistics.c: register_stat2, line 388]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:     2253944 : 3893 x [mem/shm_mem.c: _shm_resize, line 226]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:           8 : 1 x [rw_locking.h: lock_init_rw, line 45]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:          16 : 1 x [daemonize.c: set_osips_state, line 576]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:          40 : 1 x [t_hooks.c: insert_tmcb, line 92]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:     2621448 : 1 x [h_table.c: init_hash_table, line 372]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:      262144 : 32 x [net/net_tcp.c: tcp_init, line 1747]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:      251864 : 3287 x [../../ut.h: shm_str_dup, line 692]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:    33428816 : 3893 x [h_table.c: build_cell, line 244]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:          72 : 3 x [evi/event_interface.c: evi_event_subscribe, line 334]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:         176 : 1 x [event_route.c: scriptroute_parse, line 306]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:         616 : 12 x [ucontact.c: mem_update_ucontact, line 255]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:           8 : 1 x [dlg_timer.c: init_dlg_reinvite_ping_timer, line 192]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:           8 : 1 x [sl_funcs.c: sl_startup, line 80]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:    37022512 : 3893 x [sip_msg.c: sip_msg_cloner, line 534]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:        1824 : 12 x [ucontact.c: mem_update_ucontact, line 271]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:       55816 : 840 x [map.c: map_get, line 139]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:           8 : 1 x [daemonize.c: create_status_pipe, line 92]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]:        2064 : 1 x [../../lock_alloc.h: lock_set_alloc, line 66]
Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: ----------------------------------------------------


----- Ursprüngliche Mail -----
Von: "Liviu Chircu" <liviu at opensips.org>
An: "OpenSIPS users mailling list" <users at lists.opensips.org>
Gesendet: Mittwoch, 11. März 2020 11:03:16
Betreff: Re: [OpenSIPS-Users] Debugging memory leaks

On 11.03.2020 11:06, Fabian Gast wrote:
> How can we continue from the memory status on hunting down the problems? Is there any advice on this?

Hey Fabian,

When you ran the "shm_mem_dump" which produced the pasted output, what 
values did the "shmem:" statistics group hold? Based on the output, you 
were barely using 1 MB of shared memory, which is a bit strange.

The table head tells exactly what the numbers represent: total bytes, 
number of allocations and the file/func/line which allocated them.

Regards,

-- 
Liviu Chircu
www.twitter.com/liviuchircu | www.opensips-solutions.com

OpenSIPS Summit, Amsterdam, May 2020
   www.opensips.org/events


_______________________________________________
Users mailing list
Users at lists.opensips.org
http://lists.opensips.org/cgi-bin/mailman/listinfo/users

More information about the Users mailing list