[OpenSIPS-Users] Debugging memory leaks

[Fabian Gast]

Hi @all, 

according to our graphs [1] and some recent crashes it looks like we have some memory leaks 
in our opensips processes. 

We now have the memory status from one of our staging environments with about 2k devices. 
(The impact on our live machines is even more severe, but we can not enable memory debugging 
on these systems for $reasons.)

How can we continue from the memory status on hunting down the problems? Is there any advice on this?

snipped from the memory status dump - full trace available upon request... 
Mar 10 23:00:39 ireg02 /usr/sbin/opensips[42351]: Memory status (shm):
Mar 10 23:00:39 ireg02 /usr/sbin/opensips[42351]: qm_status (0x7ff2182ea000):
Mar 10 23:00:39 ireg02 /usr/sbin/opensips[42351]:  heap size= 4294967296
Mar 10 23:00:39 ireg02 /usr/sbin/opensips[42351]:  used= 19440, used+overhead=325640, free=4294641656
Mar 10 23:00:39 ireg02 /usr/sbin/opensips[42351]:  max used (+overhead)= 119231640
Mar 10 23:00:39 ireg02 /usr/sbin/opensips[42351]:  dumping summary of all alloc'ed. fragments:
Mar 10 23:00:39 ireg02 /usr/sbin/opensips[42351]: ----------------------------------------------------
Mar 10 23:00:39 ireg02 /usr/sbin/opensips[42351]: total_bytes | num_allocations x [file: func, line]
Mar 10 23:00:39 ireg02 /usr/sbin/opensips[42351]: ----------------------------------------------------
Mar 10 23:00:39 ireg02 /usr/sbin/opensips[42351]:        6040 : 366 x [statistics.c: build_stat_name, line 122]
Mar 10 23:00:39 ireg02 /usr/sbin/opensips[42351]:          32 : 1 x [dlg_timer.c: init_dlg_reinvite_ping_timer, line 185]
Mar 10 23:00:39 ireg02 /usr/sbin/opensips[42351]:         408 : 48 x [mi/mi.c: register_mi_cmd, line 174]
Mar 10 23:00:39 ireg02 /usr/sbin/opensips[42351]:         128 : 2 x [ebr_data.c: add_ebr_event, line 79]
Mar 10 23:00:39 ireg02 /usr/sbin/opensips[42351]:         168 : 14 x [map.c: map_get, line 150]
Mar 10 23:00:39 ireg02 /usr/sbin/opensips[42351]:          32 : 1 x [map.c: map_create, line 79]
Mar 10 23:00:39 ireg02 /usr/sbin/opensips[42351]:        5904 : 1 x [core_stats.c: init_pkg_stats, line 173]
Mar 10 23:00:39 ireg02 /usr/sbin/opensips[42351]:           8 : 1 x [usr_avp.c: init_extra_avps, line 83]
Mar 10 23:00:39 ireg02 /usr/sbin/opensips[42351]:           8 : 1 x [mem/shm_mem.c: shm_mem_init_mallocs, line 390]
Mar 10 23:00:39 ireg02 /usr/sbin/opensips[42351]:          80 : 10 x [evi/event_interface.c: evi_publish_event, line 75]
Mar 10 23:00:39 ireg02 /usr/sbin/opensips[42351]:        3368 : 38 x [timer.c: new_os_timer, line 146]
Mar 10 23:00:39 ireg02 /usr/sbin/opensips[42351]:          40 : 1 x [cachedb_local.c: parse_collections, line 608]
Mar 10 23:00:39 ireg02 /usr/sbin/opensips[42351]:         136 : 1 x [event_route.c: fixup_scriptroute_fetch, line 564]
Mar 10 23:00:39 ireg02 /usr/sbin/opensips[42351]:           8 : 1 x [usr_avp.c: init_extra_avps, line 74]
Mar 10 23:00:39 ireg02 /usr/sbin/opensips[42351]:         984 : 1 x [core_stats.c: init_pkg_stats, line 174]
Mar 10 23:00:39 ireg02 /usr/sbin/opensips[42351]:           8 : 1 x [timer.c: init_timer, line 83]
Mar 10 23:00:39 ireg02 /usr/sbin/opensips[42351]:         400 : 1 x [evi/event_interface.c: evi_publish_event, line 61]
Mar 10 23:00:39 ireg02 /usr/sbin/opensips[42351]:         464 : 2 x [event_routing.c: ebr_parse, line 380]
Mar 10 23:00:39 ireg02 /usr/sbin/opensips[42351]:          32 : 2 x [evi/evi_transport.c: register_event_mod, line 84]
Mar 10 23:00:39 ireg02 /usr/sbin/opensips[42351]:          16 : 1 x [daemonize.c: set_osips_state, line 576]
Mar 10 23:00:39 ireg02 /usr/sbin/opensips[42351]:          72 : 3 x [evi/event_interface.c: evi_event_subscribe, line 334]
Mar 10 23:00:39 ireg02 /usr/sbin/opensips[42351]:         176 : 1 x [event_route.c: scriptroute_parse, line 306]
Mar 10 23:00:39 ireg02 /usr/sbin/opensips[42351]:           8 : 1 x [dlg_timer.c: init_dlg_reinvite_ping_timer, line 192]
Mar 10 23:00:39 ireg02 /usr/sbin/opensips[42351]:         912 : 14 x [map.c: map_get, line 139]
Mar 10 23:00:39 ireg02 /usr/sbin/opensips[42351]:           8 : 1 x [daemonize.c: create_status_pipe, line 92]
Mar 10 23:00:39 ireg02 /usr/sbin/opensips[42351]: ----------------------------------------------------

version: opensips 2.4.6 (x86_64/linux)
flags: STATS: On, DISABLE_NAGLE, USE_MCAST, SHM_MMAP, PKG_MALLOC, QM_MALLOC, DBG_MALLOC, FAST_LOCK-ADAPTIVE_WAIT
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535
poll method support: poll, epoll, sigio_rt, select.
git revision: edef893
main.c compiled on  with gcc 4.9.2

Thanks, 

Fabian 



[1] https://imgur.com/a/9drmJHR