[OpenSIPS-Users] OpenSIPS crashed after "out of pkg memory"

[Kevin Mathy]

Hi List,

I'm facing a memory usage issue with OpenSIPS 1.9.2 on a Debian 7 server

root at asbc2:/home/kemathy# uname -a
> Linux asbc2 3.2.0-4-amd64 #1 SMP Debian 3.2.57-3+deb7u2 x86_64 GNU/Linux
>


> root at asbc2:/home/kemathy# opensips -V

version: opensips 1.9.2-notls (x86_64/linux)
> flags: STATS: On, USE_IPV6, USE_TCP, DISABLE_NAGLE, USE_MCAST, SHM_MEM,
> SHM_MMAP, PKG_MALLOC, DBG_F_MALLOC, FAST_LOCK-ADAPTIVE_WAIT
> ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16,
> MAX_URI_SIZE 1024, BUF_SIZE 65535
> poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
> svnrevision: unknown
> @(#) $Id$
> main.c compiled on 11:15:37 Jun 20 2014 with gcc 4.7


The server handles something like 70000 to 90000 calls per day, with around
20 to 40 Calls Per Second


As you can see on the following Cacti graphs, system's free memory
decreases constantly to finally reach the lower value of 160M :

https://www.dropbox.com/s/np3fnc6ni4vuoet/ASBC2_Memory_Usage_Evolution.jpg

The system can stay 2-3 days working in this state, but for example,
yesterday evening (2014-06-25 18:50:00), opensips crashed, after showing
those log messages :

Jun 25 18:51:18 asbc2 /usr/local/sbin/opensips[11390]:
> ERROR:db_mysql:db_mysql_convert_rows: no private memory left
> Jun 25 18:51:18 asbc2 /usr/local/sbin/opensips[11390]:
> ERROR:core:build_res_buf_from_sip_req: out of pkg memory  ; needs 412
> Jun 25 18:51:18 asbc2 /usr/local/sbin/opensips[11380]:
> ERROR:core:build_res_buf_from_sip_req: out of pkg memory  ; needs 355
> Jun 25 18:51:18 asbc2 /usr/local/sbin/opensips[11380]:
> ERROR:core:received_builder: out of pkg memory
> Jun 25 18:51:18 asbc2 /usr/local/sbin/opensips[11364]:
> ERROR:core:build_res_buf_from_sip_req: out of pkg memory  ; needs 355
> Jun 25 18:51:18 asbc2 /usr/local/sbin/opensips[11364]:
> ERROR:core:add_lump_rpl: no free pkg memory !
> Jun 25 18:51:18 asbc2 /usr/local/sbin/opensips[11362]:
> ERROR:core:build_res_buf_from_sip_req: out of pkg memory  ; needs 355
> Jun 25 18:51:18 asbc2 /usr/local/sbin/opensips[11362]:
> ERROR:core:db_allocate_rows: no memory left
> Jun 25 18:51:18 asbc2 /usr/local/sbin/opensips[11362]:
> ERROR:db_mysql:db_mysql_convert_rows: no private memory left
> Jun 25 18:51:18 asbc2 /usr/local/sbin/opensips[11362]:
> ERROR:core:build_res_buf_from_sip_req: out of pkg memory  ; needs 412
> Jun 25 18:51:18 asbc2 /usr/local/sbin/opensips[11356]:
> ERROR:core:build_res_buf_from_sip_req: out of pkg memory  ; needs 355
> Jun 25 18:51:18 asbc2 /usr/local/sbin/opensips[11356]:
> ERROR:core:add_lump_rpl: no free pkg memory !
>

And after the service's restart, everything was OK :

> Jun 25 19:12:39 asbc2 /usr/local/sbin/opensips[29351]: INFO:core:main:
> using 64 Mb shared memory
> Jun 25 19:12:39 asbc2 /usr/local/sbin/opensips[29351]: INFO:core:main:
> using 4 Mb private memory per process



For information, here are the memory values at this moment; opensips has
handled 5000 calls since his bootup :

> root at asbc2:/home/kemathy# free -m
>              total       used       free     shared    buffers     cached
> Mem:         16082      12887       3195          0        339      11393
> -/+ buffers/cache:       1154      14928
> Swap:        32755          0      32755



For now, OpenSIPS is launched with S_MEMORY = 64 and P_MEMORY = 4
My first idea would be to increase the P_MEMORY value to 8 for example, as
it seems to be a lack of PKG Memory, but in case it's a memory-freeing
problem, it'll not solve our issue (I think).

My questions are : How can we calculate the appropriate value for S_MEMORY
and P_MEMORY ? And how can we solve our "out of pkg memory" problem ?

If you need further informations, or anything else, feel free to ask me !

Thanks a lot for your help,


*Bien cordialement, Best Regards,  **Kevin MATHY* | Ingénieur VoIP

-- 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opensips.org/pipermail/users/attachments/20140626/ce5bbc9f/attachment-0001.htm>