<html>
<head>
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix"><tt>Hi Kevin,<br>
<br>
There is no need to wait for a crash. From time to time, you can
send a SIGUSR1 to a worker process (or a process you suspect as
running out of pkg mem) -> the process will do a pkg dump to
the log.<br>
<br>
Also, I would strongly advice upgrading to 1.11 (latest LTS) -
1.9 is no longer maintained and there were some fixes in the
memory manager since then.<br>
<br>
Regards,<br>
</tt>
<pre class="moz-signature" cols="72">Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
<a class="moz-txt-link-freetext" href="http://www.opensips-solutions.com">http://www.opensips-solutions.com</a></pre>
On 27.06.2014 10:36, Kevin Mathy wrote:<br>
</div>
<blockquote
cite="mid:CAOmzhXnD4wj44CTsyvb9mM5PDBoF1dZxgAqnBjQkY7owghNPTQ@mail.gmail.com"
type="cite">
<div dir="ltr">Hi Bogdan,
<div><br>
</div>
<div>I've set given options, and now I'm waiting for a new crash
of the service... Where the memdump will be located ? In
another logfile than opensips.log, or in the same ? </div>
<div><br>
</div>
<div>Thanks </div>
</div>
<div class="gmail_extra"><br clear="all">
<div><b>
<div><span style="font-weight:normal">Bien cordialement, </span></div>
<div><span style="font-weight:normal">Best Regards, </span></div>
<div><span style="font-weight:normal"><br>
</span></div>
</b><b>Kevin MATHY</b> |<b> </b>Ingénieur VoIP<br>
<div>
<div><b><br>
</b></div>
</div>
</div>
<br>
<br>
<div class="gmail_quote">2014-06-26 18:32 GMT+02:00
Bogdan-Andrei Iancu <span dir="ltr"><<a
moz-do-not-send="true" href="mailto:bogdan@opensips.org"
target="_blank">bogdan@opensips.org</a>></span>:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div><tt>Kevin,<br>
<br>
Restarting should not make you loose ongoing calls
(even if you use the dialog module), do do not worry
on that.<br>
<br>
Regards,<br>
</tt>
<div class="">
<pre cols="72">Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
<a moz-do-not-send="true" href="http://www.opensips-solutions.com" target="_blank">http://www.opensips-solutions.com</a></pre>
</div>
<div>
<div class="h5"> On <a moz-do-not-send="true"
href="tel:26.06.2014%2018" value="+12606201418"
target="_blank">26.06.2014 18</a>:31, Kevin Mathy
wrote:<br>
</div>
</div>
</div>
<blockquote type="cite">
<div>
<div class="h5">
<div dir="ltr">Yes, I'll set those options, and wait
'till the trafic on the server reduces a lot
(tonight I think) to restart opensips.
<div>Unless if you have another method to reload
the config without losing calls ? </div>
<div><br>
</div>
<div>Thanks a lot, <br>
<div><br>
</div>
</div>
</div>
<div class="gmail_extra"><br clear="all">
<div><b>
<div><span style="font-weight:normal">Bien
cordialement, </span></div>
<div><span style="font-weight:normal">Best
Regards, </span></div>
<div><span style="font-weight:normal"><br>
</span></div>
</b><b>Kevin MATHY</b> |<b> </b>Ingénieur VoIP<br>
<div>
<div><b><br>
</b></div>
</div>
</div>
<br>
<br>
<div class="gmail_quote">2014-06-26 17:20
GMT+02:00 Bogdan-Andrei Iancu <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:bogdan@opensips.org"
target="_blank">bogdan@opensips.org</a>></span>:<br>
<blockquote class="gmail_quote" style="margin:0
0 0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div><tt>Kevin,<br>
<br>
The trick is to set (in this order):<br>
memlog = 4<br>
memdump = 1<br>
<br>
This will do no logging at runtime, but
it will log the mem dump (at shutdown or
at signal).<br>
<br>
Could you try that ?<br>
<br>
Regards,<br>
</tt>
<div>
<pre cols="72">Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
<a moz-do-not-send="true" href="http://www.opensips-solutions.com" target="_blank">http://www.opensips-solutions.com</a></pre>
</div>
<div>
<div> On <a moz-do-not-send="true"
href="tel:26.06.2014%2018"
value="+12606201418" target="_blank">26.06.2014
18</a>:09, Kevin Mathy wrote:<br>
</div>
</div>
</div>
<blockquote type="cite">
<div>
<div>
<div dir="ltr">Hi Bogdan,
<div><br>
</div>
<div>We already have compiled
opensips as described, and set
memlog=1 in the configuration
file. </div>
<div>But as we had a lot of debug
messages about memory usage, we
unset memlog=1 in the config, so
we don't have any report now.</div>
<div>Is there a way to get those
messages in a different log file,
instead of inside the "classical"
opensips.log ? I've found an
option like "memdump" that can be
set in opensips.cfg, but what
about it ? </div>
<div><br>
</div>
<div>And as cacti doesn't seems to
be relevant to check opensips's
mem usage, do you have any other
solution to check it ?</div>
<div><br>
</div>
<div>Thanks for your help, </div>
<div class="gmail_extra"> <br
clear="all">
<div><b>
<div><span
style="font-weight:normal">Bien
cordialement, </span></div>
<div><span
style="font-weight:normal">Best
Regards, </span></div>
<div><span
style="font-weight:normal"><br>
</span></div>
</b><b>Kevin MATHY</b> |<b> </b>Ingénieur
VoIP<br>
<div>
<div><b><br>
</b></div>
</div>
</div>
<br>
<br>
<div class="gmail_quote">2014-06-26
16:46 GMT+02:00 Bogdan-Andrei
Iancu <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:bogdan@opensips.org"
target="_blank">bogdan@opensips.org</a>></span>:<br>
<blockquote class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div bgcolor="#FFFFFF"
text="#000000">
<div><tt>Hi Kevin,<br>
<br>
For debugging potential
mem leaks, please look
into:<br>
<a
moz-do-not-send="true"
href="http://www.opensips.org/Documentation/TroubleShooting-OutOfMem"
target="_blank">http://www.opensips.org/Documentation/TroubleShooting-OutOfMem</a><br>
If you need help with
that, let me know.<br>
<br>
Also, cacti is
monitoring the system
memory which is
irrelevant for OpenSIPS
(as OpenSIPS is
pre-allocating - from OS
- all the needed memory
at startup).<br>
<br>
Regards,<br>
</tt>
<pre cols="72">Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
<a moz-do-not-send="true" href="http://www.opensips-solutions.com" target="_blank">http://www.opensips-solutions.com</a></pre>
<div>
<div> On <a
moz-do-not-send="true"
href="tel:26.06.2014%2015" value="+12606201415" target="_blank">26.06.2014
15</a>:04, Kevin
Mathy wrote:<br>
</div>
</div>
</div>
<blockquote type="cite">
<div>
<div>
<div dir="ltr">
<div>
<div
style="font-family:arial,sans-serif;font-size:13px;font-weight:bold"><span
style="font-weight:normal">Hi List,</span></div>
<div
style="font-family:arial,sans-serif;font-size:13px;font-weight:bold">
<span
style="font-weight:normal"><br>
</span></div>
<div
style="font-family:arial,sans-serif;font-size:13px;font-weight:bold"><span
style="font-weight:normal">I'm facing a memory usage issue with OpenSIPS
1.9.2 on a
Debian 7
server</span></div>
<div
style="font-family:arial,sans-serif;font-size:13px;font-weight:bold"><span
style="font-weight:normal"><br>
</span></div>
<div
style="font-family:arial,sans-serif;font-size:13px;font-weight:bold"><span
style="font-weight:normal">
<blockquote
class="gmail_quote"
style="margin:0px
0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
root@asbc2:/home/kemathy#
uname -a<br>
Linux asbc2
3.2.0-4-amd64
#1 SMP Debian
3.2.57-3+deb7u2
x86_64
GNU/Linux<br>
</blockquote>
<div> </div>
<blockquote
class="gmail_quote"
style="margin:0px
0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
root@asbc2:/home/kemathy#
opensips -V</blockquote>
<blockquote
class="gmail_quote"
style="margin:0px
0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">version:
opensips
1.9.2-notls
(x86_64/linux)<br>
flags: STATS:
On, USE_IPV6,
USE_TCP,
DISABLE_NAGLE,
USE_MCAST,
SHM_MEM,
SHM_MMAP,
PKG_MALLOC,
DBG_F_MALLOC,
FAST_LOCK-ADAPTIVE_WAIT<br>
ADAPTIVE_WAIT_LOOPS=1024,
MAX_RECV_BUFFER_SIZE
262144,
MAX_LISTEN 16,
MAX_URI_SIZE
1024, BUF_SIZE
65535<br>
poll method
support: poll,
epoll_lt,
epoll_et,
sigio_rt,
select.<br>
svnrevision:
unknown<br>
@(#) $Id$<br>
main.c
compiled on
11:15:37 Jun
20 2014 with
gcc 4.7</blockquote>
<div><br>
</div>
</span></div>
<div
style="font-family:arial,sans-serif;font-size:13px;font-weight:bold">
<span
style="font-weight:normal">The
server handles
something like
70000 to 90000
calls per day,
with around 20
to 40 Calls
Per Second</span></div>
<div
style="font-family:arial,sans-serif;font-size:13px;font-weight:bold">
<span
style="font-weight:normal"><br>
</span></div>
<div
style="font-family:arial,sans-serif;font-size:13px;font-weight:bold"><br>
</div>
<div
style="font-family:arial,sans-serif;font-size:13px">As
you can see on
the following
Cacti graphs,
system's free
memory decreases
constantly to
finally reach
the lower value
of 160M :</div>
</div>
<div><br>
</div>
<a
moz-do-not-send="true"
href="https://www.dropbox.com/s/np3fnc6ni4vuoet/ASBC2_Memory_Usage_Evolution.jpg"
target="_blank">https://www.dropbox.com/s/np3fnc6ni4vuoet/ASBC2_Memory_Usage_Evolution.jpg</a>
<div><br>
</div>
<div>
<div
style="font-family:arial,sans-serif;font-size:13px">
The system can
stay 2-3 days
working in this
state, but for
example,
yesterday
evening
(2014-06-25
18:50:00),
opensips
crashed, after
showing those
log messages : </div>
<div
style="font-family:arial,sans-serif;font-size:13px">
<br>
</div>
<div
style="font-family:arial,sans-serif;font-size:13px">
<blockquote
class="gmail_quote"
style="margin:0px
0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">Jun
25 18:51:18
asbc2
/usr/local/sbin/opensips[11390]:
ERROR:db_mysql:db_mysql_convert_rows:
no private
memory left<br>
Jun 25
18:51:18 asbc2
/usr/local/sbin/opensips[11390]:
ERROR:core:build_res_buf_from_sip_req:
out of pkg
memory ;
needs 412<br>
Jun 25
18:51:18 asbc2
/usr/local/sbin/opensips[11380]:
ERROR:core:build_res_buf_from_sip_req:
out of pkg
memory ;
needs 355<br>
Jun 25
18:51:18 asbc2
/usr/local/sbin/opensips[11380]:
ERROR:core:received_builder:
out of pkg
memory<br>
Jun 25
18:51:18 asbc2
/usr/local/sbin/opensips[11364]:
ERROR:core:build_res_buf_from_sip_req:
out of pkg
memory ;
needs 355<br>
Jun 25
18:51:18 asbc2
/usr/local/sbin/opensips[11364]:
ERROR:core:add_lump_rpl:
no free pkg
memory !<br>
Jun 25
18:51:18 asbc2
/usr/local/sbin/opensips[11362]:
ERROR:core:build_res_buf_from_sip_req:
out of pkg
memory ;
needs 355<br>
Jun 25
18:51:18 asbc2
/usr/local/sbin/opensips[11362]:
ERROR:core:db_allocate_rows:
no memory left<br>
Jun 25
18:51:18 asbc2
/usr/local/sbin/opensips[11362]:
ERROR:db_mysql:db_mysql_convert_rows:
no private
memory left<br>
Jun 25
18:51:18 asbc2
/usr/local/sbin/opensips[11362]:
ERROR:core:build_res_buf_from_sip_req:
out of pkg
memory ;
needs 412<br>
Jun 25
18:51:18 asbc2
/usr/local/sbin/opensips[11356]:
ERROR:core:build_res_buf_from_sip_req:
out of pkg
memory ;
needs 355<br>
Jun 25
18:51:18 asbc2
/usr/local/sbin/opensips[11356]:
ERROR:core:add_lump_rpl:
no free pkg
memory !<br>
</blockquote>
<div><br>
</div>
<div>And after
the service's
restart,
everything was
OK : </div>
<blockquote
class="gmail_quote"
style="margin:0px
0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">Jun
25 19:12:39
asbc2
/usr/local/sbin/opensips[29351]:
<a
moz-do-not-send="true">INFO:core:main</a>:
using 64 Mb
shared memory<br>
Jun 25
19:12:39 asbc2
/usr/local/sbin/opensips[29351]:
<a
moz-do-not-send="true">INFO:core:main</a>:
using 4 Mb
private memory
per process</blockquote>
</div>
<div
style="font-family:arial,sans-serif;font-size:13px"><br>
</div>
<div
style="font-family:arial,sans-serif;font-size:13px"><br>
</div>
<div
style="font-family:arial,sans-serif;font-size:13px">For
information,
here are the
memory values at
this moment;
opensips has
handled 5000
calls since his
bootup :</div>
<div
style="font-family:arial,sans-serif;font-size:13px">
<blockquote
class="gmail_quote"
style="margin:0px
0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">root@asbc2:/home/kemathy#
free -m<br>
total
used
free
shared
buffers
cached<br>
Mem:
16082
12887
3195
0 339
11393<br>
-/+
buffers/cache:
1154
14928<br>
Swap:
32755
0 32755</blockquote>
<div><br>
</div>
<div> </div>
</div>
<div
style="font-family:arial,sans-serif;font-size:13px">For
now, OpenSIPS is
launched with
S_MEMORY = 64
and P_MEMORY = 4</div>
<div
style="font-family:arial,sans-serif;font-size:13px">My
first idea would
be to increase
the P_MEMORY
value to 8 for
example, as it
seems to be a
lack of PKG
Memory, but in
case it's a
memory-freeing
problem, it'll
not solve our
issue (I think).</div>
<div
style="font-family:arial,sans-serif;font-size:13px"><br>
</div>
<div
style="font-family:arial,sans-serif;font-size:13px">My
questions are :
How can we
calculate the
appropriate
value for
S_MEMORY and
P_MEMORY ? And
how can we solve
our "out of pkg
memory" problem
? </div>
<div
style="font-family:arial,sans-serif;font-size:13px"><br>
</div>
<div
style="font-family:arial,sans-serif;font-size:13px">If
you need further
informations, or
anything else,
feel free to ask
me !</div>
<div
style="font-family:arial,sans-serif;font-size:13px">
<br>
</div>
<div
style="font-family:arial,sans-serif;font-size:13px">Thanks
a lot for your
help, </div>
</div>
<div><br clear="all">
<div><b>
<div><span
style="font-weight:normal">Bien
cordialement, </span></div>
<div><span
style="font-weight:normal">Best
Regards, </span></div>
<div><span
style="font-weight:normal"><br>
</span></div>
</b><b>Kevin
MATHY</b> |<b> </b>Ingénieur
VoIP<br>
<div>
<div><b><br>
</b></div>
</div>
</div>
</div>
</div>
<br>
</div>
</div>
<img
moz-do-not-send="true"
src="http://www.hexanet.fr/sites/files/hexanet/files/utilities/signature_hexanet.gif"><br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
Users mailing list
<a moz-do-not-send="true" href="mailto:Users@lists.opensips.org" target="_blank">Users@lists.opensips.org</a>
<a moz-do-not-send="true" href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users" target="_blank">http://lists.opensips.org/cgi-bin/mailman/listinfo/users</a>
</pre>
</blockquote>
<br>
</div>
</blockquote>
</div>
<br>
</div>
</div>
<br>
</div>
</div>
<img moz-do-not-send="true"
src="http://www.hexanet.fr/sites/files/hexanet/files/utilities/signature_hexanet.gif"></blockquote>
<br>
</div>
</blockquote>
</div>
<br>
</div>
<br>
</div>
</div>
<img moz-do-not-send="true"
src="http://www.hexanet.fr/sites/files/hexanet/files/utilities/signature_hexanet.gif"></blockquote>
<br>
</div>
</blockquote>
</div>
<br>
</div>
<br>
<img moz-do-not-send="true"
src="http://www.hexanet.fr/sites/files/hexanet/files/utilities/signature_hexanet.gif"></blockquote>
<br>
</body>
</html>