<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Verdana;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal">Callum,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">It’s my understanding that OpenSIPS does not release memory back to the OS, but it also pre-allocates all memory at startup into its private pool and then allocates from that internally. Normally shared memory should be significantly higher
than package memory. For reference, on our system we run with “-m 1024 -M 64” and that is sufficient for us to process very high traffic volume. We don’t do registration though, so that may affect the sizes you need.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">You are setting your package memory size to 4G, so that will allocate 4G memory for every package (process) that loads and then 2G for shared memory. That will use up all the memory on your machine extremely quickly for sure. The statistics
you provided seem like the memory increase is consistent with higher traffic levels on the second reading. You can see in your case that all of your “pkmem” processes have an extremely high amount of free memory (~3GB!). But that memory is still allocated
from the OS, so you are instructing OpenSIPS to allocate much more than your system memory right at startup.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Your shared memory also has just under 2GB free, so you have a lot of headroom there too. Since OpenSIPS pre-allocates, the amount of memory being used by the system overall should be fairly steady; if it is continuously increasing that
implies a leak somewhere. IIRC there are a few processes/modules/commands in OpenSIPS or libraries it uses that do allocate memory directly from the system and not from OpenSIPS’ pool. You may need to investigate some of those to find out where your memory
is going, or look at other processes/daemons you have running that could be using that memory.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="color:black">Ben Newlin </span><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">Users <users-bounces@lists.opensips.org> on behalf of Callum Guy <callum.guy@x-on.co.uk><br>
<b>Reply-To: </b>OpenSIPS users mailling list <users@lists.opensips.org><br>
<b>Date: </b>Friday, November 29, 2019 at 10:57 AM<br>
<b>To: </b>OpenSIPS users mailling list <users@lists.opensips.org><br>
<b>Subject: </b>[OpenSIPS-Users] Memory Leak - runtime flags?<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Hi All,<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I have recently deployed a new registrar and have been seeing a gradual increase in the memory footprint - enough that I'm having to expand the RAM (its virtualised) to ensure it doesn't run out.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">You can see a diff of the statistics collected last night at 11pm and today at 3pm here: <a href="https://gist.github.com/spacetourist/2103503674e134bd598c7f1e3a82674c/revisions" target="_blank">https://gist.github.com/spacetourist/2103503674e134bd598c7f1e3a82674c/revisions</a><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Processes 5-9 are my UDP SIP receiver threads (autoscaled down from an initial footprint of 20 threads).<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Using 3.0.1 on CentOS 7 8GB RAM (soon to be 32GB!). Currently OpenSIPs is using all the RAM (minus OS usage) and 2GB of swap. Trying to use dialog and dr clustering if that is significant. Alos have NAT pings configured for all registrations
(4000 at time of writing).<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<div>
<p class="MsoNormal">I am using runtime configuration flags of "<b>-m 2048 -M 4096</b>" and am concerned that these were (way) too high, I think I've misinterpreted their meaning during initial setup. Is this a ridiculous setting for my environment? Is it just
as simple as OpenSIPs being greedy with the memory such that it doesn't bother to free anything while each process free space remaining? Should my -M value * max number of processes fit into my RAM? I guess with an 8GB system that would mean dropping this
to "-M 256"?<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I've done some research into the issue however I haven't found anything else that would be an obvious target so wondered if the community might have some ideas of where I can begin investigations. <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Many thanks,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Callum<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"><span style="border:solid windowtext 1.0pt;padding:0in"><img border="0" width="32" height="32" style="width:.3333in;height:.3333in" id="_x0000_i1028" src="cid:~WRD0002.jpg" alt="Image removed by sender."></span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><b><sup><span style="font-size:13.5pt;font-family:"Verdana",sans-serif">0333 332 0000 |
<a href="http://www.x-on.co.uk" target="_blank">
www.x-on.co.uk</a> | </span></sup></b><b><sub><span style="font-size:13.5pt;font-family:"Verdana",sans-serif"> </span></sub></b><b><sup><span style="font-size:13.5pt;font-family:"Verdana",sans-serif"><a href="https://www.linkedin.com/company/x-on" target="_blank"><span style="color:windowtext;text-decoration:none"><span style="color:blue;border:solid windowtext 1.0pt;padding:0in"><img border="0" width="24" height="24" style="width:.25in;height:.25in" id="_x0000_i1027" src="cid:~WRD0002.jpg" alt="Image removed by sender."></span></span></a>
<a href="https://www.facebook.com/XonTel" target="_blank"><span style="color:windowtext;text-decoration:none"><span style="color:blue;border:solid windowtext 1.0pt;padding:0in"><img border="0" width="24" height="24" style="width:.25in;height:.25in" id="_x0000_i1026" src="cid:~WRD0002.jpg" alt="Image removed by sender."></span></span></a>
<a href="https://twitter.com/xonuk" target="_blank"><span style="color:windowtext;text-decoration:none"><span style="color:blue;border:solid windowtext 1.0pt;padding:0in"><img border="0" width="24" height="24" style="width:.25in;height:.25in" id="_x0000_i1025" src="cid:~WRD0002.jpg" alt="Image removed by sender."></span></span></a></span></sup></b><b><span style="font-size:13.5pt"> </span></b><o:p></o:p></p>
<p><span style="font-size:6.0pt;font-family:"Verdana",sans-serif;color:black">X-on is a trading name of Storacall Technology Ltd a limited company registered in England and Wales.<br>
Registered Office : Avaland House, 110 London Road, Apsley, Hemel Hempstead, Herts, HP3 9SD. Company Registration No. 2578478.<br>
The information in this e-mail is confidential and for use by the addressee(s) only. If you are not the intended recipient, please notify X-on immediately on +44(0)333 332 0000 and delete the<br>
message from your computer. If you are not a named addressee you must not use, disclose, disseminate, distribute, copy, print or reply to this email. Views or opinions expressed by an individual<br>
within this email may not necessarily reflect the views of X-on or its associated companies. Although X-on routinely screens for viruses, addressees should scan this email and any attachments<br>
for viruses. X-on makes no representation or warranty as to the absence of viruses in this email or any attachments.</span><o:p></o:p></p>
</div>
</div>
</div>
</div>
</body>
</html>