<div dir="ltr">The same behavior has appeared on the same system, this time causing it to stop functioning. Logs show:<div><br></div><div><div>opensips[21368]: ERROR:core:parse_from_header: out of pkg_memory</div><div>opensips[21368]: ERROR:uac:restore_uris_reply: failed to find/parse FROM hdr</div>
<div>opensips[21368]: WARNING:core:fm_malloc: Not enough free memory, will atempt defragmenation</div><div>opensips[21368]: ERROR:core:parse_from_header: out of pkg_memory</div><div>opensips[21368]: ERROR:uac:restore_uris_reply: failed to find/parse FROM hdr</div>
<div>opensips[21368]: WARNING:core:fm_malloc: Not enough free memory, will atempt defragmenation</div><div>opensips[21368]: ERROR:core:build_res_buf_from_sip_res: out of pkg mem</div><div>opensips[21368]: ERROR:tm:relay_reply: no mem for outbound reply buffer</div>
<div>opensips[21368]: WARNING:core:fm_malloc: Not enough free memory, will atempt defragmenation</div><div>opensips[21368]: ERROR:core:build_res_buf_from_sip_req: out of pkg memory ; needs 410</div><div>opensips[21368]: WARNING:core:fm_malloc: Not enough free memory, will atempt defragmenation</div>
<div>opensips[21368]: ERROR:uac_auth:build_authorization_hdr: no more pkg mem</div><div>opensips[21368]: ERROR:uac_registrant:reg_tm_cback: failed to build authorization hdr</div><div>opensips[21369]: WARNING:core:fm_malloc: Not enough free memory, will atempt defragmenation</div>
<div>opensips[21369]: ERROR:uac_auth:build_authorization_hdr: no more pkg mem</div><div>opensips[21369]: ERROR:uac_registrant:reg_tm_cback: failed to build authorization hdr</div><div>...</div><div><br></div><div>The load is low, measured in "seconds between calls" rather than "calls per second". Total number of dialogs is less than 20. There are two children processes defined. The script is very simple; its main role is to relay INVITEs from one interface to another while handling the media with rtpproxy. Shared memory is 16M and pkg memory is 1M.</div>
<div class="gmail_extra"><br clear="all"><div><div dir="ltr"><div>I'll try to follow the <a href="http://www.opensips.org/Documentation/TroubleShooting-OutOfMem">memory troubleshooting steps</a> to see if anything interesting surfaces. Any other recommendations are always appreciated.</div>
<div><br></div></div></div>
<div class="gmail_extra"><br></div>Regards,</div><div class="gmail_extra">Jeff</div><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Sep 17, 2013 at 11:07 AM, Jeff Pyle <span dir="ltr"><<a href="mailto:jpyle@fidelityvoice.com" target="_blank">jpyle@fidelityvoice.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr">Update: On an OpenSIPS instance that has been running for a while (at least days), the first iteration of get_statistics will show the zero-values. Running it again, the zeros have been replaced by values that make sense.<div>
<br></div><div>Could the act of running get_statistics cause memory behavior to change? That's how it seems. Unless the reporting is wrong somehow.</div><span class=""><font color="#888888"><div><br></div><div><br>
</div><div>- Jeff</div></font></span><div><div class="h5"><div> <br><div class="gmail_extra">
<br><br><div class="gmail_quote">On Tue, Sep 17, 2013 at 11:01 AM, Jeff Pyle <span dir="ltr"><<a href="mailto:jpyle@fidelityvoice.com" target="_blank">jpyle@fidelityvoice.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div><div dir="ltr"><div>Hello,</div><div><br></div><div>We're trying to track down a memory problem on OpenSIPS 1.9.1 compiled from github (so no rev number) on Aug 13. I noticed something weird I wanted to present for opinions.</div>
<div><br></div></div></div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div><div><div><div># opensipsctl fifo get_statistics all | grep ^pkmem"</div></div></div></div><div><div><div><div>pkmem:0-total_size = 1048576</div>
</div></div></div><div><div><div><div>pkmem:0-used_size = 107128</div></div></div></div><div><div><div><div>pkmem:0-real_used_size = 163960</div></div></div></div><div><div><div><div>pkmem:0-max_used_size = 165960</div></div>
</div></div><div><div><div><div>pkmem:0-free_size = 884616</div></div></div></div><div><div><div><div>pkmem:0-fragments = 61</div></div></div></div><div><div><div><div>pkmem:1-total_size = 1048576</div></div></div></div>
<div>
<div><div><div>pkmem:1-used_size = 133960</div></div></div></div><div><div><div><div>pkmem:1-real_used_size = 192496</div></div></div></div><div><div><div><div>pkmem:1-max_used_size = 192496</div></div></div></div><div><div>
<div><div>pkmem:1-free_size = 856080</div></div></div></div><div><div><div><div>pkmem:1-fragments = 58</div></div></div></div><div><div><div><div>pkmem:2-total_size = 1048576</div></div></div></div><div><div><div><div>pkmem:2-used_size = 107112</div>
</div></div></div><div><div><div><div>pkmem:2-real_used_size = 163944</div></div></div></div><div><div><div><div>pkmem:2-max_used_size = 165960</div></div></div></div><div><div><div><div>pkmem:2-free_size = 884632</div></div>
</div></div><div><div><div><div>pkmem:2-fragments = 61</div></div></div></div><div><div><div><div><b>pkmem:3-total_size = 0</b></div></div></div></div><div><div><div><div><b>pkmem:3-used_size = 0</b></div></div></div></div>
<div><div><div><div><b>pkmem:3-real_used_size = 0</b></div></div></div></div><div><div><div><div><b>pkmem:3-max_used_size = 0</b></div></div></div></div><div><div><div><div><b>pkmem:3-free_size = 859336</b></div></div></div>
</div><div><div><div><div><b>pkmem:3-fragments = 0</b></div></div></div></div><div><div><div><div>pkmem:4-total_size = 1048576</div></div></div></div><div><div><div><div>pkmem:4-used_size = 129384</div></div></div></div>
<div>
<div><div><div>pkmem:4-real_used_size = 188544</div></div></div></div><div><div><div><div>pkmem:4-max_used_size = 197616</div></div></div></div><div><div><div><div>pkmem:4-free_size = 860032</div></div></div></div><div><div>
<div><div>pkmem:4-fragments = 112</div></div></div></div><div><div><div><div>pkmem:5-total_size = 1048576</div></div></div></div><div><div><div><div>pkmem:5-used_size = 115624</div></div></div></div><div><div><div><div>pkmem:5-real_used_size = 173656</div>
</div></div></div><div><div><div><div>pkmem:5-max_used_size = 181816</div></div></div></div><div><div><div><div>pkmem:5-free_size = 874920</div></div></div></div><div><div><div><div>pkmem:5-fragments = 102</div></div></div>
</div><div><div><div><div>pkmem:6-total_size = 1048576</div></div></div></div><div><div><div><div>pkmem:6-used_size = 115632</div></div></div></div><div><div><div><div>pkmem:6-real_used_size = 173760</div></div></div></div>
<div><div><div><div>pkmem:6-max_used_size = 181960</div></div></div></div><div><div><div><div>pkmem:6-free_size = 874816</div></div></div></div><div><div><div><div>pkmem:6-fragments = 106</div></div></div></div><div><div>
<div><div>pkmem:7-total_size = 1048576</div></div></div></div><div><div><div><div>pkmem:7-used_size = 115280</div></div></div></div><div><div><div><div>pkmem:7-real_used_size = 172112</div></div></div></div><div><div><div>
<div>pkmem:7-max_used_size = 172112</div></div></div></div><div><div><div><div>pkmem:7-free_size = 876464</div></div></div></div><div><div><div><div>pkmem:7-fragments = 59</div></div></div></div><div><div><div><div>pkmem:8-total_size = 1048576</div>
</div></div></div><div><div><div><div>pkmem:8-used_size = 115280</div></div></div></div><div><div><div><div>pkmem:8-real_used_size = 172112</div></div></div></div><div><div><div><div>pkmem:8-max_used_size = 172112</div></div>
</div></div><div><div><div><div>pkmem:8-free_size = 876464</div></div></div></div><div><div><div><div>pkmem:8-fragments = 59</div></div></div></div><div><div><div><div>pkmem:9-total_size = 1048576</div></div></div></div>
<div>
<div><div><div>pkmem:9-used_size = 106808</div></div></div></div><div><div><div><div>pkmem:9-real_used_size = 163544</div></div></div></div><div><div><div><div>pkmem:9-max_used_size = 165960</div></div></div></div><div><div>
<div><div>pkmem:9-free_size = 885032</div></div></div></div><div><div><div><div>pkmem:9-fragments = 61</div></div></div></div><div><div><div><div>pkmem:10-total_size = 1048576</div></div></div></div><div><div><div><div>pkmem:10-used_size = 115248</div>
</div></div></div><div><div><div><div>pkmem:10-real_used_size = 172128</div></div></div></div><div><div><div><div>pkmem:10-max_used_size = 173040</div></div></div></div><div><div><div><div>pkmem:10-free_size = 876448</div>
</div></div></div><div><div><div><div>pkmem:10-fragments = 62</div></div></div></div></blockquote><div><div dir="ltr"><div><br></div><div>Looking at pkmem:3, is it normal to have the 0s in total_size, used_size, etc? I restarted opensips and now they all have a total_size.</div>
<span><font color="#888888">
<div><br></div><div><br></div><div>- Jeff</div><div><br></div></font></span></div></div>
</div>
</blockquote></div><br></div></div></div></div></div>
</blockquote></div><br></div></div></div>