<html>
  <head>
    <meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    <tt>Hi, Mickael!<br>
      <br>
      This is not entirely true - you can define AVPs with the integer
      value 0. Those will have avp-&gt;flags == 0 and avp-&gt;data == 0.<br>
      What I'd do, is to note down the avp-&gt;id value of those AVPs
      and then try to see their names. To do that, you'd have to look
      into the avp_map and avp_map_shm maps to see the corresponding
      name for that id. Alternatively you can call in your script the
      avp_print() method, which prints all the AVPs for a specific
      transaction along with their id and names. Let me know how this
      goes.<br>
      <br>
      Best regards,<br>
    </tt>
    <pre class="moz-signature" cols="72">Răzvan Crainea
OpenSIPS Solutions
<a class="moz-txt-link-abbreviated" href="http://www.opensips-solutions.com">www.opensips-solutions.com</a></pre>
    <div class="moz-cite-prefix">On 06/18/2015 12:48 PM, Mickael
      Marrache wrote:<br>
    </div>
    <blockquote
cite="mid:CA+=4G22dg5_cH=11hb9nqR11my_Cc2fVtRka0S_VVSxDsjNPjg@mail.gmail.com"
      type="cite">
      <div dir="ltr">To add more information, I remember there was no
        way to define an integer AVP with value 0. I see a lot of such
        AVPs.</div>
      <div class="gmail_extra"><br>
        <div class="gmail_quote">On Thu, Jun 18, 2015 at 12:03 PM,
          Mickael Marrache <span dir="ltr">&lt;<a
              moz-do-not-send="true"
              href="mailto:mickaelmarrache@gmail.com" target="_blank"><a class="moz-txt-link-abbreviated" href="mailto:mickaelmarrache@gmail.com">mickaelmarrache@gmail.com</a></a>&gt;</span>
          wrote:<br>
          <blockquote class="gmail_quote" style="margin:0 0 0
            .8ex;border-left:1px #ccc solid;padding-left:1ex">
            <div dir="ltr">Correction of my previous email.
              <div><br>
              </div>
              <div>When I said I found AVPs without data, I may be
                wrong. avp-&gt;flags == 0 probably means the AVP data is
                an integer. So, that explains the weird values (e.g.
                0x8000) I tried to interpret as memory addresses.</div>
              <span class="HOEnZb"><font color="#888888">
                  <div><br>
                  </div>
                  <div>Mickael</div>
                </font></span></div>
            <div class="HOEnZb">
              <div class="h5">
                <div class="gmail_extra"><br>
                  <div class="gmail_quote">On Thu, Jun 18, 2015 at 11:12
                    AM, Mickael Marrache <span dir="ltr">&lt;<a
                        moz-do-not-send="true"
                        href="mailto:mickaelmarrache@gmail.com"
                        target="_blank"><a class="moz-txt-link-abbreviated" href="mailto:mickaelmarrache@gmail.com">mickaelmarrache@gmail.com</a></a>&gt;</span>
                    wrote:<br>
                    <blockquote class="gmail_quote" style="margin:0 0 0
                      .8ex;border-left:1px #ccc solid;padding-left:1ex">
                      <div dir="ltr">Hi Razvan,
                        <div><br>
                        </div>
                        <div>Here is what I've done.</div>
                        <div><br>
                        </div>
                        <div>I took a core dump of the attendant
                          process. Then, I stopped OpenSIPS so that it
                          frees allocated fragments, and at the end
                          lists all fragments that are still allocated.</div>
                        <div><br>
                        </div>
                        <div>In this list of fragments, I can see a lot
                          of AVPs.</div>
                        <div><br>
                        </div>
                        <div>I see some AVPs without data (avp-&gt;data
                          == NULL, avp-&gt;flags == 0). But something is
                          weird, it looks like that all AVPs that don't
                          have data have the same id. It looks like
                          duplicate AVPs (in different memory
                          fragments).</div>
                        <div><br>
                        </div>
                        <div>Some AVPs do have data and have a format
                          that looks valid.</div>
                        <div><br>
                        </div>
                        <div>Some AVPs looks corrupted. For example, I
                          found an AVP with same ID as the AVPs without
                          data, but avp-&gt;data == 0x8000 which looks
                          completely wrong.</div>
                        <div><br>
                        </div>
                        <div>Thanks,</div>
                        <div>Mickael</div>
                      </div>
                      <div>
                        <div>
                          <div class="gmail_extra"><br>
                            <div class="gmail_quote">On Thu, Jun 18,
                              2015 at 10:11 AM, Mickael Marrache <span
                                dir="ltr">&lt;<a moz-do-not-send="true"
href="mailto:mickaelmarrache@gmail.com" target="_blank">mickaelmarrache@gmail.com</a>&gt;</span>
                              wrote:<br>
                              <blockquote class="gmail_quote"
                                style="margin:0 0 0 .8ex;border-left:1px
                                #ccc solid;padding-left:1ex">
                                <div dir="ltr">Hi Razvan,
                                  <div><br>
                                  </div>
                                  <div>I created a core dump for the
                                    attendant process. Is it enough or
                                    we also need core dumps for other
                                    processes? Note that the leak
                                    appears in shared memory.</div>
                                  <div><br>
                                  </div>
                                  <div>We do use QM debug for this
                                    version, this is how I discovered
                                    the remaining AVPs at shutdown where
                                    the remaining allocated memory
                                    fragments are listed.</div>
                                  <div><br>
                                  </div>
                                  <div>Do you know where I should look
                                    for the AVPs in the core dump?</div>
                                  <div><br>
                                  </div>
                                  <div>Thanks,</div>
                                  <div>Mickael </div>
                                </div>
                                <div>
                                  <div>
                                    <div class="gmail_extra"><br>
                                      <div class="gmail_quote">On Tue,
                                        Jun 16, 2015 at 5:11 PM, Răzvan
                                        Crainea <span dir="ltr">&lt;<a
                                            moz-do-not-send="true"
                                            href="mailto:razvan@opensips.org"
                                            target="_blank"><a class="moz-txt-link-abbreviated" href="mailto:razvan@opensips.org">razvan@opensips.org</a></a>&gt;</span>
                                        wrote:<br>
                                        <blockquote class="gmail_quote"
                                          style="margin:0 0 0
                                          .8ex;border-left:1px #ccc
                                          solid;padding-left:1ex">
                                          <div bgcolor="#FFFFFF"
                                            text="#000000"> <tt>Hi,
                                              Mickael!<br>
                                              <br>
                                              I don't know what exactly
                                              might cause the leak. What
                                              you can do is to try to
                                              get a core dump (using
                                              tools like gcore) during
                                              low (or unexisting)
                                              traffic and try to see
                                              what do the AVPs that are
                                              leaking contain. Are you
                                              using QM debug?<br>
                                              <br>
                                              Best regards,<br>
                                            </tt>
                                            <pre cols="72">Răzvan Crainea
OpenSIPS Solutions
<a moz-do-not-send="true" href="http://www.opensips-solutions.com" target="_blank">www.opensips-solutions.com</a></pre>
                                            <div>
                                              <div>
                                                <div>On 05/27/2015 12:37
                                                  PM, Mickael Marrache
                                                  wrote:<br>
                                                </div>
                                                <blockquote type="cite">
                                                  <div dir="ltr">Any
                                                    idea? Is there
                                                    something that may
                                                    help finding the
                                                    leak cause?</div>
                                                  <div
                                                    class="gmail_extra"><br>
                                                    <div
                                                      class="gmail_quote">On
                                                      Tue, May 19, 2015
                                                      at 1:17 PM,
                                                      Mickael Marrache <span
                                                        dir="ltr">&lt;<a
moz-do-not-send="true" href="mailto:mickaelmarrache@gmail.com"
                                                          target="_blank"><a class="moz-txt-link-abbreviated" href="mailto:mickaelmarrache@gmail.com">mickaelmarrache@gmail.com</a></a>&gt;</span>
                                                      wrote:<br>
                                                      <blockquote
                                                        class="gmail_quote"
                                                        style="margin:0
                                                        0 0
                                                        .8ex;border-left:1px
                                                        #ccc
                                                        solid;padding-left:1ex">
                                                        <div dir="ltr">Any
                                                          idea what we
                                                          should do?
                                                          <div><br>
                                                          </div>
                                                          <div>I may be
                                                          doing
                                                          something
                                                          wrong but I
                                                          don't remember
                                                          I had to take
                                                          care of memory
                                                          management
                                                          when dealing
                                                          with AVPs.</div>
                                                          <div><br>
                                                          </div>
                                                          <div>Am I
                                                          right?</div>
                                                        </div>
                                                        <div>
                                                          <div>
                                                          <div
                                                          class="gmail_extra"><br>
                                                          <div
                                                          class="gmail_quote">On
                                                          Mon, May 18,
                                                          2015 at 12:08
                                                          PM, Mickael
                                                          Marrache <span
                                                          dir="ltr">&lt;<a
moz-do-not-send="true" href="mailto:mickaelmarrache@gmail.com"
                                                          target="_blank"><a class="moz-txt-link-abbreviated" href="mailto:mickaelmarrache@gmail.com">mickaelmarrache@gmail.com</a></a>&gt;</span>
                                                          wrote:<br>
                                                          <blockquote
                                                          class="gmail_quote"
                                                          style="margin:0
                                                          0 0
                                                          .8ex;border-left:1px
                                                          #ccc
                                                          solid;padding-left:1ex">
                                                          <div dir="ltr">Hi,

                                                          <div><br>
                                                          </div>
                                                          <div>We use
                                                          customized
                                                          sources based
                                                          on 1.11.2.</div>
                                                          <div><br>
                                                          </div>
                                                          <div>However,
                                                          we never
                                                          modified the
                                                          sources
                                                          related to
                                                          AVP.</div>
                                                          <span><font
                                                          color="#888888">
                                                          <div><br>
                                                          </div>
                                                          <div>Mickael</div>
                                                          </font></span></div>
                                                          <div>
                                                          <div>
                                                          <div
                                                          class="gmail_extra"><br>
                                                          <div
                                                          class="gmail_quote">On
                                                          Mon, May 18,
                                                          2015 at 12:03
                                                          PM,
                                                          Bogdan-Andrei
                                                          Iancu <span
                                                          dir="ltr">&lt;<a
moz-do-not-send="true" href="mailto:bogdan@opensips.org" target="_blank"><a class="moz-txt-link-abbreviated" href="mailto:bogdan@opensips.org">bogdan@opensips.org</a></a>&gt;</span>
                                                          wrote:<br>
                                                          <blockquote
                                                          class="gmail_quote"
                                                          style="margin:0
                                                          0 0
                                                          .8ex;border-left:1px
                                                          #ccc
                                                          solid;padding-left:1ex">
                                                          <div
                                                          text="#000000"
bgcolor="#FFFFFF"> <tt>Hi Mickael,<br>
                                                          <br>
                                                          If you do not
                                                          have any SIP
                                                          message or
                                                          transaction
                                                          under
                                                          processing,
                                                          and still have
                                                          AVPs in
                                                          memory....well,
                                                          it looks like
                                                          a leak to me.<br>
                                                          <br>
                                                          What OpenSIPS
                                                          version do you
                                                          have ?<br>
                                                          <br>
                                                          Regards,<br>
                                                          </tt><span>
                                                          <pre cols="72">Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
<a moz-do-not-send="true" href="http://www.opensips-solutions.com" target="_blank">http://www.opensips-solutions.com</a></pre>
                                                          </span>
                                                          <div>
                                                          <div>
                                                          <div>On
                                                          17.05.2015
                                                          11:55, Mickael
                                                          Marrache
                                                          wrote:<br>
                                                          </div>
                                                          <blockquote
                                                          type="cite">
                                                          <div dir="ltr">Hi,

                                                          <div><br>
                                                          </div>
                                                          <div>Here is
                                                          what I've
                                                          done:</div>
                                                          <div><br>
                                                          </div>
                                                          <div>1)
                                                          Ensured
                                                          OpenSIPS
                                                          doesn't
                                                          receive any
                                                          calls.</div>
                                                          <div>2) Waited
                                                          for all calls
                                                          to terminate.</div>
                                                          <div>3)
                                                          Checked that
                                                          the <span
                                                          style="font-family:monospace;font-size:10.4000005722046px">inuse</span><tt
style="font-size:10.4000005722046px">_tra<tt>nsaction</tt></tt> is 0.</div>
                                                          <div>4)
                                                          Stopped
                                                          OpenSIPS.</div>
                                                          <div><br>
                                                          </div>
                                                          <div>I confirm
                                                          there are
                                                          still a lot of
                                                          AVPs in
                                                          memory.</div>
                                                          <div><br>
                                                          </div>
                                                          <div>Who is
                                                          supposed to
                                                          free these
                                                          AVPs? Is it a
                                                          memory leak?</div>
                                                          <div><br>
                                                          </div>
                                                          <div>Thanks,</div>
                                                          <div>Mickael</div>
                                                          </div>
                                                          <div
                                                          class="gmail_extra"><br>
                                                          <div
                                                          class="gmail_quote">On

                                                          Fri, May 15,
                                                          2015 at 12:48
                                                          PM,
                                                          Bogdan-Andrei
                                                          Iancu <span
                                                          dir="ltr">&lt;<a
moz-do-not-send="true" href="mailto:bogdan@opensips.org" target="_blank"><a class="moz-txt-link-abbreviated" href="mailto:bogdan@opensips.org">bogdan@opensips.org</a></a>&gt;</span>
                                                          wrote:<br>
                                                          <blockquote
                                                          class="gmail_quote"
                                                          style="margin:0
                                                          0 0
                                                          .8ex;border-left:1px
                                                          #ccc
                                                          solid;padding-left:1ex">
                                                          <div
                                                          text="#000000"
bgcolor="#FFFFFF"> <tt>Hi Michael,<br>
                                                          <br>
                                                          <tt>You can do
                                                          "opensipsctl
                                                          fifo get<tt>_statistics

                                                          tm:" and you
                                                          have the
                                                          "inuse<tt>_tra<tt>nsaction"


                                                          statistic :<br>
                                                          <tt>    <a
                                                          moz-do-not-send="true"
href="http://www.opensips.org/html/docs/modules/1.11.x/tm.html#id296307"
target="_blank"><a class="moz-txt-link-freetext" href="http://www.opensips.org/html/docs/modules/1.11.x/tm.html#id296307">http://www.opensips.org/html/docs/modules/1.11.x/tm.html#id296307</a></a><br>
                                                          <br>
                                                          <tt>Regards,</tt><br>
                                                          <br>
                                                          </tt></tt></tt></tt></tt></tt><span>
                                                          <pre cols="72">Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
<a moz-do-not-send="true" href="http://www.opensips-solutions.com" target="_blank">http://www.opensips-solutions.com</a></pre>
                                                          </span>
                                                          <div>
                                                          <div>
                                                          <div>On
                                                          14.05.2015
                                                          09:45, Mickael
                                                          Marrache
                                                          wrote:<br>
                                                          </div>
                                                          <blockquote
                                                          type="cite">
                                                          <div>
                                                          <p
                                                          class="MsoNormal"><span
style="color:#1f497d" lang="EN-US">I'm not sure there were no
                                                          transactions
                                                          in memory at
                                                          shutdown.</span></p>
                                                          <p
                                                          class="MsoNormal"><span
style="color:#1f497d" lang="EN-US"> </span></p>
                                                          <p
                                                          class="MsoNormal"><span
style="color:#1f497d" lang="EN-US">How can I check this? I didn't find
                                                          an MI command
                                                          for that.</span></p>
                                                          <p
                                                          class="MsoNormal"><span
style="color:#1f497d" lang="EN-US"> </span></p>
                                                          <div>
                                                          <div
                                                          style="border:none;border-top:solid
                                                          #b5c4df
                                                          1.0pt;padding:3.0pt
                                                          0cm 0cm 0cm">
                                                          <p
                                                          class="MsoNormal"><b><span
style="font-size:10.0pt;font-family:&quot;Tahoma&quot;,&quot;sans-serif&quot;;color:windowtext">From:</span></b><span
style="font-size:10.0pt;font-family:&quot;Tahoma&quot;,&quot;sans-serif&quot;;color:windowtext">
                                                          Bogdan-Andrei
                                                          Iancu [<a
                                                          moz-do-not-send="true"
href="mailto:bogdan@opensips.org" target="_blank"><a class="moz-txt-link-freetext" href="mailto:bogdan@opensips.org">mailto:bogdan@opensips.org</a></a>]
                                                          <br>
                                                          <b>Sent:</b>
                                                          Wednesday, May
                                                          13, 2015 6:44
                                                          PM<br>
                                                          <b>To:</b>
                                                          OpenSIPS users
                                                          mailling list;
                                                          <a
                                                          moz-do-not-send="true"
href="mailto:mickaelmarrache@gmail.com" target="_blank"><a class="moz-txt-link-abbreviated" href="mailto:mickaelmarrache@gmail.com">mickaelmarrache@gmail.com</a></a><br>
                                                          <b>Subject:</b>
                                                          Re:
                                                          [OpenSIPS-Users]
                                                          Possible
                                                          memory leak
                                                          related to
                                                          AVPs</span></p>
                                                          </div>
                                                          </div>
                                                          <p
                                                          class="MsoNormal"> </p>
                                                          <p
                                                          class="MsoNormal"><tt><span
style="font-size:10.0pt">Hi Mickael,</span></tt><span><br>
                                                          <br>
                                                          <tt>What means
                                                          "a lot"  :) ?
                                                          I'm asking as
                                                          if you have
                                                          100
                                                          transactions
                                                          in memory
                                                          (when doing
                                                          shutdown) and
                                                          you use ~10
                                                          AVPs per
                                                          INVITE in
                                                          script -&gt;
                                                          you have 1000
                                                          AVPs in mem.</tt><br>
                                                          <br>
                                                          <tt>When doing
                                                          the shutdown,
                                                          are any
                                                          transaction in
                                                          memory ? (with
                                                          no active
                                                          transactions
                                                          in mem, there
                                                          are no AVPs in
                                                          mem).</tt><br>
                                                          <br>
                                                          <tt>Regards,</tt><br>
                                                          <br>
                                                          </span></p>
                                                          <pre>Bogdan-Andrei Iancu</pre>
                                                          <pre>OpenSIPS Founder and Developer</pre>
                                                          <pre><a moz-do-not-send="true" href="http://www.opensips-solutions.com" target="_blank">http://www.opensips-solutions.com</a></pre>
                                                          <div>
                                                          <p
                                                          class="MsoNormal">On

                                                          13.05.2015
                                                          17:53, Mickael
                                                          Marrache
                                                          wrote:</p>
                                                          </div>
                                                          <blockquote
                                                          style="margin-top:5.0pt;margin-bottom:5.0pt">
                                                          <p
                                                          class="MsoNormal">Hi,</p>
                                                          <p
                                                          class="MsoNormal"> </p>
                                                          <p
                                                          class="MsoNormal"><span
                                                          lang="EN-US">We

                                                          have a memory
                                                          leak and are
                                                          trying to find
                                                          the cause.</span></p>
                                                          <p
                                                          class="MsoNormal"><span
                                                          lang="EN-US"> </span></p>
                                                          <p
                                                          class="MsoNormal"><span
                                                          lang="EN-US">We


                                                          compiled the
                                                          call with
                                                          memory
                                                          debugging and
                                                          at shutdown,
                                                          we see a lot
                                                          of allocated
                                                          AVPs that are
                                                          not released.</span></p>
                                                          <p
                                                          class="MsoNormal"><span
                                                          lang="EN-US"> </span></p>
                                                          <p
                                                          class="MsoNormal"><span
                                                          lang="EN-US">opensips[27560]:    



                                                          277067. N 
                                                          address=0x2aae957a0330
                                                          frag=0x2aae957a0300

                                                          size=24 used=1</span></p>
                                                          <p
                                                          class="MsoNormal"><span
                                                          lang="EN-US">opensips[27560]:            



                                                          alloc'd from
                                                          usr_avp.c:
                                                          new_avp(119)</span></p>
                                                          <p
                                                          class="MsoNormal"><span
                                                          lang="EN-US">opensips[27560]:        



                                                          start
                                                          check=f0f0f0f0f0f0f0f0,
                                                          end check=
                                                          c0c0c0c0c0c0c0c0,
abcdefedabcdefed</span></p>
                                                          <p
                                                          class="MsoNormal"><span
                                                          lang="EN-US"> </span></p>
                                                          <p
                                                          class="MsoNormal"><span
                                                          lang="EN-US">Is

                                                          it possible to
                                                          have a memory
                                                          leak related
                                                          to AVPs? IIRC
                                                          OpenSIPS
                                                          handles it
                                                          internally.</span></p>
                                                          <p
                                                          class="MsoNormal"><span
                                                          lang="EN-US"> </span></p>
                                                          <p
                                                          class="MsoNormal"><span
                                                          lang="EN-US">Thanks,</span></p>
                                                          <p
                                                          class="MsoNormal"><span
                                                          lang="EN-US">Mickael</span></p>
                                                          <p
                                                          class="MsoNormal"><span><br>
                                                          <br>
                                                          <br>
                                                          </span></p>
                                                          <pre>_______________________________________________</pre>
                                                          <pre>Users mailing list</pre>
                                                          <pre><a moz-do-not-send="true" href="mailto:Users@lists.opensips.org" target="_blank">Users@lists.opensips.org</a></pre>
                                                          <pre><a moz-do-not-send="true" href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users" target="_blank">http://lists.opensips.org/cgi-bin/mailman/listinfo/users</a></pre>
                                                          </blockquote>
                                                          <p
                                                          class="MsoNormal"><span> </span></p>
                                                          </div>
                                                          </blockquote>
                                                          <br>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          <br>
                                                          </div>
                                                          </blockquote>
                                                          <br>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          <br>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          <br>
                                                          </div>
                                                          </div>
                                                        </div>
                                                      </blockquote>
                                                    </div>
                                                    <br>
                                                  </div>
                                                  <br>
                                                  <fieldset></fieldset>
                                                  <br>
                                                  <pre>_______________________________________________
Users mailing list
<a moz-do-not-send="true" href="mailto:Users@lists.opensips.org" target="_blank">Users@lists.opensips.org</a>
<a moz-do-not-send="true" href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users" target="_blank">http://lists.opensips.org/cgi-bin/mailman/listinfo/users</a>
</pre>
                                                </blockquote>
                                                <br>
                                              </div>
                                            </div>
                                          </div>
                                          <br>
_______________________________________________<br>
                                          Users mailing list<br>
                                          <a moz-do-not-send="true"
                                            href="mailto:Users@lists.opensips.org"
                                            target="_blank">Users@lists.opensips.org</a><br>
                                          <a moz-do-not-send="true"
                                            href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users"
                                            rel="noreferrer"
                                            target="_blank">http://lists.opensips.org/cgi-bin/mailman/listinfo/users</a><br>
                                          <br>
                                        </blockquote>
                                      </div>
                                      <br>
                                    </div>
                                  </div>
                                </div>
                              </blockquote>
                            </div>
                            <br>
                          </div>
                        </div>
                      </div>
                    </blockquote>
                  </div>
                  <br>
                </div>
              </div>
            </div>
          </blockquote>
        </div>
        <br>
      </div>
      <br>
      <fieldset class="mimeAttachmentHeader"></fieldset>
      <br>
      <pre wrap="">_______________________________________________
Users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Users@lists.opensips.org">Users@lists.opensips.org</a>
<a class="moz-txt-link-freetext" href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users">http://lists.opensips.org/cgi-bin/mailman/listinfo/users</a>
</pre>
    </blockquote>
    <br>
  </body>
</html>