<html>
  <head>
    <meta content="text/html; charset=utf-8" http-equiv="Content-Type">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    <tt>Hi <tt>Be<tt>n,<br>
          <br>
          <tt>Thank you for update. Have you tried to compile the memo<tt>ry
              debugging <tt>support ? it might speed up the <tt>detection
                  of the <tt>error</tt></tt>.<br>
                <br>
                <tt>Regards,</tt><br>
              </tt></tt></tt></tt></tt></tt>
    <pre class="moz-signature" cols="72">Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
<a class="moz-txt-link-freetext" href="http://www.opensips-solutions.com">http://www.opensips-solutions.com</a></pre>
    <div class="moz-cite-prefix">On 05.08.2016 23:00, Newlin, Ben wrote:<br>
    </div>
    <blockquote cite="mid:9B9C770C-E693-419B-B472-389474282671@inin.com"
      type="cite">
      <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
      <meta name="Title" content="">
      <meta name="Keywords" content="">
      <meta name="Generator" content="Microsoft Word 15 (filtered
        medium)">
      <style><!--
/* Font Definitions */
@font-face
        {font-family:"Courier New";
        panose-1:2 7 3 9 2 2 5 2 4 4;}
@font-face
        {font-family:"Cambria Math";
        panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
        {font-family:Calibri;
        panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
        {font-family:"Courier New \, serif ";}
@font-face
        {font-family:Menlo;
        panose-1:2 11 6 9 3 8 4 2 2 4;}
@font-face
        {font-family:"Times New Roman \, serif ";}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
        {margin:0in;
        margin-bottom:.0001pt;
        font-size:12.0pt;
        font-family:Calibri;}
a:link, span.MsoHyperlink
        {mso-style-priority:99;
        color:#0563C1;
        text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
        {mso-style-priority:99;
        color:#954F72;
        text-decoration:underline;}
pre
        {mso-style-priority:99;
        mso-style-link:"HTML Preformatted Char";
        margin:0in;
        margin-bottom:.0001pt;
        font-size:10.0pt;
        font-family:"Courier New","serif";}
tt
        {mso-style-priority:99;
        font-family:"Courier New","serif";}
span.HTMLPreformattedChar
        {mso-style-name:"HTML Preformatted Char";
        mso-style-priority:99;
        mso-style-link:"HTML Preformatted";
        font-family:Courier;}
span.EmailStyle20
        {mso-style-type:personal;
        font-family:Calibri;
        color:windowtext;}
span.EmailStyle21
        {mso-style-type:personal;
        font-family:Calibri;
        color:windowtext;}
span.EmailStyle22
        {mso-style-type:personal;
        font-family:Calibri;
        color:windowtext;}
span.EmailStyle23
        {mso-style-type:personal;
        font-family:Calibri;
        color:windowtext;}
span.EmailStyle24
        {mso-style-type:personal;
        font-family:Calibri;
        color:windowtext;}
span.EmailStyle25
        {mso-style-type:personal-reply;
        font-family:Calibri;
        color:windowtext;}
span.msoIns
        {mso-style-type:export-only;
        mso-style-name:"";
        text-decoration:underline;
        color:teal;}
.MsoChpDefault
        {mso-style-type:export-only;
        font-size:10.0pt;}
@page WordSection1
        {size:8.5in 11.0in;
        margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
        {page:WordSection1;}
--></style>
      <div class="WordSection1">
        <p class="MsoNormal"><span style="font-size:11.0pt">Bogdan,<o:p></o:p></span></p>
        <p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
        <p class="MsoNormal"><span style="font-size:11.0pt">I tried to
            reproduce this with some simple instructions, but it didn’t
            reproduce. There must be some dependency on other functions
            or configurations we are using. It would be too hard to try
            to figure out exactly what that is, so I will have to
            capture the data for you. It will just take a while for me
            to revive an older configuration that allows me to enable
            the memory debugger.
            <o:p></o:p></span></p>
        <p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
        <div>
          <div>
            <p class="MsoNormal"><span
                style="font-size:10.5pt;color:black"><o:p> </o:p></span></p>
          </div>
        </div>
        <p class="MsoNormal"><span style="font-size:10.5pt;color:black">Ben
            Newlin</span><span style="font-size:11.0pt"><o:p></o:p></span></p>
        <p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
        <div style="border:none;border-top:solid #B5C4DF
          1.0pt;padding:3.0pt 0in 0in 0in">
          <p class="MsoNormal"><b><span style="color:black">From: </span></b><span
              style="color:black">Bogdan-Andrei Iancu
              <a class="moz-txt-link-rfc2396E" href="mailto:bogdan@opensips.org">&lt;bogdan@opensips.org&gt;</a><br>
              <b>Date: </b>Tuesday, August 2, 2016 at 3:47 AM<br>
              <b>To: </b>"Newlin, Ben" <a class="moz-txt-link-rfc2396E" href="mailto:Ben.Newlin@inin.com">&lt;Ben.Newlin@inin.com&gt;</a>,
              OpenSIPS users mailling list
              <a class="moz-txt-link-rfc2396E" href="mailto:users@lists.opensips.org">&lt;users@lists.opensips.org&gt;</a><br>
              <b>Subject: </b>Re: [OpenSIPS-Users] OpenSIPS
              fix_route_dialog crashes<o:p></o:p></span></p>
        </div>
        <div>
          <p class="MsoNormal"><span style="font-family:&quot;Times New
              Roman&quot;"><o:p> </o:p></span></p>
        </div>
        <div>
          <div>
            <p class="MsoNormal">Ben,<br>
              <br>
              To make it easier, please send me the instructions on how
              to reproduce the crash.<br>
              <br>
              Thanks and Regards,<br>
              <br>
              <br>
              <o:p></o:p></p>
            <pre>Bogdan-Andrei Iancu<o:p></o:p></pre>
            <pre>OpenSIPS Founder and Developer<o:p></o:p></pre>
            <pre><a moz-do-not-send="true" href="http://www.opensips-solutions.com">http://www.opensips-solutions.com</a><o:p></o:p></pre>
            <div>
              <p class="MsoNormal">On 01.08.2016 20:17, Newlin, Ben
                wrote:<o:p></o:p></p>
            </div>
            <blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
              <p class="MsoNormal"><span style="font-size:11.0pt">Bogdan,</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">I am
                  not familiar with gdb, so I double check what you’ve
                  assessed. If there are some other steps with gdb you
                  would like me to perform, just let me know what to do.</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">Is
                  there a way to compile the memory debugger without
                  using the interactive `make menuconfig` command? Our
                  build system is completely automated, so it impossible
                  for me to do it this way. Can I pass the options as
                  build parameters or alter the makefile in some way?</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt">I can
                  provide a SIPp scenario which should reproduce the
                  issue on any basic script that uses Dialog with
                  topology hiding, if that would be easier.</span><o:p></o:p></p>
              <div>
                <div>
                  <p class="MsoNormal"><span
                      style="font-size:10.5pt;color:black"> </span><o:p></o:p></p>
                </div>
              </div>
              <p class="MsoNormal"><span
                  style="font-size:10.5pt;color:black">Ben Newlin</span><o:p></o:p></p>
              <p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
              <div style="border:none;border-top:solid #B5C4DF
                1.0pt;padding:3.0pt 0in 0in 0in">
                <p class="MsoNormal"><b><span style="color:black">From:
                    </span></b><span style="color:black">Bogdan-Andrei
                    Iancu
                    <a moz-do-not-send="true"
                      href="mailto:bogdan@opensips.org">&lt;bogdan@opensips.org&gt;</a><br>
                    <b>Date: </b>Monday, August 1, 2016 at 10:57 AM<br>
                    <b>To: </b>OpenSIPS users mailling list <a
                      moz-do-not-send="true"
                      href="mailto:users@lists.opensips.org">
                      &lt;users@lists.opensips.org&gt;</a>, "Newlin,
                    Ben" <a moz-do-not-send="true"
                      href="mailto:Ben.Newlin@inin.com">
                      &lt;Ben.Newlin@inin.com&gt;</a><br>
                    <b>Subject: </b>Re: [OpenSIPS-Users] OpenSIPS
                    fix_route_dialog crashes</span><o:p></o:p></p>
              </div>
              <div>
                <p class="MsoNormal"><span
                    style="font-family:&quot;Times New Roman \, serif
                    &quot;"> </span><o:p></o:p></p>
              </div>
              <div>
                <div>
                  <p class="MsoNormal"><tt><span
                        style="font-size:10.0pt">Hi Ben,</span></tt><span
                      style="font-size:10.0pt;font-family:&quot;Courier
                      New&quot;,&quot;serif&quot;"><br>
                      <br>
                      <tt>According to the BT, the crash is in a
                        pkg_malloc() call:</tt><br>
                      <tt>                    route = pkg_malloc(size);</tt><br>
                      <tt>Please double check this with gdb info.</tt><br>
                      <br>
                      <tt>If so, this indicate a memory corruption and
                        we have 2 options here:</tt><br>
                      <tt>    - you compile with memory debugger (see my
                        previous emails)</tt><br>
                      <tt>    - provide step-by-step indications on how
                        to reproduce this crash.</tt><br>
                      <br>
                      <tt>Thanks and Regards,</tt><br>
                      <br>
                      <br>
                    </span><o:p></o:p></p>
                  <pre>Bogdan-Andrei Iancu<o:p></o:p></pre>
                  <pre>OpenSIPS Founder and Developer<o:p></o:p></pre>
                  <pre><a moz-do-not-send="true" href="http://www.opensips-solutions.com">http://www.opensips-solutions.com</a><o:p></o:p></pre>
                  <div>
                    <p class="MsoNormal">On 29.07.2016 15:54, Newlin,
                      Ben wrote:<o:p></o:p></p>
                  </div>
                  <blockquote
                    style="margin-top:5.0pt;margin-bottom:5.0pt">
                    <p class="MsoNormal"><span style="font-size:11.0pt">This
                        is 1.11.6, running on CentOS 7.</span><o:p></o:p></p>
                    <p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
                    <div>
                      <div>
                        <p class="MsoNormal"><span
                            style="font-size:10.5pt;color:black"> </span><o:p></o:p></p>
                      </div>
                    </div>
                    <p class="MsoNormal"><span
                        style="font-size:10.5pt;color:black">Ben Newlin</span><o:p></o:p></p>
                    <p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
                    <div style="border:none;border-top:solid #B5C4DF
                      1.0pt;padding:3.0pt 0in 0in 0in">
                      <p class="MsoNormal"><b><span style="color:black">From:
                          </span></b><span style="color:black"><a
                            moz-do-not-send="true"
                            href="mailto:users-bounces@lists.opensips.org">&lt;users-bounces@lists.opensips.org&gt;</a>
                          on behalf of Bogdan-Andrei Iancu
                          <a moz-do-not-send="true"
                            href="mailto:bogdan@opensips.org">&lt;bogdan@opensips.org&gt;</a><br>
                          <b>Reply-To: </b>OpenSIPS users mailling list
                          <a moz-do-not-send="true"
                            href="mailto:users@lists.opensips.org">
                            &lt;users@lists.opensips.org&gt;</a><br>
                          <b>Date: </b>Friday, July 29, 2016 at 8:50 AM<br>
                          <b>To: </b>"Newlin, Ben" <a
                            moz-do-not-send="true"
                            href="mailto:Ben.Newlin@inin.com">&lt;Ben.Newlin@inin.com&gt;</a>,
                          OpenSIPS users mailling list
                          <a moz-do-not-send="true"
                            href="mailto:users@lists.opensips.org">&lt;users@lists.opensips.org&gt;</a><br>
                          <b>Subject: </b>Re: [OpenSIPS-Users] OpenSIPS
                          fix_route_dialog crashes</span><o:p></o:p></p>
                    </div>
                    <div>
                      <p class="MsoNormal"><span
                          style="font-family:&quot;Times New Roman \,
                          serif &quot;"> </span><o:p></o:p></p>
                    </div>
                    <div>
                      <div>
                        <p class="MsoNormal"><tt><span
                              style="font-size:10.0pt">Ben,</span></tt><span
style="font-size:10.0pt;font-family:&quot;Courier New \, serif &quot;"><br>
                            <br>
                          </span><tt><span style="font-size:10.0pt">What
                              OpenSIPS version is this (the crashing
                              one) ? 1.11 or 2.1 ?</span></tt><span
                            style="font-size:10.0pt;font-family:&quot;Courier
                            New \, serif &quot;"><br>
                            <br>
                          </span><tt><span style="font-size:10.0pt">Regards,</span></tt><span
style="font-size:10.0pt;font-family:&quot;Courier New \, serif &quot;"><br>
                            <br>
                            <br>
                            <br>
                          </span><o:p></o:p></p>
                        <pre>Bogdan-Andrei Iancu<o:p></o:p></pre>
                        <pre>OpenSIPS Founder and Developer<o:p></o:p></pre>
                        <pre><a moz-do-not-send="true" href="http://www.opensips-solutions.com">http://www.opensips-solutions.com</a><o:p></o:p></pre>
                        <div>
                          <p class="MsoNormal">On 27.07.2016 19:02,
                            Newlin, Ben wrote:<o:p></o:p></p>
                        </div>
                        <blockquote
                          style="margin-top:5.0pt;margin-bottom:5.0pt">
                          <p class="MsoNormal"><span
                              style="font-size:11.0pt">I have identified
                              that these crashes are occurring when the
                              far end system is not returning the
                              Record-Route headers in the 200 OK
                              response. The headers are present in the
                              180 response, but not the 200 OK. I have
                              reproduced the scenario using SIPp and
                              captured a SIP trace: <a
                                moz-do-not-send="true"
                                href="http://pastebin.com/ckKk3EhY">
                                http://pastebin.com/ckKk3EhY</a></span><o:p></o:p></p>
                          <p class="MsoNormal"><span
                              style="font-size:11.0pt"> </span><o:p></o:p></p>
                          <p class="MsoNormal"><span
                              style="font-size:11.0pt">The crash occurs
                              on receipt of the ACK request and attempt
                              to match the dialog.</span><o:p></o:p></p>
                          <p class="MsoNormal"><span
                              style="font-size:11.0pt"> </span><o:p></o:p></p>
                          <p class="MsoNormal"><span
                              style="font-size:11.0pt">I also captured a
                              BT for this scenario as well, in case
                              anything specific in the trace made the
                              issue easier to find:
                              <a moz-do-not-send="true"
                                href="http://pastebin.com/cM3FhPiw">http://pastebin.com/cM3FhPiw</a></span><o:p></o:p></p>
                          <p class="MsoNormal"><span
                              style="font-size:11.0pt"> </span><o:p></o:p></p>
                          <p class="MsoNormal"><span
                              style="font-size:11.0pt">I am working with
                              the other system to try to fix their
                              behavior.</span><o:p></o:p></p>
                          <p class="MsoNormal"><span
                              style="font-size:11.0pt"> </span><o:p></o:p></p>
                          <p class="MsoNormal"><span
                              style="font-size:11.0pt">Ideally the
                              Record-Route headers from previous replies
                              could be used in this case to allow the
                              call to succeed, but I don’t know if that
                              is possible.</span><o:p></o:p></p>
                          <p class="MsoNormal"><span
                              style="font-size:11.0pt"> </span><o:p></o:p></p>
                          <div>
                            <div>
                              <p class="MsoNormal"><span
                                  style="font-size:10.5pt;color:black">Thanks,</span><o:p></o:p></p>
                              <p class="MsoNormal"><span
                                  style="font-size:10.5pt;color:black"> </span><o:p></o:p></p>
                            </div>
                          </div>
                          <p class="MsoNormal"><span
                              style="font-size:10.5pt;color:black">Ben
                              Newlin</span><o:p></o:p></p>
                          <p class="MsoNormal"><span
                              style="font-size:11.0pt"> </span><o:p></o:p></p>
                          <div style="border:none;border-top:solid
                            #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
                            <p class="MsoNormal"><b><span
                                  style="color:black">From: </span></b><span
                                style="color:black">"Newlin, Ben"
                                <a moz-do-not-send="true"
                                  href="mailto:Ben.Newlin@inin.com">&lt;Ben.Newlin@inin.com&gt;</a><br>
                                <b>Date: </b>Wednesday, July 27, 2016
                                at 9:44 AM<br>
                                <b>To: </b>Bogdan-Andrei Iancu <a
                                  moz-do-not-send="true"
                                  href="mailto:bogdan@opensips.org">&lt;bogdan@opensips.org&gt;</a>,
                                OpenSIPS users mailling list
                                <a moz-do-not-send="true"
                                  href="mailto:users@lists.opensips.org">&lt;users@lists.opensips.org&gt;</a><br>
                                <b>Subject: </b>Re: [OpenSIPS-Users]
                                OpenSIPS fix_route_dialog crashes</span><o:p></o:p></p>
                          </div>
                          <div>
                            <p class="MsoNormal"><span
                                style="font-family:&quot;Times New Roman
                                \, serif &quot;"> </span><o:p></o:p></p>
                          </div>
                          <div>
                            <div>
                              <p class="MsoNormal"><span
                                  style="font-size:11.0pt">Bogdan,</span><o:p></o:p></p>
                              <p class="MsoNormal"><span
                                  style="font-size:11.0pt"> </span><o:p></o:p></p>
                              <p class="MsoNormal"><span
                                  style="font-size:11.0pt">This is a
                                  different scenario than the other you
                                  responded to. As I said, we have two
                                  types of servers that work together.
                                  One is a load-balancer and runs as a
                                  proxy. It uses double Record-Route
                                  because it sends messages between
                                  public and private networks. Then we
                                  have our other servers using TH which
                                  receive those requests. We are not
                                  using TH and RR on the same server
                                  (although I would like to).</span><o:p></o:p></p>
                              <p class="MsoNormal"><span
                                  style="font-size:11.0pt"> </span><o:p></o:p></p>
                              <p class="MsoNormal"><span
                                  style="font-size:11.0pt">If
                                  validate_dialog() and
                                  fix_route_dialog() (and possibly
                                  loose_route()) should not be called
                                  when using TH, I believe the
                                  documentation should reference that.
                                  It states that match_dialog() must be
                                  used with TH, but does not indicate
                                  that the other functions should not be
                                  used or that the functionality won’t
                                  work. There is also no documentation
                                  of the incompatibility between RR and
                                  TH.
                                </span><o:p></o:p></p>
                              <p class="MsoNormal"><span
                                  style="font-size:11.0pt"> </span><o:p></o:p></p>
                              <p class="MsoNormal"><span
                                  style="font-size:11.0pt">Either way, I
                                  ran a test where I removed all calls
                                  to loose_route(), validate_dialog(),
                                  and fix_route_dialog() from my script.
                                  The crash still occurred and the BT
                                  still pointed to fix_route_dialog()
                                  function. So it must be getting called
                                  from within Dialog module somewhere.
                                  That BT is here:
                                  <a moz-do-not-send="true"
                                    href="http://pastebin.com/wu2X2Hxh">http://pastebin.com/wu2X2Hxh</a></span><o:p></o:p></p>
                              <p class="MsoNormal"><span
                                  style="font-size:11.0pt"> </span><o:p></o:p></p>
                              <p class="MsoNormal"><span
                                  style="font-size:11.0pt">I collected
                                  this BT with loose_route() being
                                  called from my script, but not
                                  validate_dialog() or
                                  fix_route_dialog():
                                  <a moz-do-not-send="true"
                                    href="http://pastebin.com/6V7yPaHF">http://pastebin.com/6V7yPaHF</a></span><o:p></o:p></p>
                              <p class="MsoNormal"><span
                                  style="font-size:11.0pt"> </span><o:p></o:p></p>
                              <p class="MsoNormal"><span
                                  style="font-size:11.0pt">This BT was
                                  collected with all three functions
                                  being called from my script:
                                  <a moz-do-not-send="true"
                                    href="http://pastebin.com/fZYYdndn">http://pastebin.com/fZYYdndn</a></span><o:p></o:p></p>
                              <p class="MsoNormal"><span
                                  style="font-size:11.0pt"> </span><o:p></o:p></p>
                              <div>
                                <div>
                                  <p class="MsoNormal"><span
                                      style="font-size:10.5pt;color:black"> </span><o:p></o:p></p>
                                </div>
                              </div>
                              <p class="MsoNormal"><span
                                  style="font-size:10.5pt;color:black">Ben
                                  Newlin</span><o:p></o:p></p>
                              <p class="MsoNormal"><span
                                  style="font-size:11.0pt"> </span><o:p></o:p></p>
                              <div style="border:none;border-top:solid
                                #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
                                <p class="MsoNormal"><b><span
                                      style="color:black">From: </span></b><span
                                    style="color:black">Bogdan-Andrei
                                    Iancu
                                    <a moz-do-not-send="true"
                                      href="mailto:bogdan@opensips.org">&lt;bogdan@opensips.org&gt;</a><br>
                                    <b>Date: </b>Wednesday, July 27,
                                    2016 at 3:57 AM<br>
                                    <b>To: </b>OpenSIPS users mailling
                                    list <a moz-do-not-send="true"
                                      href="mailto:users@lists.opensips.org">
                                      &lt;users@lists.opensips.org&gt;</a>,
                                    "Newlin, Ben" <a
                                      moz-do-not-send="true"
                                      href="mailto:Ben.Newlin@inin.com">
                                      &lt;Ben.Newlin@inin.com&gt;</a><br>
                                    <b>Subject: </b>Re:
                                    [OpenSIPS-Users] OpenSIPS
                                    fix_route_dialog crashes</span><o:p></o:p></p>
                              </div>
                              <div>
                                <p class="MsoNormal"><span
                                    style="font-family:&quot;Times New
                                    Roman \, serif &quot;"> </span><o:p></o:p></p>
                              </div>
                              <div>
                                <div>
                                  <p class="MsoNormal"><tt><span
                                        style="font-size:10.0pt">Hi Ben,</span></tt><span
style="font-size:10.0pt;font-family:&quot;Courier New \, serif &quot;"><br>
                                      <br>
                                    </span><tt><span
                                        style="font-size:10.0pt">First,
                                        if you use TH, makes no sense to
                                        do Record-Routing - there are 2
                                        SIP concepts that overlaps. You
                                        either act as an end-point (by
                                        doing TH), either as a proxy
                                        (doing RR).</span></tt><span
                                      style="font-size:10.0pt;font-family:&quot;Courier
                                      New \, serif &quot;"><br>
                                      <br>
                                    </span><tt><span
                                        style="font-size:10.0pt">If
                                        doing TH, makes no sense to use
                                        validate + fix as these
                                        functions check and repair the
                                        routing information in the
                                        request (like Route and Contact
                                        headers). if you do TH, this
                                        routing info is actually hidden
                                        and added by OpenSIPS, so there
                                        is nothing to fix and repair.</span></tt><span
style="font-size:10.0pt;font-family:&quot;Courier New \, serif &quot;"><br>
                                      <br>
                                    </span><tt><span
                                        style="font-size:10.0pt">Nevertheless,
                                        this should not crash or corrupt
                                        OpenSIPS. HAve you managed to
                                        get a corefile ?</span></tt><span
style="font-size:10.0pt;font-family:&quot;Courier New \, serif &quot;"><br>
                                      <br>
                                    </span><tt><span
                                        style="font-size:10.0pt">Also if
                                        you suspect memory corruption,
                                        you can compile-in the memory
                                        debugger - see
                                        <a moz-do-not-send="true"
                                          href="http://www.opensips.org/Documentation/TroubleShooting-OutOfMem">http://www.opensips.org/Documentation/TroubleShooting-OutOfMem</a>
                                        .</span></tt><span
                                      style="font-size:10.0pt;font-family:&quot;Courier
                                      New \, serif &quot;"><br>
                                      <br>
                                    </span><tt><span
                                        style="font-size:10.0pt">Regards,</span></tt><span
style="font-size:10.0pt;font-family:&quot;Courier New \, serif &quot;"><br>
                                      <br>
                                      <br>
                                      <br>
                                      <br>
                                      <br>
                                    </span><o:p></o:p></p>
                                  <pre>Bogdan-Andrei Iancu<o:p></o:p></pre>
                                  <pre>OpenSIPS Founder and Developer<o:p></o:p></pre>
                                  <pre><a moz-do-not-send="true" href="http://www.opensips-solutions.com">http://www.opensips-solutions.com</a><o:p></o:p></pre>
                                  <div>
                                    <p class="MsoNormal">On 26.07.2016
                                      23:20, Newlin, Ben wrote:<o:p></o:p></p>
                                  </div>
                                  <blockquote
                                    style="margin-top:5.0pt;margin-bottom:5.0pt">
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt">I have
                                        had 3 OpenSIPS server crashes in
                                        the last week. All were due to
                                        segmentation faults. I was not
                                        able to capture core dumps; I am
                                        configuring that now to catch
                                        the next crash.</span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt"> </span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt">My logs
                                        leading up to the crash are full
                                        of errors from
                                        fix_route_dialog() complaining
                                        about invalid URIs for
                                        sequential requests:</span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt"> </span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:8.5pt;font-family:Menlo;color:#2CA221;background:white">Jul
                                        26 19:34:02 [220]
                                        ERROR:dialog:fix_route_dialog:
                                        Failed to parse SIP uri</span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:8.5pt;font-family:Menlo;color:#2CA221;background:white">Jul
                                        26 19:34:02 [220]
                                        ERROR:core:parse_uri: bad uri,
                                        state 0 parsed: &lt;ip:1&gt; (4)
                                        /
                                        &lt;ip:10.18.8.18:5060;ftag=gK0448f137;lr;r2=on&gt;&gt;
                                        (44)</span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-family:&quot;Times
                                        New Roman \, serif &quot;"> </span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:8.5pt;font-family:Menlo;color:#2CA221;background:white">Jul
                                        26 19:11:19 [218]
                                        ERROR:dialog:fix_route_dialog:
                                        Failed to parse SIP uri</span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:8.5pt;font-family:Menlo;color:#2CA221;background:white">Jul
                                        26 19:11:19 [218]
                                        ERROR:core:parse_uri: bad uri,
                                        state 0 parsed: &lt;b0i2&gt; (4)
                                        /
&lt;b0i2yjor;transport=udp&lt;sip:10.18.8.17:5060;ftag=7207ce89;lr;r2=on&gt;
                                        (65)</span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt"> </span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:8.5pt;font-family:Menlo;color:#2CA221;background:white">Jul
                                        26 17:43:19 [220]
                                        ERROR:dialog:fix_route_dialog:
                                        Failed to parse SIP uri</span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:8.5pt;font-family:Menlo;color:#2CA221;background:white">Jul
                                        26 17:43:19 [220]
                                        ERROR:core:parse_uri: bad uri,
                                        state 0 parsed: &lt;ervi&gt; (4)
                                        /
                                        &lt;ervice_id6fdbc70f-2438-4726-807c-0d081df4d87&gt;
                                        (44)</span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt"> </span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt">Many
                                        times the “URI” displayed in the
                                        error message is actually
                                        internal OpenSIPS variables, as
                                        in the last error above. When
                                        they are from the SIP message, I
                                        have verified that the messages
                                        themselves are correctly
                                        formatted. This leads me to
                                        believe there is memory
                                        corruption occurring.</span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt"> </span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt">This
                                        all started when I updated my
                                        load-balancer servers to use
                                        Record-Routing, specifically the
                                        “double_rr” mechanism for when
                                        multiple interfaces exist. The
                                        Record-Routing is occurring on
                                        different servers which have not
                                        crashed. Only the servers
                                        receiving the Record-Routed
                                        messages are experiencing the
                                        errors.</span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt"> </span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt">Here is
                                        a piece of the code processing
                                        sequential requests. I am using
                                        the topology_hiding()
                                        functionality of the Dialog
                                        module. Are validate_dialog()
                                        and fix_route_dialog() still
                                        valid in a topology_hiding
                                        scenario?</span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt"> </span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt">if
                                        (t_check_trans())</span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt">   
                                        setflag(SEQ_REQUEST);</span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt"> </span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt">  if
                                        (has_totag())</span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt">  {</span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt">   
                                        loose_route();</span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt"> </span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt">    if
                                        (match_dialog())</span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt">    {</span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt">     
                                        if (!validate_dialog())</span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt">       
                                        fix_route_dialog();</span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt"> </span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt">     
                                        if (is_method("BYE"))</span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt">       
                                        setflag(ACC_FLAG);</span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt"> </span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt">     
                                        setflag(SEQ_REQUEST);</span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt">    }</span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt">   
                                        else if
                                        (!isflagset(SEQ_REQUEST))</span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt">    {</span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt">     
                                        if (!is_method("ACK")) {</span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt">       
                                        route(rlog, LV_ERROR,
                                        "check_sequential", "Sequential
                                        request not matched");</span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt">     
                                          route(reply_error, "481",
                                        "Call Does Not Exist");</span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt">      }</span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt"> </span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt">     
                                        return(EXIT);</span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt">    }</span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt">  }</span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt"> </span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt">I will
                                        attempt to get core dumps of
                                        future crashes.</span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt"> </span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt">Thanks,</span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-size:11.0pt">Ben
                                        Newlin</span><o:p></o:p></p>
                                    <p class="MsoNormal"><span
                                        style="font-family:&quot;Times
                                        New Roman \, serif &quot;"><br>
                                        <br>
                                        <br>
                                        <br>
                                        <br>
                                        <br>
                                        <br>
                                      </span><o:p></o:p></p>
                                    <pre>_______________________________________________<o:p></o:p></pre>
                                    <pre>Users mailing list<o:p></o:p></pre>
                                    <pre><a moz-do-not-send="true" href="mailto:Users@lists.opensips.org">Users@lists.opensips.org</a><o:p></o:p></pre>
                                    <pre><a moz-do-not-send="true" href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users">http://lists.opensips.org/cgi-bin/mailman/listinfo/users</a><o:p></o:p></pre>
                                  </blockquote>
                                  <p class="MsoNormal"><span
                                      style="font-family:&quot;Times New
                                      Roman \, serif &quot;"> </span><o:p></o:p></p>
                                </div>
                              </div>
                            </div>
                          </div>
                        </blockquote>
                        <p class="MsoNormal"><span
                            style="font-family:&quot;Times New Roman \,
                            serif &quot;"> </span><o:p></o:p></p>
                      </div>
                    </div>
                    <p class="MsoNormal"><span
                        style="font-family:&quot;Times New Roman \,
                        serif &quot;"><br>
                        <br>
                        <br>
                        <br>
                      </span><o:p></o:p></p>
                    <pre>_______________________________________________<o:p></o:p></pre>
                    <pre>Users mailing list<o:p></o:p></pre>
                    <pre><a moz-do-not-send="true" href="mailto:Users@lists.opensips.org">Users@lists.opensips.org</a><o:p></o:p></pre>
                    <pre><a moz-do-not-send="true" href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users">http://lists.opensips.org/cgi-bin/mailman/listinfo/users</a><o:p></o:p></pre>
                  </blockquote>
                  <p class="MsoNormal"><span
                      style="font-family:&quot;Times New Roman \, serif
                      &quot;"> </span><o:p></o:p></p>
                </div>
              </div>
            </blockquote>
            <p class="MsoNormal"><span style="font-family:&quot;Times
                New Roman&quot;"><o:p> </o:p></span></p>
          </div>
        </div>
      </div>
    </blockquote>
    <br>
  </body>
</html>