<html>
  <head>
    <meta content="text/html; charset=windows-1252"
      http-equiv="Content-Type">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    <div class="moz-cite-prefix"><tt>Hi,<br>
        <br>
        Make a pcap of a call to see if the what's broken in that ACK.<br>
        <br>
        Regards,<br>
      </tt>
      <pre class="moz-signature" cols="72">Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
<a class="moz-txt-link-freetext" href="http://www.opensips-solutions.com">http://www.opensips-solutions.com</a></pre>
      On 02.07.2014 18:30, Gary Nyquist wrote:<br>
    </div>
    <blockquote
cite="mid:trinity-c7b63155-7b26-43b9-a649-bc1ff4650381-1404315011204@3capp-mailcom-lxa08"
      type="cite">
      <div style="font-family: Verdana;font-size: 12.0px;">
        <div>
          <div>Thanks Liviu for your advice,</div>
          <div> </div>
          <div>Wondering how can this be fixed... it is just filling up
            the log files with hundreds of repeating lines.</div>
          <div>Also, thinking how to stop the "<span style="font-family:
              monospace; font-size: 12px; line-height: 19.1875px;">out-of-dialog
              ACKs to 487 responses</span>" to be generated in the first
            place (if it is the culprit indeed).</div>
          <div>Any suggestion?</div>
          <div>Thanks again...</div>
          <div> </div>
          <div>BR</div>
          <div>- Gary</div>
          <div> 
            <div name="quote" style="margin:10px 5px 5px 10px; padding:
              10px 0 10px 10px; border-left:2px solid #C3D9E5;
              word-wrap: break-word; -webkit-nbsp-mode: space;
              -webkit-line-break: after-white-space;">
              <div style="margin:0 0 10px 0;"><b>Sent:</b> Tuesday, July
                01, 2014 at 8:05 AM<br>
                <b>From:</b> "Liviu Chircu" <a class="moz-txt-link-rfc2396E" href="mailto:liviu@opensips.org">&lt;liviu@opensips.org&gt;</a><br>
                <b>To:</b> <a class="moz-txt-link-abbreviated" href="mailto:users@lists.opensips.org">users@lists.opensips.org</a><br>
                <b>Subject:</b> Re: [OpenSIPS-Users] TCP Errors</div>
              <div name="quoted-content">
                <div style="background-color: rgb(255,255,255);">
                  <div class="moz-cite-prefix"><tt>Hello Gary,<br>
                      <br>
                      That "error" is most likely due to out-of-dialog
                      ACKs to 487 responses (CANCEL scenarios). Thank
                      you for reporting - it should actually be a
                      debugging message.<br>
                      <br>
                      Best regards,</tt>
                    <pre class="moz-signature">Liviu Chircu
OpenSIPS Developer
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://www.opensips-solutions.com" target="_blank">http://www.opensips-solutions.com</a></pre>
                    On 06/30/2014 07:00 PM, Gary Nyquist wrote:</div>
                  <blockquote>
                    <div style="font-family: Verdana;font-size: 12.0px;">
                      <div>
                        <div>Hi Vlad,</div>
                        <div> </div>
                        <div>Thanks for taking up this issue.</div>
                        <div>1.  Yes.  <span style="font-family:
                            Verdana;font-size: 12.0px;line-height:
                            19.2px;">tcp_async=1</span></div>
                        <div>2.  Running on CentOS
                          ( 2.6.32-358.6.2.el6.x86_64 )</div>
                        <div>3.  It's occurring on every INVITE</div>
                        <div> </div>
                        <div>May be unrelated to this bug; but I am also
                          seeing error messages like:</div>
                        <div>ERROR:script_helper:run_helper_logic:
                          failed to match dialog, ci
                          '4E14BD6E-53B17A8F00026071-6B1EA700'</div>
                        <div> </div>
                        <div>Thanks again...</div>
                        <div>BR</div>
                        <div>-Gary</div>
                        <div> 
                          <div style="margin: 10.0px 5.0px 5.0px
                            10.0px;padding: 10.0px 0 10.0px
                            10.0px;border-left: 2.0px solid
                            rgb(195,217,229);">
                            <div style="margin: 0 0 10.0px 0;"><b>Sent:</b> Monday,
                              June 30, 2014 at 5:06 AM<br>
                              <b>From:</b> "Vlad Paiu" <a
                                moz-do-not-send="true"
                                class="moz-txt-link-rfc2396E"
                                href="vladpaiu@opensips.org"
                                target="_parent">&lt;vladpaiu@opensips.org&gt;</a><br>
                              <b>To:</b> <a moz-do-not-send="true"
                                class="moz-txt-link-abbreviated"
                                href="users@lists.opensips.org"
                                target="_parent">users@lists.opensips.org</a><br>
                              <b>Cc:</b> <a moz-do-not-send="true"
                                class="moz-txt-link-abbreviated"
                                href="gn62@gmx.us" target="_parent">gn62@gmx.us</a><br>
                              <b>Subject:</b> Re: [OpenSIPS-Users] TCP
                              Errors</div>
                            <div>
                              <div style="background-color:
                                rgb(255,255,255);">
                                <div class="moz-cite-prefix">Hello,<br>
                                  <br>
                                  First of all, are you using the
                                  async_tcp option in your OpenSIPS
                                  script ?<br>
                                  Also, what architecture / OS are you
                                  running on ?<br>
                                  What steps are you taking in
                                  reproducing this ?<br>
                                  <br>
                                  Best Regards,
                                  <pre class="moz-signature">Vlad Paiu
OpenSIPS Developer
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://www.opensips-solutions.com" target="_blank">http://www.opensips-solutions.com</a> </pre>
                                  On 27.06.2014 19:59, Gary Nyquist
                                  wrote:</div>
                                <blockquote>
                                  <div style="font-family:
                                    Verdana;font-size: 12.0px;">
                                    <div>Thanks Bogdan for looking into
                                      it.</div>
                                    <div> </div>
                                    <div>Here is the version:</div>
                                    <div>opensips -V<br>
                                      version: opensips 1.11.1-tls
                                      (x86_64/linux)<br>
                                      flags: STATS: On, USE_TCP,
                                      USE_TLS, DISABLE_NAGLE, SHM_MEM,
                                      SHM_MMAP, PKG_MALLOC, F_MALLOC,
                                      USE_SHM_MEM,
                                      FAST_LOCK-ADAPTIVE_WAIT<br>
                                      ADAPTIVE_WAIT_LOOPS=1024,
                                      MAX_RECV_BUFFER_SIZE 262144,
                                      MAX_LISTEN 16, MAX_URI_SIZE 1024,
                                      BUF_SIZE 65535<br>
                                      poll method support: poll,
                                      epoll_lt, epoll_et, sigio_rt,
                                      select.<br>
                                      git revision: dbc8653<br>
                                      main.c compiled on 16:17:43 Jun 26
                                      2014 with gcc 4.4.7</div>
                                    <div>
                                      <div> </div>
                                      <div>Yes, this bug is
                                        reproducable.</div>
                                      <div><span>When it appears, the
                                          log file instantly gets filled
                                          with hundreds of repeating
                                          lines like this:</span></div>
                                      <div> </div>
                                      <div>Jun 26 20:30:31 ip-10-0-0-30
                                        /usr/sbin/opensips[15229]:
                                        DBG:core:handle_tcpconn_ev: data
                                        available on 0x7f40bcec7398 6<br>
                                        Jun 26 20:30:31 ip-10-0-0-30
                                        /usr/sbin/opensips[15229]:
                                        DBG:core:io_watch_del:
                                        io_watch_del op on index -1 6
                                        (0x7dff20, 6, -1, 0x0,0x1)
                                        fd_no=110 called<br>
                                        Jun 26 20:30:31 ip-10-0-0-30
                                        /usr/sbin/opensips[15229]:
                                        ERROR:core:io_watch_del: BUG -
                                        trying to del fd 6 with flags 2
                                        1<br>
                                        Jun 26 20:30:31 ip-10-0-0-30
                                        /usr/sbin/opensips[15229]:
                                        DBG:core:handle_tcpconn_ev: data
                                        available on 0x7f40bcec7398 6</div>
                                      <div>Jun 26 20:30:31 ip-10-0-0-30
                                        /usr/sbin/opensips[15229]:
                                        DBG:core:io_watch_del:
                                        io_watch_del op on index -1 6
                                        (0x7dff20, 6, -1, 0x0,0x1)
                                        fd_no=110 called<br>
                                        Jun 26 20:30:31 ip-10-0-0-30
                                        /usr/sbin/opensips[15229]:
                                        ERROR:core:io_watch_del: BUG -
                                        trying to del fd 6 with flags 2
                                        1</div>
                                      <div> </div>
                                      <div>Thanks again for your help.</div>
                                      <div> </div>
                                      <div>BR</div>
                                      <div>-Gary</div>
                                      <div> </div>
                                      <div> </div>
                                      <div style="margin: 10.0px 5.0px
                                        5.0px 10.0px;padding: 10.0px 0
                                        10.0px 10.0px;border-left: 2.0px
                                        solid rgb(195,217,229);">
                                        <div style="margin: 0 0 10.0px
                                          0;"><b>Sent:</b> Thursday,
                                          June 26, 2014 at 10:56 AM<br>
                                          <b>From:</b> "Bogdan-Andrei
                                          Iancu" <a
                                            moz-do-not-send="true"
                                            class="moz-txt-link-rfc2396E">&lt;bogdan@opensips.org&gt;</a><br>
                                          <b>To:</b> "Gary Nyquist" <a
                                            moz-do-not-send="true"
                                            class="moz-txt-link-rfc2396E">&lt;gn62@gmx.us&gt;</a><br>
                                          <b>Cc:</b> "OpenSIPS users
                                          mailling list" <a
                                            moz-do-not-send="true"
                                            class="moz-txt-link-rfc2396E">&lt;users@lists.opensips.org&gt;</a><br>
                                          <b>Subject:</b> Re:
                                          [OpenSIPS-Users] TCP Errors</div>
                                        <div>
                                          <div style="background-color:
                                            rgb(255,255,255);">
                                            <div class="moz-cite-prefix"><tt>Hi,<br>
                                                <br>
                                                That bug log actually
                                                says OpenSIPS tries to
                                                remove a connection
                                                marked as READ from a
                                                list for WRITEs :)..<br>
                                                <br>
                                                I will look into that.
                                                What exact version do
                                                you use (opensips -V) ?
                                                also, can you reproduce
                                                this  bug ?<br>
                                                <br>
                                                Thanks and regards,</tt>
                                              <pre class="moz-signature">Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://www.opensips-solutions.com" target="_blank">http://www.opensips-solutions.com</a></pre>
                                              On 25.06.2014 20:27, Gary
                                              Nyquist wrote:</div>
                                            <blockquote>
                                              <div style="font-family:
                                                Verdana;font-size:
                                                12.0px;">
                                                <div>
                                                  <div>Thanks Bogdan for
                                                    the detailed reply.</div>
                                                  <div> </div>
                                                  <div>I implemented
                                                    your advice. </div>
                                                  <div>Those error
                                                    messages are now
                                                    gone.</div>
                                                  <div> </div>
                                                  <div>But seeing a new
                                                    error now; not sure
                                                    if it is related to
                                                    that...</div>
                                                  <div> </div>
                                                  <div>ERROR:core:io_watch_del:
                                                    BUG - trying to del
                                                    fd 36 with flags 2 1</div>
                                                  <div> </div>
                                                  <div>Any advice?</div>
                                                  <div> </div>
                                                  <div>BR</div>
                                                  <div>-Gary</div>
                                                  <div> 
                                                    <div style="margin:
                                                      10.0px 5.0px 5.0px
                                                      10.0px;padding:
                                                      10.0px 0 10.0px
                                                      10.0px;border-left:
                                                      2.0px solid
                                                      rgb(195,217,229);">
                                                      <div
                                                        style="margin: 0
                                                        0 10.0px 0;"><b>Sent:</b> Wednesday,
                                                        June 25, 2014 at
                                                        5:54 AM<br>
                                                        <b>From:</b> "Bogdan-Andrei
                                                        Iancu" <a
                                                          moz-do-not-send="true"
class="moz-txt-link-rfc2396E">&lt;bogdan@opensips.org&gt;</a><br>
                                                        <b>To:</b> "OpenSIPS
                                                        users mailling
                                                        list" <a
                                                          moz-do-not-send="true"
class="moz-txt-link-rfc2396E">&lt;users@lists.opensips.org&gt;</a>, <a
moz-do-not-send="true" class="moz-txt-link-abbreviated">gn62@gmx.us</a><br>
                                                        <b>Subject:</b> Re:
                                                        [OpenSIPS-Users]
                                                        TCP Errors</div>
                                                      <div>
                                                        <div
                                                          style="background-color:
rgb(255,255,255);">
                                                          <div
                                                          class="moz-cite-prefix"><tt>Hi,<br>
                                                          <br>
                                                          Those messages
                                                          say that
                                                          OpenSIPS tried
                                                          to open a TCP
                                                          connection to
                                                          a party which
                                                          does not
                                                          respond -&gt;
                                                          there was a
                                                          timeout for
                                                          connect in 10
                                                          seconds.
                                                          Because of
                                                          this blocking
                                                          in connects,
                                                          there were not
                                                          more opensips
                                                          workers
                                                          available to
                                                          handle other
                                                          traffic.<br>
                                                          <br>
                                                          So what you
                                                          need to do is
                                                          :<br>
                                                              - minimize
                                                          the impact of
                                                          the blocking
                                                          connect - see
                                                          my previous
                                                          email on
                                                          reducing the
                                                          connect
                                                          timeout<br>
                                                              - you may
                                                          configure
                                                          OpenSIPS not
                                                          to open new
                                                          TCP connect
                                                          (but to reuse
                                                          the existing
                                                          ones, open by
                                                          clients). See
tcp_no_new_conn_bflag<br>
                                                                      <a
moz-do-not-send="true" class="moz-txt-link-freetext"
href="http://www.opensips.org/Documentation/Script-CoreParameters-1-11#toc96"
target="_blank">http://www.opensips.org/Documentation/Script-CoreParameters-1-11#toc96</a><br>
                                                              - try to
                                                          understand the
                                                          SIP patterns
                                                          where such TCP
                                                          connect fails
                                                          so you can
                                                          avoid them at
                                                          script level.<br>
                                                          <br>
                                                          Regards,</tt>
                                                          <pre class="moz-signature">Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://www.opensips-solutions.com" target="_blank">http://www.opensips-solutions.com</a></pre>
                                                          On 24.06.2014
                                                          21:34, Gary
                                                          Nyquist wrote:</div>
                                                          <blockquote>
                                                          <div
                                                          style="font-family:
                                                          Verdana;font-size:
                                                          12.0px;">
                                                          <div
                                                          style="font-family:
                                                          Verdana;font-size:
                                                          12.0px;">
                                                          <p
                                                          style="margin:
                                                          0.0px;padding:
                                                          0.0px;"><span
                                                          style="font-family:
                                                          Verdana;"><span
                                                          style="font-size:
                                                          12.0px;"><font
                                                          size="3"
                                                          face="monospace"><span>Hi,</span></font></span></span></p>
                                                          <p
                                                          style="margin:
                                                          0.0px;padding:
                                                          0.0px;"> </p>
                                                          <p
                                                          style="margin:
                                                          0.0px;padding:
                                                          0.0px;"><span
                                                          style="font-family:
                                                          Verdana;"><span
                                                          style="font-size:
                                                          12.0px;"><font
                                                          size="3"
                                                          face="monospace"><span>The
                                                          following
                                                          lines are
                                                          repeating in
                                                          the log.</span></font></span></span></p>
                                                          <p
                                                          style="margin:
                                                          0.0px;padding:
                                                          0.0px;"> </p>
                                                          <p
                                                          style="margin:
                                                          0.0px;padding:
                                                          0.0px;">ERROR:core:tcp_blocking_connect:
                                                          timeout 10 s
                                                          elapsed from
                                                          10 s<br>
                                                          ERROR:core:tcpconn_connect:
                                                          tcp_blocking_connect
                                                          failed<br>
                                                          ERROR:core:tcp_send:
                                                          connect failed<br>
                                                          ERROR:tm:msg_send:
                                                          tcp_send
                                                          failed</p>
                                                          <p
                                                          style="margin:
                                                          0.0px;padding:
                                                          0.0px;"> </p>
                                                          <p
                                                          style="margin:
                                                          0.0px;padding:
                                                          0.0px;"><a
                                                          moz-do-not-send="true"
class="moz-txt-link-freetext">INFO:core:send2child</a>: no free tcp
                                                          receiver,
                                                          connection
                                                          passed to the
                                                          least busy one</p>
                                                          <p
                                                          style="margin:
                                                          0.0px;padding:
                                                          0.0px;"> </p>
                                                          <p
                                                          style="margin:
                                                          0.0px;padding:
                                                          0.0px;"><span
                                                          style="font-family:
                                                          Verdana;"><span
                                                          style="font-size:
                                                          12.0px;"><font
                                                          size="3"
                                                          face="monospace"><span>Any
                                                          guess, what
                                                          could be the
                                                          reason?</span></font></span></span></p>
                                                          <p
                                                          style="margin:
                                                          0.0px;padding:
                                                          0.0px;"><span
                                                          style="font-family:
                                                          Verdana;"><span
                                                          style="font-size:
                                                          12.0px;"><font
                                                          size="3"
                                                          face="monospace"><span> </span></font></span></span></p>
                                                          <p
                                                          style="margin:
                                                          0.0px;padding:
                                                          0.0px;"><span
                                                          style="font-family:
                                                          Verdana;"><span
                                                          style="font-size:
                                                          12.0px;"><font
                                                          size="3"
                                                          face="monospace"><span>Best</span></font></span></span></p>
                                                          <p
                                                          style="margin:
                                                          0.0px;padding:
                                                          0.0px;"><span
                                                          style="font-family:
                                                          Verdana;"><span
                                                          style="font-size:
                                                          12.0px;"><font
                                                          size="3"
                                                          face="monospace"><span>--Gary</span></font></span></span></p>
                                                          <div><span
                                                          style="font-family:
                                                          Verdana;"><span
                                                          style="font-size:
                                                          12.0px;"> </span></span></div>
                                                          </div>
                                                          </div>
                                                           
                                                          <fieldset
                                                          class="mimeAttachmentHeader"> </fieldset>
                                                           
                                                          <pre>_______________________________________________
Users mailing list
<a moz-do-not-send="true" class="moz-txt-link-abbreviated">Users@lists.opensips.org</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users" target="_blank">http://lists.opensips.org/cgi-bin/mailman/listinfo/users</a><font size="3" face="Verdana, sans-serif, Arial, Trebuchet MS"><span style="font-style: normal;line-height: 20.796875px;white-space: normal;">
</span></font></pre>
                                                          </blockquote>
                                                        </div>
                                                      </div>
                                                    </div>
                                                  </div>
                                                </div>
                                              </div>
                                            </blockquote>
                                          </div>
                                        </div>
                                      </div>
                                    </div>
                                  </div>
                                   
                                  <fieldset class="mimeAttachmentHeader"> </fieldset>
                                   
                                  <pre>_______________________________________________
Users mailing list
<a moz-do-not-send="true" class="moz-txt-link-abbreviated">Users@lists.opensips.org</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users" target="_blank">http://lists.opensips.org/cgi-bin/mailman/listinfo/users</a><font size="3" face="Verdana, sans-serif, Arial, Trebuchet MS"><span style="font-style: normal;line-height: 20.8px;white-space: normal;">
</span></font></pre>
                                </blockquote>
                              </div>
                            </div>
                          </div>
                        </div>
                      </div>
                    </div>
                     
                    <fieldset class="mimeAttachmentHeader"> </fieldset>
                     
                    <pre>_______________________________________________
Users mailing list
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="Users@lists.opensips.org" target="_parent">Users@lists.opensips.org</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users" target="_blank">http://lists.opensips.org/cgi-bin/mailman/listinfo/users</a>
</pre>
                  </blockquote>
                  <br>
                  _______________________________________________ Users
                  mailing list <a class="moz-txt-link-abbreviated" href="mailto:Users@lists.opensips.org">Users@lists.opensips.org</a> <a
                    moz-do-not-send="true"
                    href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users"
                    target="_blank">http://lists.opensips.org/cgi-bin/mailman/listinfo/users</a></div>
              </div>
            </div>
          </div>
        </div>
      </div>
      <br>
      <fieldset class="mimeAttachmentHeader"></fieldset>
      <br>
      <pre wrap="">_______________________________________________
Users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Users@lists.opensips.org">Users@lists.opensips.org</a>
<a class="moz-txt-link-freetext" href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users">http://lists.opensips.org/cgi-bin/mailman/listinfo/users</a>
</pre>
    </blockquote>
    <br>
  </body>
</html>