<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<tt>Hi <tt>Rahul,<br>
<br>
<tt>Any lo<tt>gs will be useful (use some pastebin or file <tt>transfer
to upload).<br>
<br>
<tt>On your observations : when a connection timeout does
expire (v<tt>ia __tcpconn_lifetime<tt>()),
reactor_del_all() removes the fd with both
IO_WATCH_READ<tt> and </tt>IO_WATCH_WRIT<tt>E
flags.<tt> <tt>While in the <tt>logs it shows
"1", removing with READ only.</tt></tt></tt><br>
</tt></tt></tt></tt></tt></tt></tt><br>
<tt>Going back </tt><br>
</tt></tt>
<pre class="moz-signature" cols="72">Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
<a class="moz-txt-link-freetext" href="http://www.opensips-solutions.com">http://www.opensips-solutions.com</a></pre>
<div class="moz-cite-prefix">On 04.08.2015 16:29, Gupta, Rahul
wrote:<br>
</div>
<blockquote
cite="mid:5D7DF326E497124DACCD6F9DD6A1A2A089CDD442@NWKNJEXMBX1.corp.root.ipc.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
<meta name="Generator" content="Microsoft Word 14 (filtered
medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
@font-face
{font-family:"Times New Roman \, serif";
panose-1:0 0 0 0 0 0 0 0 0 0;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
color:black;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman","serif";
color:black;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";
color:black;}
tt
{mso-style-priority:99;
font-family:"Courier New";}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0in;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";
color:black;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
color:black;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Consolas;
color:black;}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";
color:black;}
span.EmailStyle24
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:windowtext;}
span.EmailStyle25
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle26
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle27
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle28
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal"><span style="color:#1F497D">Hi Bogdan,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">I tried the
scenario with 1.11.3 as well as 1.11.5 latest code and could
reproduce it consistently. I can turn on the higher debug
level logging and provide you the log file. Let me know
where can I upload these logs ?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Thanks<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Rahul <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF
1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span
style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext">From:</span></b><span
style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext">
Bogdan-Andrei Iancu [<a class="moz-txt-link-freetext" href="mailto:bogdan@opensips.org">mailto:bogdan@opensips.org</a>]
<br>
<b>Sent:</b> Tuesday, August 04, 2015 8:18 AM<br>
<b>To:</b> Gupta, Rahul; <a class="moz-txt-link-abbreviated" href="mailto:users@lists.opensips.org">users@lists.opensips.org</a><br>
<b>Subject:</b> Re: ERROR:core:io_watch_del: BUG -
trying to del fd 38 with flags 2 1<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><tt><span style="font-size:10.0pt">Hi
Rahul,</span></tt><span
style="font-size:10.0pt;font-family:"Courier New""><br>
<br>
<tt>I tried to reproduce your scenario during the last 2
days. Thanks to the detailed instructions and stepping, I
mananged to get (as described) to the last step. But
instead of getting the error you mentioned, the tcp conn
timeout event was properly handled:</tt><br>
<br>
<tt>[23203] DBG:core:__tcpconn_lifetime: timeout for hash=3
- 0x7f94f298fea0 (322 > 317)</tt><br>
<tt>[23203] DBG:core:io_watch_del: [TCP_main] io_watch_del
op on index -1 29 (0x8828a0, 29, -1, 0x10,0x3) fd_no=22
called</tt><br>
<br>
<tt>Are you sure you are using the latest GIT code (I tested
both 2.1 and 1.11).</tt><br>
<br>
<tt>Regards,</tt><br>
<br>
<br>
</span><o:p></o:p></p>
<pre>Bogdan-Andrei Iancu<o:p></o:p></pre>
<pre>OpenSIPS Founder and Developer<o:p></o:p></pre>
<pre><a moz-do-not-send="true" href="http://www.opensips-solutions.com">http://www.opensips-solutions.com</a><o:p></o:p></pre>
<div>
<p class="MsoNormal">On 31.07.2015 17:40, Gupta, Rahul wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal"><span style="color:#1F497D">Hi Bogdan,
thanks for looking into this issue. Let me know if anyway
I can help to get to the bottom of this issue, like
providing logs etc</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Thanks</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Rahul</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #B5C4DF
1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span
style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext">From:</span></b><span
style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext">
Bogdan-Andrei Iancu [<a moz-do-not-send="true"
href="mailto:bogdan@opensips.org">mailto:bogdan@opensips.org</a>]
<br>
<b>Sent:</b> Tuesday, July 28, 2015 1:13 PM<br>
<b>To:</b> Gupta, Rahul; <a moz-do-not-send="true"
href="mailto:users@lists.opensips.org">users@lists.opensips.org</a><br>
<b>Subject:</b> Re: ERROR:core:io_watch_del: BUG -
trying to del fd 38 with flags 2 1</span><o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><tt><span style="font-size:10.0pt">Hi
Rahul,</span></tt><span
style="font-size:10.0pt;font-family:"Courier
New""><br>
<br>
<tt>These are great news - that you find a way to
reproduce it. Let me do it and start debugging.</tt><br>
<br>
<tt>I really do appreciate your effort in dealing with
this error.</tt><br>
<br>
<tt>Regards,</tt><br>
<br>
<br>
</span><o:p></o:p></p>
<pre>Bogdan-Andrei Iancu<o:p></o:p></pre>
<pre>OpenSIPS Founder and Developer<o:p></o:p></pre>
<pre><a moz-do-not-send="true" href="http://www.opensips-solutions.com">http://www.opensips-solutions.com</a><o:p></o:p></pre>
<div>
<p class="MsoNormal">On 24.07.2015 00:19, Gupta, Rahul
wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal"><span style="color:#1F497D">Hi Bogdan,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Almost a
month ago, I have raised the issue regarding the
opensips log file filling up with the following message
</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">ERROR:core:io_watch_del:
BUG - trying to del fd 38 with flags 2 1</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Now I can
reproduce it every time with the following scenario:</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in"><span
style="color:#1F497D">1)</span><span
style="font-size:7.0pt">
</span><span style="color:#1F497D">UAC and UAS are
connected to opensips proxy with TCP transport</span><o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in"><span
style="color:#1F497D">2)</span><span
style="font-size:7.0pt">
</span><span style="color:#1F497D">Connections looks good</span><o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in"><span
style="color:#1F497D">3)</span><span
style="font-size:7.0pt">
</span><span style="color:#1F497D">Netstat shows the
connection to UAC in ESTABLISHED state and the
tcp_conn_lists of opensips also looks fine for that UAC</span><o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in"><span
style="color:#1F497D">4)</span><span
style="font-size:7.0pt">
</span><span style="color:#1F497D">Now unplug the Ethernet
cable on UAC</span><o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in"><span
style="color:#1F497D">5)</span><span
style="font-size:7.0pt">
</span><span style="color:#1F497D">After tcp connection
timeout (set to 5 mins in opensips.cfg), the tcp
connection goes away from netstat as well as from
tcp_conn_lists</span><o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in"><span
style="color:#1F497D">6)</span><span
style="font-size:7.0pt">
</span><span style="color:#1F497D">UAS tries another tcp
call to the UAC which is still unplugged</span><o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in"><span
style="color:#1F497D">7)</span><span
style="font-size:7.0pt">
</span><span style="color:#1F497D">tcp_conn_lists shows
the tcp connection to the UAC and netstat shows the
connection in SYN_SENT state</span><o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in"><span
style="color:#1F497D">8)</span><span
style="font-size:7.0pt">
</span><span style="color:#1F497D">After the tcp
connection timeout (set to 5 mins in opensips.cfg), the
connection goes away from netstat however it remains
there in tcp_conn_lists and at that moment the “BUG -
trying to del fd 38 with flags 2 1” starts printing in
infinite loop.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">I looked at
the source code and observed the following</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in"><span
style="color:#1F497D">1)</span><span
style="font-size:7.0pt">
</span><span style="color:#1F497D">When the network cable
is plugged in </span><o:p></o:p></p>
<p class="MsoListParagraph"><span style="color:#1F497D">The
io_watch_add happens with flag IO_WATCH_READ in
tcp_main.c when the command is CONN_NEW</span><o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in"><span
style="color:#1F497D">2)</span><span
style="font-size:7.0pt">
</span><span style="color:#1F497D">When the network cable
is unplugged</span><o:p></o:p></p>
<p class="MsoListParagraph"><span style="color:#1F497D">The
io_watch_add happens with flag IO_WATCH_WRITE in
tcp_main.c when the command is ASYNC_CONNECT</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in"><span
style="color:#1F497D">3)</span><span
style="font-size:7.0pt">
</span><span style="color:#1F497D">While doing
io_watch_del after timeout, from handle_tcpconn_ev
method, io_watch_del always uses IO_WATCH_READ to delete
the fd, which gives this error in case of unplugged
cable.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Please look
into this and suggest how can I fix this issue ?</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Thanks</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Rahul Gupta</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoListParagraph"><span style="color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<div>
<p><span
style="font-size:10.0pt;font-family:"Arial","sans-serif";color:gray">------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------</span><o:p></o:p></p>
<p><span
style="font-size:10.0pt;font-family:"Arial","sans-serif";color:gray">DISCLAIMER:
This e-mail may contain information that is
confidential, privileged or otherwise protected from
disclosure. If you are not an intended recipient of
this e-mail, do not duplicate or redistribute it by
any means. Please delete it and any attachments and
notify the sender that you have received it in error.
Unintended recipients are prohibited from taking
action on the basis of information in this
e-mail.E-mail messages may contain computer viruses or
other defects, may not be accurately replicated on
other systems, or may be intercepted, deleted or
interfered with without the knowledge of the sender or
the intended recipient. If you are not comfortable
with the risks associated with e-mail messages, you
may decide not to use e-mail to communicate with IPC.
IPC reserves the right, to the extent and under
circumstances permitted by applicable law, to retain,
monitor and intercept e-mail messages to and from its
systems.</span><o:p></o:p></p>
</div>
</blockquote>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New Roman
, serif","serif""> </span><o:p></o:p></p>
</blockquote>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman","serif""><o:p> </o:p></span></p>
</div>
</blockquote>
<br>
</body>
</html>