<div dir="ltr">Bogdan, I've collected some "opensipsctl trap" output here for you, see line 2611 onwards:<div><br></div><div><a href="https://travis-ci.org/sippy/voiptests/jobs/261951017">https://travis-ci.org/sippy/voiptests/jobs/261951017</a><br></div><div><br></div><div>This is 10 seconds after starting the test, CANCELs arrive at about 2-3s mark, so it's well into "dead zone". Yet, I see some CANCEL messages in the traces, so it looks like some locking issue indeed.</div><div><br></div><div>Please let me know if it helps or if you need any more debug. For the time being, I am successfully running with 2.3 and master by reverting change 1eb4ec0f78f43f6ff546de49bc72e513876fb86b</div><div><br></div><div>-Max</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Aug 2, 2017 at 8:03 AM, Bogdan-Andrei Iancu <span dir="ltr"><<a href="mailto:bogdan@opensips.org" target="_blank">bogdan@opensips.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<tt>Thank you Maxim for your troubleshooting.<br>
<br>
If you say this code makes a difference in your test, I have the
say that the newly added locking may do the "blocking" . To see if
this is true, while OpenSIPS is "blocked" could you run several
times the "opensipsctl trap" ?<br>
<br>
Best regards,<br>
</tt><span class="">
<pre class="m_278828822372435955moz-signature" cols="72">Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
<a class="m_278828822372435955moz-txt-link-freetext" href="http://www.opensips-solutions.com" target="_blank">http://www.opensips-solutions.<wbr>com</a>
OpenSIPS Bootcamp 2017, Houston, US
<a class="m_278828822372435955moz-txt-link-freetext" href="http://opensips.org/training/OpenSIPS_Bootcamp_2017.html" target="_blank">http://opensips.org/training/<wbr>OpenSIPS_Bootcamp_2017.html</a>
</pre>
</span><div><div class="h5"><div class="m_278828822372435955moz-cite-prefix">On 08/02/2017 03:59 AM, Maxim Sobolev
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi Bogdan & Co, I think I've narrowed down
failure to a specific revision in the 2.3 branch, see <a href="https://travis-ci.org/sippy/voiptests/builds/260009545" target="_blank">https://travis-ci.org/<wbr>sippy/voiptests/builds/<wbr>260009545</a>:
<div><br>
</div>
<div>
<div>commit 1eb4ec0f78f43f6ff546de49bc72e5<wbr>13876fb86b</div>
<div>Author: Bogdan-Andrei Iancu <<a href="mailto:bogdan@opensips.org" target="_blank">bogdan@opensips.org</a>></div>
<div>Date: Thu Mar 9 15:37:01 2017 +0200</div>
<div><br>
</div>
<div> Added support for dynamic branch manipulation</div>
<div><br>
</div>
<div> New functions were added to allow remote injecting of
new branches into an ongoing transaction:</div>
<div> * t_inject_branches(source, flags) - injects new
branches;</div>
<div> * t_wait_for_new_branches() - instruct transaction to
wait more for possible new injections.</div>
</div>
<div><br>
</div>
<div>Looking at the diff in question, the following piece of
code looks suspicious, but it might be something else from
that change:<br>
</div>
<div><br>
</div>
<div>
<div>@@ -1244,7 +1301,18 @@ inline static int w_t_relay(
struct sip_msg *p_msg , char *proxy, char *flags)</div>
<div><br>
</div>
<div> update_cloned_msg_from_msg(
t->uas.request, p_msg);</div>
<div><br>
</div>
<div>- ret = t_forward_nonack( t, p_msg, p);</div>
<div>+ if (route_type==FAILURE_ROUTE) {</div>
<div>+ /* If called from failure route
we need reset the branch counter to</div>
<div>+ * ignore the previous set of
branches (already terminated) */</div>
<div>+ ret = t_forward_nonack( t, p_msg,
p, 1/*reset*/);</div>
<div>+ } else {</div>
<div>+ /* if called from request route
and the transaction was previously</div>
<div>+ * created, better lock here to
avoid any overlapping with</div>
<div>+ * branch injection from other
processes */</div>
<div>+ LOCK_REPLIES(t);</div>
<div>+ ret = t_forward_nonack( t, p_msg,
p, 1/*reset*/);</div>
<div>+ UNLOCK_REPLIES(t);</div>
<div>+ }</div>
<div> if (ret<=0 ) {</div>
<div> LM_ERR("t_forward_nonack
failed\n");</div>
<div> ret = t_relay_inerr2scripterr();</div>
<div>@@ -1262,7 +1330,8 @@ route_err:</div>
</div>
<div><br>
</div>
<div>The revisions right before that work, none after does.
Hopefully it will give you some clues. Let me know if any
other info is needed.</div>
<div><br>
</div>
<div>-Max</div>
</div>
</blockquote>
<br>
</div></div></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr">Maksym Sobolyev<br>Sippy Software, Inc.<br>Internet Telephony (VoIP) Experts<br>Tel (Canada): +1-778-783-0474<br>Tel (Toll-Free): +1-855-747-7779<br>Fax: +1-866-857-6942<br>Web: <a href="http://www.sippysoft.com" target="_blank">http://www.sippysoft.com</a><br>MSN: <a href="mailto:sales@sippysoft.com" target="_blank">sales@sippysoft.com</a><br>Skype: SippySoft<br></div></div>
</div>