<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    <p>Hi Pasan,</p>
    <p>I don't see anything in the logs or your cluster configuration
      which could indicate that the dialogs don't replicate. Are you
      sure that the problem is consistently reproducible and that the
      INVITE for the call in question receives a final reply? Dialogs
      are replicated when they get to the confirmed state, and judging
      from the logs, this doesn't appear to be happening for that call.</p>
    <p>Regards,<br>
    </p>
    <pre class="moz-signature" cols="72">Vlad Patrascu
OpenSIPS Developer
<a class="moz-txt-link-freetext" href="http://www.opensips-solutions.com">http://www.opensips-solutions.com</a></pre>
    <div class="moz-cite-prefix">On 27.02.2018 04:15, Pasan Meemaduma
      via Users wrote:<br>
    </div>
    <blockquote type="cite"
      cite="mid:2103041839.6602135.1519697741901@mail.yahoo.com">
      <div style="color:#000; background-color:#fff; font-family:標楷體,
        dfkai-sb;font-size:16px">
        <div id="yui_3_16_0_ym19_1_1519637009882_57129">Hi Vlad,</div>
        <div id="yui_3_16_0_ym19_1_1519637009882_57130"><br>
        </div>
        <div id="yui_3_16_0_ym19_1_1519637009882_57131" dir="ltr">I have
          sent you the full debug logs as requested. Also clusterer_list
          mi command gives following output when dialog replication
          stopped working.</div>
        <div dir="ltr" id="yui_3_16_0_ym19_1_1519637009882_57282"><br>
        </div>
        <div dir="ltr" id="yui_3_16_0_ym19_1_1519637009882_57283">node1>>opensipsctl
          fifo clusterer_list<br
            id="yui_3_16_0_ym19_1_1519637009882_57335">
          Cluster:: 1<br id="yui_3_16_0_ym19_1_1519637009882_57336">
                  Node:: 2 DB_ID=5 URL=bin:10.3.1.137:5566 Enabled=1
          Link_state=Up      Next_hop=2 Description=Node 2<br
            id="yui_3_16_0_ym19_1_1519637009882_57337">
          <br>
        </div>
        <div dir="ltr" id="yui_3_16_0_ym19_1_1519637009882_57284"><br>
        </div>
        <div dir="ltr" id="yui_3_16_0_ym19_1_1519637009882_57253"> node2>>
          opensipsctl fifo clusterer_list<br
            id="yui_3_16_0_ym19_1_1519637009882_57249">
          Cluster:: 1<br id="yui_3_16_0_ym19_1_1519637009882_57250">
                  Node:: 1 DB_ID=4 URL=bin:10.3.1.136:5566 Enabled=1
          Link_state=Up      Next_hop=1 Description=Node 1<br
            id="yui_3_16_0_ym19_1_1519637009882_57251">
          1<br>
        </div>
        <div dir="ltr" id="yui_3_16_0_ym19_1_1519637009882_57176"><br>
        </div>
        <div id="yui_3_16_0_ym19_1_1519637009882_57081"><span></span></div>
        <div class="qtdSeparateBR"><br>
          <br>
        </div>
        <div class="yahoo_quoted" style="display: block;">
          <div style="font-family: 標楷體, dfkai-sb; font-size: 16px;">
            <div style="font-family: HelveticaNeue, Helvetica Neue,
              Helvetica, Arial, Lucida Grande, Sans-Serif; font-size:
              16px;">
              <div dir="ltr"><font size="2" face="Arial"> On Monday, 26
                  February 2018, 18:02, Vlad Patrascu
                  <a class="moz-txt-link-rfc2396E" href="mailto:vladp@opensips.org"><vladp@opensips.org></a> wrote:<br>
                </font></div>
              <br>
              <br>
              <div class="y_msg_container">
                <div id="yiv0666455548">
                  <div>
                    <div>Hi,</div>
                    <div>Can you send the full logs for both nodes from
                      the time that node1 restarts onwards? Also, what
                      is the output of 'clusterer_list' mi command on
                      the instances?</div>
                    <div>Regards,<br clear="none">
                    </div>
                    <pre class="yiv0666455548moz-signature">Vlad Patrascu
OpenSIPS Developer
<a rel="nofollow" shape="rect" class="yiv0666455548moz-txt-link-freetext" target="_blank" href="http://www.opensips-solutions.com/" moz-do-not-send="true">http://www.opensips-solutions.com</a></pre>
                    <div class="yiv0666455548yqt8664929634"
                      id="yiv0666455548yqt71956">
                      <div class="yiv0666455548moz-cite-prefix">On
                        26.02.2018 08:38, Pasan Meemaduma via Users
                        wrote:<br clear="none">
                      </div>
                      <blockquote type="cite">
                        <div
                          style="color:#000;background-color:#fff;font-family:標楷體,
                          dfkai-sb;font-size:16px;">
                          <div
                            id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_6812">Hi
                            Guys,</div>
                          <div
                            id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_3101"><br
                              clear="none">
                          </div>
                          <div dir="ltr"
                            id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_6811">Its
                            me again :). I'm using dialog replication in
                            opensips 2.3.3 and it appears it doesn't
                            work after a node goes down and come back
                            online. The recovered node</div>
                          <div dir="ltr"
                            id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_3102">doesn't
                            seems to receiving dialog info via binary
                            interface.</div>
                          <div dir="ltr"
                            id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_3103"><br
                              clear="none">
                          </div>
                          <div dir="ltr"
                            id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_6810">I
                            have node1 and node2 with dialog replication
                            on. everything works fine and if I shutdown
                            node1 and bring it back online after a while</div>
                          <div dir="ltr"
                            id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_3104">node2
                            doesn't send new call dialog info via binary
                            interface. <br clear="none">
                          </div>
                          <div dir="ltr"
                            id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4987"><br
                              clear="none">
                          </div>
                          <div dir="ltr"
                            id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4986">I
                            have attach the debug trace from both nodes,
                            if you need anything else let me know. I'm
                            also using an anycast ip as the service ip.</div>
                          <div dir="ltr"
                            id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4996"><br
                              clear="none">
                          </div>
                          <div dir="ltr"
                            id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_3105"><br
                              clear="none">
                          </div>
                          <div dir="ltr"
                            id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4985">on
                            node2 for new call</div>
                          <div dir="ltr"
                            id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4997"><br
                              clear="none">
                          </div>
                          <div dir="ltr"
                            id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4847">Feb
                            26 17:28:35 voip2-sip23b
                            /usr/sbin/opensips[4703]:
                            DBG:dialog:build_new_dlg: new dialog
                            0x7f188afd3bf8
(c=ktQ0Pkdwz50qGYjED6Brpw..,f=sip:XXXXXXXXX@somedomain;transport=UDP,t=sip:+YYYYYYYYYYY@somedomain;transport=UDP,ft=2b161508)
                            on hash 2317<br
                              id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4888"
                              clear="none">
                            Feb 26 17:28:35 voip2-sip23b
                            /usr/sbin/opensips[4703]:
                            DBG:dialog:init_leg_info: route_set ,
                            contact <a rel="nofollow" shape="rect"
                              class="yiv0666455548moz-txt-link-abbreviated"
ymailto="mailto:sip:XXXXXXXXX@192.168.27.11:56419;transport=UDP"
                              target="_blank"
                              href="mailto:sip:XXXXXXXXX@192.168.27.11:56419;transport=UDP"
                              moz-do-not-send="true">sip:XXXXXXXXX@192.168.27.11:56419;transport=UDP</a>,
                            cseq 2 and bind_addr udp:10.3.3.1:5060<br
                              id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4889"
                              clear="none">
                            Feb 26 17:28:35 voip2-sip23b
                            /usr/sbin/opensips[4703]:
                            DBG:dialog:dlg_add_leg_info: set leg 0 for
                            0x7f188afd3bf8: tag=<2b161508>
                            rcseq=<0><br
                              id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4890"
                              clear="none">
                            Feb 26 17:28:35 voip2-sip23b
                            /usr/sbin/opensips[4703]:
                            DBG:dialog:link_dlg: ref dlg 0x7f188afd3bf8
                            with 3 -> 3 in h_entry 0x7f188afc3828 -
                            2317<br
                              id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4891"
                              clear="none">
                            Feb 26 17:28:35 voip2-sip23b
                            /usr/sbin/opensips[4703]:
                            DBG:dialog:new_dlg_val: inserting
                            <exe-callid>=<45ad4c76-1abe-11e8-9410-831894b67d0c><br
id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4892" clear="none">
                            Feb 26 17:28:35 voip2-sip23b
                            /usr/sbin/opensips[4703]:
                            DBG:dialog:dlg_onreq: t hash_index = 47425,
                            t label = 2069082013<br
                              id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4893"
                              clear="none">
                            Feb 26 17:28:35 voip2-sip23b
                            /usr/sbin/opensips[4703]:
                            DBG:dialog:dlg_update_contact: Updated
                            dialog 0x7f188afd3bf8 contact to <a
                              rel="nofollow" shape="rect"
                              class="yiv0666455548moz-txt-link-rfc2396E"
ymailto="mailto:sip:XXXXXXXXX@192.168.27.11:56419;transport=UDP"
                              target="_blank"
                              href="mailto:sip:XXXXXXXXX@192.168.27.11:56419;transport=UDP"
                              moz-do-not-send="true"><sip:XXXXXXXXX@192.168.27.11:56419;transport=UDP></a><br
id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4894" clear="none">
                            Feb 26 17:28:35 voip2-sip23b
                            /usr/sbin/opensips[4703]:
                            DBG:dialog:unref_dlg: unref dlg
                            0x7f188afd3bf8 with 1 -> 2 in entry
                            0x7f188afc3828<br
                              id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4895"
                              clear="none">
                            Feb 26 17:28:52 voip2-sip23b
                            /usr/sbin/opensips[4702]:
                            DBG:dialog:ref_dlg: ref dlg 0x7f188afd3bf8
                            with 1 -> 3<br
                              id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4896"
                              clear="none">
                            Feb 26 17:28:52 voip2-sip23b
                            /usr/sbin/opensips[4702]:
                            DBG:dialog:ref_dlg: ref dlg 0x7f188afd3bf8
                            with 1 -> 4<br
                              id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4897"
                              clear="none">
                            Feb 26 17:28:52 voip2-sip23b
                            /usr/sbin/opensips[4702]:
                            DBG:dialog:unref_dlg: unref dlg
                            0x7f188afd3bf8 with 1 -> 3 in entry
                            0x7f188afc3828<br
                              id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4898"
                              clear="none">
                            Feb 26 17:28:56 voip2-sip23b
                            /usr/sbin/opensips[4711]:
                            DBG:dialog:unref_dlg: unref dlg
                            0x7f188afd3bf8 with 1 -> 2 in entry
                            0x7f188afc3828<br
                              id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4899"
                              clear="none">
                            Feb 26 17:29:05 voip2-sip23b
                            /usr/sbin/opensips[4711]:
                            DBG:dialog:next_state_dlg: dialog
                            0x7f188afd3bf8 changed from state 1 to state
                            5, due event 4<br
                              id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4900"
                              clear="none">
                            Feb 26 17:29:05 voip2-sip23b
                            /usr/sbin/opensips[4711]:
                            DBG:dialog:dlg_onreply: dialog
                            0x7f188afd3bf8 failed (negative reply)<br
                              id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4901"
                              clear="none">
                            Feb 26 17:29:05 voip2-sip23b
                            /usr/sbin/opensips[4711]:
                            DBG:dialog:unref_dlg: unref dlg
                            0x7f188afd3bf8 with 1 -> 1 in entry
                            0x7f188afc3828<br
                              id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4902"
                              clear="none">
                            Feb 26 17:29:11 voip2-sip23b
                            /usr/sbin/opensips[4711]:
                            DBG:dialog:next_state_dlg: dialog
                            0x7f188afd3bf8 changed from state 5 to state
                            5, due event 1<br
                              id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4903"
                              clear="none">
                            Feb 26 17:29:11 voip2-sip23b
                            /usr/sbin/opensips[4711]:
                            DBG:dialog:unref_dlg: unref dlg
                            0x7f188afd3bf8 with 1 -> 0 in entry
                            0x7f188afc3828<br
                              id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4904"
                              clear="none">
                            Feb 26 17:29:11 voip2-sip23b
                            /usr/sbin/opensips[4711]:
                            DBG:dialog:unref_dlg: ref <=0 for dialog
                            0x7f188afd3bf8<br
                              id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4905"
                              clear="none">
                            Feb 26 17:29:11 voip2-sip23b
                            /usr/sbin/opensips[4711]:
                            DBG:dialog:destroy_dlg: destroying dialog
                            0x7f188afd3bf8<br
                              id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4906"
                              clear="none">
                            Feb 26 17:29:11 voip2-sip23b
                            /usr/sbin/opensips[4711]:
                            DBG:dialog:destroy_dlg: dlg expired or not
                            in list - dlg 0x7f188afd3bf8
                            [2317:677053409] with clid
                            'ktQ0Pkdwz50qGYjED6Brpw..' and tags
                            '2b161508' 'NULL'<br
                              id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4907"
                              clear="none">
                          </div>
                          <div dir="ltr"
                            id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4908"><br
                              clear="none">
                          </div>
                          <div dir="ltr"
                            id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_6813"><br
                              clear="none">
                          </div>
                          <div dir="ltr"
                            id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_6814">on
                            node1 <br clear="none">
                          </div>
                          <div dir="ltr"
                            id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_6815"><br
                              clear="none">
                          </div>
                          <div dir="ltr"
                            id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_6767">Feb
                            26 17:28:41 voip2-sip23a
                            /usr/sbin/opensips[15820]:
                            DBG:dialog:dlg_onroute: route param is
                            'd09.1e30b582' (len=12)<br
                              id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4970"
                              clear="none">
                            Feb 26 17:28:41 voip2-sip23a
                            /usr/sbin/opensips[15820]:
                            DBG:dialog:lookup_dlg: no dialog
                            id=677053409 found on entry 2317<br
                              id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4971"
                              clear="none">
                            Feb 26 17:28:41 voip2-sip23a
                            /usr/sbin/opensips[15820]:
                            DBG:dialog:dlg_onroute: unable to find
                            dialog for BYE with route param
                            'd09.1e30b582'<br
                              id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4972"
                              clear="none">
                            Feb 26 17:28:41 voip2-sip23a
                            /usr/sbin/opensips[15822]:
                            DBG:dialog:dlg_onroute: route param is
                            'd09.1e30b582' (len=12)<br
                              id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4973"
                              clear="none">
                            Feb 26 17:28:41 voip2-sip23a
                            /usr/sbin/opensips[15822]:
                            DBG:dialog:lookup_dlg: no dialog
                            id=677053409 found on entry 2317<br
                              id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4974"
                              clear="none">
                            Feb 26 17:28:41 voip2-sip23a
                            /usr/sbin/opensips[15822]:
                            DBG:dialog:dlg_onroute: unable to find
                            dialog for BYE with route param
                            'd09.1e30b582'<br
                              id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4975"
                              clear="none">
                            Feb 26 17:28:41 voip2-sip23a
                            /usr/sbin/opensips[15822]:
                            DBG:dialog:dlg_onroute: route param is
                            'd09.1e30b582' (len=12)<br
                              id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4976"
                              clear="none">
                            Feb 26 17:28:41 voip2-sip23a
                            /usr/sbin/opensips[15822]:
                            DBG:dialog:lookup_dlg: no dialog
                            id=677053409 found on entry 2317<br
                              id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4977"
                              clear="none">
                            Feb 26 17:28:41 voip2-sip23a
                            /usr/sbin/opensips[15822]:
                            DBG:dialog:dlg_onroute: unable to find
                            dialog for BYE with route param
                            'd09.1e30b582'<br
                              id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4978"
                              clear="none">
                            Feb 26 17:28:42 voip2-sip23a
                            /usr/sbin/opensips[15820]:
                            DBG:dialog:dlg_onroute: route param is
                            'd09.1e30b582' (len=12)<br
                              id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4979"
                              clear="none">
                            Feb 26 17:28:42 voip2-sip23a
                            /usr/sbin/opensips[15820]:
                            DBG:dialog:lookup_dlg: no dialog
                            id=677053409 found on entry 2317<br
                              id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4980"
                              clear="none">
                            Feb 26 17:28:42 voip2-sip23a
                            /usr/sbin/opensips[15820]:
                            DBG:dialog:dlg_onroute: unable to find
                            dialog for BYE with route param
                            'd09.1e30b582'<br
                              id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4981"
                              clear="none">
                            Feb 26 17:28:42 voip2-sip23a
                            /usr/sbin/opensips[15820]:
                            DBG:dialog:dlg_onroute: route param is
                            'd09.1e30b582' (len=12)<br
                              id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4982"
                              clear="none">
                            Feb 26 17:28:42 voip2-sip23a
                            /usr/sbin/opensips[15820]:
                            DBG:dialog:lookup_dlg: no dialog
                            id=677053409 found on entry 2317<br
                              id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_4983"
                              clear="none">
                            <br clear="none">
                          </div>
                          <div dir="ltr"
                            id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_6769"><br
                              clear="none">
                          </div>
                          <div dir="ltr"
                            id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_6770">If
                            I purposely take down node1 again and try to
                            make a call I can see that it indicates the
                            failure to replicate dialog info on the
                            cluster as below,</div>
                          <div dir="ltr"
                            id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_6793"><br
                              clear="none">
                          </div>
                          <div dir="ltr"
                            id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_6800">Feb
                            26 17:27:01 voip2-sip23b
                            /usr/sbin/opensips[4701]:
                            ERROR:dialog:replicate_dialog_updated: All
                            destinations in cluster: 1 are down or
                            probing<br
                              id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_6798"
                              clear="none">
                            Feb 26 17:27:01 voip2-sip23b
                            /usr/sbin/opensips[4701]:
                            ERROR:dialog:replicate_dialog_updated:
                            Failed to replicate updated dialog<br
                              id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_6799"
                              clear="none">
                          </div>
                          <div dir="ltr"
                            id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_6801"><br
                              clear="none">
                          </div>
                          <div
                            id="yiv0666455548yui_3_16_0_ym19_1_1519626563729_3106"><br
                              clear="none">
                          </div>
                        </div>
                        <br clear="none">
                        <fieldset
                          class="yiv0666455548mimeAttachmentHeader"></fieldset>
                        <br clear="none">
                        <pre>_______________________________________________
Users mailing list
<a rel="nofollow" shape="rect" class="yiv0666455548moz-txt-link-abbreviated" ymailto="mailto:Users@lists.opensips.org" target="_blank" href="mailto:Users@lists.opensips.org" moz-do-not-send="true">Users@lists.opensips.org</a>
<a rel="nofollow" shape="rect" class="yiv0666455548moz-txt-link-freetext" target="_blank" href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users" moz-do-not-send="true">http://lists.opensips.org/cgi-bin/mailman/listinfo/users</a>
</pre>
                      </blockquote>
                    </div>
                    <br clear="none">
                  </div>
                </div>
                <div class="yqt8664929634" id="yqt52019">_______________________________________________<br
                    clear="none">
                  Users mailing list<br clear="none">
                  <a shape="rect"
                    ymailto="mailto:Users@lists.opensips.org"
                    href="mailto:Users@lists.opensips.org"
                    moz-do-not-send="true">Users@lists.opensips.org</a><br
                    clear="none">
                  <a shape="rect"
                    href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users"
                    target="_blank" moz-do-not-send="true">http://lists.opensips.org/cgi-bin/mailman/listinfo/users</a><br
                    clear="none">
                </div>
                <br>
                <br>
              </div>
            </div>
          </div>
        </div>
      </div>
      <br>
      <fieldset class="mimeAttachmentHeader"></fieldset>
      <br>
      <pre wrap="">_______________________________________________
Users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Users@lists.opensips.org">Users@lists.opensips.org</a>
<a class="moz-txt-link-freetext" href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users">http://lists.opensips.org/cgi-bin/mailman/listinfo/users</a>
</pre>
    </blockquote>
    <br>
  </body>
</html>