<html><head><style type='text/css'>p { margin: 0; }</style></head><body><div style='font-family: courier new,courier,monaco,monospace,sans-serif; font-size: 12pt; color: #000000'>Hello Bogdan,<div>Yes, I will do pull today and test again auto scaling feature.</div><div><br></div><div>Thank you.</div><div><br></div><div>volga629 <br><hr id="zwchr"><div style="color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;"><b>From: </b>"Bogdan-Andrei Iancu" <bogdan@opensips.org><br><b>To: </b>"Slava Bendersky" <volga629@networklab.ca><br><b>Cc: </b>"OpenSIPS users mailling list" <users@lists.opensips.org><br><b>Sent: </b>Friday, May 10, 2019 5:02:48 AM<br><b>Subject: </b>Re: [OpenSIPS-Users] auto scaling<br><br>
  
    
  
  
    <tt>Hi Slava,<br>
      <br>
      There were some fixes in the auto-scaling in the last weeks, could
      you confirm or not your issue against the latest 3.0 code ?<br>
      <br>
      Regards,<br>
    </tt>
    <pre class="moz-signature">Bogdan-Andrei Iancu

OpenSIPS Founder and Developer
  <a class="moz-txt-link-freetext" href="https://www.opensips-solutions.com" target="_blank">https://www.opensips-solutions.com</a>
OpenSIPS Summit 2019
  <a class="moz-txt-link-freetext" href="https://www.opensips.org/events/Summit-2019Amsterdam/" target="_blank">https://www.opensips.org/events/Summit-2019Amsterdam/</a>
</pre>
    <div class="moz-cite-prefix">On 03/15/2019 12:47 PM, Slava Bendersky
      wrote:<br>
    </div>
    <blockquote cite="mid:1824870661.406.1552646859543.JavaMail.bendersky@nlws01.networklab.lan">
      <style>p { margin: 0; }</style>
      <div style="font-family: courier
        new,courier,monaco,monospace,sans-serif; font-size: 12pt; color:
        #000000"><font face="courier new, courier, monaco, monospace,
          sans-serif">Hello Bogdan,</font>
        <div style="font-family: "courier new", courier,
          monaco, monospace, sans-serif;"><span style="font-family:
            monospace;">Upscaling error is didn't show up any more in
            logs at least I didn't so it. Might need trigger bigger
            load.</span></div>
        <div><font face="monospace"><br>
          </font></div>
        <div><span style="font-family: monospace;"><b>do you do
              auto-scaling only for TCP ?</b></span></div>
        <div><font face="monospace"><br>
          </font></div>
        <div><font face="monospace">Our  setup in most  multi homed, as
            example public side TCP private side UDP</font></div>
        <div><font face="monospace"><br>
          </font></div>
        <div><font face="monospace"><br>
          </font></div>
        <div><span style="font-family: monospace;"><b>Now, about the TCP
              downscaling - I guess you see as terminated by OpenSIPS
              the TCP conns anchored in the process that is terminated ?</b></span></div>
        <div><font face="courier new, courier, monaco, monospace,
            sans-serif"><br>
            Yes, that how is look like end result that opensips start
            blocking TCP connections. If specific process  handle sip
            registration via tcp and opensips trying close to down scale
            end point need send re register in order do not loose
            registration. Other wise it cause stale connections. Just
            close properly connection not enough I think.</font></div>
        <div><font face="courier new, courier, monaco, monospace,
            sans-serif"><br>
          </font></div>
        <div><font face="courier new, courier, monaco, monospace,
            sans-serif"><br>
          </font></div>
        <div><font face="courier new, courier, monaco, monospace,
            sans-serif">volga629</font></div>
        <div><br>
        </div>
        <div><br>
        </div>
        <div><br>
        </div>
        <div><br>
          <hr id="zwchr" style="font-family: "courier new",
            courier, monaco, monospace, sans-serif;">
          <div style="font-family: Helvetica, Arial, sans-serif; color:
            rgb(0, 0, 0); font-weight: normal; font-style: normal;
            text-decoration: none; font-size: 12pt;"><b>From: </b>"Bogdan-Andrei
            Iancu" <a class="moz-txt-link-rfc2396E" href="mailto:bogdan@opensips.org" target="_blank"><bogdan@opensips.org></a><br>
            <b>To: </b>"OpenSIPS users mailling list"
            <a class="moz-txt-link-rfc2396E" href="mailto:users@lists.opensips.org" target="_blank"><users@lists.opensips.org></a>, "Slava Bendersky"
            <a class="moz-txt-link-rfc2396E" href="mailto:volga629@networklab.ca" target="_blank"><volga629@networklab.ca></a><br>
            <b>Sent: </b>Thursday, March 14, 2019 1:45:20 PM<br>
            <b>Subject: </b>Re: [OpenSIPS-Users] auto scaling<br>
            <br>
            <tt>Slava,<br>
              <br>
              What about the upscaling error you originally reported ?
              is there any way to reproduce it ? do you do auto-scaling
              only for TCP (no UDP,timer) ?<br>
              <br>
              Now, about the TCP downscaling - I guess you see as
              terminated by OpenSIPS the TCP conns anchored in the
              process that is terminated ? Is is a proper close of the
              connection ? or ?<br>
              <br>
              Regards,<br>
            </tt>
            <pre class="moz-signature">Bogdan-Andrei Iancu

OpenSIPS Founder and Developer
  <a class="moz-txt-link-freetext" href="https://www.opensips-solutions.com" target="_blank">https://www.opensips-solutions.com</a>
OpenSIPS Summit 2019
  <a class="moz-txt-link-freetext" href="https://www.opensips.org/events/Summit-2019Amsterdam/" target="_blank">https://www.opensips.org/events/Summit-2019Amsterdam/</a>
</pre>
            <div class="moz-cite-prefix">On 03/14/2019 02:14 PM, Slava
              Bendersky wrote:<br>
            </div>
            <blockquote cite="mid:902388553.152.1552565672232.JavaMail.bendersky@nlws01.networklab.lan">
              <style>p { margin: 0; }</style>
              <div style="font-family: courier
                new,courier,monaco,monospace,sans-serif; font-size:
                12pt; color: #000000">Hello Bogdan,
                <div>After we doing more testing with auto scaling we
                  observed issue with TCP, where opensips try scale down
                  the processes it tear down them and break TCP
                  connections with end points and that cause blocked
                  connection state where opensips stops process TCP
                  connections.</div>
                <div><br>
                </div>
                <div>volga629 <br>
                  <br>
                  <hr id="zwchr">
                  <div style="color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;"><b>From:
                    </b>"Slava Bendersky" <a class="moz-txt-link-rfc2396E" href="mailto:volga629@networklab.ca" target="_blank"><volga629@networklab.ca></a><br>
                    <b>To: </b>"Bogdan-Andrei Iancu" <a class="moz-txt-link-rfc2396E" href="mailto:bogdan@opensips.org" target="_blank"><bogdan@opensips.org></a><br>
                    <b>Cc: </b>"OpenSIPS users mailling list" <a class="moz-txt-link-rfc2396E" href="mailto:users@lists.opensips.org" target="_blank"><users@lists.opensips.org></a><br>
                    <b>Sent: </b>Wednesday, March 13, 2019 12:21:14 AM<br>
                    <b>Subject: </b>Re: [OpenSIPS-Users] auto scaling<br>
                    <br>
                    <style>p { margin: 0; }</style>
                    <div style="font-family: courier
                      new,courier,monaco,monospace,sans-serif;
                      font-size: 12pt; color: #000000">I will do pull
                      today.
                      <div><br>
                      </div>
                      <div>volga629<br>
                        <hr id="zwchr">
                        <div style="color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;"><b>From:
                          </b>"Bogdan-Andrei Iancu" <a class="moz-txt-link-rfc2396E" href="mailto:bogdan@opensips.org" target="_blank"><bogdan@opensips.org></a><br>
                          <b>To: </b>"Slava Bendersky" <a class="moz-txt-link-rfc2396E" href="mailto:volga629@networklab.ca" target="_blank"><volga629@networklab.ca></a><br>
                          <b>Cc: </b>"OpenSIPS users mailling list" <a class="moz-txt-link-rfc2396E" href="mailto:users@lists.opensips.org" target="_blank"><users@lists.opensips.org></a><br>
                          <b>Sent: </b>Tuesday, March 12, 2019 4:52:06
                          AM<br>
                          <b>Subject: </b>Re: [OpenSIPS-Users] auto
                          scaling<br>
                          <br>
                          <tt>Slava,<br>
                            <br>
                            Have you pulled my change on the improved
                            log (just update from master) ? and post
                            here the new output you get for the error.<br>
                            <br>
                            Thanks,<br>
                          </tt>
                          <pre class="moz-signature">Bogdan-Andrei Iancu

OpenSIPS Founder and Developer
  <a class="moz-txt-link-freetext" href="https://www.opensips-solutions.com" target="_blank">https://www.opensips-solutions.com</a>
OpenSIPS Summit 2019
  <a class="moz-txt-link-freetext" href="https://www.opensips.org/events/Summit-2019Amsterdam/" target="_blank">https://www.opensips.org/events/Summit-2019Amsterdam/</a>
</pre>
                          <div class="moz-cite-prefix">On 03/12/2019
                            05:51 AM, Slava Bendersky wrote:<br>
                          </div>
                          <blockquote cite="mid:423067196.274.1552362703793.JavaMail.bendersky@nlws01.networklab.lan">
                            <style>p { margin: 0; }</style>
                            <div style="font-family: courier
                              new,courier,monaco,monospace,sans-serif;
                              font-size: 12pt; color: #000000">Hello
                              Bogdan,
                              <div>My question how to calculate
                                correctly </div>
                              <div><br>
                              </div>
                              <div><b>tcp_workers</b></div>
                              <div><b>udp_workers</b></div>
                              <div><b><br>
                                </b></div>
                              <div>to properly assign with in  auto
                                scaling profile. Which parameters need
                                use as base ?</div>
                              <div><br>
                              </div>
                              <div>As example if vm 4GB memory up what
                                will be max number for scale up
                                processes and what should be value for
                                tcp and udp workers.</div>
                              <div>It message is appear in log on first
                                attempt to fork in scale up.</div>
                              <div><br>
                              </div>
                              <div>volga629 </div>
                              <div>   </div>
                              <div>
                                <hr id="zwchr">
                                <div style="color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;"><b>From:
                                  </b>"Bogdan-Andrei Iancu" <a class="moz-txt-link-rfc2396E" href="mailto:bogdan@opensips.org" target="_blank"><bogdan@opensips.org></a><br>
                                  <b>To: </b>"OpenSIPS users mailling
                                  list" <a class="moz-txt-link-rfc2396E" href="mailto:users@lists.opensips.org" target="_blank"><users@lists.opensips.org></a>,
                                  "Slava Bendersky" <a class="moz-txt-link-rfc2396E" href="mailto:volga629@networklab.ca" target="_blank"><volga629@networklab.ca></a><br>
                                  <b>Sent: </b>Monday, March 11, 2019
                                  4:44:57 AM<br>
                                  <b>Subject: </b>Re: [OpenSIPS-Users]
                                  auto scaling<br>
                                  <br>
                                  <tt>Hi Slava,<br>
                                    <br>
                                    Based on the OpenSIPS logs, did you
                                    get that error on the first forking
                                    attempt or sometime later after some
                                    number of new tcp processes forking
                                    ? More or less, my question is :
                                    what was the process
                                    creation/termination history before
                                    getting that bug log ?<br>
                                    <br>
                                    Also I just pushed a small commit to
                                    improve the date provided by the bug
                                    log.<br>
                                    <br>
                                    In regards to the "not clear how to
                                    calculate"...could you be specific ?<br>
                                    <br>
                                    Best regards,<br>
                                  </tt>
                                  <pre class="moz-signature">Bogdan-Andrei Iancu

OpenSIPS Founder and Developer
  <a class="moz-txt-link-freetext" href="https://www.opensips-solutions.com" target="_blank">https://www.opensips-solutions.com</a>
OpenSIPS Summit 2019
  <a class="moz-txt-link-freetext" href="https://www.opensips.org/events/Summit-2019Amsterdam/" target="_blank">https://www.opensips.org/events/Summit-2019Amsterdam/</a>
</pre>
                                  <div class="moz-cite-prefix">On
                                    03/10/2019 05:12 PM, Slava Bendersky
                                    wrote:<br>
                                  </div>
                                  <blockquote cite="mid:1387008888.101.1552230768174.JavaMail.bendersky@nlws01.networklab.lan">
                                    <style>p { margin: 0; }</style>
                                    <div style="font-family: courier
                                      new,courier,monaco,monospace,sans-serif;
                                      font-size: 12pt; color: #000000"><font face="courier new, courier,
                                        monaco, monospace, sans-serif"><span style="font-size: 12pt;">Hello
                                          Everyone,</span></font>
                                      <div style="color: rgb(0, 0, 0);
                                        font-family: "courier
                                        new", courier, monaco,
                                        monospace, sans-serif;
                                        font-size: 12pt;">Trying test
                                        auto scaling feature with
                                        following config</div>
                                      <div style="color: rgb(0, 0, 0);
                                        font-family: "courier
                                        new", courier, monaco,
                                        monospace, sans-serif;
                                        font-size: 12pt;"><br>
                                      </div>
                                      <div>
                                        <div><font face="courier new,
                                            courier, monaco, monospace,
                                            sans-serif">auto_scaling_profile
                                            = PRF_SIP</font></div>
                                        <div><font face="courier new,
                                            courier, monaco, monospace,
                                            sans-serif">     scale up to
                                            24 on 70% for 4 cycles
                                            within 5</font></div>
                                        <div><font face="courier new,
                                            courier, monaco, monospace,
                                            sans-serif">     scale down
                                            to 4 on 20% for 8 cycles</font></div>
                                      </div>
                                      <div><font face="courier new,
                                          courier, monaco, monospace,
                                          sans-serif"><br>
                                        </font></div>
                                      <div><font face="courier new,
                                          courier, monaco, monospace,
                                          sans-serif">On box 4096 Mb
                                          memory and 4 virtual CPU</font></div>
                                      <div><font face="courier new,
                                          courier, monaco, monospace,
                                          sans-serif"><br>
                                        </font></div>
                                      <div><font face="courier new,
                                          courier, monaco, monospace,
                                          sans-serif">Getting in log </font></div>
                                      <div><font face="courier new,
                                          courier, monaco, monospace,
                                          sans-serif"><br>
                                        </font></div>
                                      <div><font face="courier new,
                                          courier, monaco, monospace,
                                          sans-serif">CRITICAL:core:fork_dynamic_tcp_process:
                                          #012>>> trying to
                                          fork one more TCP worker but
                                          no free slots in the TCP
                                          table#012#012It seems you have
                                          hit a programming
                                          bug.#012Please help us make
                                          OpenSIPS better by reporting
                                          it at <a class="moz-txt-link-freetext" href="https://github.com/OpenSIPS/opensips/issues" target="_blank">https://github.com/OpenSIPS/opensips/issues</a></font></div>
                                      <div><font face="courier new,
                                          courier, monaco, monospace,
                                          sans-serif"><br>
                                        </font></div>
                                      <div><font face="courier new,
                                          courier, monaco, monospace,
                                          sans-serif"><br>
                                        </font></div>
                                      <div><font face="courier new,
                                          courier, monaco, monospace,
                                          sans-serif"><b>Not clear how
                                            to calculate workers for tcp
                                            or udp</b></font></div>
                                      <div><font face="courier new,
                                          courier, monaco, monospace,
                                          sans-serif"><br>
                                        </font></div>
                                      <div><font face="courier new,
                                          courier, monaco, monospace,
                                          sans-serif"><b>tcp_workers=10</b>
                                          use_auto_scaling_profile
                                          PRF_SIP    </font></div>
                                      <div><font face="courier new,
                                          courier, monaco, monospace,
                                          sans-serif"><b>listen=udp:lan_ip:5060</b>
                                          use_auto_scaling_profile
                                          PRF_SIP</font></div>
                                      <div><font face="courier new,
                                          courier, monaco, monospace,
                                          sans-serif"><br>
                                        </font></div>
                                      <div><font face="courier new,
                                          courier, monaco, monospace,
                                          sans-serif"><br>
                                        </font></div>
                                      <div><font face="courier new,
                                          courier, monaco, monospace,
                                          sans-serif">
                                          <div>[root@vprx00 ~]# opensips
                                            -V</div>
                                          <div>version: opensips
                                            3.0.0-dev (x86_64/linux)</div>
                                          <div>flags: STATS: On,
                                            SHM_EXTRA_STATS,
                                            DISABLE_NAGLE, USE_MCAST,
                                            SHM_MMAP, PKG_MALLOC,
                                            QM_MALLOC, DBG_MALLOC,
                                            FAST_LOCK-ADAPTIVE_WAIT</div>
                                          <div>ADAPTIVE_WAIT_LOOPS=1024,
                                            MAX_RECV_BUFFER_SIZE 262144,
                                            MAX_LISTEN 16, MAX_URI_SIZE
                                            1024, BUF_SIZE 65535</div>
                                          <div>poll method support:
                                            poll, epoll, sigio_rt,
                                            select.</div>
                                          <div>main.c compiled on
                                            10:06:47 Mar  7 2019 with
                                            gcc 8</div>
                                          <div><br>
                                          </div>
                                          <div><br>
                                          </div>
                                        </font></div>
                                      <div><font face="courier new,
                                          courier, monaco, monospace,
                                          sans-serif">volga629</font></div>
                                    </div>
                                    <br>
                                    <fieldset class="mimeAttachmentHeader"></fieldset>
                                    <br>
                                    <pre>_______________________________________________
Users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Users@lists.opensips.org" target="_blank">Users@lists.opensips.org</a>
<a class="moz-txt-link-freetext" href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users" target="_blank">http://lists.opensips.org/cgi-bin/mailman/listinfo/users</a>
</pre>
                                  </blockquote>
                                  <br>
                                </div>
                              </div>
                            </div>
                          </blockquote>
                          <br>
                        </div>
                      </div>
                    </div>
                    <br>
                    _______________________________________________<br>
                    Users mailing list<br>
                    <a class="moz-txt-link-abbreviated" href="mailto:Users@lists.opensips.org" target="_blank">Users@lists.opensips.org</a><br>
                    <a class="moz-txt-link-freetext" href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users" target="_blank">http://lists.opensips.org/cgi-bin/mailman/listinfo/users</a><br>
                  </div>
                  <br>
                </div>
              </div>
              <br>
              <fieldset class="mimeAttachmentHeader"></fieldset>
              <br>
              <pre>_______________________________________________
Users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Users@lists.opensips.org" target="_blank">Users@lists.opensips.org</a>
<a class="moz-txt-link-freetext" href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users" target="_blank">http://lists.opensips.org/cgi-bin/mailman/listinfo/users</a>
</pre>
            </blockquote>
            <br>
          </div>
          <br>
        </div>
      </div>
    </blockquote>
    <br>
  

</div><br></div></div></body></html>