<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<tt>Hi Slava,<br>
<br>
There were some fixes in the auto-scaling in the last weeks, could
you confirm or not your issue against the latest 3.0 code ?<br>
<br>
Regards,<br>
</tt>
<pre class="moz-signature" cols="72">Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
<a class="moz-txt-link-freetext" href="https://www.opensips-solutions.com">https://www.opensips-solutions.com</a>
OpenSIPS Summit 2019
<a class="moz-txt-link-freetext" href="https://www.opensips.org/events/Summit-2019Amsterdam/">https://www.opensips.org/events/Summit-2019Amsterdam/</a>
</pre>
<div class="moz-cite-prefix">On 03/15/2019 12:47 PM, Slava Bendersky
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:1824870661.406.1552646859543.JavaMail.bendersky@nlws01.networklab.lan">
<style type="text/css">p { margin: 0; }</style>
<div style="font-family: courier
new,courier,monaco,monospace,sans-serif; font-size: 12pt; color:
#000000"><font face="courier new, courier, monaco, monospace,
sans-serif">Hello Bogdan,</font>
<div style="font-family: "courier new", courier,
monaco, monospace, sans-serif;"><span style="font-family:
monospace;">Upscaling error is didn't show up any more in
logs at least I didn't so it. Might need trigger bigger
load.</span></div>
<div><font face="monospace"><br>
</font></div>
<div><span style="font-family: monospace;"><b>do you do
auto-scaling only for TCP ?</b></span></div>
<div><font face="monospace"><br>
</font></div>
<div><font face="monospace">Our setup in most multi homed, as
example public side TCP private side UDP</font></div>
<div><font face="monospace"><br>
</font></div>
<div><font face="monospace"><br>
</font></div>
<div><span style="font-family: monospace;"><b>Now, about the TCP
downscaling - I guess you see as terminated by OpenSIPS
the TCP conns anchored in the process that is terminated ?</b></span></div>
<div><font face="courier new, courier, monaco, monospace,
sans-serif"><br>
Yes, that how is look like end result that opensips start
blocking TCP connections. If specific process handle sip
registration via tcp and opensips trying close to down scale
end point need send re register in order do not loose
registration. Other wise it cause stale connections. Just
close properly connection not enough I think.</font></div>
<div><font face="courier new, courier, monaco, monospace,
sans-serif"><br>
</font></div>
<div><font face="courier new, courier, monaco, monospace,
sans-serif"><br>
</font></div>
<div><font face="courier new, courier, monaco, monospace,
sans-serif">volga629</font></div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
<hr id="zwchr" style="font-family: "courier new",
courier, monaco, monospace, sans-serif;">
<div style="font-family: Helvetica, Arial, sans-serif; color:
rgb(0, 0, 0); font-weight: normal; font-style: normal;
text-decoration: none; font-size: 12pt;"><b>From: </b>"Bogdan-Andrei
Iancu" <a class="moz-txt-link-rfc2396E" href="mailto:bogdan@opensips.org"><bogdan@opensips.org></a><br>
<b>To: </b>"OpenSIPS users mailling list"
<a class="moz-txt-link-rfc2396E" href="mailto:users@lists.opensips.org"><users@lists.opensips.org></a>, "Slava Bendersky"
<a class="moz-txt-link-rfc2396E" href="mailto:volga629@networklab.ca"><volga629@networklab.ca></a><br>
<b>Sent: </b>Thursday, March 14, 2019 1:45:20 PM<br>
<b>Subject: </b>Re: [OpenSIPS-Users] auto scaling<br>
<br>
<tt>Slava,<br>
<br>
What about the upscaling error you originally reported ?
is there any way to reproduce it ? do you do auto-scaling
only for TCP (no UDP,timer) ?<br>
<br>
Now, about the TCP downscaling - I guess you see as
terminated by OpenSIPS the TCP conns anchored in the
process that is terminated ? Is is a proper close of the
connection ? or ?<br>
<br>
Regards,<br>
</tt>
<pre class="moz-signature">Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
<a class="moz-txt-link-freetext" href="https://www.opensips-solutions.com" target="_blank" moz-do-not-send="true">https://www.opensips-solutions.com</a>
OpenSIPS Summit 2019
<a class="moz-txt-link-freetext" href="https://www.opensips.org/events/Summit-2019Amsterdam/" target="_blank" moz-do-not-send="true">https://www.opensips.org/events/Summit-2019Amsterdam/</a>
</pre>
<div class="moz-cite-prefix">On 03/14/2019 02:14 PM, Slava
Bendersky wrote:<br>
</div>
<blockquote
cite="mid:902388553.152.1552565672232.JavaMail.bendersky@nlws01.networklab.lan">
<style>p { margin: 0; }</style>
<div style="font-family: courier
new,courier,monaco,monospace,sans-serif; font-size:
12pt; color: #000000">Hello Bogdan,
<div>After we doing more testing with auto scaling we
observed issue with TCP, where opensips try scale down
the processes it tear down them and break TCP
connections with end points and that cause blocked
connection state where opensips stops process TCP
connections.</div>
<div><br>
</div>
<div>volga629 <br>
<br>
<hr id="zwchr">
<div
style="color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;"><b>From:
</b>"Slava Bendersky" <a
class="moz-txt-link-rfc2396E"
href="mailto:volga629@networklab.ca"
target="_blank" moz-do-not-send="true"><volga629@networklab.ca></a><br>
<b>To: </b>"Bogdan-Andrei Iancu" <a
class="moz-txt-link-rfc2396E"
href="mailto:bogdan@opensips.org" target="_blank"
moz-do-not-send="true"><bogdan@opensips.org></a><br>
<b>Cc: </b>"OpenSIPS users mailling list" <a
class="moz-txt-link-rfc2396E"
href="mailto:users@lists.opensips.org"
target="_blank" moz-do-not-send="true"><users@lists.opensips.org></a><br>
<b>Sent: </b>Wednesday, March 13, 2019 12:21:14 AM<br>
<b>Subject: </b>Re: [OpenSIPS-Users] auto scaling<br>
<br>
<style>p { margin: 0; }</style>
<div style="font-family: courier
new,courier,monaco,monospace,sans-serif;
font-size: 12pt; color: #000000">I will do pull
today.
<div><br>
</div>
<div>volga629<br>
<hr id="zwchr">
<div
style="color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;"><b>From:
</b>"Bogdan-Andrei Iancu" <a
class="moz-txt-link-rfc2396E"
href="mailto:bogdan@opensips.org"
target="_blank" moz-do-not-send="true"><bogdan@opensips.org></a><br>
<b>To: </b>"Slava Bendersky" <a
class="moz-txt-link-rfc2396E"
href="mailto:volga629@networklab.ca"
target="_blank" moz-do-not-send="true"><volga629@networklab.ca></a><br>
<b>Cc: </b>"OpenSIPS users mailling list" <a
class="moz-txt-link-rfc2396E"
href="mailto:users@lists.opensips.org"
target="_blank" moz-do-not-send="true"><users@lists.opensips.org></a><br>
<b>Sent: </b>Tuesday, March 12, 2019 4:52:06
AM<br>
<b>Subject: </b>Re: [OpenSIPS-Users] auto
scaling<br>
<br>
<tt>Slava,<br>
<br>
Have you pulled my change on the improved
log (just update from master) ? and post
here the new output you get for the error.<br>
<br>
Thanks,<br>
</tt>
<pre class="moz-signature">Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
<a class="moz-txt-link-freetext" href="https://www.opensips-solutions.com" target="_blank" moz-do-not-send="true">https://www.opensips-solutions.com</a>
OpenSIPS Summit 2019
<a class="moz-txt-link-freetext" href="https://www.opensips.org/events/Summit-2019Amsterdam/" target="_blank" moz-do-not-send="true">https://www.opensips.org/events/Summit-2019Amsterdam/</a>
</pre>
<div class="moz-cite-prefix">On 03/12/2019
05:51 AM, Slava Bendersky wrote:<br>
</div>
<blockquote
cite="mid:423067196.274.1552362703793.JavaMail.bendersky@nlws01.networklab.lan">
<style>p { margin: 0; }</style>
<div style="font-family: courier
new,courier,monaco,monospace,sans-serif;
font-size: 12pt; color: #000000">Hello
Bogdan,
<div>My question how to calculate
correctly </div>
<div><br>
</div>
<div><b>tcp_workers</b></div>
<div><b>udp_workers</b></div>
<div><b><br>
</b></div>
<div>to properly assign with in auto
scaling profile. Which parameters need
use as base ?</div>
<div><br>
</div>
<div>As example if vm 4GB memory up what
will be max number for scale up
processes and what should be value for
tcp and udp workers.</div>
<div>It message is appear in log on first
attempt to fork in scale up.</div>
<div><br>
</div>
<div>volga629 </div>
<div> </div>
<div>
<hr id="zwchr">
<div
style="color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;"><b>From:
</b>"Bogdan-Andrei Iancu" <a
class="moz-txt-link-rfc2396E"
href="mailto:bogdan@opensips.org"
target="_blank"
moz-do-not-send="true"><bogdan@opensips.org></a><br>
<b>To: </b>"OpenSIPS users mailling
list" <a
class="moz-txt-link-rfc2396E"
href="mailto:users@lists.opensips.org"
target="_blank"
moz-do-not-send="true"><users@lists.opensips.org></a>,
"Slava Bendersky" <a
class="moz-txt-link-rfc2396E"
href="mailto:volga629@networklab.ca"
target="_blank"
moz-do-not-send="true"><volga629@networklab.ca></a><br>
<b>Sent: </b>Monday, March 11, 2019
4:44:57 AM<br>
<b>Subject: </b>Re: [OpenSIPS-Users]
auto scaling<br>
<br>
<tt>Hi Slava,<br>
<br>
Based on the OpenSIPS logs, did you
get that error on the first forking
attempt or sometime later after some
number of new tcp processes forking
? More or less, my question is :
what was the process
creation/termination history before
getting that bug log ?<br>
<br>
Also I just pushed a small commit to
improve the date provided by the bug
log.<br>
<br>
In regards to the "not clear how to
calculate"...could you be specific ?<br>
<br>
Best regards,<br>
</tt>
<pre class="moz-signature">Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
<a class="moz-txt-link-freetext" href="https://www.opensips-solutions.com" target="_blank" moz-do-not-send="true">https://www.opensips-solutions.com</a>
OpenSIPS Summit 2019
<a class="moz-txt-link-freetext" href="https://www.opensips.org/events/Summit-2019Amsterdam/" target="_blank" moz-do-not-send="true">https://www.opensips.org/events/Summit-2019Amsterdam/</a>
</pre>
<div class="moz-cite-prefix">On
03/10/2019 05:12 PM, Slava Bendersky
wrote:<br>
</div>
<blockquote
cite="mid:1387008888.101.1552230768174.JavaMail.bendersky@nlws01.networklab.lan">
<style>p { margin: 0; }</style>
<div style="font-family: courier
new,courier,monaco,monospace,sans-serif;
font-size: 12pt; color: #000000"><font
face="courier new, courier,
monaco, monospace, sans-serif"><span
style="font-size: 12pt;">Hello
Everyone,</span></font>
<div style="color: rgb(0, 0, 0);
font-family: "courier
new", courier, monaco,
monospace, sans-serif;
font-size: 12pt;">Trying test
auto scaling feature with
following config</div>
<div style="color: rgb(0, 0, 0);
font-family: "courier
new", courier, monaco,
monospace, sans-serif;
font-size: 12pt;"><br>
</div>
<div>
<div><font face="courier new,
courier, monaco, monospace,
sans-serif">auto_scaling_profile
= PRF_SIP</font></div>
<div><font face="courier new,
courier, monaco, monospace,
sans-serif"> scale up to
24 on 70% for 4 cycles
within 5</font></div>
<div><font face="courier new,
courier, monaco, monospace,
sans-serif"> scale down
to 4 on 20% for 8 cycles</font></div>
</div>
<div><font face="courier new,
courier, monaco, monospace,
sans-serif"><br>
</font></div>
<div><font face="courier new,
courier, monaco, monospace,
sans-serif">On box 4096 Mb
memory and 4 virtual CPU</font></div>
<div><font face="courier new,
courier, monaco, monospace,
sans-serif"><br>
</font></div>
<div><font face="courier new,
courier, monaco, monospace,
sans-serif">Getting in log </font></div>
<div><font face="courier new,
courier, monaco, monospace,
sans-serif"><br>
</font></div>
<div><font face="courier new,
courier, monaco, monospace,
sans-serif">CRITICAL:core:fork_dynamic_tcp_process:
#012>>> trying to
fork one more TCP worker but
no free slots in the TCP
table#012#012It seems you have
hit a programming
bug.#012Please help us make
OpenSIPS better by reporting
it at <a
class="moz-txt-link-freetext"
href="https://github.com/OpenSIPS/opensips/issues" target="_blank"
moz-do-not-send="true">https://github.com/OpenSIPS/opensips/issues</a></font></div>
<div><font face="courier new,
courier, monaco, monospace,
sans-serif"><br>
</font></div>
<div><font face="courier new,
courier, monaco, monospace,
sans-serif"><br>
</font></div>
<div><font face="courier new,
courier, monaco, monospace,
sans-serif"><b>Not clear how
to calculate workers for tcp
or udp</b></font></div>
<div><font face="courier new,
courier, monaco, monospace,
sans-serif"><br>
</font></div>
<div><font face="courier new,
courier, monaco, monospace,
sans-serif"><b>tcp_workers=10</b>
use_auto_scaling_profile
PRF_SIP </font></div>
<div><font face="courier new,
courier, monaco, monospace,
sans-serif"><b>listen=udp:lan_ip:5060</b>
use_auto_scaling_profile
PRF_SIP</font></div>
<div><font face="courier new,
courier, monaco, monospace,
sans-serif"><br>
</font></div>
<div><font face="courier new,
courier, monaco, monospace,
sans-serif"><br>
</font></div>
<div><font face="courier new,
courier, monaco, monospace,
sans-serif">
<div>[root@vprx00 ~]# opensips
-V</div>
<div>version: opensips
3.0.0-dev (x86_64/linux)</div>
<div>flags: STATS: On,
SHM_EXTRA_STATS,
DISABLE_NAGLE, USE_MCAST,
SHM_MMAP, PKG_MALLOC,
QM_MALLOC, DBG_MALLOC,
FAST_LOCK-ADAPTIVE_WAIT</div>
<div>ADAPTIVE_WAIT_LOOPS=1024,
MAX_RECV_BUFFER_SIZE 262144,
MAX_LISTEN 16, MAX_URI_SIZE
1024, BUF_SIZE 65535</div>
<div>poll method support:
poll, epoll, sigio_rt,
select.</div>
<div>main.c compiled on
10:06:47 Mar 7 2019 with
gcc 8</div>
<div><br>
</div>
<div><br>
</div>
</font></div>
<div><font face="courier new,
courier, monaco, monospace,
sans-serif">volga629</font></div>
</div>
<br>
<fieldset
class="mimeAttachmentHeader"></fieldset>
<br>
<pre>_______________________________________________
Users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Users@lists.opensips.org" target="_blank" moz-do-not-send="true">Users@lists.opensips.org</a>
<a class="moz-txt-link-freetext" href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users" target="_blank" moz-do-not-send="true">http://lists.opensips.org/cgi-bin/mailman/listinfo/users</a>
</pre>
</blockquote>
<br>
</div>
</div>
</div>
</blockquote>
<br>
</div>
</div>
</div>
<br>
_______________________________________________<br>
Users mailing list<br>
<a class="moz-txt-link-abbreviated"
href="mailto:Users@lists.opensips.org"
target="_blank" moz-do-not-send="true">Users@lists.opensips.org</a><br>
<a class="moz-txt-link-freetext"
href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users"
target="_blank" moz-do-not-send="true">http://lists.opensips.org/cgi-bin/mailman/listinfo/users</a><br>
</div>
<br>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre>_______________________________________________
Users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Users@lists.opensips.org" target="_blank" moz-do-not-send="true">Users@lists.opensips.org</a>
<a class="moz-txt-link-freetext" href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users" target="_blank" moz-do-not-send="true">http://lists.opensips.org/cgi-bin/mailman/listinfo/users</a>
</pre>
</blockquote>
<br>
</div>
<br>
</div>
</div>
</blockquote>
<br>
</body>
</html>