<html><head><style type='text/css'>p { margin: 0; }</style></head><body><div style='font-family: courier new,courier,monaco,monospace,sans-serif; font-size: 12pt; color: #000000'><font face="courier new, courier, monaco, monospace, sans-serif">Hello Bogdan,</font><div style="font-family: "courier new", courier, monaco, monospace, sans-serif;"><span style="font-family: monospace;">Upscaling error is didn't show up any more in logs at least I didn't so it. Might need trigger bigger load.</span></div><div><font face="monospace"><br></font></div><div><span style="font-family: monospace;"><b>do you do auto-scaling only for TCP ?</b></span></div><div><font face="monospace"><br></font></div><div><font face="monospace">Our setup in most multi homed, as example public side TCP private side UDP</font></div><div><font face="monospace"><br></font></div><div><font face="monospace"><br></font></div><div><span style="font-family: monospace;"><b>Now, about the TCP downscaling - I guess you see as terminated by OpenSIPS the TCP conns anchored in the process that is terminated ?</b></span></div><div><font face="courier new, courier, monaco, monospace, sans-serif"><br>Yes, that how is look like end result that opensips start blocking TCP connections. If specific process handle sip registration via tcp and opensips trying close to down scale end point need send re register in order do not loose registration. Other wise it cause stale connections. Just close properly connection not enough I think.</font></div><div><font face="courier new, courier, monaco, monospace, sans-serif"><br></font></div><div><font face="courier new, courier, monaco, monospace, sans-serif"><br></font></div><div><font face="courier new, courier, monaco, monospace, sans-serif">volga629</font></div><div><br></div><div><br></div><div><br></div><div><br><hr id="zwchr" style="font-family: "courier new", courier, monaco, monospace, sans-serif;"><div style="font-family: Helvetica, Arial, sans-serif; color: rgb(0, 0, 0); font-weight: normal; font-style: normal; text-decoration: none; font-size: 12pt;"><b>From: </b>"Bogdan-Andrei Iancu" <bogdan@opensips.org><br><b>To: </b>"OpenSIPS users mailling list" <users@lists.opensips.org>, "Slava Bendersky" <volga629@networklab.ca><br><b>Sent: </b>Thursday, March 14, 2019 1:45:20 PM<br><b>Subject: </b>Re: [OpenSIPS-Users] auto scaling<br><br>
<tt>Slava,<br>
<br>
What about the upscaling error you originally reported ? is there
any way to reproduce it ? do you do auto-scaling only for TCP (no
UDP,timer) ?<br>
<br>
Now, about the TCP downscaling - I guess you see as terminated by
OpenSIPS the TCP conns anchored in the process that is terminated
? Is is a proper close of the connection ? or ?<br>
<br>
Regards,<br>
</tt>
<pre class="moz-signature">Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
<a class="moz-txt-link-freetext" href="https://www.opensips-solutions.com" target="_blank">https://www.opensips-solutions.com</a>
OpenSIPS Summit 2019
<a class="moz-txt-link-freetext" href="https://www.opensips.org/events/Summit-2019Amsterdam/" target="_blank">https://www.opensips.org/events/Summit-2019Amsterdam/</a>
</pre>
<div class="moz-cite-prefix">On 03/14/2019 02:14 PM, Slava Bendersky
wrote:<br>
</div>
<blockquote cite="mid:902388553.152.1552565672232.JavaMail.bendersky@nlws01.networklab.lan">
<style>p { margin: 0; }</style>
<div style="font-family: courier
new,courier,monaco,monospace,sans-serif; font-size: 12pt; color:
#000000">Hello Bogdan,
<div>After we doing more testing with auto scaling we observed
issue with TCP, where opensips try scale down the processes it
tear down them and break TCP connections with end points and
that cause blocked connection state where opensips stops
process TCP connections.</div>
<div><br>
</div>
<div>volga629 <br>
<br><hr id="zwchr">
<div style="color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;"><b>From:
</b>"Slava Bendersky" <a class="moz-txt-link-rfc2396E" href="mailto:volga629@networklab.ca" target="_blank"><volga629@networklab.ca></a><br>
<b>To: </b>"Bogdan-Andrei Iancu"
<a class="moz-txt-link-rfc2396E" href="mailto:bogdan@opensips.org" target="_blank"><bogdan@opensips.org></a><br>
<b>Cc: </b>"OpenSIPS users mailling list"
<a class="moz-txt-link-rfc2396E" href="mailto:users@lists.opensips.org" target="_blank"><users@lists.opensips.org></a><br>
<b>Sent: </b>Wednesday, March 13, 2019 12:21:14 AM<br>
<b>Subject: </b>Re: [OpenSIPS-Users] auto scaling<br>
<br>
<style>p { margin: 0; }</style>
<div style="font-family: courier
new,courier,monaco,monospace,sans-serif; font-size: 12pt;
color: #000000">I will do pull today.
<div><br>
</div>
<div>volga629<br>
<hr id="zwchr">
<div style="color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;"><b>From:
</b>"Bogdan-Andrei Iancu" <a class="moz-txt-link-rfc2396E" href="mailto:bogdan@opensips.org" target="_blank"><bogdan@opensips.org></a><br>
<b>To: </b>"Slava Bendersky"
<a class="moz-txt-link-rfc2396E" href="mailto:volga629@networklab.ca" target="_blank"><volga629@networklab.ca></a><br>
<b>Cc: </b>"OpenSIPS users mailling list"
<a class="moz-txt-link-rfc2396E" href="mailto:users@lists.opensips.org" target="_blank"><users@lists.opensips.org></a><br>
<b>Sent: </b>Tuesday, March 12, 2019 4:52:06 AM<br>
<b>Subject: </b>Re: [OpenSIPS-Users] auto scaling<br>
<br>
<tt>Slava,<br>
<br>
Have you pulled my change on the improved log (just
update from master) ? and post here the new output
you get for the error.<br>
<br>
Thanks,<br>
</tt>
<pre class="moz-signature">Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
<a class="moz-txt-link-freetext" href="https://www.opensips-solutions.com" target="_blank">https://www.opensips-solutions.com</a>
OpenSIPS Summit 2019
<a class="moz-txt-link-freetext" href="https://www.opensips.org/events/Summit-2019Amsterdam/" target="_blank">https://www.opensips.org/events/Summit-2019Amsterdam/</a>
</pre>
<div class="moz-cite-prefix">On 03/12/2019 05:51 AM,
Slava Bendersky wrote:<br>
</div>
<blockquote cite="mid:423067196.274.1552362703793.JavaMail.bendersky@nlws01.networklab.lan">
<style>p { margin: 0; }</style>
<div style="font-family: courier
new,courier,monaco,monospace,sans-serif;
font-size: 12pt; color: #000000">Hello Bogdan,
<div>My question how to calculate correctly </div>
<div><br>
</div>
<div><b>tcp_workers</b></div>
<div><b>udp_workers</b></div>
<div><b><br>
</b></div>
<div>to properly assign with in auto scaling
profile. Which parameters need use as base ?</div>
<div><br>
</div>
<div>As example if vm 4GB memory up what will be
max number for scale up processes and what
should be value for tcp and udp workers.</div>
<div>It message is appear in log on first attempt
to fork in scale up.</div>
<div><br>
</div>
<div>volga629 </div>
<div> </div>
<div>
<hr id="zwchr">
<div style="color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;"><b>From:
</b>"Bogdan-Andrei Iancu" <a class="moz-txt-link-rfc2396E" href="mailto:bogdan@opensips.org" target="_blank"><bogdan@opensips.org></a><br>
<b>To: </b>"OpenSIPS users mailling list" <a class="moz-txt-link-rfc2396E" href="mailto:users@lists.opensips.org" target="_blank"><users@lists.opensips.org></a>,
"Slava Bendersky" <a class="moz-txt-link-rfc2396E" href="mailto:volga629@networklab.ca" target="_blank"><volga629@networklab.ca></a><br>
<b>Sent: </b>Monday, March 11, 2019 4:44:57
AM<br>
<b>Subject: </b>Re: [OpenSIPS-Users] auto
scaling<br>
<br>
<tt>Hi Slava,<br>
<br>
Based on the OpenSIPS logs, did you get that
error on the first forking attempt or
sometime later after some number of new tcp
processes forking ? More or less, my
question is : what was the process
creation/termination history before getting
that bug log ?<br>
<br>
Also I just pushed a small commit to improve
the date provided by the bug log.<br>
<br>
In regards to the "not clear how to
calculate"...could you be specific ?<br>
<br>
Best regards,<br>
</tt>
<pre class="moz-signature">Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
<a class="moz-txt-link-freetext" href="https://www.opensips-solutions.com" target="_blank">https://www.opensips-solutions.com</a>
OpenSIPS Summit 2019
<a class="moz-txt-link-freetext" href="https://www.opensips.org/events/Summit-2019Amsterdam/" target="_blank">https://www.opensips.org/events/Summit-2019Amsterdam/</a>
</pre>
<div class="moz-cite-prefix">On 03/10/2019
05:12 PM, Slava Bendersky wrote:<br>
</div>
<blockquote cite="mid:1387008888.101.1552230768174.JavaMail.bendersky@nlws01.networklab.lan">
<style>p { margin: 0; }</style>
<div style="font-family: courier
new,courier,monaco,monospace,sans-serif;
font-size: 12pt; color: #000000"><font face="courier new, courier, monaco,
monospace, sans-serif"><span style="font-size: 12pt;">Hello
Everyone,</span></font>
<div style="color: rgb(0, 0, 0);
font-family: "courier new",
courier, monaco, monospace, sans-serif;
font-size: 12pt;">Trying test auto
scaling feature with following config</div>
<div style="color: rgb(0, 0, 0);
font-family: "courier new",
courier, monaco, monospace, sans-serif;
font-size: 12pt;"><br>
</div>
<div>
<div><font face="courier new, courier,
monaco, monospace, sans-serif">auto_scaling_profile
= PRF_SIP</font></div>
<div><font face="courier new, courier,
monaco, monospace, sans-serif">
scale up to 24 on 70% for 4 cycles
within 5</font></div>
<div><font face="courier new, courier,
monaco, monospace, sans-serif">
scale down to 4 on 20% for 8 cycles</font></div>
</div>
<div><font face="courier new, courier,
monaco, monospace, sans-serif"><br>
</font></div>
<div><font face="courier new, courier,
monaco, monospace, sans-serif">On box
4096 Mb memory and 4 virtual CPU</font></div>
<div><font face="courier new, courier,
monaco, monospace, sans-serif"><br>
</font></div>
<div><font face="courier new, courier,
monaco, monospace, sans-serif">Getting
in log </font></div>
<div><font face="courier new, courier,
monaco, monospace, sans-serif"><br>
</font></div>
<div><font face="courier new, courier,
monaco, monospace, sans-serif">CRITICAL:core:fork_dynamic_tcp_process:
#012>>> trying to fork one
more TCP worker but no free slots in
the TCP table#012#012It seems you have
hit a programming bug.#012Please help
us make OpenSIPS better by reporting
it at <a class="moz-txt-link-freetext" href="https://github.com/OpenSIPS/opensips/issues" target="_blank">https://github.com/OpenSIPS/opensips/issues</a></font></div>
<div><font face="courier new, courier,
monaco, monospace, sans-serif"><br>
</font></div>
<div><font face="courier new, courier,
monaco, monospace, sans-serif"><br>
</font></div>
<div><font face="courier new, courier,
monaco, monospace, sans-serif"><b>Not
clear how to calculate workers for
tcp or udp</b></font></div>
<div><font face="courier new, courier,
monaco, monospace, sans-serif"><br>
</font></div>
<div><font face="courier new, courier,
monaco, monospace, sans-serif"><b>tcp_workers=10</b>
use_auto_scaling_profile PRF_SIP </font></div>
<div><font face="courier new, courier,
monaco, monospace, sans-serif"><b>listen=udp:lan_ip:5060</b>
use_auto_scaling_profile PRF_SIP</font></div>
<div><font face="courier new, courier,
monaco, monospace, sans-serif"><br>
</font></div>
<div><font face="courier new, courier,
monaco, monospace, sans-serif"><br>
</font></div>
<div><font face="courier new, courier,
monaco, monospace, sans-serif">
<div>[root@vprx00 ~]# opensips -V</div>
<div>version: opensips 3.0.0-dev
(x86_64/linux)</div>
<div>flags: STATS: On,
SHM_EXTRA_STATS, DISABLE_NAGLE,
USE_MCAST, SHM_MMAP, PKG_MALLOC,
QM_MALLOC, DBG_MALLOC,
FAST_LOCK-ADAPTIVE_WAIT</div>
<div>ADAPTIVE_WAIT_LOOPS=1024,
MAX_RECV_BUFFER_SIZE 262144,
MAX_LISTEN 16, MAX_URI_SIZE 1024,
BUF_SIZE 65535</div>
<div>poll method support: poll, epoll,
sigio_rt, select.</div>
<div>main.c compiled on 10:06:47 Mar
7 2019 with gcc 8</div>
<div><br>
</div>
<div><br>
</div>
</font></div>
<div><font face="courier new, courier,
monaco, monospace, sans-serif">volga629</font></div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre>_______________________________________________
Users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Users@lists.opensips.org" target="_blank">Users@lists.opensips.org</a>
<a class="moz-txt-link-freetext" href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users" target="_blank">http://lists.opensips.org/cgi-bin/mailman/listinfo/users</a>
</pre>
</blockquote>
<br>
</div>
</div>
</div>
</blockquote>
<br>
</div>
</div>
</div>
<br>
_______________________________________________<br>
Users mailing list<br>
<a class="moz-txt-link-abbreviated" href="mailto:Users@lists.opensips.org" target="_blank">Users@lists.opensips.org</a><br>
<a class="moz-txt-link-freetext" href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users" target="_blank">http://lists.opensips.org/cgi-bin/mailman/listinfo/users</a><br>
</div>
<br>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre>_______________________________________________
Users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Users@lists.opensips.org" target="_blank">Users@lists.opensips.org</a>
<a class="moz-txt-link-freetext" href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users" target="_blank">http://lists.opensips.org/cgi-bin/mailman/listinfo/users</a>
</pre>
</blockquote>
<br>
</div><br></div></div></body></html>