<html><head></head><body><div class="ydpac33dc08yahoo-style-wrap" style="font-family: Helvetica Neue, Helvetica, Arial, sans-serif; font-size: 13px;"><div></div>
<div dir="ltr" data-setdir="false">Enough proof. Thank you Alex for sharing your experiences with us.</div><div dir="ltr" data-setdir="false"><br></div><div dir="ltr" data-setdir="false">Regards,</div><div dir="ltr" data-setdir="false">HYavari<br></div><div dir="ltr" data-setdir="false"><br></div><div><br></div>
</div><div id="ydp688a2fd4yahoo_quoted_9270223770" class="ydp688a2fd4yahoo_quoted">
<div style="font-family:'Helvetica Neue', Helvetica, Arial, sans-serif;font-size:13px;color:#26282a;">
<div>
On Monday, May 4, 2020, 10:52:49 PM GMT+4:30, Aleksandar Sosic <alex@canyan.io> wrote:
</div>
<div><br></div>
<div><br></div>
<div><div dir="ltr">Hi,<br clear="none"><br clear="none">I'm new to OpenSIPS but been running Kamailio on Kubernetes in<br clear="none">production on different projects. The same applies to Asterisk and<br clear="none">RTPengine.<br clear="none"><br clear="none">The networking issue can be easily solved using another networking<br clear="none">plugin for k8s like macvlan.<br clear="none">You can find a video from a couple of years ago here describing some<br clear="none">of the issues we had at EvoSip and how we solved them:<br clear="none"><a shape="rect" href="https://www.youtube.com/watch?v=QOB998rAyUM" rel="nofollow" target="_blank">https://www.youtube.com/watch?v=QOB998rAyUM</a><br clear="none">Although I would not use anycast anymore, it works but it's a pain to<br clear="none">deploy and maintain. So much complexity...I like simple things ;)<br clear="none"><br clear="none">RTPEngine can be run in kernel space without issues and with the same<br clear="none">performances as on VMs.<br clear="none"><br clear="none">Working ad Wazo we developed an opensource solution of a C4 running on<br clear="none">k8s and it was running 600cps without issues with one SBC and two<br clear="none">Router nodes with two RTPs.<br clear="none">Here are a few articles describing some of the implementations we've done there:<br clear="none"><a shape="rect" href="https://wazo-platform.org/blog/wazo-platform-c4-overview" rel="nofollow" target="_blank">https://wazo-platform.org/blog/wazo-platform-c4-overview</a><br clear="none"><a shape="rect" href="https://wazo-platform.org/blog/kamailio-ha-dispatcher-and-dmq" rel="nofollow" target="_blank">https://wazo-platform.org/blog/kamailio-ha-dispatcher-and-dmq</a><br clear="none"><a shape="rect" href="https://wazo-platform.org/blog/kamailio-routing-with-rtjson-and-http-async-client" rel="nofollow" target="_blank">https://wazo-platform.org/blog/kamailio-routing-with-rtjson-and-http-async-client</a><br clear="none"><br clear="none">All of our implementations have an SBC doing only some basic filtering<br clear="none">and dispatching the requests to another layer of multiple "routers"<br clear="none">that are applying the routing logic.<br clear="none"><br clear="none">For the SBC I would recommend an HA installation as you would do with<br clear="none">VMs, for the rest of the infra you can have everything on the k8s<br clear="none">local networking and scale as much as you want without issues.<br clear="none">Mind that the SBC should do just the balancing of the SIP messages,<br clear="none">all the logic should be on the "routing" layer beneath.<br clear="none">We almost always had an API that did the routing logic. So the<br clear="none">"router" nodes would query the API which responded what to do with the<br clear="none">SIP packet and where to route it. Much easier to maintain and develop<br clear="none">and obviously scale.<br clear="none"><br clear="none">For auto-scaling we used a custom k8s controller in the EvoSIP<br clear="none">project, but in Wazo we used Consul for service discovery and<br clear="none">populating the dispatcher list in Kamailio which is a better solution<br clear="none">IMHO because you can run it on VMs also. I can confirm that it worked<br clear="none">like a charm. When you would scale the RTPs or Routers the dispatcher<br clear="none">and RTP list will be updated and an rpc call will trigger the reload<br clear="none">in Kamailio without restarting it or changing things "manually".<br clear="none">Pretty neat.<br clear="none"><br clear="none">I suppose you can do the same with OpenSIPS, but as told I'm pretty<br clear="none">new with it. Just did an integration of a Rating Billing System with<br clear="none">the rest_client module and tracer.<br clear="none"><br clear="none">Please mind that I don't work anymore with the guys at Wazo so I'm not<br clear="none">sure about the status of their C4 project which had also a helm chart<br clear="none">for deploying it in k8s. Not sure if it is maintained now.<br clear="none"><br clear="none">We will be publishing some articles regarding your questions in the<br clear="none">next months and probably release a Canyan C4 as an open-source project<br clear="none">as we did with the Rating Engine.<br clear="none"><br clear="none">So to recap with your question, yes it is possible and yes it can run<br clear="none">in production but it's not quite that simple to do if you have no<br clear="none">experience with k8s. You can do it with a default k8s deployment but<br clear="none">will not have the performances as you would have with VM as the other<br clear="none">guys here mentioned.<br clear="none">To have those performances and the scalability and all the nice<br clear="none">features of k8s you need some customization of the k8s cluster and<br clear="none">moving your logic on an API in order to drive multiple instances of<br clear="none">your OpenIPS/Kamailio as "stateless" components to be able to scale.<br clear="none"><br clear="none">With a deployment on k8s you than have multiple advantages as you can<br clear="none">add or remove nodes, have them partially deployed on-premises or in<br clear="none">the cloud as you need them. Mind that the RTPs should be as near as<br clear="none">possible to the client in order to have a good MOS and so a good QoS.<br clear="none">The rest of the infra is messaging so you can deploy it wherever you<br clear="none">like. You can, for example, have some nodes on-premises near the<br clear="none">client and use the nodeSelector k8s feature to deploy RTPs there.<br clear="none"><br clear="none">To conclude I would suggest you to try it out. It has multiple<br clear="none">advantages that a standard VM deployment does not. It will make your<br clear="none">life easier later ;)<br clear="none"><br clear="none">For any other questions don't hesitate to contact me here or in private.<br clear="none"><br clear="none">Regards,<br clear="none">Alex<br clear="none"><div class="ydp688a2fd4yqt5686847018" id="ydp688a2fd4yqtfd56833"><br clear="none">On Sat, May 2, 2020 at 7:56 AM H Yavari via Users<br clear="none"><<a shape="rect" href="mailto:users@lists.opensips.org" rel="nofollow" target="_blank">users@lists.opensips.org</a>> wrote:<br clear="none">><br clear="none">> Hi to all,<br clear="none">><br clear="none">> As you know docker and K8s, are growing quickly. So we dockerized Asterisk and OpenSIPS also.<br clear="none">> But I see some community members are against it. They have some reasons like NAT, RTP ports and performance.<br clear="none">><br clear="none">> Do you agree with them ?<br clear="none">> Is there any successful large scale OpenSIPS cluster based on K8s ?<br clear="none">><br clear="none">><br clear="none">> Thanks for sharing your experiences.<br clear="none">><br clear="none">><br clear="none">> Regards,<br clear="none">> HY</div><br clear="none">> _______________________________________________<br clear="none">> Users mailing list<br clear="none">> <a shape="rect" href="mailto:Users@lists.opensips.org" rel="nofollow" target="_blank">Users@lists.opensips.org</a><br clear="none">> <a shape="rect" href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users" rel="nofollow" target="_blank">http://lists.opensips.org/cgi-bin/mailman/listinfo/users</a><div class="ydp688a2fd4yqt5686847018" id="ydp688a2fd4yqtfd71515"><br clear="none"></div></div></div>
</div>
</div></body></html>