[OpenSIPS-Users] Rest Client Async operation

[Ben Newlin]

Liviu,

Thank you. This makes complete sense now and the documentation updates are excellent. Thanks as well for the workaround suggestion; we will investigate whether it is necessary. You do make a good point in the docs about TCP connection reuse in the workers, which shows my example as a bit of an extreme case. In practical use it is more likely the connection would exist and the transfer would timeout instead, which would be async.

Even though the example may be extreme, in the current world of cloud computing having the remote server “disappear” without closing TCP connections is becoming a much more common concern. Cloud instances can and do just disappear and the connections will hang; cloud networks have issues and can leave connections hung as well. We have been wrestling with these issues with the db_postgres and db_virtual modules for some time and have an open ticket for it now. While it is impossible to avoid some service impairment when a necessary remote component is no longer available, having the entire call processing application block on the resource is also not a very good response. We are always looking for better ways to gracefully handle the unexpected.

Thanks for the time and quick responses!

Ben Newlin

From: Users <users-bounces at lists.opensips.org> on behalf of Liviu Chircu <liviu at opensips.org>
Reply-To: OpenSIPS users mailling list <users at lists.opensips.org>
Date: Thursday, June 27, 2019 at 5:05 AM
To: OpenSIPS users mailling list <users at lists.opensips.org>
Subject: Re: [OpenSIPS-Users] Rest Client Async operation


On 26.06.2019 17:48, Ben Newlin wrote:
Thanks again for the info. I think what you are saying is that the async operation is not launching a new process to handle the called function, but is performing the function in the original worker thread and only taking advantage of any suspend/resume or polling functionality already exposed by the underlying function itself.
Just to clear this up:  the underlying functions themselves need not offer any polling functionality,
they just need to meet two criteria: (1) be non-blocking; (2) provide a valid fd for the async engine to poll on.
Any blocking I/O functionality, be it within libcurl, MySQL, etc. that meets the above can be adapted to
work with the async engine available in OpenSIPS 2.1+.

I understand that the practicalities of the implementation in OpenSIPS may have required this design, but I must re-iterate that these limitations need to be documented very carefully as they are very important to understand when designing OpenSIPS scripts with async functionality and are not described anywhere. I could not find anywhere in the documentation that indicates that async operations can potentially still block the original worker thread and block call processing. The closest is:

“The current OpenSIPS worker will launch the asynchronous operation, after which it will continue to process other pending tasks”

But this provides no elaboration on what it means for the worker to “launch” the operation, and more importantly it does not indicate that the launching itself can block, which is the key issue here.
Agreed - will try to find a way to integrate this corner-case into the docs, somehow.

As I said, this unfortunately makes async processing mostly useless for us. For both DB and REST queries if only the data transfer is async then it is only useful when the data being transferred is extremely large or prone to delays/jitter. Such transfers should be avoided during realtime processing whether async or not, as they will still delay the individual call even if not others. For small payloads, like the single JSON object common in REST responses, it is the connection itself that is the concern. Once connected, running the data transfer in async mode represents no real gain.
Then I recommend you stop using rest_client, which currently optimizes system resource
usage for setups where the TCP connect cannot possibly ever hang, and resort to forking
a process for each HTTP request, using a construct such as:

    async(exec("curl your_connect_hanging_http_endpoint"), resume_route);

Although forking a process for each request is a costly operation which will
eat more system resources during normal operation, at least this solution optimizes
for the worst case, when the HTTP server is down.  In this latter case, the
throughput of your SIP server won't be hindered that much, as the hanging connect
will be done asynchronously.

Best regards,

Liviu Chircu

OpenSIPS Developer

http://www.opensips-solutions.com<http://www.opensips-solutions.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opensips.org/pipermail/users/attachments/20190627/03519d14/attachment-0001.html>