[OpenSIPS-Users] Fine tuning high CPS and msyql queries

[Calvin Ellison]

I attempted to reproduce the original breakdown around 3000 CPS using the
default 212992 byte receive buffer and could not, which tells me I broke a
cardinal rule of load testing and changed more than one thing at a time.
Also, don't do load testing when tired. I suspect that I had also made a
change to the sipp scenario recv/sched loops, or I had unknowingly
broken something while checking out the tuned package.

I deeply appreciate Alex's instance that I was wrong and to keep digging. I
am happy to retract my claim regarding "absolutely terrible sysctl
defaults". Using synchronous/blocking DB queries, the 8-core server reached
14,000 CPS, at which point I declared it fixed and went to bed. It could
probably go higher: there's only one DB query with a <10ms response time,
Memcache for the query response, and some logic to decide how to respond.
There's only a single non-200 final response, so it's probably as
minimalist as it gets.

If anyone else is trying to tune their setup, I think Alex's advice to "not
run more than 2 * (CPU threads) [children]" is the best place to start. I
had inherited this project from someone else's work under version 1.11 and
they had used 128 children. They were using remote DB servers with much
higher latency than the local DBs we have today, so that might have been
the reason. Or they were just wrong to being with.

The Description for Asynchronous Statements is extremely tempting and was
what started me down that path; it might be missing a qualification that
Async can be an improvement for slow blocking operations, but the
additional overhead may be a disadvantage for very fast blocking
operations.

Thank you to everyone who responded to this topic.

Regards,

*Calvin Ellison*
Senior Voice Operations Engineer
calvin.ellison at voxox.com


On Fri, Jun 12, 2020 at 6:42 PM Alex Balashov <abalashov at evaristesys.com>
wrote:
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opensips.org/pipermail/users/attachments/20200615/41d4ccba/attachment.html>