[OpenSIPS-Users] Mysql Crashes and panics

[Geoffrey Mina]

It sounds to me like your rating script is doing a full table lock and
thus not allowing opensips to insert/update rows into the table until
it completes.  Are you using InnoDB or MyISAM tables?  If you are
using InnoDB I would do a "SHOW INNODB STATUS" at the time of the
'lockup' to verify that you have, in fact, locked the entire table,
and namely the next index.

On Thu, Apr 16, 2009 at 4:27 AM, Bogdan-Andrei Iancu
<bogdan at voice-system.ro> wrote:
> Hi Brett,
>
> Brett Nemeroff wrote:
>> All,
>> Ok, so I fixed my kernel panic issue
> Cool, what was the cause ?
>
>
>> .. But I'm not real happy with the fix. I'm loading up like 157,000
>> routes.. each route has between 1-4 gateways in the gwlist. however,
>> **I only have 1 actual route in the dr_gateways table**.. I hadn't
>> added the other 3 gateways..
>>
>> Now when dr_reload runs, it sucks up routes, but if there is a missing
>> gateway for that route, the entire route is dropped. I didn't expect
>> that, I thought it would have just skipped loading that gateway for
>> that route.
> Make sense - please open a bug report on the tracker about this.
>> Seems that all of that loading / unloading of routes halted my system.
>> I have no idea why.. But it's consistent and repeatable **on Ubuntu
>> 8.04** I did the same setup, same routes in CentOS 5.2 and could not
>> for the life of me reproduce the problem.
> So, do you think it is OS or setup-only specific ?
>>
>> It's also worth mentioning that I hadn't increased the SHM in that
>> config...
> but do you get any error about mem (like running out of mem).
>>
>>
>> Ok, so different issue.. This is the same "opensips crashes under
>> heavy db load" issue.. I load up Opensips with 100 CPS with SIPP. It's
>> real busy, and processing real happily.
> OK
>>
>> Then I run an external perl based rating script.. For about 15-20
>> second, all is good.. then I start getting retrans, then opensips
>> dies. I reported this like a week ago, and I think bogdan pushed out
>> some fixes that are in 1.5.1 (which I am running)... Well, my rating
>> script was running, and I got a DateTime error.. it was telling me
>> that a Date/Time written in the DB wasn't valid..
> Could you post the exact error from the log ?
>> so I checked it out.. Look at this from my acc table:
>>
>>
>> +-------+--------+------------+-------------------------+----------------------------+----------+----------------------------+---------------------+-----------------------+------------+--------+------------------+------------+-----------+------+-------+
>> | id    | method | from_tag   | to_tag                  | callid
>>               | sip_code | sip_reason                 | time
>>      | ua                    | from_did   | to_did | srcip
>>  | gw         | harvested | pdd  | trace |
>> +-------+--------+------------+-------------------------+----------------------------+----------+----------------------------+---------------------+-----------------------+------------+--------+------------------+------------+-----------+------+-------+
>> | 69622 | 0      | 1510444289 | 22645-3410 at 1.2.3.4
>> <mailto:22645-3410 at 1.2.3.4> | sip:sipp at 1.2.3.4:5061
>> <http://sip:sipp@1.2.3.4:5061> | 341      | sip:9999 at 1.2.3.4:5060
>> <http://sip:9999@1.2.3.4:5060> | 0000-00-00 00:00:00 |
>> udp:1.2.3.4:5060 <http://1.2.3.4:5060> | 1239854732 | 4      |
>> udp:1.2.3.4 | 1239897933 |         0 |    1 |     1 |
>> +-------+--------+------------+-------------------------+----------------------------+----------+----------------------------+---------------------+-----------------------+------------+--------+------------------+------------+-----------+------+-------+
>> 1 row in set (0.00 sec)
>>
>> The data is all in the wrong fields!! This happened on exactly ONE record.
> yeap....it looks messy...
>>
>> In addition, when I watch "SHOW PROCESSLIST" in mysql, i normally
>> (even at 100CPS) just see like 4-6 "acc" inserts
> do not fully follow you here - 100CPS should produce (INVITE+BYE)*100
> acc records per sec -> 200 acc per sec.
>> .. when my rating script runs, I see my screen fill up (completely)
>> with "acc" inserts. So I'm about to test this with the max connections
>> in mysql raised up to.. say... 1000. Maybe the cause is running out of
>> mysql connections for acc transactions under high load..
> opensips does not dynamically create more DB connection - whatever the
> load is, the number of DB connection is the same - their number depends
> only of the number of processes in opensips.
>
> Regards,
> Bogdan
>>
>> -Brett
>>
>>
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> Users mailing list
>> Users at lists.opensips.org
>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>>
>
>
> _______________________________________________
> Users mailing list
> Users at lists.opensips.org
> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>