[OpenSIPS-Users] Opensips stops responding

[Schneur Rosenberg]

Hi, I've had recently a few times that OpenSIPSs didnt crash but it
stopped responding to certain requests, perhaps INVITES that requeired
DB access, but I havent verified it becuase I always wanted to bring
it back up ASAP because it seems like it did respond to my requests
that my keepalived VRRP script which sends a notify to OpenSIPs and
OpenSIPS does a DB lookup and only on success it returns a true thats
how I make sure that MYSQL is working in addition to  OpenSIPs, but
opensips seems to have responded because it did not failover, and
thats why I didnt have time to investigate further to which requests
it responds and to which it didnt.

Here are the syslogs from the time of the crash, maybe it can shed
some light to the issue, the query in there that failed is the query
from keepalived, I assume it failed only once and then did respond,
otherwise the IP would of failed over to the backup, but its only an
assumption.

Jul 10 20:13:13 sipsvr1 /sbin/opensips[29446]:
INFO:db_mysql:switch_state_to_disconnected: disconnect event for
0x7f2ba43e7600
Jul 10 20:13:13 sipsvr1 /sbin/opensips[29446]:
INFO:db_mysql:reset_all_statements: resetting all statements on
connection: (0x7f2ba44062a0) 0x7f2ba43e7600
Jul 10 20:13:13 sipsvr1 /sbin/opensips[29446]:
INFO:db_mysql:connect_with_retry: re-connected successful for
0x7f2ba43e7600
Jul 10 20:13:13 sipsvr1 /sbin/opensips[29446]:
INFO:db_mysql:db_mysql_do_prepared_query: reconnected to mysql server
-> re-init the statement
Jul 10 20:13:13 sipsvr1 /sbin/opensips[29486]:
WARNING:core:handle_timer_job: utimer job <tm-utimer> has a 30000 us
delay in execution
Jul 10 20:13:15 sipsvr1 /sbin/opensips[29400]:
INFO:clusterer:do_action_trans_2: Ping reply not received, node [2] is
down
Jul 10 20:13:15 sipsvr1 /sbin/opensips[29400]:
INFO:clusterer:do_action_trans_2: Ping reply not received, node [2] is
down
Jul 10 20:13:15 sipsvr1 /sbin/opensips[29478]:
INFO:clusterer:handle_internal_msg: Node [2] is UP
Jul 10 20:13:15 sipsvr1 /sbin/opensips[29478]:
INFO:clusterer:handle_internal_msg: Node [2] is UP
Jul 10 20:13:15 sipsvr1 /sbin/opensips[29422]:
INFO:presence:update_presentity: *** found in db but not in htable
[a.1561468443.29425.374568.2]
Jul 10 20:13:15 sipsvr1 /sbin/opensips[29419]:
INFO:presence:update_presentity: *** found in db but not in htable
[a.1561468443.29431.1000819.1]
Jul 10 20:13:15 sipsvr1 /sbin/opensips[29428]:
INFO:presence:update_presentity: *** found in db but not in htable
[a.1561468443.29411.161103.9]
Jul 10 20:13:15 sipsvr1 /sbin/opensips[29446]:
CRITICAL:db_mysql:wrapper_single_mysql_stmt_execute: driver error
(1180): Got error 35 "Resource deadlock avoided" during COMMIT
Jul 10 20:13:15 sipsvr1 /sbin/opensips[29446]:
ERROR:usrloc:db_update_ucontact: updating database failed
Jul 10 20:13:15 sipsvr1 /sbin/opensips[29446]: ERROR:usrloc:wb_timer:
updating contact in db failed
Jul 10 20:13:15 sipsvr1 /sbin/opensips[29388]:
CRITICAL:db_mysql:wrapper_single_mysql_real_query: driver error
(1047): WSREP has not yet prepared node for application use
Jul 10 20:13:15 sipsvr1 /sbin/opensips[29388]:
ERROR:core:db_do_raw_query: error while submitting query
Jul 10 20:13:15 sipsvr1 /sbin/opensips[29388]:
ERROR:avpops:db_query_avp: raw_query failed: db0(usr_preferences)
select domain from domain where domain='...
Jul 10 20:13:15 sipsvr1 Keepalived_vrrp[514]: pid 2092 exited with status 1
Jul 10 20:13:16 sipsvr1 /sbin/opensips[29428]:
INFO:presence:update_presentity: *** found in db but not in htable
[a.1561468443.29422.274079.61]
Jul 10 20:13:16 sipsvr1 /sbin/opensips[29429]:
INFO:presence:update_presentity: *** found in db but not in htable
[a.1561468443.29421.236751.61]
Jul 10 20:13:16 sipsvr1 /sbin/opensips[29425]:
INFO:presence:update_presentity: *** found in db but not in htable
[a.1561468443.29419.224887.28]
Jul 10 20:13:16 sipsvr1 /sbin/opensips[29422]:
INFO:presence:update_presentity: *** found in db but not in htable
[a.1561468443.29413.165448.9]
Jul 10 20:13:16 sipsvr1 /sbin/opensips[29419]:
INFO:presence:update_presentity: *** found in db but not in htable
[a.1561468443.29429.711965.1]
Jul 10 20:13:16 sipsvr1 /sbin/opensips[29411]:
INFO:presence:update_presentity: *** found in db but not in htable
[a.1561468443.29431.1000786.2]
Jul 10 20:13:16 sipsvr1 /sbin/opensips[29428]:
INFO:presence:update_presentity: *** found in db but not in htable
[a.1561468443.29411.161104.10]
Jul 10 20:13:16 sipsvr1 /sbin/opensips[29421]:
INFO:presence:update_presentity: *** found in db but not in htable
[a.1561468443.29419.224888.10]
Jul 10 20:13:16 sipsvr1 /sbin/opensips[29433]:
INFO:presence:update_presentity: *** found in db but not in htable
[a.1561468443.29421.236752.28]