<div>Hello,</div><div> </div><div>I've got a weird one here.</div><div><br></div><div>I have 8 Opensips 1.6 instances in Xen domUs spread across two dom0s. Linux on dom0s and domUs is Debian Lenny. Apparently all at one time they became kernel blocked, and two minutes later I got messages like the following on the domUs:</div>
<div><br></div><blockquote class="gmail_quote" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex; ">
INFO: task opensips:17856 blocked for more than 120 seconds<br>"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.</blockquote><div><br></div><div>It's still unclear to me whether all Opensips processes hung on all machines. During this time, Opensips through syslog was complaining about DB connectivity issues across the board, including messages like this:</div>
<div><br></div><blockquote class="gmail_quote" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex; ">
ERROR:db_mysql:db_mysql_connect: driver error(2003): Can't connect to MySQL server on 'dbcluster' (111)</blockquote><div><br></div><div>'dbcluster' is referenced in /etc/hosts to point to a mysqld instance of MySQL cluster, different for each domU depending on which dom0 it resides on. I was able to connect to 'dbcluster' IPs from the mysql cli client, however. There were other indicators the cluster itself was fine, including other connections from non-Opensips clients on other machines. Only Opensips was affected, and it was affected everywhere in my network.</div>
<div><br></div><div>The first time we tried to reset the opensips processes with the rc.d script it failed to return. Possibly because it was still having trouble connecting to the db. But, then it did restart with no explanation.</div>
<div><br></div><div>Does any of this sound familiar to anyone?</div><div><br></div><div><br></div><div>- Jeff</div>