[OpenSIPS-Users] Opensips 1.6 crash

Jeff Pyle jpyle at fidelityvoice.com
Thu Jan 21 06:14:48 CET 2010 

Hi Bogdan,

A reversion to an earlier build where I didn't have problems didn't solve the problem.  It must have been triggered by a recent config change.  I recently added the media_relay_avp option to the mediaproxy module.  I've commented it out to see if that's relevant.

I sent you a new backtrace from the core dumped from the earlier build.


- Jeff


On Jan 20, 2010, at 1:26 PM, Bogdan-Andrei Iancu wrote:

> yeah, sometimes a double coredump happens: a worker process crashes due 
> a bug -> opensips starts shutdown in main process -> this one crashes 
> too mainly because inconsistency in data/mem/locks from the first 
> crashed process.
> 
> I got your email -> I will take a look.
> 
> Thanks and regards,
> Bogdan
> 
> Jeff Pyle wrote:
>> Sorry, you asked for logs.  Here they are:
>> 
>> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27560]: INFO:core:handle_sigs: child process 27595 exited by a signal 11 
>> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27560]: INFO:core:handle_sigs: core was generated 
>> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27560]: INFO:core:handle_sigs: terminating due to SIGCHLD 
>> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27588]: INFO:core:sig_usr: signal 15 received 
>> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27597]: INFO:core:sig_usr: signal 15 received 
>> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27596]: INFO:core:sig_usr: signal 15 received 
>> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27598]: INFO:core:sig_usr: signal 15 received 
>> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27604]: INFO:core:sig_usr: signal 15 received 
>> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27605]: INFO:core:sig_usr: signal 15 received 
>> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27572]: INFO:core:sig_usr: signal 15 received 
>> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27580]: INFO:core:sig_usr: signal 15 received 
>> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27570]: INFO:core:sig_usr: signal 15 received 
>> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27573]: INFO:core:sig_usr: signal 15 received 
>> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27581]: INFO:core:sig_usr: signal 15 received 
>> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27577]: INFO:core:sig_usr: signal 15 received 
>> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27587]: INFO:core:sig_usr: signal 15 received 
>> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27578]: INFO:core:sig_usr: signal 15 received 
>> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27579]: INFO:core:sig_usr: signal 15 received 
>> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27589]: INFO:core:sig_usr: signal 15 received 
>> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27590]: INFO:core:sig_usr: signal 15 received 
>> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27591]: INFO:core:sig_usr: signal 15 received 
>> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27606]: INFO:core:sig_usr: signal 15 received 
>> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27571]: INFO:core:sig_usr: signal 15 received 
>> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27594]: INFO:core:sig_usr: signal 15 received 
>> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27607]: INFO:core:sig_usr: signal 15 received 
>> Jan 20 11:40:57 hostname /usr/local/sbin/opensips[27560]: INFO:db_mysql:get_new_stmt_ctx: disconect event for 0x8248390 
>> Jan 20 11:40:57 hostname /usr/local/sbin/opensips[27560]: INFO:db_mysql:reset_all_statements: reseting all statements on connection: (0x8248798) 0x8248390 
>> Jan 20 11:40:57 hostname /usr/local/sbin/opensips[27560]: INFO:db_mysql:get_new_stmt_ctx: re-connected successful for 0x8248390 
>> 
>> I wrote it very poorly before... there were two cores dumped.  I posted the backtrace for the one mentioned in the syslog.  The backtrace for other I can send you privately.
>> 
>> 
>> Thanks,
>> Jeff
>> 
>> 
>> On Jan 20, 2010, at 12:56 PM, Jeff Pyle wrote:
>> 
>> 
>>> Hi Bogdan,
>>> 
>>> The only mention of a core in the syslog was the 27560 one:
>>> 
>>> /usr/local/sbin/opensips[27560]: INFO:core:handle_sigs: core was generated
>>> 
>>> The backtrace I sent was for this one.  The only other core generated.  Its backtrace is significantly longer.  Because of the semi-private nature of some of the headers it contains I can send it to you privately if you think it would be helpful.
>>> 
>>> 
>>> - Jeff
>>> 
>>> 
>>> On Jan 20, 2010, at 12:36 PM, Bogdan-Andrei Iancu wrote:
>>> 
>>> 
>>>> Hi Jeff,
>>>> 
>>>> For the records, if no otherwise configured (via the -w cli param), 
>>>> opensips (when forking) will change the working dir to /   (root 
>>>> filesystem).
>>>> 
>>>> Now, about the core- the back trace you see here is showing a crash 
>>>> during shutdown (main->handle_sigs->cleanup...). So, you need to 
>>>> identify if indeed the crash was during a normal shutdown or if the 
>>>> crash you have the core for is just a side effect of a previous crash of 
>>>> another process (that triggered shutdown).
>>>> 
>>>> Could you post the relevant (to crash) logs ?
>>>> 
>>>> Regards,
>>>> Bogdan
>>>> 
>>>> Jeff Pyle wrote:
>>>> 
>>>>> Never mind, I'm a little slow today.  I found /core.27560.  A backtrace shows the following:
>>>>> 
>>>>> #0  0x00a16827 in free () from /lib/i686/nosegneg/libc.so.6
>>>>> No symbol table info available.
>>>>> #1  0x0017e021 in my_no_flags_free (ptr=0x9b00000) at my_malloc.c:59
>>>>> No locals.
>>>>> #2  0x0017f863 in free_root (root=0x9b1dc30, MyFlags=0) at my_alloc.c:355
>>>>> 	next = <value optimized out>
>>>>> #3  0x0017b218 in mysql_stmt_close (stmt=0x9bffd00) at libmysql.c:5058
>>>>> 	mysql = (MYSQL *) 0x82483c4
>>>>> #4  0x0016cc2c in db_mysql_free_stmt_list (head=0x0) at dbase.c:160
>>>>> No locals.
>>>>> #5  0x00173186 in db_mysql_free_connection (con=0x8248390) at my_con.c:138
>>>>> No locals.
>>>>> #6  0x08126449 in db_do_close (_h=0x82482d4, free_connection=0x17315c <db_mysql_free_connection>) at db/db.c:310
>>>>> 	con = (struct pool_con *) 0x8248390
>>>>> 	__FUNCTION__ = "db_do_close"
>>>>> #7  0x0016cbb3 in db_mysql_close (_h=0x82482d4) at dbase.c:583
>>>>> No locals.
>>>>> #8  0x00b537ae in destroy () at ul_mod.c:369
>>>>> 	__FUNCTION__ = "destroy"
>>>>> #9  0x080be211 in destroy_modules () at sr_module.c:370
>>>>> 	t = (struct sr_module *) 0x81b7cc8
>>>>> 	foo = (struct sr_module *) 0x81b7c20
>>>>> #10 0x0806b0a0 in cleanup (show_status=1) at main.c:336
>>>>> No locals.
>>>>> #11 0x0806bfd4 in handle_sigs () at main.c:533
>>>>> 	chld = 0
>>>>> 	chld_status = 139
>>>>> 	i = 6
>>>>> 	do_exit = 1
>>>>> ---Type <return> to continue, or q <return> to quit---
>>>>> 	__FUNCTION__ = "handle_sigs"
>>>>> #12 0x08070564 in main (argc=3, argv=0xbfdc6654) at main.c:913
>>>>> 	cfg_log_stderr = 0
>>>>> 	cfg_stream = (FILE *) 0x9b0a008
>>>>> 	c = <value optimized out>
>>>>> 	r = 0
>>>>> 	tmp = 0xbfdc7f74 ""
>>>>> 	tmp_len = <value optimized out>
>>>>> 	port = 10309893
>>>>> 	proto = <value optimized out>
>>>>> 	ret = <value optimized out>
>>>>> 	seed = 266001097
>>>>> 	rfd = 4
>>>>> 	__FUNCTION__ = "main"
>>>>> 
>>>>> 
>>>>> This is on 1.6 rev 6511.  I the mysql libraries come from RPMs installed, version 6.3.20 (cluster) from Mysql's site.
>>>>> 
>>>>> Any thoughts?
>>>>> 
>>>>> 
>>>>> Thanks,
>>>>> Jeff
>>>>> 
>>>>> 
>>>>> 
>>>>> On Jan 20, 2010, at 12:00 PM, Jeff Pyle wrote:
>>>>> 
>>>>> 
>>>>> 
>>>>>> Hello,
>>>>>> 
>>>>>> Opensips 1.6 dumped on me.  I see the following in the syslog:
>>>>>> 
>>>>>> /usr/local/sbin/opensips[27560]: INFO:core:handle_sigs: core was generated
>>>>>> 
>>>>>> Where might this file be, and what might it be called?  I'm having trouble locating it.  I start Opensips with the Fedora init.d file, calling the daemon like this:
>>>>>> 
>>>>>> daemon $oser -m 128 $OPTIONS 2>/dev/null | tail -1
>>>>>> 
>>>>>> There is nothing in $OPTIONS.  $oser = /usr/local/sbin/opensips
>>>>>> 
>>>>>> I've seen cores dump into /tmp before but that's not the case this time.  Any suggestions would be appreciated it.
>>>>>> 
>>>>>> 
>>>>>> Thanks,
>>>>>> Jeff  
>>>>>> _______________________________________________
>>>>>> Users mailing list
>>>>>> Users at lists.opensips.org
>>>>>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>>>>>> 
>>>>>> 
>>>>> _______________________________________________
>>>>> Users mailing list
>>>>> Users at lists.opensips.org
>>>>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>>>>> 
>>>>> 
>>>>> 
>>>> -- 
>>>> Bogdan-Andrei Iancu
>>>> www.voice-system.ro
>>>> 
>>>> 
>>>> _______________________________________________
>>>> Users mailing list
>>>> Users at lists.opensips.org
>>>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>>>> 
>> 
>> 
>> _______________________________________________
>> Users mailing list
>> Users at lists.opensips.org
>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>> 
>> 
> 
> 
> -- 
> Bogdan-Andrei Iancu
> www.voice-system.ro
> 
> 
> _______________________________________________
> Users mailing list
> Users at lists.opensips.org
> http://lists.opensips.org/cgi-bin/mailman/listinfo/users

More information about the Users mailing list