[OpenSIPS-Users] Opensips 1.6 crash
Bogdan-Andrei Iancu
bogdan at voice-system.ro
Wed Jan 20 19:26:12 CET 2010
yeah, sometimes a double coredump happens: a worker process crashes due
a bug -> opensips starts shutdown in main process -> this one crashes
too mainly because inconsistency in data/mem/locks from the first
crashed process.
I got your email -> I will take a look.
Thanks and regards,
Bogdan
Jeff Pyle wrote:
> Sorry, you asked for logs. Here they are:
>
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27560]: INFO:core:handle_sigs: child process 27595 exited by a signal 11
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27560]: INFO:core:handle_sigs: core was generated
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27560]: INFO:core:handle_sigs: terminating due to SIGCHLD
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27588]: INFO:core:sig_usr: signal 15 received
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27597]: INFO:core:sig_usr: signal 15 received
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27596]: INFO:core:sig_usr: signal 15 received
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27598]: INFO:core:sig_usr: signal 15 received
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27604]: INFO:core:sig_usr: signal 15 received
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27605]: INFO:core:sig_usr: signal 15 received
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27572]: INFO:core:sig_usr: signal 15 received
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27580]: INFO:core:sig_usr: signal 15 received
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27570]: INFO:core:sig_usr: signal 15 received
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27573]: INFO:core:sig_usr: signal 15 received
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27581]: INFO:core:sig_usr: signal 15 received
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27577]: INFO:core:sig_usr: signal 15 received
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27587]: INFO:core:sig_usr: signal 15 received
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27578]: INFO:core:sig_usr: signal 15 received
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27579]: INFO:core:sig_usr: signal 15 received
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27589]: INFO:core:sig_usr: signal 15 received
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27590]: INFO:core:sig_usr: signal 15 received
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27591]: INFO:core:sig_usr: signal 15 received
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27606]: INFO:core:sig_usr: signal 15 received
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27571]: INFO:core:sig_usr: signal 15 received
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27594]: INFO:core:sig_usr: signal 15 received
> Jan 20 11:40:56 hostname /usr/local/sbin/opensips[27607]: INFO:core:sig_usr: signal 15 received
> Jan 20 11:40:57 hostname /usr/local/sbin/opensips[27560]: INFO:db_mysql:get_new_stmt_ctx: disconect event for 0x8248390
> Jan 20 11:40:57 hostname /usr/local/sbin/opensips[27560]: INFO:db_mysql:reset_all_statements: reseting all statements on connection: (0x8248798) 0x8248390
> Jan 20 11:40:57 hostname /usr/local/sbin/opensips[27560]: INFO:db_mysql:get_new_stmt_ctx: re-connected successful for 0x8248390
>
> I wrote it very poorly before... there were two cores dumped. I posted the backtrace for the one mentioned in the syslog. The backtrace for other I can send you privately.
>
>
> Thanks,
> Jeff
>
>
> On Jan 20, 2010, at 12:56 PM, Jeff Pyle wrote:
>
>
>> Hi Bogdan,
>>
>> The only mention of a core in the syslog was the 27560 one:
>>
>> /usr/local/sbin/opensips[27560]: INFO:core:handle_sigs: core was generated
>>
>> The backtrace I sent was for this one. The only other core generated. Its backtrace is significantly longer. Because of the semi-private nature of some of the headers it contains I can send it to you privately if you think it would be helpful.
>>
>>
>> - Jeff
>>
>>
>> On Jan 20, 2010, at 12:36 PM, Bogdan-Andrei Iancu wrote:
>>
>>
>>> Hi Jeff,
>>>
>>> For the records, if no otherwise configured (via the -w cli param),
>>> opensips (when forking) will change the working dir to / (root
>>> filesystem).
>>>
>>> Now, about the core- the back trace you see here is showing a crash
>>> during shutdown (main->handle_sigs->cleanup...). So, you need to
>>> identify if indeed the crash was during a normal shutdown or if the
>>> crash you have the core for is just a side effect of a previous crash of
>>> another process (that triggered shutdown).
>>>
>>> Could you post the relevant (to crash) logs ?
>>>
>>> Regards,
>>> Bogdan
>>>
>>> Jeff Pyle wrote:
>>>
>>>> Never mind, I'm a little slow today. I found /core.27560. A backtrace shows the following:
>>>>
>>>> #0 0x00a16827 in free () from /lib/i686/nosegneg/libc.so.6
>>>> No symbol table info available.
>>>> #1 0x0017e021 in my_no_flags_free (ptr=0x9b00000) at my_malloc.c:59
>>>> No locals.
>>>> #2 0x0017f863 in free_root (root=0x9b1dc30, MyFlags=0) at my_alloc.c:355
>>>> next = <value optimized out>
>>>> #3 0x0017b218 in mysql_stmt_close (stmt=0x9bffd00) at libmysql.c:5058
>>>> mysql = (MYSQL *) 0x82483c4
>>>> #4 0x0016cc2c in db_mysql_free_stmt_list (head=0x0) at dbase.c:160
>>>> No locals.
>>>> #5 0x00173186 in db_mysql_free_connection (con=0x8248390) at my_con.c:138
>>>> No locals.
>>>> #6 0x08126449 in db_do_close (_h=0x82482d4, free_connection=0x17315c <db_mysql_free_connection>) at db/db.c:310
>>>> con = (struct pool_con *) 0x8248390
>>>> __FUNCTION__ = "db_do_close"
>>>> #7 0x0016cbb3 in db_mysql_close (_h=0x82482d4) at dbase.c:583
>>>> No locals.
>>>> #8 0x00b537ae in destroy () at ul_mod.c:369
>>>> __FUNCTION__ = "destroy"
>>>> #9 0x080be211 in destroy_modules () at sr_module.c:370
>>>> t = (struct sr_module *) 0x81b7cc8
>>>> foo = (struct sr_module *) 0x81b7c20
>>>> #10 0x0806b0a0 in cleanup (show_status=1) at main.c:336
>>>> No locals.
>>>> #11 0x0806bfd4 in handle_sigs () at main.c:533
>>>> chld = 0
>>>> chld_status = 139
>>>> i = 6
>>>> do_exit = 1
>>>> ---Type <return> to continue, or q <return> to quit---
>>>> __FUNCTION__ = "handle_sigs"
>>>> #12 0x08070564 in main (argc=3, argv=0xbfdc6654) at main.c:913
>>>> cfg_log_stderr = 0
>>>> cfg_stream = (FILE *) 0x9b0a008
>>>> c = <value optimized out>
>>>> r = 0
>>>> tmp = 0xbfdc7f74 ""
>>>> tmp_len = <value optimized out>
>>>> port = 10309893
>>>> proto = <value optimized out>
>>>> ret = <value optimized out>
>>>> seed = 266001097
>>>> rfd = 4
>>>> __FUNCTION__ = "main"
>>>>
>>>>
>>>> This is on 1.6 rev 6511. I the mysql libraries come from RPMs installed, version 6.3.20 (cluster) from Mysql's site.
>>>>
>>>> Any thoughts?
>>>>
>>>>
>>>> Thanks,
>>>> Jeff
>>>>
>>>>
>>>>
>>>> On Jan 20, 2010, at 12:00 PM, Jeff Pyle wrote:
>>>>
>>>>
>>>>
>>>>> Hello,
>>>>>
>>>>> Opensips 1.6 dumped on me. I see the following in the syslog:
>>>>>
>>>>> /usr/local/sbin/opensips[27560]: INFO:core:handle_sigs: core was generated
>>>>>
>>>>> Where might this file be, and what might it be called? I'm having trouble locating it. I start Opensips with the Fedora init.d file, calling the daemon like this:
>>>>>
>>>>> daemon $oser -m 128 $OPTIONS 2>/dev/null | tail -1
>>>>>
>>>>> There is nothing in $OPTIONS. $oser = /usr/local/sbin/opensips
>>>>>
>>>>> I've seen cores dump into /tmp before but that's not the case this time. Any suggestions would be appreciated it.
>>>>>
>>>>>
>>>>> Thanks,
>>>>> Jeff
>>>>> _______________________________________________
>>>>> Users mailing list
>>>>> Users at lists.opensips.org
>>>>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>>>>>
>>>>>
>>>> _______________________________________________
>>>> Users mailing list
>>>> Users at lists.opensips.org
>>>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>>>>
>>>>
>>>>
>>> --
>>> Bogdan-Andrei Iancu
>>> www.voice-system.ro
>>>
>>>
>>> _______________________________________________
>>> Users mailing list
>>> Users at lists.opensips.org
>>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>>>
>
>
> _______________________________________________
> Users mailing list
> Users at lists.opensips.org
> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>
>
--
Bogdan-Andrei Iancu
www.voice-system.ro
More information about the Users
mailing list