I hope this should go to the devel mailing list - it was not clear to me. <br><br>We recently implemented Opensips 1.6.4 to begin use the ability to search across dialogs in order to allow for attended transfers with multiple Asterisk servers in a load balanced setup. Today we had a crash. Below is the log entries and the full backtrace. I mention the dialog search feature only because even I can tell that is a big part of the backtrace. :-) I XXXX'ed out a few host names and those are available on request if necessary.<br>
<br>If this should have been in the users list my apologies. Let me know if there is anything I can provide or clues to the issue.<br><br>-James<br><br><br><log><br>Feb 16 18:05:16 ip-10-194-153-140 opensips[20576]: INFO:core:send2child: no free tcp receiver, connection passed to the least busy one (1)<br>
Feb 16 18:05:21 ip-10-194-153-140 opensips[20576]: last message repeated 4 times<br>Feb 16 18:05:21 ip-10-194-153-140 opensips[20576]: CRITICAL:core:receive_fd: EOF on 19<br>Feb 16 18:05:21 ip-10-194-153-140 opensips[20496]: INFO:core:handle_sigs: child process 20507 exited by a signal 11<br>
Feb 16 18:05:21 ip-10-194-153-140 opensips[20496]: INFO:core:handle_sigs: core was generated<br>Feb 16 18:05:21 ip-10-194-153-140 opensips[20496]: INFO:core:handle_sigs: terminating due to SIGCHLD<br>Feb 16 18:05:21 ip-10-194-153-140 opensips[20571]: INFO:core:sig_usr: signal 15 received<br>
Feb 16 18:05:21 ip-10-194-153-140 opensips[20558]: INFO:core:sig_usr: signal 15 received<br><br><bt><br>gdb /usr/local/sbin/opensips /home/ubuntu/core.opensips.0216 <br>GNU gdb (GDB) 7.1-ubuntu<br>Copyright (C) 2010 Free Software Foundation, Inc.<br>
License GPLv3+: GNU GPL version 3 or later <<a href="http://gnu.org/licenses/gpl.html">http://gnu.org/licenses/gpl.html</a>><br>This is free software: you are free to change and redistribute it.<br>There is NO WARRANTY, to the extent permitted by law. Type "show copying"<br>
and "show warranty" for details.<br>This GDB was configured as "i486-linux-gnu".<br>For bug reporting instructions, please see:<br><<a href="http://www.gnu.org/software/gdb/bugs/">http://www.gnu.org/software/gdb/bugs/</a>>...<br>
Reading symbols from /usr/local/sbin/opensips...done.<br>[New Thread 20496]<br><br>warning: Can't read pathname for load map: Input/output error.<br>Reading symbols from /lib/tls/i686/nosegneg/libdl.so.2...(no debugging symbols found)...done.<br>
Loaded symbols for /lib/tls/i686/nosegneg/libdl.so.2<br>Reading symbols from /lib/tls/i686/nosegneg/libresolv.so.2...(no debugging symbols found)...done.<br>Loaded symbols for /lib/tls/i686/nosegneg/libresolv.so.2<br>Reading symbols from /lib/i686/cmov/libssl.so.0.9.8...(no debugging symbols found)...done.<br>
Loaded symbols for /lib/i686/cmov/libssl.so.0.9.8<br>Reading symbols from /lib/i686/cmov/libcrypto.so.0.9.8...(no debugging symbols found)...done.<br>Loaded symbols for /lib/i686/cmov/libcrypto.so.0.9.8<br>Reading symbols from /lib/tls/i686/nosegneg/libc.so.6...(no debugging symbols found)...done.<br>
Loaded symbols for /lib/tls/i686/nosegneg/libc.so.6<br>Reading symbols from /lib/ld-linux.so.2...(no debugging symbols found)...done.<br>Loaded symbols for /lib/ld-linux.so.2<br>Reading symbols from /lib/libz.so.1...(no debugging symbols found)...done.<br>
Loaded symbols for /lib/libz.so.1<br>Reading symbols from /usr/local/lib/opensips/modules/db_mysql.so...done.<br>Loaded symbols for /usr/local/lib/opensips/modules/db_mysql.so<br>Reading symbols from /usr/lib/libmysqlclient.so.16...(no debugging symbols found)...done.<br>
Loaded symbols for /usr/lib/libmysqlclient.so.16<br>Reading symbols from /lib/tls/i686/nosegneg/libpthread.so.0...(no debugging symbols found)...done.<br>Loaded symbols for /lib/tls/i686/nosegneg/libpthread.so.0<br>Reading symbols from /lib/tls/i686/nosegneg/libcrypt.so.1...(no debugging symbols found)...done.<br>
Loaded symbols for /lib/tls/i686/nosegneg/libcrypt.so.1<br>Reading symbols from /lib/tls/i686/nosegneg/libnsl.so.1...(no debugging symbols found)...done.<br>Loaded symbols for /lib/tls/i686/nosegneg/libnsl.so.1<br>Reading symbols from /lib/tls/i686/nosegneg/libm.so.6...(no debugging symbols found)...done.<br>
Loaded symbols for /lib/tls/i686/nosegneg/libm.so.6<br>Reading symbols from /usr/local/lib/opensips/modules/signaling.so...done.<br>Loaded symbols for /usr/local/lib/opensips/modules/signaling.so<br>Reading symbols from /usr/local/lib/opensips/modules/sl.so...done.<br>
Loaded symbols for /usr/local/lib/opensips/modules/sl.so<br>Reading symbols from /usr/local/lib/opensips/modules/tm.so...done.<br>Loaded symbols for /usr/local/lib/opensips/modules/tm.so<br>Reading symbols from /usr/local/lib/opensips/modules/rr.so...done.<br>
Loaded symbols for /usr/local/lib/opensips/modules/rr.so<br>Reading symbols from /usr/local/lib/opensips/modules/maxfwd.so...done.<br>Loaded symbols for /usr/local/lib/opensips/modules/maxfwd.so<br>Reading symbols from /usr/local/lib/opensips/modules/usrloc.so...done.<br>
Loaded symbols for /usr/local/lib/opensips/modules/usrloc.so<br>Reading symbols from /usr/local/lib/opensips/modules/registrar.so...done.<br>Loaded symbols for /usr/local/lib/opensips/modules/registrar.so<br>Reading symbols from /usr/local/lib/opensips/modules/textops.so...done.<br>
Loaded symbols for /usr/local/lib/opensips/modules/textops.so<br>Reading symbols from /usr/local/lib/opensips/modules/mi_fifo.so...done.<br>Loaded symbols for /usr/local/lib/opensips/modules/mi_fifo.so<br>Reading symbols from /usr/local/lib/opensips/modules/mi_datagram.so...done.<br>
Loaded symbols for /usr/local/lib/opensips/modules/mi_datagram.so<br>Reading symbols from /usr/local/lib/opensips/modules/uri.so...done.<br>Loaded symbols for /usr/local/lib/opensips/modules/uri.so<br>Reading symbols from /usr/local/lib/opensips/modules/dialog.so...done.<br>
Loaded symbols for /usr/local/lib/opensips/modules/dialog.so<br>Reading symbols from /usr/local/lib/opensips/modules/load_balancer.so...done.<br>Loaded symbols for /usr/local/lib/opensips/modules/load_balancer.so<br>Reading symbols from /usr/local/lib/opensips/modules/nathelper.so...done.<br>
Loaded symbols for /usr/local/lib/opensips/modules/nathelper.so<br>Reading symbols from /usr/local/lib/opensips/modules/nat_traversal.so...done.<br>Loaded symbols for /usr/local/lib/opensips/modules/nat_traversal.so<br>Reading symbols from /usr/local/lib/opensips/modules/auth.so...done.<br>
Loaded symbols for /usr/local/lib/opensips/modules/auth.so<br>Reading symbols from /usr/local/lib/opensips/modules/auth_db.so...done.<br>Loaded symbols for /usr/local/lib/opensips/modules/auth_db.so<br>Reading symbols from /usr/local/lib/opensips/modules/domain.so...done.<br>
Loaded symbols for /usr/local/lib/opensips/modules/domain.so<br>Reading symbols from /usr/local/lib/opensips/modules/presence.so...done.<br>Loaded symbols for /usr/local/lib/opensips/modules/presence.so<br>Reading symbols from /usr/lib/libxml2.so.2...(no debugging symbols found)...done.<br>
Loaded symbols for /usr/lib/libxml2.so.2<br>Reading symbols from /usr/local/lib/opensips/modules/presence_xml.so...done.<br>Loaded symbols for /usr/local/lib/opensips/modules/presence_xml.so<br>Reading symbols from /usr/local/lib/opensips/modules/presence_mwi.so...done.<br>
Loaded symbols for /usr/local/lib/opensips/modules/presence_mwi.so<br>Reading symbols from /usr/local/lib/opensips/modules/permissions.so...done.<br>Loaded symbols for /usr/local/lib/opensips/modules/permissions.so<br>Reading symbols from /lib/tls/i686/nosegneg/libnss_compat.so.2...(no debugging symbols found)...done.<br>
Loaded symbols for /lib/tls/i686/nosegneg/libnss_compat.so.2<br>Reading symbols from /lib/tls/i686/nosegneg/libnss_nis.so.2...(no debugging symbols found)...done.<br>Loaded symbols for /lib/tls/i686/nosegneg/libnss_nis.so.2<br>
Reading symbols from /lib/tls/i686/nosegneg/libnss_files.so.2...(no debugging symbols found)...done.<br>Loaded symbols for /lib/tls/i686/nosegneg/libnss_files.so.2<br>Reading symbols from /lib/libnss_mdns4_minimal.so.2...(no debugging symbols found)...done.<br>
Loaded symbols for /lib/libnss_mdns4_minimal.so.2<br>Reading symbols from /lib/tls/i686/nosegneg/libnss_dns.so.2...(no debugging symbols found)...done.<br>Loaded symbols for /lib/tls/i686/nosegneg/libnss_dns.so.2<br>Core was generated by `/usr/local/sbin/opensips -P /var/run/opensips/opensips.pid -m 256 -u root -g ww'.<br>
Program terminated with signal 6, Aborted.<br>#0 0xb787c416 in __kernel_vsyscall ()<br>(gdb) bt full<br>#0 0xb787c416 in __kernel_vsyscall ()<br>No symbol table info available.<br>#1 0xb758a851 in raise () from /lib/tls/i686/nosegneg/libc.so.6<br>
No symbol table info available.<br>#2 0xb758dd42 in abort () from /lib/tls/i686/nosegneg/libc.so.6<br>No symbol table info available.<br>#3 0x0806e958 in sig_alarm_abort (signo=14) at main.c:427<br> __FUNCTION__ = "sig_alarm_abort"<br>
#4 <signal handler called><br>No symbol table info available.<br>#5 0xb787c416 in __kernel_vsyscall ()<br>No symbol table info available.<br>#6 0xb760594c in sched_yield () from /lib/tls/i686/nosegneg/libc.so.6<br>
No symbol table info available.<br>#7 0xb71c24eb in get_lock (ticks=0, param=0x0) at ../../mem/../fastlock.h:182<br> i = 0<br>#8 dialog_update_db (ticks=0, param=0x0) at dlg_db_handler.c:907<br> my_ps_update = 0x81e276c<br>
my_ps_insert = 0x0<br> index = 1551<br> values = {{type = DB_INT, nul = 0, free = 2, val = {int_val = 1519, double_val = 4.2439923324162607e-314, time_val = 1519, string_val = 0x5ef <Address 0x5ef out of bounds>, <br>
str_val = {s = 0x5ef <Address 0x5ef out of bounds>, len = 2}, blob_val = {s = 0x5ef <Address 0x5ef out of bounds>, len = 2}, bitmap_val = 1519}}, {type = DB_INT, nul = 0, <br> free = 0, val = {int_val = 1285491238, double_val = 2.6164059934745423e-312, time_val = 1285491238, string_val = 0x4c9f0a26 <Address 0x4c9f0a26 out of bounds>, str_val = {<br>
s = 0x4c9f0a26 <Address 0x4c9f0a26 out of bounds>, len = 123}, blob_val = {s = 0x4c9f0a26 <Address 0x4c9f0a26 out of bounds>, len = 123}, bitmap_val = 1285491238}}, {<br> type = DB_STR, nul = 0, free = 0, val = {int_val = 0, double_val = 5.4200094642041093e-270, time_val = 0, string_val = 0x0, str_val = {s = 0x0, len = 134670378}, <br>
blob_val = {s = 0x0, len = 134670378}, bitmap_val = 0}}, {type = DB_STR, nul = 0, free = 3, val = {int_val = 0, double_val = 0, time_val = 0, string_val = 0x0, str_val = {<br> s = 0x0, len = 0}, blob_val = {s = 0x0, len = 0}, bitmap_val = 0}}, {type = DB_STR, nul = 845914100, free = 0, val = {int_val = 0, double_val = -0.044442176818847656, <br>
time_val = 0, string_val = 0x0, str_val = {s = 0x0, len = -1079590624}, blob_val = {s = 0x0, len = -1079590624}, bitmap_val = 0}}, {type = DB_STR, nul = -1217834395, <br> free = 0, val = {int_val = 0, double_val = 1.7215751852101254e-310, time_val = 0, string_val = 0x0, str_val = {s = 0x0, len = 8113}, blob_val = {s = 0x0, len = 8113}, <br>
bitmap_val = 0}}, {type = DB_STR, nul = -1217678400, free = 3, val = {int_val = 0, double_val = 1.7215751852101254e-310, time_val = 0, string_val = 0x0, str_val = {<br> s = 0x0, len = 8113}, blob_val = {s = 0x0, len = 8113}, bitmap_val = 0}}, {type = DB_STR, nul = -1218635830, free = 0, val = {int_val = 0, <br>
double_val = -5.1493448982678538e-42, time_val = 0, string_val = 0x0, str_val = {s = 0x0, len = -1218661020}, blob_val = {s = 0x0, len = -1218661020}, bitmap_val = 0}}, {<br> type = DB_STR, nul = 8200, free = 0, val = {int_val = -1217683468, double_val = -9.9099209757563708e-42, time_val = -1217683468, string_val = 0xb76b9ff4 "|\235\025", <br>
str_val = {s = 0xb76b9ff4 "|\235\025", len = -1217683468}, blob_val = {s = 0xb76b9ff4 "|\235\025", len = -1217683468}, bitmap_val = 3077283828}}, {type = DB_INT, <br> nul = -1218652746, free = -1223437532, val = {int_val = -1223437620, double_val = -2.2222703724075424e-43, time_val = -1223437620, <br>
string_val = 0xb713d2cc "\fW\023\267\006", str_val = {s = 0xb713d2cc "\fW\023\267\006", len = -1223437628}, blob_val = {s = 0xb713d2cc "\fW\023\267\006", <br> len = -1223437628}, bitmap_val = 3071529676}}, {type = DB_STR, nul = -1223437724, free = -1223437684, val = {int_val = -1223437676, <br>
double_val = -2.2210387624291097e-43, time_val = -1223437676, string_val = 0xb713d294 "\316V\023\267\020", str_val = {s = 0xb713d294 "\316V\023\267\020", <br> len = -1223438348}, blob_val = {s = 0xb713d294 "\316V\023\267\020", len = -1223438348}, bitmap_val = 3071529620}}, {type = DB_STR, nul = -1493721088, <br>
free = -1079590600, val = {int_val = -1223663627, double_val = -5.7181266402473013e-121, time_val = -1223663627, string_val = 0xb7105ff5 "[\201\303\377o\003", str_val = {<br>---Type <return> to continue, or q <return> to quit---<br>
s = 0xb7105ff5 "[\201\303\377o\003", len = -1493721088}, blob_val = {s = 0xb7105ff5 "[\201\303\377o\003", len = -1493721088}, bitmap_val = 3071303669}}, {type = DB_STR, <br> nul = -1223437564, free = -1223437596, val = {int_val = -1223437588, double_val = -5.1278976835577225e-42, time_val = -1223437588, string_val = 0xb713d2ec "XW\023\267\a", <br>
str_val = {s = 0xb713d2ec "XW\023\267\a", len = -1218668857}, blob_val = {s = 0xb713d2ec "XW\023\267\a", len = -1218668857}, bitmap_val = 3071529708}}, {type = DB_STR, <br> nul = 0, free = 160289568, val = {int_val = -1079590596, double_val = -5.18431919678883e-42, time_val = -1079590596, string_val = 0xbfa6c13c "", str_val = {<br>
s = 0xbfa6c13c "", len = -1218648242}, blob_val = {s = 0xbfa6c13c "", len = -1218648242}, bitmap_val = 3215376700}}, {type = DB_INT, nul = 0, free = -1218776128, val = {<br> int_val = 4, double_val = 1.9762625833649862e-323, time_val = 4, string_val = 0x4 <Address 0x4 out of bounds>, str_val = {s = 0x4 <Address 0x4 out of bounds>, len = 0}, <br>
blob_val = {s = 0x4 <Address 0x4 out of bounds>, len = 0}, bitmap_val = 4}}, {type = DB_INT, nul = 0, free = -1218315245, val = {int_val = 1297922670, <br> double_val = -5.1111603421990452e-42, time_val = 1297922670, string_val = 0x4d5cba6e <Address 0x4d5cba6e out of bounds>, str_val = {<br>
s = 0x4d5cba6e <Address 0x4d5cba6e out of bounds>, len = -1218674972}, blob_val = {s = 0x4d5cba6e <Address 0x4d5cba6e out of bounds>, len = -1218674972}, <br> bitmap_val = 1297922670}}, {type = DB_STR, nul = 0, free = 77, val = {int_val = -1487010572, double_val = 7.7533023252331478e-314, time_val = -1487010572, <br>
string_val = 0xa75e04f4 "1021ba69\030", str_val = {s = 0xa75e04f4 "1021ba69\030", len = 3}, blob_val = {s = 0xa75e04f4 "1021ba69\030", len = 3}, <br> bitmap_val = 2807956724}}, {type = DB_STR, nul = 0, free = -1215881216, val = {int_val = -1488176656, double_val = 7.7527262031885827e-314, time_val = -1488176656, <br>
string_val = 0xa74c39f0 "1026611c\024", str_val = {s = 0xa74c39f0 "1026611c\024", len = 3}, blob_val = {s = 0xa74c39f0 "1026611c\024", len = 3}, <br> bitmap_val = 2806790640}}, {type = DB_STR, nul = 1, free = 77, val = {int_val = 160298304, double_val = 2.0591278960872443e-312, time_val = 160298304, <br>
string_val = 0x98df540 "destCP#sip:XXXXXXXXXXXXXX|calleruri#sip:XXXXXXXXXXXXXXXXX|\r]\267\021", str_val = {<br> s = 0x98df540 "destCP#sip:XXXXXXXXXXXXXXXX|calleruri#sip:XXXXXXXXXXXXXXXXXXXXX|\r]\267\021", len = 97}, blob_val = {<br>
s = 0x98df540 "destCP#sip:XXXXXXXXXXXXXXXXXX|calleruri#sip:XXXXXXXXXXXXXXXXXX|\r]\267\021", len = 97}, bitmap_val = 160298304}}, {<br> type = DB_STR, nul = 1, free = -1217683468, val = {int_val = 160298408, double_val = 2.3421151637093839e-313, time_val = 160298408, string_val = 0x98df5a8 "lbXpstn#B9|", <br>
str_val = {s = 0x98df5a8 "lbXpstn#B9|", len = 11}, blob_val = {s = 0x98df5a8 "lbXpstn#B9|", len = 11}, bitmap_val = 160298408}}, {type = DB_INT, nul = 0, free = 96, <br> val = {int_val = 0, double_val = -9.9099170538395911e-42, time_val = 0, string_val = 0x0, str_val = {s = 0x0, len = -1217683468}, blob_val = {s = 0x0, len = -1217683468}, <br>
bitmap_val = 0}}}<br> cell = <value optimized out><br> on_shutdown = 1 '\001'<br> callee_leg = <value optimized out><br> insert_keys = {0xb71e382c, 0xb71e3824, 0xb71e37fc, 0xb71e3804, 0xb71e380c, 0xb71e3814, 0xb71e381c, 0xb71e388c, 0xb71e3884, 0xb71e3844, 0xb71e386c, 0xb71e3864, 0xb71e387c, <br>
0xb71e3874, 0xb71e3834, 0xb71e384c, 0xb71e385c, 0xb71e3854, 0xb71e3894, 0xb71e389c, 0xb71e38a4}<br> __FUNCTION__ = "dialog_update_db"<br>#9 0xb71b9c29 in mod_destroy () at dialog.c:734<br>No locals.<br>
#10 0x080c3bf0 in destroy_modules () at sr_module.c:370<br> t = 0x81ce3b8<br> foo = 0x81ce310<br>#11 0x0806e3f6 in cleanup (show_status=1) at main.c:336<br>No locals.<br>#12 0x0806ef57 in handle_sigs () at main.c:534<br>
chld = 0<br> chld_status = 139<br> i = <value optimized out><br> do_exit = 1<br> __FUNCTION__ = "handle_sigs"<br>---Type <return> to continue, or q <return> to quit---<br>
#13 0x0807289f in main_loop (argc=9, argv=0xbfa6c444) at main.c:914<br> i = 9<br> pid = <value optimized out><br> si = 0x0<br> startup_done = 0x0<br> chd_rank = 44<br>#14 main (argc=9, argv=0xbfa6c444) at main.c:1393<br>
cfg_log_stderr = 0<br> cfg_stream = 0x0<br> c = <value optimized out><br> r = <value optimized out><br> tmp = 0xbfa6d9ec ""<br> tmp_len = <value optimized out><br>
port = <value optimized out><br> proto = <value optimized out><br> ret = <value optimized out><br> seed = 715879913<br> rfd = <value optimized out><br> __FUNCTION__ = "main"<br>
(gdb) <br><br><br><br><br>-James<br><br><br><br>