<div dir="ltr"><div dir="ltr">OK, I figured out my mistake with gdb so I now have a 'bt full' output of the core dump:<div><br></div><div><div>GNU gdb (Ubuntu 8.1-0ubuntu3.1) 8.1.0.20180409-git</div><div>Copyright (C) 2018 Free Software Foundation, Inc.</div><div>License GPLv3+: GNU GPL version 3 or later <<a href="http://gnu.org/licenses/gpl.html">http://gnu.org/licenses/gpl.html</a>></div><div>This is free software: you are free to change and redistribute it.</div><div>There is NO WARRANTY, to the extent permitted by law. Type "show copying"</div><div>and "show warranty" for details.</div><div>This GDB was configured as "x86_64-linux-gnu".</div><div>Type "show configuration" for configuration details.</div><div>For bug reporting instructions, please see:</div><div><<a href="http://www.gnu.org/software/gdb/bugs/">http://www.gnu.org/software/gdb/bugs/</a>>.</div><div>Find the GDB manual and other documentation resources online at:</div><div><<a href="http://www.gnu.org/software/gdb/documentation/">http://www.gnu.org/software/gdb/documentation/</a>>.</div><div>For help, type "help".</div><div>Type "apropos word" to search for commands related to "word"...</div><div>Reading symbols from /usr/local/sbin/opensips...done.</div><div>[New LWP 12152]</div><div>[Thread debugging using libthread_db enabled]</div><div>Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".</div><div>Core was generated by `/usr/local/sbin/opensips -P /var/run/opensips/opensips.pid -m 128 -M 4 -u opens'.</div><div>Program terminated with signal SIGSEGV, Segmentation fault.</div><div>#0 __strlen_sse2 () at ../sysdeps/x86_64/multiarch/../strlen.S:120</div><div>#0 __strlen_sse2 () at ../sysdeps/x86_64/multiarch/../strlen.S:120</div><div>No locals.</div><div>#1 0x0000562dc721ae56 in resolvehost (name=0x706475 <error: Cannot access memory at address 0x706475>, no_ip_test=0) at resolve.c:464</div><div> he = 0x0</div><div> start = {tv_sec = 139802802612144, tv_usec = 0}</div><div> ip = 0x0</div><div> s = {s = 0x706475 <error: Cannot access memory at address 0x706475>, len = -2013680896}</div><div>#2 0x0000562dc71e477a in fix_actions (a=0x7f266dc3d1e0) at route.c:1320</div><div> t = 0x7f266dc3de38</div><div> ret = 0</div><div> cmd = 0x7f2660636740 <cmds+480></div><div> acmd = 0xc51aa61587f9ab00</div><div> he = 0x562dc718c990 <_start></div><div> ip = {af = 1417257328, len = 32764, u = {addrl = {139803026513288, 1841548528}, addr32 = {1841028488, 32550, 1841548528, 0}, addr16 = {56712, 28091, 32550, 0, 52464, 28099, 0, 0},</div><div> addr = "\210ݻm&\177\000\000\360\314\303m\000\000\000"}}</div><div> si = 0x7ffc5479a970</div><div> host = {s = 0x7ffc5479a970 "\017", len = 0}</div><div> proto = 0</div><div> port = 32</div><div> p = 0x562dc718c990 <_start></div><div> blh = 0x7f266dbda998</div><div> i = 0</div><div> s = {s = 0x7f266dc3d9d8 "CUSTOM_LOG: Rewriting host:port", len = 31}</div><div> model = 0x7f266dc4b8e0</div><div> models = {0x2f74, 0x562dc718c990 <_start>, 0x7ffc5479a170, 0x562dc71e2df5 <fix_expr+253>, 0x7f2660418f38 <cmds+184>}</div><div> sp = 0x0</div><div> xlp = 0x2f74</div><div> ev_id = 32550</div><div> __FUNCTION__ = "fix_actions"</div><div>#3 0x0000562dc71e3edd in fix_actions (a=0x7f266dc40c48) at route.c:1227</div><div> t = 0x7f266dc40c48</div><div> ret = 0</div><div> cmd = 0x7f2660418e80 <cmds></div><div> acmd = 0xc51aa61587f9ab00</div><div> he = 0x562dc718c990 <_start></div><div> ip = {af = 1417257632, len = 32764, u = {addrl = {139803026515496, 1841545040}, addr32 = {1841030696, 32550, 1841545040, 0}, addr16 = {58920, 28091, 32550, 0, 48976, 28099, 0, 0},</div><div> addr = "(\346\273m&\177\000\000P\277\303m\000\000\000"}}</div><div> si = 0x7ffc5479a970</div><div> host = {s = 0x7ffc5479a970 "\017", len = 0}</div><div> proto = 0</div><div> port = 32</div><div> p = 0x562dc718c990 <_start></div><div> blh = 0x7f266dbda620</div><div> i = 0</div><div> s = {s = 0x7f266dc3c740 "CUSTOM_LOG: Engaging rtpengine_offer with flags: $var(rtpflags)", len = 63}</div><div> model = 0x0</div><div> models = {0x2f74, 0x562dc718c990 <_start>, 0x7ffc5479a2a0, 0x562dc71e2df5 <fix_expr+253>, 0x7f2660418f38 <cmds+184>}</div><div> sp = 0x0</div><div> xlp = 0x2f74</div><div> ev_id = 32550</div><div> __FUNCTION__ = "fix_actions"</div><div>#4 0x0000562dc71e3f31 in fix_actions (a=0x7f266dc40d70) at route.c:1231</div><div> t = 0x7f266dc40d70</div><div> ret = 0</div><div> cmd = 0x7f2660418e80 <cmds></div><div> acmd = 0xc51aa61587f9ab00</div><div> he = 0x562dc71e3127 <fix_expr+1071></div><div> ip = {af = 1417257936, len = 32764, u = {addrl = {139803026518464, 1841510768}, addr32 = {1841033664, 32550, 1841510768, 0}, addr16 = {61888, 28091, 32550, 0, 14704, 28099, 0, 0},</div><div> addr = "\300\361\273m&\177\000\000p9\303m\000\000\000"}}</div><div> si = 0x0</div><div> host = {s = 0x7ffc5479a970 "\017", len = 0}</div><div> proto = 0</div><div> port = 1417259376</div><div> p = 0x562dc718c990 <_start></div><div> blh = 0x7ffc5479a970</div><div> i = 0</div><div> s = {s = 0x7f266dc3ba38 "CUSTOM_LOG: Engaging rtpengine_offer with flags: $var(rtpflags)", len = 63}</div><div> model = 0x0</div><div> models = {0x2f74, 0x562dc718c990 <_start>, 0x7ffc5479a3d0, 0x562dc71e2df5 <fix_expr+253>, 0x7f266577cf98 <cmds+1240>}</div><div> sp = 0x0</div><div> xlp = 0x7ffc5479a350</div><div> ev_id = -988109291</div><div> __FUNCTION__ = "fix_actions"</div><div>#5 0x0000562dc71e3f31 in fix_actions (a=0x7f266dc378c8) at route.c:1231</div><div> t = 0x7f266dc40e98</div><div> ret = 0</div><div> cmd = 0x7f266577cf70 <cmds+1200></div><div> acmd = 0x1</div><div> he = 0xc51aa61587f9ab00</div><div> ip = {af = 12148, len = 0, u = {addrl = {140721725744416, 94754613807504}, addr32 = {1417258272, 32764, 3340290448, 22061}, addr16 = {42272, 21625, 32764, 0, 51600, 50968, 22061, 0},</div><div> addr = " \245yT\374\177\000\000\220\311\030\307-V\000"}}</div><div> si = 0x562dc73564d0 <__FUNCTION__.10125></div><div> host = {s = 0x7ffc5479a970 "\017", len = 0}</div><div> proto = 0</div><div> port = 0</div><div> p = 0x7f266f207740</div><div> blh = 0x0</div><div> i = 0</div><div> s = {s = 0x7f266dc33630 "CUSTOM_LOG: Checking for Media Services Routing...", len = 50}</div><div> model = 0x0</div><div> models = {0x0, 0x7ffc5479a550, 0x0, 0x7ffc5479af23, 0x2f74}</div><div> sp = 0x0</div><div> xlp = 0x7ffc5479a970</div><div> ev_id = 0</div><div> __FUNCTION__ = "fix_actions"</div><div>#6 0x0000562dc71e68a8 in fix_rls () at route.c:1767</div><div> i = 4</div><div> ret = 0</div><div>#7 0x0000562dc724659b in reload_routing_script () at cfg_reload.c:508</div><div> sr = 0x7f266dc21428</div><div> sr_bk = 0x0</div><div> curr_wdir = 0x0</div><div> cfg_buf = {</div><div> s = 0x562dc8c83e40 "__OSSPP_FILEBEGIN__ \"/usr/local/etc/opensips/opensips.cfg\"\n__OSSPP_LINE__ 1\n#\n__OSSPP_LINE__ 2\n# OpenSIPS residential configuration script\n__OSSPP_LINE__ 3\n# by OpenSIPS Solutions <team@opensips-s"..., len = 43994}</div><div> cnt_sleep = 1854342416</div><div> ret = 0</div><div> __FUNCTION__ = "reload_routing_script"</div><div>#8 0x0000562dc72d41e6 in w_reload_routes (params=0x7ffc5479a600, async_hdl=0x0) at mi/mi_core.c:762</div><div>No locals.</div><div>#9 0x0000562dc72d0e9c in handle_mi_request (req=0x7ffc5479a6e0, cmd=0x7f266dbf2ea8, async_hdl=0x0) at mi/mi.c:407</div><div> resp = 0x7f266dbcf3a0</div><div> cmd_recipe = 0x562dc75c2bc0 <mi_core_cmds+18272></div><div> cmd_params = {item = 0x0, list = 0x562dc75c2bc8 <mi_core_cmds+18280>}</div><div> is_ambiguous = 0</div><div> pos_params = 0</div><div> __FUNCTION__ = "handle_mi_request"</div><div>#10 0x00007f2664cdc3bf in mi_fifo_server (fifo_stream=0x562dc8c04670) at fifo_fnc.c:757</div><div> parse_end = 0x0</div><div> request = {req_obj = 0x562dc8c21700, id = 0x562dc8c21660, method = 0x562dc8c21610, params = 0x0, invalid = 0}</div><div> read_len = 100</div><div> parse_len = 0</div><div> req_method = 0x562dc8c5f590 "reload_routes"</div><div> file_sep = 0x7f266dc20409 ":{\"jsonrpc\": \"2.0\", \"id\": \"13372\", \"method\": \"reload_routes\", \"params\": {}}"</div><div> file = 0x7f266dbcf3a0 "/tmp/opensips_fifo_reply_4224"</div><div> p = 0x7f266dc2040a "{\"jsonrpc\": \"2.0\", \"id\": \"13372\", \"method\": \"reload_routes\", \"params\": {}}"</div><div> cmd = 0x7f266dbf2ea8</div><div> reply_stream = 0x562dc8c1ada0</div><div> remain_len = 0</div><div> hdl = 0x0</div><div> response = 0x562dc8c6f0c0</div><div> rc = 0</div><div> buf = {s = 0x7f266dc203f0 ":opensips_fifo_reply_4224:{\"jsonrpc\": \"2.0\", \"id\": \"13372\", \"method\": \"reload_routes\", \"params\": {}}", len = 4096}</div><div> __FUNCTION__ = "mi_fifo_server"</div><div>#11 0x00007f2664cd7d89 in fifo_process (rank=0) at mi_fifo.c:223</div><div> fifo_stream = 0x562dc8c04670</div><div> __FUNCTION__ = "fifo_process"</div><div>#12 0x0000562dc722f8d6 in start_module_procs () at sr_module.c:853</div><div> m = 0x7f266dbacc10</div><div> n = 0</div><div> l = 0</div><div> x = 0</div><div> __FUNCTION__ = "start_module_procs"</div><div>#13 0x0000562dc72287d6 in main_loop () at main.c:774</div><div> chd_rank = 0</div><div> startup_done = 0x0</div><div> last_check = 0</div><div> rc = 1001</div><div> __FUNCTION__ = "main_loop"</div><div>#14 0x0000562dc722be12 in main (argc=15, argv=0x7ffc5479a978) at main.c:1480</div><div> cfg_log_stderr = 1</div><div> c = -1</div><div> r = 0</div><div> tmp = 0x7ffc5479af07 ""</div><div> tmp_len = 9</div><div> port = 0</div><div> proto = 1862227552</div><div> protos_no = 1</div><div> options = 0x562dc734bb98 "f:cCm:M:b:l:n:N:rRvdDFEVhw:t:u:g:p:P:G:W:o:a:k:s:"</div><div> ret = -1</div><div> seed = 2480117594</div><div> rfd = 3</div><div> __FUNCTION__ = "main"</div><div>(gdb)</div></div><div><br></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, 25 Sep 2019 at 15:05, Mark Farmer <<a href="mailto:farmorg@gmail.com">farmorg@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">To add a bit more, I've enabled core dumps but I've not been able to see anything useful:<div><br></div><div><div>(gdb) core-file /var/log/opensips/core.12152</div><div>[New LWP 12152]</div><div>Core was generated by `/usr/local/sbin/opensips -P /var/run/opensips/opensips.pid -m 128 -M 4 -u opens'.</div><div>Program terminated with signal SIGSEGV, Segmentation fault.</div><div>#0 0x00007f266e894646 in ?? ()</div><div>(gdb) bt full</div><div>#0 0x00007f266e894646 in ?? ()</div><div>No symbol table info available.</div><div>#1 0x0000562dc721ae56 in ?? ()</div><div>No symbol table info available.</div><div>#2 0x000000006dc3d9f7 in ?? ()</div><div>No symbol table info available.</div><div>#3 0x0000000000706475 in ?? ()</div><div>No symbol table info available.</div><div>#4 0x0000000000000000 in ?? ()</div><div>No symbol table info available.</div></div><div><br></div><div>I also 'upgraded' to a git clone so now running latest code.</div><div><br></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, 25 Sep 2019 at 13:33, Mark Farmer <<a href="mailto:farmorg@gmail.com" target="_blank">farmorg@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr">Hi all<div><br></div><div>I've seen reports of this elsewhere but didn't find a solution. When I use opensips-cli to do a 'mi reload_routes' OpenSIPS segfaults.</div><div><br></div><div>I did a git pull and install clean yesterday on opensips-cli.</div><div><br></div><div><div>opensips -V</div><div>version: opensips 3.0.0 (x86_64/linux)</div><div>flags: STATS: On, DISABLE_NAGLE, USE_MCAST, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, HP_MALLOC, DBG_MALLOC, CC_O0, FAST_LOCK-ADAPTIVE_WAIT</div><div>ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535</div><div>poll method support: poll, epoll, sigio_rt, select.</div><div>git revision: 435890a06</div><div>main.c compiled on 13:19:25 Jun 21 2019 with gcc 7</div></div><div><br></div><div>Oddly, it doesn't seem to matter what is in the last line, it errors regardless but this is the part of the route that seems to trigger it:</div><div><br></div><div><div>else if (isflagset(Voicemail)) {</div><div> $var(rtpflags) ="replace-session-connection replace-origin trust-address force symmetric in-face=external out-face=internal";</div><div> xlog("CUSTOM_LOG: Call goes to Voicemail service - Route to Asterisk.");</div><div> xlog("CUSTOM_LOG: Engaging rtpengine_offer with flags: $var(rtpflags)");</div><div> rtpengine_offer($var(rtpflags));</div></div><div> xlog("CUSTOM_LOG: Rewriting host:port");</div><div> rewritehostport("<a href="http://10.150.50.53:5060" target="_blank">10.150.50.53:5060</a>");</div><div> force_send_socket(udp:10.150.50.51);</div><div> return(1);</div><div><br></div><div><div>opensips-cli -d -x mi reload_routes</div><div>DEBUG: using config file /etc/opensips-cli.cfg</div><div>DEBUG: Loaded module 'database'</div><div>DEBUG: Loaded module 'diagnose'</div><div>DEBUG: Loaded module 'instance'</div><div>DEBUG: Loaded module 'mi'</div><div>DEBUG: sent command ':opensips_fifo_reply_23259:{"jsonrpc": "2.0", "id": "16512", "method": "which", "params": []}'</div><div>DEBUG: Loaded module 'tls'</div><div>DEBUG: Loaded module 'trace'</div><div>DEBUG: Skipping module 'trap' - excluded on purpose</div><div>DEBUG: Loaded module 'user'</div><div>DEBUG: running in non-interactive mode '['mi', 'reload_routes']'</div><div>DEBUG: running command 'reload_routes' '[]'</div><div>DEBUG: named parameters are used</div><div>DEBUG: running command 'reload_routes' '{}'</div><div>DEBUG: sent command ':opensips_fifo_reply_20789:{"jsonrpc": "2.0", "id": "10120", "method": "reload_routes", "params": {}}'</div><div>ERROR: communication exception for 'reload_routes' returned:</div><div>ERROR: Is OpenSIPS running?</div></div><div><br></div><div><div>DBG:core:decode_mime_type: Decoding MIME type for:[application/sdp]</div><div>DBG:core:fix_actions: fixing t_on_reply, /usr/local/etc/opensips/opensips.cfg:735</div><div>DBG:core:fix_actions: fixing is_registered, /usr/local/etc/opensips/opensips.cfg:749</div><div>DBG:core:fix_actions: fixing is_registered, /usr/local/etc/opensips/opensips.cfg:749</div><div>DBG:core:fix_actions: fixing insert_hf, /usr/local/etc/opensips/opensips.cfg:751</div><div>DBG:core:fix_actions: fixing lookup, /usr/local/etc/opensips/opensips.cfg:753</div><div>DBG:core:fix_actions: fixing is_from_gw, /usr/local/etc/opensips/opensips.cfg:753</div><div>DBG:core:fix_actions: fixing rtpengine_offer, /usr/local/etc/opensips/opensips.cfg:768</div><div>DBG:core:fix_actions: fixing do_routing, /usr/local/etc/opensips/opensips.cfg:769</div><div>DBG:core:fix_actions: fixing rtpengine_offer, /usr/local/etc/opensips/opensips.cfg:774</div><div>DBG:core:fix_actions: fixing do_routing, /usr/local/etc/opensips/opensips.cfg:775</div><div>DBG:core:fix_actions: fixing rtpengine_offer, /usr/local/etc/opensips/opensips.cfg:780</div><div>CRITICAL:core:sig_usr: segfault in process pid: 30907, id: 2</div><div>DBG:core:restore_segv_handler: restoring SIGSEGV handler...</div><div>DBG:core:restore_segv_handler: successfully restored system SIGSEGV handler</div><div>DBG:core:handle_sigs: OpenSIPS exit status = 139</div><div>INFO:core:handle_sigs: child process 30907 exited by a signal 11</div><div>INFO:core:handle_sigs: core was generated</div><div>INFO:core:handle_sigs: terminating due to SIGCHLD</div><div><br></div></div><div><br></div><div>Thanks and regards</div><div>Mark.</div><div><br></div></div></div></div></div></div></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail-m_8507956091282646903gmail_signature">Mark Farmer<br><a href="mailto:farmorg@gmail.com" target="_blank">farmorg@gmail.com</a></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature">Mark Farmer<br><a href="mailto:farmorg@gmail.com" target="_blank">farmorg@gmail.com</a></div>