<div dir="ltr">Bogdan,<div><br></div><div>Here is the backtrace with those flags enabled</div><div><br></div><div>
<p class=""><span class="">gdb /usr/sbin/opensips core.2685</span></p>
<p class=""><span class="">GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-80.el7</span></p>
<p class=""><span class="">Copyright (C) 2013 Free Software Foundation, Inc.</span></p>
<p class=""><span class="">License GPLv3+: GNU GPL version 3 or later <<a href="http://gnu.org/licenses/gpl.html">http://gnu.org/licenses/gpl.html</a>></span></p>
<p class=""><span class="">This is free software: you are free to change and redistribute it.</span></p>
<p class=""><span class="">There is NO WARRANTY, to the extent permitted by law. Type "show copying"</span></p>
<p class=""><span class="">and "show warranty" for details.</span></p>
<p class=""><span class="">This GDB was configured as "x86_64-redhat-linux-gnu".</span></p>
<p class=""><span class="">For bug reporting instructions, please see:</span></p>
<p class=""><span class=""><<a href="http://www.gnu.org/software/gdb/bugs/">http://www.gnu.org/software/gdb/bugs/</a>>...</span></p>
<p class=""><span class="">Reading symbols from /usr/sbin/opensips...done.</span></p>
<p class=""><span class="">[New LWP 2685]</span></p>
<p class=""><span class="">[Thread debugging using libthread_db enabled]</span></p>
<p class=""><span class="">Using host libthread_db library "/lib64/libthread_db.so.1".</span></p>
<p class=""><span class="">Core was generated by `/sbin/opensips -P /var/run/opensips.pid -u opensips -g opensips -M 1024 -f /etc'.</span></p>
<p class=""><span class="">Program terminated with signal 6, Aborted.</span></p>
<p class=""><span class="">#0 0x00007fc95ab1d5f7 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56</span></p>
<p class=""><span class="">56<span class=""> </span> return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig);</span></p>
<p class=""><span class="">(gdb) </span></p><p class=""><span class=""><br></span></p><p class="">Let me know if you need anything else</p><p class=""><br></p></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Jul 4, 2016 at 5:54 AM, Bogdan-Andrei Iancu <span dir="ltr"><<a href="mailto:bogdan@opensips.org" target="_blank">bogdan@opensips.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<tt>Thank you Tito,<br>
<br>
It looks like a crash in the memory manager, while trying to
allocate a new structure. Do debug such problems there is no other
way than enabling debugging for the memory manager (QM_MALLOC +
DBG_MALLOC flags) - is this a production system with considerable
load on it ?<br>
<br>
Regards,<br>
</tt><span class="">
<pre cols="72">Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
<a href="http://www.opensips-solutions.com" target="_blank">http://www.opensips-solutions.com</a></pre>
</span><div><div class="h5"><div>On 01.07.2016 18:44, Tito Cumpen wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Bogdan,
<div><br>
</div>
<div><br>
Correction I was using :</div>
<div><br>
</div>
<div>
<p><span>version: opensips 2.2.0
(x86_64/linux)</span></p>
<p><span>flags: STATS: On, DISABLE_NAGLE,
USE_MCAST, SHM_MMAP, PKG_MALLOC, F_MALLOC,
FAST_LOCK-ADAPTIVE_WAIT</span></p>
<p><span>ADAPTIVE_WAIT_LOOPS=1024,
MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE
1024, BUF_SIZE 65535</span></p>
<p><span>poll method support: poll,
epoll_lt, epoll_et, sigio_rt, select.</span></p>
<p><span>git revision: d835721</span></p>
<p><span>main.c compiled on 15:24:26 Jun 28
2016 with gcc 4.8.5</span></p>
<p><span><br>
</span></p>
<p><span><br>
</span></p>
<p><span><br>
</span></p>
</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Fri, Jul 1, 2016 at 9:23 AM, Tito
Cumpen <span dir="ltr"><<a href="mailto:tito@xsvoce.com" target="_blank">tito@xsvoce.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">Bogdan,
<div><br>
</div>
<div>Here is the backtrace:</div>
<div><br>
</div>
<div><br>
</div>
<div>
<p><span>Reading symbols from /usr/sbin/opensips...done.</span></p>
<p><span>[New LWP 2648]</span></p>
<p><span>[Thread debugging using libthread_db enabled]</span></p>
<p><span>Using host libthread_db library
"/lib64/libthread_db.so.1".</span></p>
<p><span>Core was generated by `/sbin/opensips -P
/var/run/opensips.pid -u opensips -g opensips -M
1024 -f /etc'.</span></p>
<p><span>Program terminated with signal 11, Segmentation
fault.</span></p>
<p><span>#0 0x0000000000512782 in fm_remove_free
(n=0x7fadc0898210, qm=0x7fadc0809010) at
mem/f_malloc.c:187</span></p>
<p><span>187<span> </span><span> </span>*pf=n->u.nxt_free;</span></p>
<p><span>Missing separate debuginfos, use:
debuginfo-install
cyrus-sasl-lib-2.1.26-20.el7_2.x86_64
glibc-2.17-106.el7_2.4.x86_64
gmp-6.0.0-12.el7_1.x86_64
gnutls-3.3.8-14.el7_2.x86_64
keyutils-libs-1.5.8-3.el7.x86_64
krb5-libs-1.13.2-12.el7_2.x86_64
libcom_err-1.42.9-7.el7.x86_64
libcurl-7.29.0-25.el7.centos.x86_64
libffi-3.0.13-16.el7.x86_64
libgcc-4.8.5-4.el7.x86_64
libgcrypt-1.5.3-12.el7_1.1.x86_64
libgpg-error-1.12-3.el7.x86_64
libidn-1.28-4.el7.x86_64
libmicrohttpd-0.9.33-2.el7.x86_64
librabbitmq-0.5.2-1.el7.x86_64
libselinux-2.2.2-6.el7.x86_64
libssh2-1.4.3-10.el7_2.1.x86_64
libstdc++-4.8.5-4.el7.x86_64
libtasn1-3.8-2.el7.x86_64 nettle-2.7.1-4.el7.x86_64
nspr-4.11.0-1.el7_2.x86_64 nss-3.21.0-9.el7_2.x86_64
nss-softokn-freebl-3.16.2.3-14.2.el7_2.x86_64
nss-util-3.21.0-2.2.el7_2.x86_64
openldap-2.4.40-9.el7_2.x86_64
openssl-libs-1.0.1e-51.el7_2.4.x86_64
p11-kit-0.20.7-3.el7.x86_64 pcre-8.32-15.el7.x86_64
trousers-0.3.13-1.el7.x86_64
xz-libs-5.1.2-12alpha.el7.x86_64
zlib-1.2.7-15.el7.x86_64</span></p>
<p><span>(gdb) bt full</span></p>
<p><span>#0 0x0000000000512782 in fm_remove_free
(n=0x7fadc0898210, qm=0x7fadc0809010) at
mem/f_malloc.c:187</span></p>
<p><span> pf = 0x0</span></p>
<p><span> hash = 2051</span></p>
<p><span>#1 fm_malloc (qm=0x7fadc0809010,
size=size@entry=65608) at mem/f_malloc.c:415</span></p>
<p><span> frag = 0x7fadc0898210</span></p>
<p><span> n = <optimized out></span></p>
<p><span> hash = 2051</span></p>
<p><span> __FUNCTION__ = "fm_malloc"</span></p>
<p><span>#2 0x00007fadb4615091 in ws_process
(con=0x7fadbeb3f718) at ../proto_ws/ws_common.h:576</span></p>
<p><span> size = <optimized out></span></p>
<p><span> req = <optimized out></span></p>
<p><span> ret_code = WS_ERR_NONE</span></p>
<p><span> bk = <optimized out></span></p>
<p><span> msg_len = <optimized out></span></p>
<p><span> local_rcv = {src_ip = {af = <a href="tel:3082146304" value="+13082146304" target="_blank">3082146304</a>,
len = 32685, u = {addrl = {34359801655,
140720703251544}, addr32 = {63287, 8, 394765400,
32764}, addr16 = {63287, 0, 8, 0, 42072, 6023,
32764, 0}, </span></p>
<p><span> addr =
"7\367\000\000\b\000\000\000X\244\207\027\374\177\000"}},
dst_ip = {af = 3080002780, len = 32685, u = {addrl =
{4, 63119970000}, addr32 = {4, 0, 2990427856, 14},
addr16 = {4, 0, 0, </span></p>
<p><span> 0, 20176, 45630, 14, 0}, addr =
"\004\000\000\000\000\000\000\000\320N>\262\016\000\000"}},
src_port = 52976, dst_port = 49287, proto = 32685,
proto_reserved1 = -1214962633, </span></p>
<p><span> proto_reserved2 = 32685, src_su = {s
= {sa_family = 2, sa_data =
")\350n\343\276C\360#%\275\016\000\000"}, sin =
{sin_family = 2, sin_port = 59433, sin_addr =
{s_addr = 1136583534}, </span></p>
<p><span> sin_zero =
"\360#%\275\016\000\000"}, sin6 = {sin6_family = 2,
sin6_port = 59433, sin6_flowinfo = 1136583534,
sin6_addr = {__in6_u = {</span></p>
<p><span> __u6_addr8 =
"\360#%\275\016\000\000\000\360·\300\255\177\000",
__u6_addr16 = {9200, 48421, 14, 0, 52976, 49287,
32685, 0}, __u6_addr32 = {3173327856, 14,
3230125808, 32685}}}, </span></p>
<p><span> sin6_scope_id = 1}}, bind_address
= 0x7fadc0882ea8}</span></p>
<p><span> newreq = <optimized out></span></p>
<p><span> msg_buf = <optimized out></span></p>
<p><span>#3 wss_read_req (con=0x7fadbeb3f718,
bytes_read=<optimized out>) at proto_wss.c:428</span></p>
<p><span> size = <optimized out></span></p>
<p><span> __FUNCTION__ = "wss_read_req"</span></p>
<p><span>#4 0x000000000059c374 in handle_io
(fm=<optimized out>, idx=idx@entry=0,
event_type=event_type@entry=1) at
net/net_tcp_proc.c:205</span></p>
<p><span> ret = 0</span></p>
<p><span> n = <optimized out></span></p>
<p><span> con = 0x7fadbeb3f718</span></p>
<p><span> s = 57</span></p>
<p><span> rw = <optimized out></span></p>
<p><span> resp = <optimized out></span></p>
<p><span> response = {140384203757736, 1}</span></p>
<p><span> __FUNCTION__ = "handle_io"</span></p>
<p><span>#5 0x000000000059d914 in io_wait_loop_epoll
(h=<optimized out>, t=<optimized out>,
repeat=<optimized out>) at
net/../io_wait_loop.h:221</span></p>
<p><span> ret = <optimized out></span></p>
<p><span> e = <optimized out></span></p>
<p><span> n = 1</span></p>
<p><span> r = 0</span></p>
<p><span>#6 tcp_worker_proc (unix_sock=<optimized
out>) at net/net_tcp_proc.c:312</span></p>
<p><span> __FUNCTION__ = "tcp_worker_proc"</span></p>
<p><span>#7 0x00000000005a7fc9 in tcp_start_processes
(chd_rank=chd_rank@entry=0x8417e0
<chd_rank.11018>,
startup_done=startup_done@entry=0x7fadbe978738) at
net/net_tcp.c:1761</span></p>
<p><span> r = 5</span></p>
<p><span> reader_fd = {50, 52}</span></p>
<p><span> pid = 0</span></p>
<p><span> load_p = 0x7fadbe979398</span></p>
<p><span> __FUNCTION__ = "tcp_start_processes"</span></p>
<p><span>#8 0x0000000000419ef5 in main_loop () at
main.c:677</span></p>
<p><span> startup_done = 0x7fadbe978738</span></p>
<p><span> chd_rank = 18</span></p>
<p><span>#9 main (argc=<optimized out>,
argv=<optimized out>) at main.c:1249</span></p>
<p><span> cfg_stream = <optimized out></span></p>
<p><span> c = <optimized out></span></p>
<p><span> r = <optimized out></span></p>
<p><span> tmp = 0x7ffc1787bf65 ""</span></p>
<p><span> tmp_len = <optimized out></span></p>
<p><span> port = <optimized out></span></p>
<p><span> proto = <optimized out></span></p>
<p><span> protos_no = <optimized out></span></p>
<p><span> options = 0x5dae60
"f:cCm:M:b:l:n:N:rRvdDFETSVhw:t:u:g:P:G:W:o:"</span></p>
<p><span>---Type <return> to continue, or q
<return> to quit---</span></p>
<p><span> ret = -1</span></p>
<p><span> seed = 995413301</span></p>
<p><span> rfd = <optimized out></span></p>
<p><span> __FUNCTION__ = "main"</span></p>
<p><span><br>
</span></p>
<p><span><br>
</span></p>
<p><span>currently using </span>version: opensips
2.3.0-dev (x86_64/linux)</p>
<p><span>flags: STATS: On, DISABLE_NAGLE, USE_MCAST,
SHM_MMAP, PKG_MALLOC, QM_MALLOC, DBG_MALLOC,
FAST_LOCK-ADAPTIVE_WAIT</span></p>
<p><span>ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE
262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE
65535</span></p>
<p><span>poll method support: poll, epoll_lt, epoll_et,
sigio_rt, select.</span></p>
<p><span>git revision: 92434ef</span></p>
<p><span>main.c compiled on 15:05:06 Jun 28 2016 with
gcc 4.8.5</span></p>
</div>
</div>
<div>
<div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Tue, Jun 28, 2016 at 8:26
AM, Bogdan-Andrei Iancu <span dir="ltr"><<a href="mailto:bogdan@opensips.org" target="_blank"></a><a href="mailto:bogdan@opensips.org" target="_blank">bogdan@opensips.org</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF"> <tt>Hi
Tito,<br>
<br>
If opensips crashes, were you able to extract
a backtrace from the core file(s) ?<br>
<br>
Thanks and regards,<br>
</tt>
<pre cols="72">Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
<a href="http://www.opensips-solutions.com" target="_blank">http://www.opensips-solutions.com</a></pre>
<div>
<div>
<div>On 27.06.2016 21:20, Tito Cumpen wrote:<br>
</div>
</div>
</div>
<blockquote type="cite">
<div>
<div>
<div dir="ltr">Group,
<div><br>
</div>
<div><br>
</div>
<div>I am experiencing strange behavior
when configuring sip tls on port 443.
At time opensips crashes or stops
accepting new connections. Here are
the tcp configs I am using:</div>
<div><br>
</div>
<p><span>#disable_tcp=no</span></p>
<p><span>tcp_connection_lifetime=3600</span></p>
<p><span>tcp_connect_timeout=3</span></p>
<p><span>tcp_keepidle = 30</span></p>
<p><span>tcp_keepinterval = 5</span></p>
<p><span>tcp_keepalive = 1</span></p>
<p><span>tcp_keepcount = 5</span></p>
<p>tcp_max_msg_time = 8<br>
</p>
<div>tcp_children=10</div>
<div><br>
</div>
<div><br>
</div>
<div>Any idea what would case this? I am
assuming there are probes out in the
internet that eventually make opensips
crash?</div>
<div><br>
</div>
<div>Thanks,</div>
<div>Tito </div>
</div>
<br>
<fieldset></fieldset>
<br>
</div>
</div>
<pre>_______________________________________________
Users mailing list
<a href="mailto:Users@lists.opensips.org" target="_blank">Users@lists.opensips.org</a>
<a href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users" target="_blank">http://lists.opensips.org/cgi-bin/mailman/listinfo/users</a>
</pre>
</blockquote>
<br>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</div></div></div>
</blockquote></div><br></div>