<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<tt>Hi Denis,<br>
<br>
Many thanks for the output - this confirm a suspicious of mine in
regards to the cause of the crash - let me try to see if we can
reproduce it on our side or if we need more info from your side. <br>
Please do not delete the corefile for the moment :) <br>
<br>
Best regards,<br>
</tt>
<pre class="moz-signature" cols="72">Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
<a class="moz-txt-link-freetext" href="http://www.opensips-solutions.com">http://www.opensips-solutions.com</a>
OpenSIPS Summit May 2017 Amsterdam
<a class="moz-txt-link-freetext" href="http://www.opensips.org/events/Summit-2017Amsterdam.html">http://www.opensips.org/events/Summit-2017Amsterdam.html</a>
</pre>
<div class="moz-cite-prefix">On 04/05/2017 11:03 AM, Denis wrote:<br>
</div>
<blockquote cite="mid:5350841491379420@web18o.yandex.ru" type="cite">
<div>Hello, Bogdan!</div>
<div> </div>
<div>
<div>f 5</div>
<div>#5 0x00007fdf6844c4f2 in free_dlg_dlg (dlg=0x7fdf6c4ac158)
at dlg_hash.c:183</div>
<div>183
destroy_dlg_callbacks_list(dlg->cbs.first);</div>
<div> </div>
<div>
<div>p *dlg->cbs.first</div>
<div>$1 = {types = 96, callback = 0x7fdf67531300
<acc_dlg_callback>, param = 0x0, callback_param_free =
0x7fdf675318fe <dlg_free_acc_mask>, next =
0x7fdf84191080}</div>
<div> </div>
<div>
<div>p *dlg->cbs.first->next</div>
<div>$2 = {types = 96, callback = 0x7fdf67531300
<acc_dlg_callback>, param = 0x7fdf83c9f340,
callback_param_free = 0x7fdf675318fe
<dlg_free_acc_mask>, next = 0x0}</div>
<div> </div>
<div>
<div>p *dlg->cbs.first->next->next</div>
<div>Cannot access memory at address 0x0</div>
</div>
</div>
</div>
</div>
<div> </div>
<div>
<div>p *dlg->cbs.first->next->next->next</div>
<div>Cannot access memory at address 0x20</div>
<div> </div>
<div>and so on with ....at address 0x20</div>
<div> </div>
</div>
<div>-- </div>
<div>С уважением, Денис.</div>
<div>Best regards, Denis</div>
<div> </div>
<div> </div>
<div> </div>
<div>04.04.2017, 18:43, "Bogdan-Andrei Iancu"
<a class="moz-txt-link-rfc2396E" href="mailto:bogdan@opensips.org"><bogdan@opensips.org></a>:</div>
<blockquote type="cite">
<div bgcolor="#FFFFFF">Super :)....we got to something useful.<br>
<br>
Now, again in GDB please do:<br>
<br>
f 5<br>
p *dlg->cbs.first<br>
p *dlg->cbs.first->next<br>
p *dlg->cbs.first->next->next<br>
<br>
and keep adding "->next" until you get error from gdb.<br>
<br>
Thanks,
<pre>Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
<a moz-do-not-send="true" href="http://www.opensips-solutions.com/">http://www.opensips-solutions.com</a>
OpenSIPS Summit May 2017 Amsterdam
<a moz-do-not-send="true" href="http://www.opensips.org/events/Summit-2017Amsterdam.html">http://www.opensips.org/events/Summit-2017Amsterdam.html</a>
</pre>
<div>On 04/04/2017 06:07 PM, Денис Путято wrote:</div>
<blockquote cite="mid:3037001491318426@web47g.yandex.ru"
type="cite">
<div>Sorry, Bogdan, i missed that</div>
<div> </div>
<div>I found such CRITICAL message</div>
<div>"CRITICAL:core:qm_free: freeing already freed pointer,
first free: acc_logic.c: dlg_free_acc_mask(211) -
aborting"</div>
<div> </div>
<div>And after that and till crash only
"WARNING:core:utimer_ticker:...." reports in syslog.</div>
<div> </div>
<div>
<div>(gdb)</div>
<div>f 2</div>
<div>#2 0x<span>0000000000515592</span> in qm_free
(qm=<optimized out>, p=p@entry=0x7fdf83c9f340,
file=file@entry=0x7fdf6753ad62 "acc_logic.c",</div>
<div> func=func@entry=0x7fdf6753af90
<__FUNCTION__.12529> "dlg_free_acc_mask",
line=line@entry=211) at mem/q_malloc.c:483</div>
<div>483 abort();</div>
<div> </div>
<div> </div>
<div>
<div>p *((struct qm_frag*) ((char*)p-sizeof(struct
qm_frag)))</div>
<div>$1 = {size = 8, u = {nxt_free = 0x7fdf71a8c6b0,
is_free = <span>140597661320880</span>}, file =
0x7fdf6753ad62 "acc_logic.c", func = 0x7fdf6753af90
<__FUNCTION__.12529> "dlg_free_acc_mask", line =
211,</div>
<div> check = <span>17361641481138401520</span>}</div>
</div>
</div>
<div> </div>
<div>-- </div>
<div>С уважением, Денис.</div>
<div>Best regards, Denis</div>
<div> </div>
<div> </div>
<div> </div>
<div>04.04.2017, 17:58, "Bogdan-Andrei Iancu" <a
moz-do-not-send="true" href="mailto:bogdan@opensips.org"><a class="moz-txt-link-rfc2396E" href="mailto:bogdan@opensips.org"><bogdan@opensips.org></a></a>:</div>
<blockquote type="cite">
<div bgcolor="#FFFFFF">The RADIUS part is not relevant.<br>
<br>
Still you didn;t answer to my question on the CRITICAL
error - according to the backtrace, you should have a
CRITICAL message in the opensips log, just before the
crash.<br>
<br>
In GDB, do:<br>
f 2<br>
p *((struct qm_frag*) ((char*)p-sizeof(struct
qm_frag))) <br>
<br>
Regards,
<pre>Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
<a moz-do-not-send="true" href="http://www.opensips-solutions.com/">http://www.opensips-solutions.com</a>
OpenSIPS Summit May 2017 Amsterdam
<a moz-do-not-send="true" href="http://www.opensips.org/events/Summit-2017Amsterdam.html">http://www.opensips.org/events/Summit-2017Amsterdam.html</a>
</pre>
<div>On 04/04/2017 05:55 PM, Denis wrote:</div>
<blockquote cite="mid:2993391491317739@web47g.yandex.ru"
type="cite">
<div>During crash there was another problem with LAN
network.</div>
<div>Because of this problem Opensips could not
connect to radius server.</div>
<div>
<div>rc_ip_hostname: couldn't look up host by addr:
xxxxxx</div>
<div>rc_send_server: no reply from RADIUS server
unknown:1812</div>
</div>
<div> </div>
<div>No more critical messages.</div>
<div> </div>
<div>Bogdan, i am sorry, but</div>
<div>"in frame 2". What does it mean?</div>
<div> </div>
<div> </div>
<div>-- </div>
<div>С уважением, Денис.</div>
<div>Best regards, Denis</div>
<div> </div>
<div> </div>
<div> </div>
<div>04.04.2017, 16:35, "Bogdan-Andrei Iancu" <a
moz-do-not-send="true"
href="mailto:bogdan@opensips.org"><a class="moz-txt-link-rfc2396E" href="mailto:bogdan@opensips.org"><bogdan@opensips.org></a></a>:</div>
<blockquote type="cite">
<div bgcolor="#FFFFFF">Any CRITICAL message into the
log before the crash ?<br>
<br>
also, in gdb in frame 2, please print:<br>
p *((struct qm_frag*) ((char*)p-sizeof(struct
qm_frag)))<br>
<br>
<br>
Thanks,
<pre>Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
<a moz-do-not-send="true" href="http://www.opensips-solutions.com/">http://www.opensips-solutions.com</a>
OpenSIPS Summit May 2017 Amsterdam
<a moz-do-not-send="true" href="http://www.opensips.org/events/Summit-2017Amsterdam.html">http://www.opensips.org/events/Summit-2017Amsterdam.html</a>
</pre>
<div>On 04/04/2017 03:19 PM, Denis wrote:</div>
<blockquote
cite="mid:2219671491308376@web4m.yandex.ru"
type="cite">
<div>
<div>p shm_block->first_frag</div>
<div>$1 = (struct qm_frag *) 0x7fdf6b9d44f8</div>
<div>
<div> </div>
<div>p shm_block->last_frag_end</div>
<div>$2 = (struct qm_frag_end *)
0x7fe02899efd0</div>
<div> </div>
<div>
<div> p mem_block->first_frag</div>
<div>$3 = (struct qm_frag *) 0x7fe<span><span><span>029214508</span></span></span></div>
<div> </div>
<div>
<div>p mem_block->last_frag_end</div>
<div>$4 = (struct qm_frag_end *)
0x7fe0691defe0</div>
</div>
</div>
</div>
<div> </div>
</div>
<div> </div>
<div>-- </div>
<div>С уважением, Денис.</div>
<div>Best regards, Denis</div>
<div> </div>
<div> </div>
<div> </div>
<div>04.04.2017, 14:51, "Bogdan-Andrei Iancu" <a
moz-do-not-send="true"
href="mailto:bogdan@opensips.org"><a class="moz-txt-link-rfc2396E" href="mailto:bogdan@opensips.org"><bogdan@opensips.org></a></a>:</div>
<blockquote type="cite">
<div bgcolor="#FFFFFF">OK,please enter gdb and
print:<br>
p shm_block->first_frag<br>
p shm_block->last_frag_end<br>
<br>
p mem_block->first_frag<br>
p mem_block->last_frag_end<br>
<br>
Thanks,
<pre>Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
<a moz-do-not-send="true" href="http://www.opensips-solutions.com/">http://www.opensips-solutions.com</a>
OpenSIPS Summit May 2017 Amsterdam
<a moz-do-not-send="true" href="http://www.opensips.org/events/Summit-2017Amsterdam.html">http://www.opensips.org/events/Summit-2017Amsterdam.html</a>
</pre>
<div>On 04/04/2017 01:42 PM, Denis wrote:</div>
<blockquote
cite="mid:1579141491302539@web17o.yandex.ru"
type="cite">
<div>Hello, Bogdan!</div>
<div> </div>
<div>No, i cannot see in the log any
messages you mentioned it your letter.</div>
<div>The situation dealing with crash had
such stage.</div>
<div> </div>
<div>1. I prepared to migrate to 2.2.3
from 2.2.2. But do not have a time to
restart Opensips</div>
<div>2. As a result "i waited" when
Opensips have crashed. During crash of
2.2.2 no core file has been generated,
but Opensips has been restarted using
cron.</div>
<div>3. After 2) Opensips became 2.2.3. It
was working about 1 minute and crashed
again, but in that time, core file has
been generated and i attached
information from it.</div>
<div>So, between 2) and 3) , as i
mentioned above, there were no messages
about memory, but i see many messages
like</div>
<div>'WARNING:core:utimer_ticker: utimer
task <tm-utimer> already scheduled
for 71500 ms</div>
<div>WARNING:core:timer_ticker: timer task
<pike-clean> already scheduled for
72400 ms</div>
<div>WARNING:core:timer_ticker: timer task
<nh-timer> already scheduled for
72400 ms</div>
<div>WARNING:core:timer_ticker: timer task
<dlg-timer> already scheduled for
72400 ms</div>
<div>WARNING:core:timer_ticker: timer task
<blcore-expire> already scheduled
for 72400 ms'</div>
<div>and so on</div>
<div> </div>
<div>As for about version</div>
<div> </div>
<div>
<div>version: opensips 2.2.3
(x86_64/linux)</div>
<div>flags: STATS: On, DISABLE_NAGLE,
USE_MCAST, SHM_MMAP, PKG_MALLOC,
QM_MALLOC, DBG_MALLOC,
FAST_LOCK-ADAPTIVE_WAIT</div>
<div>ADAPTIVE_WAIT_LOOPS=1024,
MAX_RECV_BUFFER_SIZE 262144,
MAX_LISTEN 16, MAX_URI_SIZE 1024,
BUF_SIZE 65535</div>
<div>poll method support: poll,
epoll_lt, epoll_et, sigio_rt, select.</div>
<div>git revision: <span><span><span><span>7922417</span></span></span></span></div>
<div>main.c compiled on 06:29:28 Mar 23
2017 with gcc 5.4.0</div>
<div> </div>
</div>
<div>-- </div>
<div>С уважением, Денис.</div>
<div>Best regards, Denis</div>
<div> </div>
<div> </div>
<div> </div>
<div>04.04.2017, 12:43, "Bogdan-Andrei
Iancu" <a moz-do-not-send="true"
href="mailto:bogdan@opensips.org"><bogdan@opensips.org></a>:</div>
<blockquote type="cite">
<div bgcolor="#FFFFFF">Hello Denis,<br>
<br>
Thank you for the report. Do you see
any CRITICAL log just before the
crash, something like:<br>
"bad pointer 0xxxxxxxx (out of
memory block!) - aborting"<br>
?<br>
If yes, please post here the exact
line.<br>
<br>
Also, do you use the latest OpenSIPS
2.2 from GIT repo ? (run "opensips
-v")<br>
<br>
Regards,
<pre>Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
<a moz-do-not-send="true" href="http://www.opensips-solutions.com/">http://www.opensips-solutions.com</a>
OpenSIPS Summit May 2017 Amsterdam
<a moz-do-not-send="true" href="http://www.opensips.org/events/Summit-2017Amsterdam.html">http://www.opensips.org/events/Summit-2017Amsterdam.html</a>
</pre>
<div>On 04/04/2017 10:58 AM, Denis via
Users wrote:</div>
<blockquote
cite="mid:325131491292730@web16g.yandex.ru"
type="cite">
<div>Hello!</div>
<div> </div>
<div>Server:: OpenSIPS (2.2.3
(x86_64/linux))</div>
<div> </div>
<div>Thank you for any help.</div>
<div> </div>
<div>P.S. Opensips has been compiled
with QM_MALLOC and DBG_MALLOC
flags</div>
<div> </div>
<div>-- </div>
<div>С уважением, Денис.</div>
<div>Best regards, Denis</div>
<div> </div>
<div> </div>
<div> </div>
<pre>_______________________________________________
Users mailing list
<a moz-do-not-send="true" href="mailto:Users@lists.opensips.org">Users@lists.opensips.org</a>
<a moz-do-not-send="true" href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users">http://lists.opensips.org/cgi-bin/mailman/listinfo/users</a>
</pre>
</blockquote>
</div>
</blockquote>
</blockquote>
</div>
</blockquote>
</blockquote>
</div>
</blockquote>
</blockquote>
</div>
</blockquote>
</blockquote>
</div>
</blockquote>
</blockquote>
<br>
</body>
</html>