[OpenSIPS-Devel] OpenSIPS Crash
Bogdan-Andrei Iancu
bogdan at opensips.org
Wed Jun 26 09:12:01 EDT 2019
Oh, just do in gdb:
f 2
p *((struct qm_frag*)((char*)(p)-sizeof(struct qm_frag)))
In regards to async, I was more talking about the script async() (to try
to remove it) , not about the async in the protocols.
Best regards,
Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
https://www.opensips-solutions.com
OpenSIPS Summit 2019
https://www.opensips.org/events/Summit-2019Amsterdam/
On 06/26/2019 04:00 PM, Ben Newlin wrote:
>
> Bogdan,
>
> Thanks for the response. I will get the info you need. When you say to
> print in frame 2, is there some command I must issue to put the
> current context into frame 2 (or 7 or 8) or do I just issue the
> command you provided? Apologies, I use GDB very seldomly.
>
> Also, while we are using async support in this proxy, none of our
> current TLS traffic is using any async functions. Do you think other
> calls using async could be interfering with the TLS calls?
>
> Ben Newlin
>
> *From: *Bogdan-Andrei Iancu <bogdan at opensips.org>
> *Date: *Wednesday, June 26, 2019 at 5:25 AM
> *To: *Ben Newlin <Ben.Newlin at genesys.com>, OpenSIPS devel mailling
> list <devel at lists.opensips.org>
> *Subject: *Re: [OpenSIPS-Devel] OpenSIPS Crash
>
> Hi Ben,
>
> In any of the corefile, in frame 2 could you print
> p *((struct qm_frag*)((char*)(p)-sizeof(struct qm_frag)))
>
>
> Now, I see that the backtrace via libcurl & libcrypto may go via some
> error cleanup - see frame #7 and #8 - so , do you see any curl error
> prior to the crash (ideally from the same process) ?
>
> If you can reproduce it easily, could you try without the async
> support, just as a PoC only.
>
> Thanks,
>
>
> Bogdan-Andrei Iancu
> OpenSIPS Founder and Developer
> https://www.opensips-solutions.com
> OpenSIPS Summit 2019
> https://www.opensips.org/events/Summit-2019Amsterdam/
>
> On 06/25/2019 10:04 PM, Ben Newlin wrote:
>
> Yes, we are using tls_mgm and proto_tls. We have the following
> version of openssl-devel installed:
>
> # yum info openssl-devel
>
> Name : openssl-devel
>
> Arch : x86_64
>
> Epoch : 1
>
> Version : 1.0.2k
>
> Release : 16.amzn2.1.1
>
> Size : 3.1 M
>
> Repo : installed
>
> From repo : amzn2-core
>
> Summary : Files for development of applications which will use
> OpenSSL
>
> URL : http://www.openssl.org/ <http://www.openssl.org/>
>
> License : OpenSSL
>
> Description : OpenSSL is a toolkit for supporting cryptography.
> The openssl-devel
>
> : package contains include files needed to develop
> applications which
>
> : support various cryptographic algorithms and protocols.
>
> Ben Newlin
>
> *From: *Bogdan-Andrei Iancu <bogdan at opensips.org>
> <mailto:bogdan at opensips.org>
> *Date: *Tuesday, June 25, 2019 at 2:54 PM
> *To: *Ben Newlin <Ben.Newlin at genesys.com>
> <mailto:Ben.Newlin at genesys.com>, OpenSIPS devel mailling list
> <devel at lists.opensips.org> <mailto:devel at lists.opensips.org>
> *Subject: *Re: [OpenSIPS-Devel] OpenSIPS Crash
>
> Thank you Ben :)
>
> The backtraces report a double free, but the mem ops are performed
> via libcrypto ( opensips registers mem-specific functions to lib
> crypto).
>
> Are you using in your opensips cfg any other module that might
> require ssl/crtypto, like tls_mgm, proto_wss ?
>
> Regards,
>
>
> Bogdan-Andrei Iancu
>
>
>
> OpenSIPS Founder and Developer
>
> https://www.opensips-solutions.com
>
> OpenSIPS Summit 2019
>
> https://www.opensips.org/events/Summit-2019Amsterdam/
>
> On 06/25/2019 06:49 PM, Ben Newlin wrote:
>
> Bogdan,
>
> You have good timing. :) I was just collecting the data from a
> crash this morning with the extra flags set.
>
> # opensips -V
>
> version: opensips 2.4.5 (x86_64/linux)
>
> flags: STATS: On, DISABLE_NAGLE, USE_MCAST, SHM_MMAP,
> PKG_MALLOC, QM_MALLOC, DBG_MALLOC, FAST_LOCK-ADAPTIVE_WAIT
>
> ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144,
> MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535
>
> poll method support: poll, epoll, sigio_rt, select.
>
> git revision: 60097425d
>
> main.c compiled on 22:27:12 Jun 24 2019 with gcc 7
>
> [1] https://pastebin.com/aj1mvk0N <https://pastebin.com/aj1mvk0N>
>
> [2] https://pastebin.com/kfSz9Ks0 <https://pastebin.com/kfSz9Ks0>
>
> [3] https://pastebin.com/hptH9yiA <https://pastebin.com/hptH9yiA>
>
> [4] https://pastebin.com/GfYYZxzB <https://pastebin.com/GfYYZxzB>
>
> [5] https://pastebin.com/CzYiNAch <https://pastebin.com/CzYiNAch>
>
> Ben Newlin
>
> *From: *Bogdan-Andrei Iancu <bogdan at opensips.org>
> <mailto:bogdan at opensips.org>
> *Date: *Tuesday, June 25, 2019 at 11:39 AM
> *To: *Ben Newlin <Ben.Newlin at genesys.com>
> <mailto:Ben.Newlin at genesys.com>, OpenSIPS devel mailling list
> <devel at lists.opensips.org> <mailto:devel at lists.opensips.org>
> *Subject: *Re: [OpenSIPS-Devel] OpenSIPS Crash
>
> Hi Ben,
>
> The backtrace + corefile should do it ;).
>
> Best regards,
>
>
>
> Bogdan-Andrei Iancu
>
>
>
> OpenSIPS Founder and Developer
>
> https://www.opensips-solutions.com
>
> OpenSIPS Summit 2019
>
> https://www.opensips.org/events/Summit-2019Amsterdam/
>
> On 06/20/2019 07:19 PM, Ben Newlin wrote:
>
> Bogdan,
>
> Yes I will enable those. This seems to be reproducing
> fairly often so hopefully we will get some more info
> quickly. Is there anything else I need to collect at time
> of the crash with this enabled or is the core dump sufficient?
>
> Ben Newlin
>
> *From: *Bogdan-Andrei Iancu <bogdan at opensips.org>
> <mailto:bogdan at opensips.org>
> *Date: *Thursday, June 20, 2019 at 12:15 PM
> *To: *Ben Newlin <Ben.Newlin at genesys.com>
> <mailto:Ben.Newlin at genesys.com>, OpenSIPS devel mailling
> list <devel at lists.opensips.org>
> <mailto:devel at lists.opensips.org>
> *Subject: *Re: [OpenSIPS-Devel] OpenSIPS Crash
>
> Hi Ben,
>
> This looks like very generic - a crash due a previous
> memory corruption - is it possible for you to run with
> QM_MALLOC + DBG_MALLOC ? otherwise it is rather impossible
> to identify the original source of corruption.
>
> Thanks and regards,
>
>
>
>
> Bogdan-Andrei Iancu
>
>
>
> OpenSIPS Founder and Developer
>
> https://www.opensips-solutions.com
>
> OpenSIPS Summit 2019
>
> https://www.opensips.org/events/Summit-2019Amsterdam/
>
> On 06/20/2019 05:34 PM, Ben Newlin wrote:
>
> We have had another crash this morning:
> https://pastebin.com/dmXYbhzC
>
> OpenSIPS version is the same as below.
>
> Ben Newlin
>
> *From: *Ben Newlin <Ben.Newlin at genesys.com>
> <mailto:Ben.Newlin at genesys.com>
> *Date: *Wednesday, June 19, 2019 at 2:38 PM
> *To: *OpenSIPS devel mailling list
> <devel at lists.opensips.org>
> <mailto:devel at lists.opensips.org>, Bogdan-Andrei Iancu
> <bogdan at opensips.org> <mailto:bogdan at opensips.org>
> *Subject: *Re: [OpenSIPS-Devel] OpenSIPS Crash
>
> The Github issue has the version info for that, for
> all the other crashes the version is below.
>
> $ opensips -V
>
> version: opensips 2.4.5 (x86_64/linux)
>
> flags: STATS: On, DISABLE_NAGLE, USE_MCAST, SHM_MMAP,
> PKG_MALLOC, F_MALLOC, FAST_LOCK-ADAPTIVE_WAIT
>
> ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144,
> MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535
>
> poll method support: poll, epoll, sigio_rt, select.
>
> git revision: 60097425d
>
> main.c compiled on 18:06:35 Jun 13 2019 with gcc 7
>
> Ben Newlin
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opensips.org/pipermail/devel/attachments/20190626/72ff1964/attachment-0001.html>
More information about the Devel
mailing list