[OpenSIPS-Users] OpenSIPS core dumps

thrillerbee thrillerbee at gmail.com
Wed Nov 10 17:19:33 CET 2010


Bogdan,

Well, I spoke too soon - it's not just an issue with the opensipsctl fifo
calls - looks more like a memory leak.  It crashed again today, but I did
get some errors in the syslog this time right before the crash:
Nov 10 15:42:32 core1 /usr/local/sbin/opensips[27044]:
ERROR:db_flatstore:new_flat_id: no pkg memory left
Nov 10 15:42:32 core1 kernel: [5508366.582447] opensips[27044]: segfault at
10 ip 7fa7ff74c21f sp 7fffdc101700 error 4 in
db_flatstore.so[7fa7ff749000+5000]
To be thorough, I've attached the backtrace & output from print commands
(although they're the same as before).

To answer your question, yes - I do use the flat_rotate MI command.

Thanks again.

On Wed, Nov 10, 2010 at 4:04 AM, Bogdan-Andrei Iancu <bogdan at voice-system.ro
> wrote:

> Hi,
>
> opensipsctl takes care that each command takes a separate fifo reply, so
> here it should be no problem. But the problem may be when comes with sending
> multiple commands (via FIFO) in the same time - this translates into
> parallel writes to the same file and depends on the atomicity of the write
> op.
>
> But in the worst case, a mixture at the FIFO level may lead to bogus
> command and not in any kind of crash....Do you use the "flat_rotate" MI
> command ?
>
> Regards,
> Bogdan
>
> thrillerbee wrote:
>
>> Bogdan,
>>
>> It seems the issue is with 'opensipsctl fifo' - it's very sensitive to
>> simultaneous calls.  Basically, I've combined all my scripts to prevent
>> 'opensipsctl fifo' from being called too frequently and that seems (so far)
>> to have mitigated the issue.  Is there anything one should know about how
>> (not) to use /opensipsctl/?
>>
>> Thanks.
>>
>> On Mon, Nov 8, 2010 at 6:07 AM, Bogdan-Andrei Iancu <
>> bogdan at voice-system.ro <mailto:bogdan at voice-system.ro>> wrote:
>>
>>    Hi,
>>
>>    strange if you do not have any errors :(....
>>
>>    I just made a fix on both trunk and 1.6 to extend some checks in
>>    flatstore and prevent crashing (even if the DB op will not be
>>    executed).
>>
>>    Could you update from SVN and see if stops crashing ?
>>
>>    Regards,
>>    Bogdan
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opensips.org/pipermail/users/attachments/20101110/5ca57ab4/attachment-0001.htm>
-------------- next part --------------
Core was generated by `/usr/local/sbin/opensips -P /var/run/opensips/opensips.pid -m 512 -u root -g ro'.
Program terminated with signal 11, Segmentation fault.
[New process 27044]
#0  0x00007fa7ff74c21f in flat_db_insert (h=0x7efb38, k=0x7fa7fe48ca60, v=0x7fa7fe48cd20, n=19) at flatstore.c:165
165             f = CON_FILE(h);
(gdb) bt full
#0  0x00007fa7ff74c21f in flat_db_insert (h=0x7efb38, k=0x7fa7fe48ca60, v=0x7fa7fe48cd20, n=19) at flatstore.c:165
        f = <value optimized out>
        i = <value optimized out>
        l = <value optimized out>
        p = <value optimized out>
        __FUNCTION__ = "flat_db_insert"
#1  0x00007fa7fe2737ef in acc_db_request (rq=0x7fa7de27a978, rpl=<value optimized out>) at acc.c:364
        m = 19
        n = <value optimized out>
        i = <value optimized out>
        my_ps = (db_ps_t) 0x0
        __FUNCTION__ = "acc_db_request"
#2  0x00007fa7fe27869e in tmcb_func (t=<value optimized out>, type=<value optimized out>, ps=<value optimized out>) at acc_logic.c:386
No locals.
#3  0x00007fa7ff1014a2 in run_trans_callbacks (type=256, trans=0x7fa7df8a6e00, req=0x7fa7de27a978, rpl=0x7efef0, code=200) at t_hooks.c:208
        cbp = (struct tm_callback *) 0x7fa7dc84f6f0
        backup = (struct usr_avp **) 0x771f48
        trans_backup = (struct cell *) 0x7fa7df8a6e00
        __FUNCTION__ = "run_trans_callbacks"
#4  0x00007fa7ff1015c7 in run_trans_callbacks_locked (type=256, trans=0x7fa7df8a6e00, req=0x7fa7de27a978, rpl=0x7efef0, code=200) at t_hooks.c:254
No locals.
#5  0x00007fa7ff11ea38 in relay_reply (t=0x7fa7df8a6e00, p_msg=<value optimized out>, branch=1, msg_status=200, cancel_bitmap=0x7fffdc101ad8) at t_reply.c:1257
        relay = 1
        save_clone = 0
        buf = 0x1184390 "SIP/2.0 200 OK\r\nVia: SIP/2.0/UDP 24.121.80.36;branch=z9hG4bKadcb.35a7dc2.0\r\nVia: SIP/2.0/UDP 184.106.205.223;branch=z9hG4bKadcb.be5e7083.0\r\nVia: SIP/2.0/UDP 216.18.222.3;branch=z9hG4bKadcb.18d6b484.0\r"...
        res_len = 1391
        relayed_code = 200
        relayed_msg = (struct sip_msg *) 0x7efef0
        bm = {to_tag_val = {s = 0x1 <Address 0x1 out of bounds>, len = 8322800}}
        totag_retr = <value optimized out>
        reply_status = RPS_COMPLETED
        cb_s = {
  s = 0x1184390 "SIP/2.0 200 OK\r\nVia: SIP/2.0/UDP 24.121.80.36;branch=z9hG4bKadcb.35a7dc2.0\r\nVia: SIP/2.0/UDP 184.106.205.223;branch=z9hG4bKadcb.be5e7083.0\r\nVia: SIP/2.0/UDP 216.18.222.3;branch=z9hG4bKadcb.18d6b484.0\r"..., len = 1391}
        text = {s = 0x18f <Address 0x18f out of bounds>, len = 1452}
        __FUNCTION__ = "relay_reply"
#6  0x00007fa7ff11f548 in reply_received (p_msg=0x7efef0) at t_reply.c:1502
        last_uac_status = <value optimized out>
        branch = 1
        reply_status = <value optimized out>
        timer = <value optimized out>
        cancel_bitmap = 0
        t = (struct cell *) 0x7fa7df8a6e00
        backup_list = <value optimized out>
        has_reply_route = 3692375232
        __FUNCTION__ = "reply_received"
#7  0x000000000042519d in forward_reply (msg=0x7efef0) at forward.c:559
        new_buf = <value optimized out>
        to = <value optimized out>
        new_len = <value optimized out>
        mod = (struct sr_module *) 0x78dfe0
        proto = <value optimized out>
        id = <value optimized out>
        send_sock = <value optimized out>
        len = <value optimized out>
        __FUNCTION__ = "forward_reply"
#8  0x000000000046663d in receive_msg (
    buf=0x761f40 "SIP/2.0 200 OK\r\nVia: SIP/2.0/UDP 24.121.80.38;branch=z9hG4bKadcb.c144d801.1\r\nVia: SIP/2.0/UDP 24.121.80.36;branch=z9hG4bKadcb.35a7dc2.0\r\nVia: SIP/2.0/UDP 184.106.205.223;branch=z9hG4bKadcb.be5e7083.0\r"..., len=1452, rcv_info=0x7fffdc101bd0) at receive.c:200
        msg = (struct sip_msg *) 0x7efef0
        __FUNCTION__ = "receive_msg"
#9  0x00000000004ac48a in udp_rcv_loop () at udp_server.c:492
        len = 1452
        tmp = <value optimized out>
        from = (union sockaddr_union *) 0x7efeb8
        fromlen = 16
        ri = {src_ip = {af = 2, len = 4, u = {addrl = {5327868006277719247, 0}, addr32 = {878322895, 1240490937, 0, 0}, addr16 = {9423, 13402, 25529, 18928, 0, 0, 0, 
        0}, addr = "##Z4#c####00\000\000\000\000\000\000"}}, dst_ip = {af = 2, len = 4, u = {addrl = {642808138, 0}, addr32 = {642808138, 0, 0, 0}, addr16 = {31050, 
        9808, 0, 0, 0, 0, 0, 0}, addr = "JyP&", '\0' <repeats 11 times>}}, src_port = 5070, dst_port = 5060, proto = 1, proto_reserved1 = 0, proto_reserved2 = 0, 
  src_su = {s = {sa_family = 2, sa_data = "\023##$Z4\000\000\000\000\000\000\000"}, sin = {sin_family = 2, sin_port = 52755, sin_addr = {s_addr = 878322895}, 
      sin_zero = "\000\000\000\000\000\000\000"}, sin6 = {sin6_family = 2, sin6_port = 52755, sin6_flowinfo = 878322895, sin6_addr = {in6_u = {
          u6_addr8 = '\0' <repeats 15 times>, u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, u6_addr32 = {0, 0, 0, 0}}}, sin6_scope_id = 0}}, bind_address = 0x7db170}
        p = (callback_list *) 0x7fffdc101bd8
        buf = "SIP/2.0 200 OK\r\nVia: SIP/2.0/UDP 24.121.80.38;branch=z9hG4bKadcb.c144d801.1\r\nVia: SIP/2.0/UDP 24.121.80.36;branch=z9hG4bKadcb.35a7dc2.0\r\nVia: SIP/2.0/UDP 184.106.205.223;branch=z9hG4bKadcb.be5e7083.0\r"...
        __FUNCTION__ = "udp_rcv_loop"
#10 0x000000000042c1b7 in main (argc=<value optimized out>, argv=0x7fffdc101db8) at main.c:818
        cfg_log_stderr = <value optimized out>
        cfg_stream = <value optimized out>
        c = <value optimized out>
        r = <value optimized out>
        tmp = 0x7fffdc102a3f ""
        tmp_len = <value optimized out>
        port = <value optimized out>
        proto = <value optimized out>
        ret = <value optimized out>
        seed = 522039592
        rfd = <value optimized out>
        __FUNCTION__ = "main"
(gdb) print h
$1 = (const db_con_t *) 0x7efb38
(gdb) print h->tail
$2 = 0
(gdb) print ((struct flat_con*)(h->tail))->file
Cannot access memory at address 0x10


More information about the Users mailing list