[OpenSIPS-Users] Opensips crashes when retrieving AVPs from Radius requests

Samuel Muller sml at l33.fr
Sun Sep 30 14:52:24 CEST 2012


Hey Bogdan,

tomorrow morning i'll re-install the full opensips 1.8.1 with all the
needed libs to debug - actually we made a simple debian package and
the debug package of opensips is not usefull for the modules we are
using. I've to admit that I'm a bit afraid to see that crashes happens
randomly.

++

Samuel MULLER
Telecom & Media Consultant
co-owner
L33 NETWORKS
+33 663 128 505
sml at l33.fr
www.l33.fr


On Sun, Sep 30, 2012 at 1:17 PM, Bogdan-Andrei Iancu
<bogdan at opensips.org> wrote:
> Hi Samuel,
>
> This bt looks a bit corrupted, so we need a different approach - have a way
> of reproducing this in a deterministic way ?
>
>
> Regards,
>
> Bogdan-Andrei Iancu
> OpenSIPS Founder and Developer
> http://www.opensips-solutions.com
>
>
> On 09/28/2012 03:05 PM, Samuel Muller wrote:
>>
>> Hello Bogdan,
>>
>> i've a small update: it's not crashing with acc radius, only with
>> auth_aaa and aaa_radius.
>>
>> Here's the result of (gdb) bt full :
>>
>> └ /tmp $ sudo gdb --core=/tmp/core.opensips.sig11.1758
>> /usr/lib/debug/usr/sbin/opensips
>> GNU gdb (GDB) 7.0.1-debian
>> Core was generated by `/usr/sbin/opensips -P
>> /var/run/opensips/opensips.pid -m 512 -M 16 -u opensips -g opensips'.
>> Program terminated with signal 11, Segmentation fault.
>> #0  0x00007f51de596dd0 in ?? () from /lib/libc.so.6
>> (gdb)
>> (gdb) bt full
>>
>> #0  0x00007f51de596dd0 in ?? () from /lib/libc.so.6
>> No symbol table info available.
>> #1  0x000000000042d34b in str_cmp (tree=<value optimized out>,
>> key=...) at map.c:62
>> No locals.
>> #2  map_find (tree=<value optimized out>, key=...) at map.c:101
>>          cmp = 16
>>          p = 0x7f51b46a1ab0
>> #3  0x00000000013615d0 in ?? ()
>> No symbol table info available.
>> #4  0x000000000000000c in ?? ()
>> No symbol table info available.
>> #5  0x0000000000000002 in ?? ()
>> No symbol table info available.
>> #6  0x00007f51d908abf0 in values.8998 () from
>> /usr/lib/opensips/modules/aaa_radius.so
>> No symbol table info available.
>> #7  0x00007f51d908abf0 in values.8998 () from
>> /usr/lib/opensips/modules/aaa_radius.so
>> No symbol table info available.
>> #8  0x00007f51b45576c8 in ?? ()
>> No symbol table info available.
>> #9  0x00000000ffffffff in ?? ()
>> No symbol table info available.
>> #10 0x000000000048db36 in __search_avp_map (alias=0xe, m=0xc) at
>> usr_avp.c:482
>>          id =<value optimized out>
>>          __FUNCTION__ = '\000'<repeats 16 times>
>> #11 0x00000030ffffffff in ?? ()
>> No symbol table info available.
>> #12 0x00007fffd2b792a0 in ?? ()
>> No symbol table info available.
>> #13 0x00007fffd2b791d0 in ?? ()
>> No symbol table info available.
>> #14 0xd8a4b6eb360f80e6 in ?? ()
>> No symbol table info available.
>> #15 0xe13853fa7600e35f in ?? ()
>> No symbol table info available.
>> #16 0xd8a4b6eb360f80e6 in ?? ()
>> No symbol table info available.
>> #17 0x00007f51d8e8a1e1 in __FUNCTION__.9143 () from
>> /usr/lib/opensips/modules/aaa_radius.so
>> No symbol table info available.
>> #18 0x00007f51d908abf0 in values.8998 () from
>> /usr/lib/opensips/modules/aaa_radius.so
>> No symbol table info available.
>> #19 0x00007fffd2b7927c in ?? ()
>> No symbol table info available.
>> #20 0x0000000000000001 in ?? ()
>> No symbol table info available.
>> #21 0x00000000ffffffff in ?? ()
>> No symbol table info available.
>> ---Type<return>  to continue, or q<return>  to quit---
>> #22 0x00007f51dd339148 in ?? ()
>> No symbol table info available.
>> #23 0x000000000000002a in ?? ()
>> No symbol table info available.
>> #24 0x000000000048de78 in lookup_avp_alias_str (name=0x7f51b46a1ab0,
>> avp_name=0xe, extra=12) at usr_avp.c:499
>> No locals.
>> #25 parse_avp_spec_aux (name=0x7f51b46a1ab0, avp_name=0xe, extra=12)
>> at usr_avp.c:559
>>          id = 20321744
>>          __FUNCTION__ = '\000'<repeats 18 times>
>> #26 0x000000000136171e in ?? ()
>> No symbol table info available.
>> #27 0x5d9e214c00000001 in ?? ()
>> No symbol table info available.
>> #28 0x00007fffd2b792a0 in ?? ()
>> No symbol table info available.
>> #29 0x0000000000000002 in ?? ()
>> No symbol table info available.
>> #30 0x0000000000000002 in ?? ()
>> No symbol table info available.
>> #31 0x0000000000000002 in ?? ()
>> No symbol table info available.
>> #32 0x0000000000000002 in ?? ()
>> No symbol table info available.
>> #33 0x00000000013615a0 in ?? ()
>> No symbol table info available.
>> #34 0x00007f51d8433620 in ?? () from /usr/lib/opensips/modules/auth_aaa.so
>> No symbol table info available.
>> #35 0x000000000048e164 in get_avp_id (name=0x13615d0) at usr_avp.c:580
>>          id =<value optimized out>
>>          __FUNCTION__ = "\000\000\000\000\000\000\000\000\000\000"
>> #36 0x000000000136171e in ?? ()
>> No symbol table info available.
>> #37 0x0000004100000001 in ?? ()
>> No symbol table info available.
>> #38 0x0000000000000002 in ?? ()
>> No symbol table info available.
>> #39 0x00007f51d8e882b0 in extract_avp (vp=<value optimized out>) at
>> rad.c:201
>>          names = {s = 0x13615d0 "caller_stime:none", len = 12}
>>          values = {s = 0x13615dd "none", len = 4}
>>          name =<value optimized out>
>>          r =<value optimized out>
>>          p = 0x0
>>          end =<value optimized out>
>>          value = {n = 20321757, s = {s = 0x13615dd "none", len = 4}}
>>          flags =<value optimized out>
>>          __FUNCTION__ = "extract_avp"
>> ---Type<return>  to continue, or q<return>  to quit---
>> #40 0x00007f51d8e888ee in rad_send_message (rh=<value optimized out>,
>> request=<value optimized out>, reply=<value optimized out>) at
>> rad.c:262
>>          msg =
>> "\000\224\267\322\377\177\000\000\000\000\000\000\000\000\000\000藷\322\377\177\000\000\000\000\000\000\000\000\000\000p\230\267\322\377\177\000\000䗷\322\322\006\000\000X\377o\332Q\177\000\000\377\377\377\377\377\377\377\377d\227\267\322",
>> '\000'<repeats 24 times>"\377, \177\000\000
>>
>> \224\267\322\377\177\000\000\000\000\000\000藷\322H\230\267\322\377\177\000\000\300\260U\336\n\000\000\000_\"\000\000\000\000\000\000G\230\267\322\003\000\000\000\331\bP\000\000\000\000\000\377\377\377\377\377\377\377\377䗷\322\377\177",
>> '\000'<repeats 18 times>"\320,
>>
>> q3\335Q\177\000\000䗷\322\377\177\000\000\000\000\000\000H\230\267\322d\000\000\000\004\000\000\000藷\322\377\177\000\000
>>
>> \377\065\001\000\000\000\000\260\232\267\322\377\177\000\000\005\000\000\000\000\000\000\000\300\232\267\322\377\177\000\000\017c[\336Q\177\000\000\270\022M\336Q\177\000\000\001\000\000\000\000\000\000\000\020c[\336Q\177\000\000\020c[\336Q\177\000\000\002\000\000\000\000\000\000\000d6j\264Q\177\000\000"...
>>          vp =<value optimized out>
>>          attr =<value optimized out>
>>          result = 0
>>          __FUNCTION__ = "rad_send_message"
>> #41 0x00007f51d8230c3f in aaa_authorize_sterman (_msg=<value optimized
>> out>, _cred=0x7f51dd3394f8, _method=<value optimized out>,
>> _user=<value optimized out>) at sterman.c:253
>>          send =<value optimized out>
>>          received = 0x7f51dd338c90
>>          service = 15
>>          ruri =<value optimized out>
>>          __FUNCTION__ = "aaa_authorize_sterman"
>> #42 0x00007f51d82301fe in authorize (_msg=0x7f51dd335cc0,
>> _realm=<value optimized out>, _uri_user=<value optimized out>,
>> _hftype=5) at authorize.c:132
>>          res =<value optimized out>
>>          ret =<value optimized out>
>>          h = 0x7f51dd338ff0
>>          cred = 0x7f51dd3368b0
>>          user = {s = 0x7f51dd310248 "10907", len = 5}
>>          domain = {
>>            s = 0x76631d "ibtel.vp.ibrowse.com>;tag=1c1314559488\r\nTo:
>> <sip:00344745615 at 192.168.1.50>\r\nCall-ID:
>> 131455892322420100263 at 192.168.1.55\r\nCSeq: 2 INVITE\r\nContact:
>> <sip:10907 at 192.168.1.55:5060>\r\nRecord-Route:<sip:192."..., len =
>> 20}
>>          pv_val = {rs = {s = 0x7fffd2b7a930 "\300\251\267\322\377\177",
>> len = 0}, ri = 1758, flags = 0}
>>          __FUNCTION__ = "authorize"
>> #43 0x00000000004160c7 in do_action (a=0x7f51dd2ac640,
>> msg=0x7f51dd335cc0) at action.c:1483
>>          val_s = {s = 0x0, len = -562115008}
>>          increment =<value optimized out>
>>          decrement =<value optimized out>
>>          aux = {s = 0x0, len = -1}
>>          adv_addr = {s = 0x1360170 "`\002", len = 96}
>>          ret =<value optimized out>
>>          v =<value optimized out>
>>          to =<value optimized out>
>>          p =<value optimized out>
>>          tmp =<value optimized out>
>>          new_uri =<value optimized out>
>>          end =<value optimized out>
>>          crt =<value optimized out>
>>          len =<value optimized out>
>>          i =<value optimized out>
>>          user =<value optimized out>
>>          expires =<value optimized out>
>>          vals = {{s = 0x3000000010<Address 0x3000000010 out of
>> bounds>, len = -759714864}, {s = 0x7fffd2b7ab10 "\200\001\066\001",
>> len = 0}, {s = 0xffffffffffffffa8<Address 0xffffffffffffffa8 out of
>> bounds>, len = 20316528}, {
>> ---Type<return>  to continue, or q<return>  to quit---
>>              s = 0x2010<Address 0x2010 out of bounds>, len =
>> 20324736}, {s = 0x7f51de7ece40 "", len = 96}}
>>          result = {s = 0x7fffd2b7aaf0 "\030", len = -565159604}
>>          uri = {user = {s = 0x50ffd8 "", len = 0}, passwd = {s =
>> 0x7fffd2b7acb8 "(Α\332Q\177", len = -759714664}, host = {s =
>> 0x7f51d79fbd20 "DBG:permissions:%s: subnet table is empty\n", len =
>> -759714692}, port = {
>>              s = 0x7fffd2b7ab04 "\377\177", len = 0}, params = {s =
>> 0x0, len = -565017482}, headers = {s = 0x7fffd2b7a800
>> "0\251\267\322\377\177", len = 0}, port_no = 42997, proto = 53943,
>> type = 32767, transport = {s = 0x0,
>>              len = -759714704}, ttl = {s = 0x2<Address 0x2 out of
>> bounds>, len = -759715424}, user_param = {s = 0x1ffffffff<Address
>> 0x1ffffffff out of bounds>, len = -759715850}, maddr = {
>>              s = 0xffffffff00000000<Address 0xffffffff00000000 out of
>> bounds>, len = -759715872}, method = {s = 0x7f51de528476
>> "L\213L$(\351\030\355\377\377Hc\355L\211\357\276\060", len =
>> -562130912}, lr = {
>>              s = 0x7fff00000000<Address 0x7fff00000000 out of bounds>,
>> len = -759715755}, r2 = {s = 0x7fff00000000<Address 0x7fff00000000
>> out of bounds>, len = 16}, gr = {s = 0x2<Address 0x2 out of bounds>,
>> len = -759715328},
>>            transport_val = {s = 0x1d2b7ad3c<Address 0x1d2b7ad3c out of
>> bounds>, len = -759715754}, ttl_val = {s = 0x0, len = -759715776},
>> user_param_val = {s = 0x7fffd2b7a856 "58樷\322\377\177", len =
>> -562130912}, maddr_val = {
>>              s = 0x7fffd2b7a9df "", len = 19093424}, method_val = {s =
>> 0x7fffd2b7a800 "0\251\267\322\377\177", len = 16}, lr_val = {s =
>> 0x7fffd2b7a9f0 "\020 ", len = -759715536}, r2_val = {
>>              s = 0x7f5100000000<Address 0x7f5100000000 out of bounds>,
>> len = -759715611}, gr_val = {s = 0x7f5100000000<Address
>> 0x7f5100000000 out of bounds>, len = 20316556}, u_name = {{s = 0x2
>> <Address 0x2 out of bounds>,
>>                len = -759715184}, {s = 0x166666666<Address 0x166666666
>> out of bounds>, len = -759715610}, {s = 0x0, len = -759715632}, {s =
>> 0x7fffd2b7a8e6 "", len = -562130912}, {s = 0x7f51de5b6309 "T ", len =
>> 20316556}},
>>            u_val = {{s = 0x7f51de7e93e0 " \215~\336Q\177", len =
>> 20316556}, {s = 0x7<Address 0x7 out of bounds>, len = 8187}, {s =
>> 0x7f51de527289 "HcD$@D\213\\$\bH\215D\005", len = -562115008}, {s =
>> 0x0, len = 8192}, {
>>                s = 0x7f5100000000<Address 0x7f5100000000 out of
>> bounds>, len = 3}}, u_params_no = 2}
>>          next_hop = {user = {s = 0x7f51d5331cdd "%s: nothing to
>> trace...\n", len = 0}, passwd = {s = 0x7fffd2b7aa20
>> "\220\\3\335Q\177", len = 0}, host = {s = 0x7f51de5b630c "[%d]", len =
>> -759715220}, port = {s = 0x0, len = 0},
>>            params = {s = 0x0, len = 0}, headers = {s = 0x7fffd2b7a937
>> "", len = 5248458}, port_no = 0, proto = 0, type = ERROR_URI_T,
>> transport = {s = 0xa00000002<Address 0xa00000002 out of bounds>, len
>> = -759715529}, ttl = {
>>              s = 0x7f51de4ff326 "H\211E(\353\277\017\037@", len =
>> 20316004}, user_param = {s = 0x7fffd2b7a938 "@\316~\336Q\177", len =
>> -627974649}, maddr = {s = 0x0, len = 5308380}, method = {
>>              s = 0x7f51de4d12b8
>>
>> "L\213\205p\371\377\377H\213\215\200\371\377\377L\213\215x\371\377\377I9\300\017\205\017\373\377\377IcԸ\377\377\377\177H)\320I9\300\017\207\373\372\377\377E)\364A\001\314A\200\071",
>> len = 0},
>>            lr = {s = 0x50ffdc "", len = 5308380}, r2 = {s = 0xa00000000
>> <Address 0xa00000000 out of bounds>, len = 1758}, gr = {s = 0xd2b7ac40
>> <Address 0xd2b7ac40 out of bounds>, len = -759715104}, transport_val =
>> {
>>              s = 0x7fffd2b7ac50 "\001", len = -759714936}, ttl_val = {s
>> = 0x7fffd2b7ab68 "x\254\267\322\377\177", len = 5308376},
>> user_param_val = {s = 0x7fffd2b7ab4c "", len = -759715372}, maddr_val
>> = {s = 0x0, len = 0},
>>            method_val = {s = 0x7fff00000000<Address 0x7fff00000000 out
>> of bounds>, len = -759716144}, lr_val = {s = 0x0, len = -759715144},
>> r2_val = {s = 0x7fffd2b7abb8 "", len = -759715008}, gr_val = {
>>              s = 0x6d200000000<Address 0x6d200000000 out of bounds>,
>> len = -564436212}, u_name = {{s = 0xffffffffffffffff<Address
>> 0xffffffffffffffff out of bounds>, len = 0}, {s = 0x0, len = 0}, {
>>                s = 0x7f5100000000<Address 0x7f5100000000 out of
>> bounds>, len = -759715040}, {s = 0x0, len = -759714928}, {s =
>> 0xde4ff326<Address 0xde4ff326 out of bounds>, len = 0}}, u_val = {{
>>                s = 0xd2b7ad20<Address 0xd2b7ad20 out of bounds>, len =
>> 34}, {s = 0x7fffd2b7ae30 "\200\263\267\322\377\177", len = 0}, {s =
>> 0xde4d12b8<Address 0xde4d12b8 out of bounds>, len = 0}, {s =
>> 0x7f51d79fbd4a "",
>>                len = 42}, {s = 0xad79fbc9f<Address 0xad79fbc9f out of
>> bounds>, len = 1651}}, u_params_no = 43704}
>>          u =<value optimized out>
>>          port =<value optimized out>
>>          cmatch =<value optimized out>
>>          aitem =<value optimized out>
>>          adefault =<value optimized out>
>>          spec =<value optimized out>
>>          model =<value optimized out>
>>          val = {rs = {s = 0x2010<Address 0x2010 out of bounds>, len =
>> -565156589}, ri = -759715392, flags = 32767}
>>          pve =<value optimized out>
>>          name_s = {s = 0x0, len = -88}
>>          start = {tv_sec = 1348751368, tv_usec = 690416}
>>          end_time =<value optimized out>
>>          __FUNCTION__ = "\000\000\000\000\000\000\000\000\000"
>> #44 0x0000000000000000 in ?? ()
>> No symbol table info available.
>> (gdb)
>>
>>
>> hope it helps ...
>>
>> Samuel MULLER
>> Telecom&  Media Consultant
>>
>> co-owner
>> L33 NETWORKS
>> +33 663 128 505
>> sml at l33.fr
>> www.l33.fr
>>
>>
>> On Fri, Sep 28, 2012 at 12:52 PM, Bogdan-Andrei Iancu
>> <bogdan at opensips.org>  wrote:
>>>
>>> Hi Samuel,
>>>
>>> If you have the core file, could you get a backtrace and post it ?
>>>
>>> Regards,
>>>
>>> Bogdan-Andrei Iancu
>>> OpenSIPS Founder and Developer
>>> http://www.opensips-solutions.com
>>>
>>>
>>>
>>> On 09/27/2012 03:43 PM, Samuel Muller wrote:
>>>>
>>>> Hello,
>>>>
>>>> I see something really strange in v.1.8.x that not happened in
>>>> v.1.7.x, concerning radius request.
>>>>
>>>> I'm using aaa_radius, acct and auth_aaa modules.
>>>> I've 2 radius requests where i need to retrieve AVPs : one for the
>>>> auth, antoher one to check the callee's stuff using "call-check"
>>>> custom request (radius_sent_auth().
>>>>
>>>> my sets are correctly defined, freeradius is sent back exactly what i
>>>> expect to, and the process looks really clean (PostgreSQL ok, ...).
>>>>
>>>> but it crashes randomly when the function tries to extract AVPs:
>>>>
>>>> 2012-09-27T08:14:24.231112+00:00 vpp-scp [1296]:
>>>> 31235822142010193058 at 192.168.1.55: - AUTH
>>>> 2012-09-27T08:14:24.231120+00:00 vpp-scp [1296]:
>>>> DBG:core:parse_headers: flags=ffffffffffffffff
>>>> 2012-09-27T08:14:24.231675+00:00 vpp-scp [1296]:
>>>> DBG:sl:run_sl_callbacks: callback id 0 entered
>>>> 2012-09-27T08:14:24.231692+00:00 vpp-scp [1296]:
>>>> DBG:siptrace:trace_sl_onreply_out: trace slonreply out
>>>> 2012-09-27T08:14:24.231703+00:00 vpp-scp [1296]:
>>>> DBG:siptrace:trace_sl_onreply_out: nothing to trace...
>>>> 2012-09-27T08:14:24.231714+00:00 vpp-scp [1296]: DBG:auth:check_nonce:
>>>> comparing [50640afe000000033e97d5256be5383c2ea043e416ed4559] and
>>>> [50640afe000000033e97d5256be5383c2ea043e416ed4559]
>>>> ...
>>>> 2012-09-27T08:14:24.247104+00:00 vpp-scp [1296]:
>>>> DBG:core:__search_avp_map: looking for [caller_username] avp  - found
>>>> 43
>>>> 2012-09-27T08:14:24.247114+00:00 vpp-scp [1296]:
>>>> DBG:aaa_radius:extract_avp: AVP 'caller_username'='10907'/0 has been
>>>> added
>>>> 2012-09-27T08:14:24.248501+00:00 vpp-scp [1296]:
>>>> DBG:core:__search_avp_map: looking for [caller_stimeout] avp  - found
>>>> 40
>>>> 2012-09-27T08:14:24.248513+00:00 vpp-scp [1296]:
>>>> DBG:aaa_radius:extract_avp: AVP 'caller_stimeout'='3600'/0 has been
>>>> added
>>>> 2012-09-27T08:14:24.248581+00:00 vpp-scp [1296]:
>>>> DBG:core:__search_avp_map: looking for [caller_stime] avp  - found -1
>>>> 2012-09-27T08:14:32.802250+00:00 vpp-scp [1290]: DBG:core:handle_sigs:
>>>> status = 139
>>>> 2012-09-27T08:14:32.802279+00:00 vpp-scp [1290]:
>>>> INFO:core:handle_sigs: child process 1296 exited by a signal 11
>>>> 2012-09-27T08:14:32.802284+00:00 vpp-scp [1290]:
>>>> INFO:core:handle_sigs: core was generated
>>>> 2012-09-27T08:14:32.802288+00:00 vpp-scp [1290]:
>>>> INFO:core:handle_sigs: terminating due to SIGCHLD
>>>> 2012-09-27T08:14:32.802291+00:00 vpp-scp [1300]: INFO:core:sig_usr:
>>>> signal 15 received
>>>>
>>>>
>>>> I've core dumps, but nothing usefull ...
>>>>
>>>> this happens with all radius modules (acct, auth_aaa, aaa_radius), and
>>>> only since I use the v.1.8.0 and now 1.8.1 too. (from 1.7.2, no
>>>> problems)
>>>>
>>>> Any clues ?
>>>>
>>>>
>>>> Samuel MULLER
>>>> sml at l33.fr
>>>> www.l33.fr
>>>>
>>>> _______________________________________________
>>>> Users mailing list
>>>> Users at lists.opensips.org
>>>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>>>>
>



More information about the Users mailing list