[OpenSIPS-Users] Siptrace Issue
Bogdan-Andrei Iancu
bogdan at opensips.org
Wed Feb 6 20:56:16 CET 2013
Cool :)
In gdb, please do :
f 5
p trace_local_ip
p send_sock
p to
f 6
p ((struct dest_info*)ps->extra2)->send_sock
p *((struct dest_info*)ps->extra2)->send_sock
Thanks,
Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
http://www.opensips-solutions.com
On 02/06/2013 09:48 PM, Seth Schultz wrote:
> Thanks for all your help. I was finally able to get the system to
> dump a core file. Here is the output from the backtrace.
>
> #0 0x00007f4820da1425 in raise () from /lib/x86_64-linux-gnu/libc.so.6
> #1 0x00007f4820da4b8b in abort () from /lib/x86_64-linux-gnu/libc.so.6
> #2 0x00007f4815ee2d30 in pipport2su (pipport=<optimized out>,
> tmp_su=0x7fff9f5d4230, proto=<optimized out>) at siptrace.c:1798
> #3 0x00007f4815ee4ba3 in trace_send_hep_duplicate
> (toip=0x7f48160ff9e0 "udp:xxx.xxx.xxx.xxx:5060", fromip=0x7f4818da11b8
> "udp:172.16.1.115", body=<optimized out>) at siptrace.c:1619
> #4 save_siptrace (avp=0x7f4794ebf600, first_val=0x7fff9f5d4300,
> vals=0x7f48160ffdc0, keys=0x7f48160ffcc0, msg=<optimized out>) at
> siptrace.c:533
> #5 0x00007f4815ef8a61 in trace_msg_out (msg=0x7f4817464340,
> sbuf=<optimized out>, send_sock=<optimized out>, proto=1,
> to=<optimized out>) at siptrace.c:1053
> #6 0x00007f4815efbe5c in trace_onreq_out (t=<optimized out>,
> type=<optimized out>, ps=0x7fff9f5d43d0) at siptrace.c:937
> #7 0x00007f481721e8b2 in run_trans_callbacks (type=1024,
> trans=0x7f4794eba310, req=<optimized out>, rpl=<optimized out>,
> code=<optimized out>) at t_hooks.c:212
> #8 0x00007f481721d76d in t_forward_nonack (t=0x7f4794eba310,
> p_msg=0x7f4817464340, proxy=<optimized out>) at t_fwd.c:753
> #9 0x00007f481723c248 in w_t_relay (p_msg=0x7f4817464340, proxy=0x0,
> flags=<optimized out>) at tm.c:1160
> #10 0x0000000000418b47 in do_action (a=0x7f4818d8ea88,
> msg=0x7f4817464340) at action.c:1714
> #11 0x000000000041deee in run_action_list (a=<optimized out>,
> msg=0x7f4817464340) at action.c:170
> #12 0x0000000000499d9e in eval_elem (e=0x7f4818d8eb60,
> msg=0x7f4817464340, val=0x0) at route.c:1499
> #13 0x00000000004998bd in eval_expr (e=0x7f4818d8eb60,
> msg=0x7f4817464340, val=0x0) at route.c:1844
> #14 0x0000000000499974 in eval_expr (e=0x7f4818d8ebb0,
> msg=0x7f4817464340, val=0x0) at route.c:1860
> #15 0x00000000004998aa in eval_expr (e=0x7f4818d8ec00,
> msg=0x7f4817464340, val=0x0) at route.c:1865
> #16 0x000000000041875a in do_action (a=0x7f4818d8eef0,
> msg=0x7f4817464340) at action.c:992
> #17 0x000000000041deee in run_action_list (a=<optimized out>,
> msg=0x7f4817464340) at action.c:170
> #18 0x000000000041b7f0 in do_action (a=0x7f4818d8f418,
> msg=0x7f4817464340) at action.c:1009
> #19 0x000000000041deee in run_action_list (a=<optimized out>,
> msg=0x7f4817464340) at action.c:170
> #20 0x000000000041b7f0 in do_action (a=0x7f4818d8f4f0,
> msg=0x7f4817464340) at action.c:1009
> #21 0x000000000041deee in run_action_list (a=<optimized out>,
> msg=0x7f4817464340) at action.c:170
> #22 0x000000000041e420 in run_actions (msg=0x7f4817464340,
> a=0x7f4818d8c3c0) at action.c:136
> #23 run_actions (msg=0x7f4817464340, a=0x7f4818d8c3c0) at action.c:189
> #24 run_top_route (a=0x7f4818d8c3c0, msg=0x7f4817464340) at action.c:210
> #25 0x00007f4817232f22 in run_failure_handlers (t=0x7f4794eba310) at
> t_reply.c:657
> #26 t_should_relay_response (Trans=0x7f4794eba310, new_code=<optimized
> out>, branch=0, should_store=0x7fff9f5d6480, should_relay=<optimized
> out>, cancel_bitmap=<optimized out>,
> reply=0xffffffffffffffff) at t_reply.c:956
> #27 0x00007f4817232fb9 in relay_reply (t=0x7f4794eba310,
> p_msg=0xffffffffffffffff, branch=0, msg_status=408,
> cancel_bitmap=0x7fff9f5d6520) at t_reply.c:1170
> #28 0x00007f4817237a75 in fake_reply (branch=<optimized out>,
> t=0x7f4794eba310, code=<optimized out>) at timer.c:259
> #29 final_response_handler (fr_tl=<optimized out>) at timer.c:370
> #30 timer_routine (ticks=84, attr=<optimized out>) at timer.c:1011
> #31 0x00000000004d8cf2 in timer_ticker (drift=<synthetic pointer>,
> timer_list=<optimized out>) at timer.c:384
> #32 run_timer_process (tpl=<optimized out>) at timer.c:500
> #33 start_timer_processes () at timer.c:610
> #34 0x000000000041523a in main_loop () at main.c:945
> #35 main (argc=<optimized out>, argv=<optimized out>) at main.c:1541
>
> On 2/6/2013 2:27 PM, Bogdan-Andrei Iancu wrote:
>> That's because you need to have "root" permissions to write into
>> /proc/sys/kernel/core_pattern (used for forcing the core file
>> pattern). You should comment in the init.d file the line for writting
>> into that file.
>>
>> Regards,
>>
>> Bogdan-Andrei Iancu
>> OpenSIPS Founder and Developer
>> http://www.opensips-solutions.com
>>
>>
>> On 02/06/2013 08:52 PM, Seth Schultz wrote:
>>> Unfortunately I am having trouble getting the system to dump a core
>>> file. Here is the error message I am getting once I enable core dumps.
>>>
>>> /etc/init.d/opensips: 103: /etc/init.d/opensips: cannot create
>>> /proc/sys/kernel/core_pattern: Permission denied
>>>
>>> I have also tried starting opensips as the root user, but it still
>>> throws this error.
>>>
>>> Here is my /etc/defaults/opensips:
>>> RUN_OPENSIPS=yes
>>> USER=opensips
>>> GROUP=opensips
>>> S_MEMORY=1024
>>> P_MEMORY=32
>>> DUMP_CORE=yes
>>>
>>> And here are the relevant lines in /etc/init.d/opensips:
>>> if test "$DUMP_CORE" = "yes" ; then
>>> # set proper ulimit
>>> ulimit -c unlimited
>>>
>>> # directory for the core dump files
>>> COREDIR=/home/corefiles
>>> [ -d $COREDIR ] || mkdir $COREDIR
>>> chmod 777 $COREDIR
>>> echo "$COREDIR/core.%e.sig%s.%p" > /proc/sys/kernel/core_pattern
>>> fi
>>>
>>> Thanks,
>>> Seth
>>> On 2/6/2013 12:27 PM, Bogdan-Andrei Iancu wrote:
>>>> yes, you have to, if using the debian init.d files.
>>>>
>>>> The core will be dumped into the opensips working directory - if
>>>> not configured one, it will be in root file system "/".
>>>>
>>>> Regards,
>>>>
>>>> Bogdan-Andrei Iancu
>>>> OpenSIPS Founder and Developer
>>>> http://www.opensips-solutions.com
>>>>
>>>>
>>>> On 02/06/2013 07:02 PM, Seth Schultz wrote:
>>>>> I assume I also need to set the following variable in
>>>>> /etc/default/opensips.
>>>>>
>>>>> change
>>>>> DUMP_CORE=no
>>>>> to
>>>>> DUMP_CORE=yes
>>>>>
>>>>> This may be a silly question, but where will it dump the core file
>>>>> to?
>>>>>
>>>>> Thanks,
>>>>> Seth
>>>>>
>>>>>
>>>>> On 2/6/2013 11:56 AM, Bogdan-Andrei Iancu wrote:
>>>>>> Where you changed the LM_ERR() in code, after it, simply put :
>>>>>> abort(); . Recompile and run again -> when you get the error,
>>>>>> opensips should stop by itself with a core dump. Once you get the
>>>>>> core file, use gdb to get a backtrace (run "bt" on gdb).
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Bogdan-Andrei Iancu
>>>>>> OpenSIPS Founder and Developer
>>>>>> http://www.opensips-solutions.com
>>>>>>
>>>>>>
>>>>>> On 02/06/2013 06:23 PM, Seth Schultz wrote:
>>>>>>> Bogdan-Andrei,
>>>>>>>
>>>>>>> Thank you for the reply. I have seen this error occur in both
>>>>>>> 1.8.2 and 1.9.0. Would you please explain exactly how I can
>>>>>>> catch the error and call abort()? Also, is there anything else I
>>>>>>> can enable which would help us track down the cause?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Seth
>>>>>>>
>>>>>>> Seth Schultz
>>>>>>> E-Mail: sschultz at scholarchip.com
>>>>>>> Phone: 212.255.8005 x 124
>>>>>>> Fax: 212.255.8091
>>>>>>>
>>>>>>> On 2/6/2013 11:09 AM, Bogdan-Andrei Iancu wrote:
>>>>>>>> Hi Seth,
>>>>>>>>
>>>>>>>> That is really strange - using 1.9 or 1.8 ?
>>>>>>>>
>>>>>>>> Do make it short, could you put an "abort()" when the error is
>>>>>>>> triggered ? -> we could look into backtrace to see where the
>>>>>>>> faulty string comes from.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>>
>>>>>>>> Bogdan-Andrei Iancu
>>>>>>>> OpenSIPS Founder and Developer
>>>>>>>> http://www.opensips-solutions.com
>>>>>>>>
>>>>>>>>
>>>>>>>> On 02/06/2013 01:09 AM, Seth Schultz wrote:
>>>>>>>>> Hello,
>>>>>>>>>
>>>>>>>>> Could someone please help me resolve the following error
>>>>>>>>> message "ERROR:siptrace:pipport2su: no port specified". This
>>>>>>>>> error only happens on some of the sip packets, but I can't
>>>>>>>>> determine why. When this error occurs, the trace for those
>>>>>>>>> packets never make it to my homer capture server (I have to
>>>>>>>>> use ngrep to see them).
>>>>>>>>>
>>>>>>>>> Furthermore, I modified siptrace.c to output the value of
>>>>>>>>> pipport and the error message returned this
>>>>>>>>> "ERROR:siptrace:pipport2su: udp:172.16.1.115 no port
>>>>>>>>> specified". I am just not sure where it is getting the
>>>>>>>>> udp:172.16.1.115 value from.
>>>>>>>>>
>>>>>>>>> Thanks in advance,
>>>>>>>>> Seth
>>>>>>>>>
>>>>>>>>> Here are my module parameters.
>>>>>>>>> ...
>>>>>>>>> port=5060
>>>>>>>>> listen=udp:172.16.1.115:5060
>>>>>>>>> ...
>>>>>>>>>
>>>>>>>>> loadmodule "siptrace.so"
>>>>>>>>> modparam("siptrace", "enable_ack_trace", 1)
>>>>>>>>> modparam("siptrace", "trace_flag", "TRACE")
>>>>>>>>> modparam("siptrace", "trace_on", 1)
>>>>>>>>> modparam("siptrace", "trace_to_database", 0)
>>>>>>>>> modparam("siptrace", "traced_user_avp", "$avp(called)")
>>>>>>>>> modparam("siptrace", "hep_version", 2)
>>>>>>>>> modparam("siptrace", "hep_capture_id", 338)
>>>>>>>>> modparam("siptrace", "duplicate_uri", "sip:172.16.1.99:9060")
>>>>>>>>> modparam("siptrace", "duplicate_with_hep", 1)
>>>>>>>>>
>>>>>>>>> and here is how I am using siptrace in my script.
>>>>>>>>>
>>>>>>>>> route{
>>>>>>>>> ...
>>>>>>>>> setflag(TRACE);
>>>>>>>>> sip_trace();
>>>>>>>>> ...
>>>>>>>>>
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> Below is a capture (from ngrep) of a packet that through this
>>>>>>>>> error.
>>>>>>>>>
>>>>>>>>> INVITE sip:12063430011 at yyy.yyy.yyy.yyy SIP/2.0.
>>>>>>>>> Record-Route:
>>>>>>>>> <sip:xxx.xxx.xxx.xxx;lr;ftag=0m15597FNecKe;schip=d4c.b840a454>.
>>>>>>>>> Via: SIP/2.0/UDP
>>>>>>>>> xxx.xxx.xxx.xxx:5060;branch=z9hG4bK0533.69b5df66.1.
>>>>>>>>> Via: SIP/2.0/UDP
>>>>>>>>> 172.16.1.105;received=172.16.1.105;rport=5060;branch=z9hG4bK3Uj0ye5ve15SK.
>>>>>>>>> Max-Forwards: 69.
>>>>>>>>> From: "Unknown" <sip:9999999999 at 172.16.1.115>;tag=0m15597FNecKe.
>>>>>>>>> To: <sip:19999999999 at 172.16.1.115>.
>>>>>>>>> Call-ID: a7747eca-ea88-1230-e489-57cd493474a3.
>>>>>>>>> CSeq: 39716116 INVITE.
>>>>>>>>> Contact:
>>>>>>>>> <sip:gw+opensips at 172.16.1.105:5060;transport=udp;gw=opensips>.
>>>>>>>>> User-Agent: FS.
>>>>>>>>> Allow: INVITE, ACK, BYE, CANCEL, OPTIONS, MESSAGE, UPDATE,
>>>>>>>>> INFO, REGISTER, REFER, NOTIFY.
>>>>>>>>> Supported: timer, precondition, path, replaces.
>>>>>>>>> Allow-Events: talk, hold, refer.
>>>>>>>>> Session-Expires: 120;refresher=uac.
>>>>>>>>> Min-SE: 120.
>>>>>>>>> Content-Type: application/sdp.
>>>>>>>>> Content-Disposition: session.
>>>>>>>>> Content-Length: 247.
>>>>>>>>> P-Call-Type: Notification.
>>>>>>>>> X-FS-Support: update_display,send_info.
>>>>>>>>> Remote-Party-ID: "Unknown"
>>>>>>>>> <sip:9999999999 at 172.16.1.115>;party=calling;screen=yes;privacy=off.
>>>>>>>>> .
>>>>>>>>> v=0.
>>>>>>>>> o=FreeSWITCH 1360080758 1360080759 IN IP4 zzz.zzz.zzz.zzz.
>>>>>>>>> s=FreeSWITCH.
>>>>>>>>> c=IN IP4 zzz.zzz.zzz.zzz.
>>>>>>>>> t=0 0.
>>>>>>>>> m=audio 29516 RTP/AVP 0 8 3 101.
>>>>>>>>> a=rtpmap:101 telephone-event/8000.
>>>>>>>>> a=fmtp:101 0-16.
>>>>>>>>> a=silenceSupp:off - - - -.
>>>>>>>>> a=ptime:20.
>>>>>>>>> a=schipmangled:yes.
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Users mailing list
>>>>>>>>> Users at lists.opensips.org
>>>>>>>>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>>>>>>>>>
>>>>>>>
>>>>>
>>>
>
More information about the Users
mailing list