[OpenSIPS-Users] 1.6 rev 6147 and dp_translate memory error?

Bogdan-Andrei Iancu bogdan at voice-system.ro
Thu Sep 24 14:06:54 CEST 2009


Hi Ron,

Are you sure the DB content is what you used during the crash? The log says:

"string 4085557777 matched the match_exp ^([2-9][0-9]{2}[2-9][0-9]{6}) 
but not the subst_exp ^([2-9][0-9]{2}[2-9][0-9]{6})(.+)!" ,

But I see no record with match_exp="([2-9][0-9]{2}[2-9][0-9]{6})" and 
subst_exp = "^([2-9][0-9]{2}[2-9][0-9]{6})(.+)" in the DB dump you posted !

Also, could you also print (from dgb) the match_nb and subst_comp and  
subst_comp->_matches[match_nb]  vars?

To speed up this, will it be possible to grant me access to the machine 
where the core is ? only the core file is useless for me as I need the 
binaries, the modules, and some libraries from your server :(...

Regards,
Bogdan

Ron McCarthy wrote:
> I ran one more test and this is the final results:
>
> Sep 22 08:15:13 sips /usr/local/sbin/opensips[52643]: 
> ERROR:dialplan:rule_translate: the string 4085557777 matched the 
> match_exp ^([2-9][0-9]{2}[2-9][0-9]{6}) but not the subst_exp 
> ^([2-9][0-9]{2}[2-9][0-9]{6})(.+)!
> Sep 22 08:15:13 sips /usr/local/sbin/opensips[52643]: 
> ERROR:dialplan:translate: could not build the output
> Sep 22 08:15:13 sips /usr/local/sbin/opensips[52643]: 
> ERROR:core:do_assign: no value in right expression
> Sep 22 08:15:13 sips /usr/local/sbin/opensips[52643]: 
> ERROR:core:do_assign: error at line: 432
>
> Line 432 is: $rU = $avp(s:carto_did);
>
> Then we get:
>
> Sep 22 08:17:48 sips /usr/local/sbin/opensips[52643]: 
> ERROR:core:do_assign: error at line: 299
> Sep 22 08:17:48 sips /usr/local/sbin/opensips[52643]: 
> ERROR:dialplan:dp_get_svalue: no PV or NULL or non-STR val found 
> (error in scripts)
> Sep 22 08:17:48 sips /usr/local/sbin/opensips[52643]: 
> ERROR:dialplan:dp_translate_f: invalid param 2
>
> Line 299: $avp(s:aninpa)=$(avp(s:newfrom_did){s.substr,0,3});
>
> At this point the switch is still taking on new calls, we get those 
> errors some more times, then we core dump
>
> Here is the gdb output:
> (gdb) print result
> $1 = (str *) 0x7fffffffe170
> (gdb) print result->len
> $2 = 0
> (gdb) print result->s
> $3 = 0x801fe3f00 "5679990000"
> (gdb) print match
> $4 = {begin = 0x802f7e0a8 "5679990000", len = -43123475}
>
> Core file is 2GB but you can download via HTTP if that will help.
>
> Thanks and let me know what ever else I can do.
>
>
>
>
> Dialplan rules are:
>
> dpid   pr     match_op     match_exp     match_len     subst_exp     
> repl_exp
> 200     40     1     ^([2-9][0-9]{2}[2-9][0-9]{6})     0     
> ^([2-9][0-9]{2}[2-9][0-9]{6})     \1          
> 200     50     1     ^1([2-9][0-9]{2}[2-9][0-9]{6})     0     
> ^1([2-9][0-9]{2}[2-9][0-9]{6})(.+)     \1        
>
> On Tue, Sep 22, 2009 at 7:51 AM, Bogdan-Andrei Iancu 
> <bogdan at voice-system.ro <mailto:bogdan at voice-system.ro>> wrote:
>
>     Hi Ron,
>
>     there couple of options:
>        1) we try remove debugging
>        2) give me access to the core file so that I can investigate it
>        3) send the the DB content and the input for the dp_translate() to
>     try to reproduce it
>
>     so, let's try 1)
>      please print in gdb the following values:  result , result->len,
>     result->s, match.
>
>     Thanks and regards,
>     Bogdan
>
>     Ron McCarthy wrote:
>     > Hi,
>     >
>     > Just updated and ran a test and now it's core dumping with no errors
>     > in syslog, a backtrace shows:
>     >
>     > #0  0x0000000801ed9e41 in rule_translate (msg=0x775c88, string={s =
>     > 0x8029e4e99 "6025504004", len = 10}, rule=Variable "rule" is not
>     > available.
>     > ) at dp_repl.c:192
>     > #1  0x0000000801edb124 in translate (msg=0x775c88, input={s =
>     > 0x8029e4e99 "6025504004", len = 10}, output=0x7fffffffd570,
>     > idp=Variable "idp" is not available.
>     > ) at dp_repl.c:346
>     > #2  0x0000000801ed2472 in dp_translate_f (msg=0x775c88,
>     str1=Variable
>     > "str1" is not available.
>     > ) at dialplan.c:368
>     > #3  0x000000000040dbb1 in do_action (a=0x6dfb90, msg=0x775c88) at
>     > action.c:962
>     > #4  0x000000000040c6a7 in run_action_list (a=Variable "a" is not
>     > available.
>     > ) at action.c:139
>     > #5  0x000000000040fb8e in do_action (a=0x6e3d10, msg=0x775c88) at
>     > action.c:706
>     > #6  0x000000000040c6a7 in run_action_list (a=Variable "a" is not
>     > available.
>     > ) at action.c:139
>     > #7  0x000000000040eebe in do_action (a=0x6d4d08, msg=0x775c88) at
>     > action.c:119
>     > #8  0x000000000040c6a7 in run_action_list (a=Variable "a" is not
>     > available.
>     > ) at action.c:139
>     > #9  0x000000000040fb8e in do_action (a=0x6d4f48, msg=0x775c88) at
>     > action.c:706
>     > #10 0x000000000040c6a7 in run_action_list (a=Variable "a" is not
>     > available.
>     > ) at action.c:139
>     > #11 0x000000000040fb8e in do_action (a=0x6d6f90, msg=0x775c88) at
>     > action.c:706
>     > #12 0x000000000040c6a7 in run_action_list (a=Variable "a" is not
>     > available.
>     > ) at action.c:139
>     > #13 0x0000000000410d29 in run_top_route (a=0x6bd588,
>     msg=0x775c88) at
>     > action.c:119
>     > #14 0x0000000000455d6b in receive_msg (
>     >     buf=0x65dd80 "INVITE sip:1234568888 at 192.168.1.100:5060
>     <http://sip:1234568888@192.168.1.100:5060>
>     > <http://sip:1234568888@192.168.1.100:5060> SIP/2.0\r\nVia:
>     SIP/2.0/UDP
>     > 192.168.1.100:5061;branch=z9hG4bK-33820-611-0\r\nFrom: sipp
>     > <sip:1234569999 at 192.168.1.100:5060
>     <http://sip:1234569999@192.168.1.100:5060>
>     >
>     <http://sip:1234569999@192.168.1.100:5060>>;tag=33820SIPpTag00611\r\nTo:
>     > sut <sip:12345"..., len=555, rcv_info=0x7fffffffea70) at
>     receive.c:162
>     > #15 0x0000000000499f3e in udp_rcv_loop () at udp_server.c:490
>     > #16 0x0000000000428a2a in main (argc=7, argv=Variable "argv" is not
>     > available.
>     > ) at main.c:818
>     >
>     > That was about 800 calls into it at 10 CPS.
>     >
>     > What else can I try?
>     >
>     > Thanks
>     >
>     >
>     > On Tue, Sep 22, 2009 at 5:26 AM, Bogdan-Andrei Iancu
>     > <bogdan at voice-system.ro <mailto:bogdan at voice-system.ro>
>     <mailto:bogdan at voice-system.ro <mailto:bogdan at voice-system.ro>>>
>     wrote:
>     >
>     >     Hi Ron,
>     >
>     >     If you update from SVN trunk, this should be fixed. Please
>     let me know
>     >     if it works for you or not.
>     >
>     >     Regards,
>     >     Bogdan
>     >
>     >
>     >     Ron McCarthy wrote:
>     >     > Hi,
>     >     >
>     >     >  dp_translate("200", "$avp(s:from_did)/$avp(s:newfrom_did)");
>     >     >
>     >     > We call that after the INVITE and allow_trusted, etc. It gets
>     >     called 4
>     >     > times total, twice to normalize the $fU and $fU vars then
>     two more
>     >     > times to change the values if needed, (adds a 1, +1, etc).
>     >     >
>     >     > Thanks
>     >     >
>     >     >
>     >     > On Fri, Sep 18, 2009 at 1:24 AM, Bogdan-Andrei Iancu
>     >     > <bogdan at voice-system.ro <mailto:bogdan at voice-system.ro>
>     <mailto:bogdan at voice-system.ro <mailto:bogdan at voice-system.ro>>
>     >     <mailto:bogdan at voice-system.ro
>     <mailto:bogdan at voice-system.ro> <mailto:bogdan at voice-system.ro
>     <mailto:bogdan at voice-system.ro>>>>
>     >     wrote:
>     >     >
>     >     >     Hi Ron,
>     >     >
>     >     >     Hoe do you call the dp_translate function from the script?
>     >     >
>     >     >     Regards,
>     >     >     Bogdan
>     >     >
>     >     >     Ron McCarthy wrote:
>     >     >     > Hi list,
>     >     >     >
>     >     >     > Ive done quite a bit of troubleshooting and ive
>     found the
>     >     switch
>     >     >     runs
>     >     >     > clean with not using dp_translate, but when we do the
>     >     errors appear.
>     >     >     >
>     >     >     > After a few thousand calls we start getting: (no errors
>     >     before this)
>     >     >     >
>     >     >     > Sep 18 00:09:13 sips /usr/local/sbin/opensips[68260]:
>     >     >     > ERROR:dialplan:dp_get_svalue: no AVP or SCRIPTVAR
>     found (error
>     >     >     in scripts)
>     >     >     > Sep 18 00:09:13 sips /usr/local/sbin/opensips[68260]:
>     >     >     > ERROR:dialplan:dp_translate_f: invalid param 2
>     >     >     > Sep 18 00:09:13 sips /usr/local/sbin/opensips[68260]:
>     >     >     > ERROR:core:do_assign: no value in right expression
>     >     >     > Sep 18 00:09:13 sips /usr/local/sbin/opensips[68260]:
>     >     >     > ERROR:core:do_assign: error at line: 298
>     >     >     >
>     >     >     > Backtrace shows:
>     >     >     > #0  0x0000000801ff0211 in rule_translate (msg=0x6fe600,
>     >     string={s =
>     >     >     > 0x80282a9c3 "1234569999", len = 10}, rule=Variable
>     "rule"
>     >     is not
>     >     >     > available.
>     >     >     > ) at dp_repl.c:192
>     >     >     > 192 memcpy(result->s + result->len, match.begin,
>     match.len);
>     >     >     > (gdb)
>     >     >     >
>     >     >     > Were using sipP to test this, im setting the source and
>     >     dest number
>     >     >     > manually with a AVP var then having dp_translate run on
>     >     it, its
>     >     >     taking
>     >     >     > a 10 digit number and turning it into 11 digits, we have
>     >     about 45
>     >     >     > rules loaded into the database for the dialplan,
>     with this
>     >     >     particular
>     >     >     > dialplan ID their is 2 rules total, we call
>     dp_translate a
>     >     total
>     >     >     of 4
>     >     >     > times for each new call.
>     >     >     >
>     >     >     > vmstat is basically all 0's when dp_translate disabled,
>     >     when enabled
>     >     >     > it looks like:
>     >     >     >
>     >     >     > 0 9 0   2891M  2574M  1484   0   0   0  3737   0   0   0
>     >     2744 29807
>     >     >     > 11711 13 15 72
>     >     >     >  1 7 0   2899M  2569M  1493   0   0   0  1983   0  
>     0   0
>     >     2678 39221
>     >     >     > 11355 13 11 76
>     >     >     >  0 8 0   2891M  2568M  1119   0   0   0  2821   0  
>     0   0
>     >     2360 28331
>     >     >     > 10401 13 15 72
>     >     >     >  0 8 0   2901M  2565M  1477   0   0   0  2086   0  
>     0   0
>     >     2226 39722
>     >     >     > 9430 11 15 74
>     >     >     >  1 8 0   2893M  2560M  1250   0   0   0  1993   0  
>     0   0
>     >     2912 23983
>     >     >     > 12123 11 15 74
>     >     >     >  4 6 0   2901M  2551M  1557   0   0   0  2035   0  
>     0   0
>     >     3075 38446
>     >     >     > 13035 12 18 70
>     >     >     >  0 9 0   2893M  2548M  1103   0   0   0  1877   0  
>     0   0
>     >     2772 26050
>     >     >     > 11474 12 12 76
>     >     >     >  0 8 0   2901M  2539M  1434   0   0   0   743   0  
>     0   0
>     >     3289 34833
>     >     >     > 13759  8 17 75
>     >     >     >  0 9 0   2893M  2534M   943   0   0   0  1533   0  
>     0   0
>     >     3372 23843
>     >     >     > 14379  8 24 68
>     >     >     >  2 7 0   2901M  2528M  1252   0   0   0  1207   0  
>     0   0
>     >     2762 39615
>     >     >     > 11275 12 13 75
>     >     >     >  0 8 0   2902M  2521M  1134   0   0   0   703   0  
>     0   0
>     >     3364 18464
>     >     >     > 14069  6 18 76
>     >     >     >  0 8 0   2901M  2514M  1670   0   0   0  1737   0  
>     0   0
>     >     3771 17832
>     >     >     > 17211  1 16 82
>     >     >     >  0 8 0   2902M  2508M  1212   0   0   0   803   0  
>     0   0
>     >     3141 5263
>     >     >     > 13990  1 14 85
>     >     >     >  0 8 0   2901M  2499M  1542   0   0   0  1241   0  
>     0   0
>     >     3720 17120
>     >     >     > 16641  1 17 82
>     >     >     >  0 7 0   2902M  2497M  1260   0   0   0  2027   0  
>     0   0
>     >     2561 6328
>     >     >     > 11863  1 14 85
>     >     >     >  0 7 0   2901M  2499M  1979   0   0   0  3653   0  
>     0   0
>     >     2442 19121
>     >     >     > 11724  3 13 85
>     >     >     >  1 8 0   2902M  2498M  1387   0   0   0  3062   0  
>     0   0
>     >     2183 6172
>     >     >     > 10662  0 13 87
>     >     >     >
>     >     >     >
>     >     >     > We have ran this at 5CPS and the switch will run
>     fine for
>     >     several
>     >     >     > thousand calls, then at 60+ CPS and runs for several
>     thousand
>     >     >     calls as
>     >     >     > well, so it appears to be a memory issue to me as
>     when the
>     >     total
>     >     >     > number of processed calls goes up is when it dies on us.
>     >     >     >
>     >     >     > Let me know what else I can do to test/debug on my
>     side to
>     >     help
>     >     >     with this.
>     >     >     >
>     >     >     > Thanks
>     >     >     >
>     >     >
>




More information about the Users mailing list