[OpenSIPS-Users] opensips Crashing v2.2.2
Richard Robson
rrobson at greenlightcrm.com
Mon Jan 16 07:57:44 EST 2017
Hi,
I've done some more testing and the carrier is now responding with a
shorter message, despite them saying nothing was wrong. and the crashing
isn't happening.
Therefore I'm unable to reproduce the original error on our live system.
I've put the node back into service and I'm monitoring the situation.
Regards,
Richard
On 06/01/2017 12:19, Richard Robson wrote:
> I've some more back traces.
>
> this could be a resource problem. The back story is we want to check
> for channel limits and Rate limits for all calls. Channel limit hits
> should failover to the next gateway and rate limits should delat for
> up to 2 seconds before failing over to the next gateway. This is all
> working ok as far as I can tell. We have one provider that has 600
> channels and cps of 25. We now have a second provider with 3 nodes of
> 100 channels and 4 cps. All nodes are being checked for Channels and
> CPS. I noticed that the failover of GW was not being checked and added
> new route to do the checking. Also to confuse things more it looks
> like one of the new carriers nodes may be incorrectly configured asnd
> we are getting failures (503 Service unavailable).
> Yesterday we were using all four gateways and no crashes, but after
> adding in the new route for the failover checking it started to crash
> this morning. the logs indicate that the failover is working correctly.
> We are checking the outbound calls twice once for the customer limits
> and then for the carrier limits. This has been fine for the last few
> days, since the system was updated ( it was only carrier limits
> checked before)
>
> i've removed the suspect node and the extra route, but I notice the
> error from the node in the backtraces with an address out of bounds
>
> I've more back traces if you require
>
> http://pastebin.com/BB9RP6rv
> http://pastebin.com/QSTbYgNQ
> http://pastebin.com/vUmQLT0k
>
> On 06/01/2017 09:30, Richard Robson wrote:
>> Hi,
>>
>> we are getting a crash in opensips. V2.2.2
>>
>>
>> bt all:
>>
>>
>> #0 0x0000000000425de2 in context_put_int (type=CONTEXT_GLOBAL,
>> ctx=0x0, pos=4, data=1) at context.c:173
>> No locals.
>> #1 0x00007f1c121642cc in dlg_onreply (t=0x7f1c1d86e938,
>> type=<optimized out>, param=<optimized out>) at dlg_handlers.c:490
>> rpl = 0x7f1c3be9f3b8
>> req = 0x7f1c1cf97070
>> dlg = 0x7f1c20b39040
>> new_state = 0
>> old_state = 472847928
>> unref = 495381272
>> event = 4
>> mangled_from = {s = 0x0, len = 0}
>> mangled_to = {s = 0x0, len = 0}
>> req_out_buff = <optimized out>
>> #2 0x00007f1c1b5a4069 in run_trans_callbacks (type=type at entry=64,
>> trans=trans at entry=0x7f1c1d86e938, req=req at entry=0x7f1c1cf97070,
>> rpl=<optimized out>, code=<optimized out>) at t_hooks.c:209
>> params = {req = 0x7f1c1cf97070, rpl = 0x7f1c3be9f3b8, code =
>> 500, param = 0x7f1c1c6110c0, extra1 = 0x7ffc05c6ebe0, extra2 =
>> 0x7f1c1d86ea50}
>> cbp = 0x7f1c1c6110b0
>> backup = 0x845628 <global_avps>
>> trans_backup = 0x7f1c1d86e938
>> __FUNCTION__ = "run_trans_callbacks"
>> #3 0x00007f1c1b5a43c2 in run_trans_callbacks_locked
>> (type=type at entry=64, trans=trans at entry=0x7f1c1d86e938,
>> req=0x7f1c1cf97070, rpl=rpl at entry=0x7f1c3be9f3b8,
>> code=code at entry=500) at t_hooks.c:262
>> No locals.
>> #4 0x00007f1c1b55bf9c in relay_reply (t=<optimized out>,
>> p_msg=<optimized out>, branch=<optimized out>, msg_status=<optimized
>> out>, cancel_bitmap=<optimized out>) at t_reply.c:1244
>> relay = <optimized out>
>> save_clone = <optimized out>
>> buf = 0x7f1c3be954b0 "SIP/2.0 500 Service Unavailable-No
>> Bandwidth Available\r\nVia: SIP/2.0/UDP
>> 192.168.36.12:5060;received=192.168.36.12;rport=5060;branch=z9hG4bK32b724a4\r\nTo:
>> <sip:07724149184 at 192.168.36.82>;tag=3692683275"...
>> res_len = 484
>> relayed_code = 500
>> relayed_msg = <optimized out>
>> bm = {to_tag_val = {s = 0x1 <Address 0x1 out of bounds>, len
>> = 8897760}}
>> totag_retr = 0
>> uas_rb = <optimized out>
>> cb_s = {
>> s = 0x7f1c3be954b0 "SIP/2.0 500 Service Unavailable-No
>> Bandwidth Available\r\nVia: SIP/2.0/UDP
>> 192.168.36.12:5060;received=192.168.36.12;rport=5060;branch=z9hG4bK32b724a4\r\nTo:
>> <sip:07724149184 at 192.168.36.82>;tag=3692683275"..., len = 484}
>> text = {s = 0x1 <Address 0x1 out of bounds>, len = 1002627336}
>> __FUNCTION__ = "relay_reply"
>> #5 0x00007f1c1b562406 in reply_received (p_msg=0x7f1c3be9f3b8) at
>> t_reply.c:1505
>> msg_status = 500
>> last_uac_status = 100
>> branch = 0
>> reply_status = <optimized out>
>> timer = 139758694817869
>> cancel_bitmap = 0
>> uac = 0x7f1c1d86eb10
>> t = 0x7f1c1d86e938
>> backup_list = <optimized out>
>> has_reply_route = <optimized out>
>> __FUNCTION__ = "reply_received"
>> #6 0x0000000000458fed in forward_reply
>> (msg=msg at entry=0x7f1c3be9f3b8) at forward.c:495
>> new_buf = 0x0
>> to = 0x0
>> new_len = 0
>> mod = 0x7f1c3bbf6e90
>> proto = <optimized out>
>> id = 0
>> send_sock = <optimized out>
>> s = <optimized out>
>> len = <optimized out>
>> __FUNCTION__ = "forward_reply"
>> #7 0x0000000000439126 in receive_msg (
>> buf=0x858c00 <buf.8277> "SIP/2.0 503 Service Unavailable-No
>> Bandwidth Available\r\nVia: SIP/2.0/UDP
>> 141.170.9.157:5060;branch=z9hG4bK817f.88c248e3.0\r\nTo:
>> <sip:+447724149184 at 109.159.137.74>;tag=3692683275-7994\r\nFrom:
>> <sip:016185"..., len=<optimized out>,
>> rcv_info=rcv_info at entry=0x7ffc05c6ee60,
>> existing_context=existing_context at entry=0x0) at receive.c:257
>> ctx = 0x7f1c3be8f3d0
>> msg = 0x7f1c3be9f3b8
>> start = {tv_sec = 50509, tv_usec = 139758636819258}
>> rc = 3
>> in_buff = {
>> s = 0x7f1c3be92b40 "SIP/2.0 500 Service Unavailable-No
>> Bandwidth Available\r\nVia: SIP/2.0/UDP
>> 141.170.9.157:5060;branch=z9hG4bK817f.88c248e3.0\r\nTo:
>> <sip:+447724149184 at 109.159.137.74>;tag=3692683275-7994\r\nFrom:
>> <sip:016185"..., len = 444}
>> __FUNCTION__ = "receive_msg"
>> #8 0x00000000005b0b13 in udp_read_req (si=<optimized out>,
>> bytes_read=<optimized out>) at net/proto_udp/proto_udp.c:192
>> ri = {src_ip = {af = 2, len = 4, u = {addrl =
>> {139759486345069, 0}, addr32 = {1250533229, 32540, 0, 0}, addr16 =
>> {40813, 19081, 32540, 0, 0, 0, 0, 0}, addr =
>> "m\237\211J\034\177\000\000\000\000\000\000\000\000\000"}}, dst_ip = {
>> af = 2, len = 4, u = {addrl = {1378134208, 0}, addr32 =
>> {1378134208, 0, 0, 0}, addr16 = {43200, 21028, 0, 0, 0, 0, 0, 0},
>> addr = "\300\250$R", '\000' <repeats 11 times>}}, src_port = 5060,
>> dst_port = 5060, proto = 1,
>> proto_reserved1 = 0, proto_reserved2 = 0, src_su = {s =
>> {sa_family = 2, sa_data =
>> "\023\304m\237\211J\000\000\000\000\000\000\000"}, sin = {sin_family
>> = 2, sin_port = 50195, sin_addr = {s_addr = 1250533229},
>> sin_zero = "\000\000\000\000\000\000\000"}, sin6 =
>> {sin6_family = 2, sin6_port = 50195, sin6_flowinfo = 1250533229,
>> sin6_addr = {__in6_u = {__u6_addr8 =
>> "\000\000\000\000\000\000\000\000\001\000\000\000\000\000\000",
>> __u6_addr16 = {0, 0, 0, 0, 1, 0, 0, 0}, __u6_addr32
>> = {0, 0, 1, 0}}}, sin6_scope_id = 4851503}}, bind_address =
>> 0x7f1c3bbf6720}
>> len = <optimized out>
>> buf = "SIP/2.0 503 Service Unavailable-No Bandwidth
>> Available\r\nVia: SIP/2.0/UDP
>> 141.170.9.157:5060;branch=z9hG4bK817f.88c248e3.0\r\nTo:
>> <sip:+447724149184 at 109.159.137.74>;tag=3692683275-7994\r\nFrom:
>> <sip:016185"...
>> fromlen = 16
>> p = <optimized out>
>> msg = {
>> s = 0x858c00 <buf.8277> "SIP/2.0 503 Service Unavailable-No
>> Bandwidth Available\r\nVia: SIP/2.0/UDP
>> 141.170.9.157:5060;branch=z9hG4bK817f.88c248e3.0\r\nTo:
>> <sip:+447724149184 at 109.159.137.74>;tag=3692683275-7994\r\nFrom:
>> <sip:016185"..., len = 467}
>> __FUNCTION__ = "udp_read_req"
>> #9 0x000000000059a420 in handle_io (idx=<optimized out>,
>> event_type=<optimized out>, fm=<optimized out>) at net/net_udp.c:259
>> read = 1023068896
>> fm = <optimized out>
>> #10 io_wait_loop_epoll (h=<optimized out>, t=<optimized out>,
>> repeat=<optimized out>) at net/../io_wait_loop.h:225
>> ret = <optimized out>
>> e = <optimized out>
>> n = 1
>> r = 0
>> #11 udp_rcv_loop (si=si at entry=0x7f1c3bbf6720) at net/net_udp.c:308
>> __FUNCTION__ = "udp_rcv_loop"
>> #12 0x000000000059b948 in udp_start_processes
>> (chd_rank=chd_rank at entry=0x8457a0 <chd_rank.11028>,
>> startup_done=startup_done at entry=0x0) at net/net_udp.c:372
>> si = 0x7f1c3bbf6720
>> load_p = 0x7f1c1bee92e8
>> pid = <optimized out>
>> i = <optimized out>
>> __FUNCTION__ = "udp_start_processes"
>> #13 0x0000000000419f50 in main_loop () at main.c:671
>> startup_done = 0x0
>> chd_rank = 3
>> #14 main (argc=<optimized out>, argv=<optimized out>) at main.c:1261
>> cfg_stream = <optimized out>
>> c = <optimized out>
>> r = <optimized out>
>> tmp = 0x7ffc05c70f66 ""
>> tmp_len = <optimized out>
>> port = <optimized out>
>> proto = <optimized out>
>> protos_no = <optimized out>
>> options = 0x5df1d0 "f:cCm:M:b:l:n:N:rRvdDFETSVhw:t:u:g:P:G:W:o:"
>> ret = -1
>> seed = 1763847833
>> rfd = <optimized out>
>> __FUNCTION__ = "main"
>>
>
>
--
Richard Robson
Greenlight Support
01382 843843
support at greenlightcrm.com
More information about the Users
mailing list