[OpenSIPS-Users] dialog bye_on_timeout and other issues

Alex Massover alex at jajah.com
Wed Jun 23 10:27:24 CEST 2010 

Hi Bogdan,

I was able to reproduce the corrupted BYE problem even with children=1. With single working child the problem happens only when OpenSIPS is under massive stress/at maximum performance level (with blocking auth 20 cps is enough to push OpenSIPS into a corner).

With children>=1 the problem is sometimes reproducible even with single call.

> -----Original Message-----
> From: users-bounces at lists.opensips.org [mailto:users-
> bounces at lists.opensips.org] On Behalf Of Alex Massover
> Sent: יום ד 23 יוני 2010 11:09
> To: OpenSIPS users mailling list
> Subject: Re: [OpenSIPS-Users] dialog bye_on_timeout and other issues
> 
> Hi Bogdan,
> 
> Have you received the trace and the log I sent?
> 
> Also one more observation. The problem is not reproducible with
> children=1 but is reproducible with children>=1.
> 
> I noticed long time ago that children>=1 misbehaves:
> 
> a) performance is worse instead of getting better
> b) it can hang
> c) with big number of children it hangs/deadlocks very easily
> 
> I might be mistaken but it looks for me that something is wrong either
> with interprocessing or with locks or with both of them.
> 
> A corrupted BYE is just a symptom of bigger problem.
> 
> For example in this scenario 180 is processed within one child and 200
> within another child, there's race condition/locks misbehave/something
> wrong with memory, and as a result something gets corrupted and wrong
> BYE is sent.
> 
> > -----Original Message-----
> > From: users-bounces at lists.opensips.org [mailto:users-
> > bounces at lists.opensips.org] On Behalf Of Alex Massover
> > Sent: יום ג 22 יוני 2010 12:04
> > To: OpenSIPS users mailling list
> > Subject: Re: [OpenSIPS-Users] dialog bye_on_timeout and other issues
> >
> > Hi,
> >
> > > You mentioned you generate these calls with SIPP - right ? are all
> of
> > > them the same ?
> > >
> > [Alex] Yes, they all are the same.
> >
> > > looking at signalling, we should see why that calls have the
> > > callee_route_set (some RR done after your proxy)..
> > >
> > [Alex] UAS is also a SIPP script. But it's not the reason for
> OpenSIPS
> > to send corrupted BYE anyway ;)
> >
> > > also, have you checked the patch I sent you?
> >
> > [Alex]
> > Yes, I get no warnings.
> >
> > > Regards,
> > > Bogdan
> > >
> > > Alex Massover wrote:
> > > > Hi Bogdan,
> > > >
> > > > At signaling level 200 OK to INVITE contains RR header (always):
> > > >
> > > > Record-Route: <sip:212.179.159.9:7640>;lr
> > > >
> > > > But at dialog level I have only rare appearance of route set:
> > > >
> > > > 	callee_route_set:: <sip:212.179.159.9:7640>;lr
> > > >
> > > > absolutely most of the dialogs do not have it:
> > > >
> > > > opensipsctl fifo dlg_list | grep route_set gives:
> > > >
> > > > 	caller_route_set::
> > > > 	callee_route_set::
> > > > 	caller_route_set::
> > > > 	callee_route_set::
> > > > 	caller_route_set::
> > > > 	callee_route_set::
> > > > 	caller_route_set::
> > > > 	callee_route_set::
> > > > 	caller_route_set::
> > > > 	callee_route_set::
> > > > 	caller_route_set::
> > > > 	callee_route_set::
> > > > 	caller_route_set::
> > > > 	callee_route_set::
> > > > 	caller_route_set::
> > > > 	callee_route_set::
> > > > 	caller_route_set::
> > > > 	callee_route_set:: <sip:212.179.159.9:7640>;lr
> > > > 	caller_route_set::
> > > > 	callee_route_set::
> > > > 	caller_route_set::
> > > > 	callee_route_set::
> > > >
> > > > And looks that it corresponds with the corrupted BYEs. Most of
> the
> > > BYEs do not have Route headers and they are not corrupted, but some
> > of
> > > them have it and they are corrupted.
> > > >
> > > > And there's no warning after applying the patch.
> > > >
> > > >
> > > >
> > > >> -----Original Message-----
> > > >> From: users-bounces at lists.opensips.org [mailto:users-
> > > >> bounces at lists.opensips.org] On Behalf Of Bogdan-Andrei Iancu
> > > >> Sent: יום ב 21 יוני 2010 15:12
> > > >> To: OpenSIPS users mailling list
> > > >> Subject: Re: [OpenSIPS-Users] dialog bye_on_timeout and other
> > issues
> > > >>
> > > >> Hi Alex,
> > > >>
> > > >> First, about the Route hdr - opensips adds a Route hdr in BYE
> only
> > > if
> > > >> the dialog (on that specific leg) received a 200 OK INVITE with
> RR
> > > >> header - can you confirm this at (1) signalling level and (2) at
> > > dialog
> > > >> level (do a dlg_list via MI).
> > > >>
> > > >> Now, about the bogus BYE - indeed, it is strange - do you use a
> > > local
> > > >> route for accessing the BYEs requests? Attached is a small
> > debugging
> > > >> patch - please apply it a nd see if you get any WARNINGs at
> > runtime.
> > > >>
> > > >> Regards,
> > > >> Bogdan
> > > >>
> > > >> --
> > > >> Bogdan-Andrei Iancu
> > > >> OpenSIPS Bootcamp
> > > >> 20 - 24 September 2010, Frankfurt, Germany www.voice-system.ro
> > > >>
> > > >>
> > > >>
> > > >> Alex Massover wrote:
> > > >>
> > > >>> Hi,
> > > >>>
> > > >>> I have a strange behavior of OpenSIPS 1.6.2. First dialog
> module
> > > >>> _/sometimes/_ sends a wrong bye (generated by dialog module on
> > > >>>
> > > >> timeout):
> > > >>
> > > >>> Here’s a correct one:
> > > >>>
> > > >>> BYE sip:972123456789 at 212.179.159.9:7640;transport=UDP SIP/2.0
> > > >>>
> > > >>> Via: SIP/2.0/UDP 212.179.159.18;branch=z9hG4bKd6c7.7f7a3d36.0
> > > >>>
> > > >>> To: <sip:+496925420990 at 212.179.159.18:5060>;tag=8548
> > > >>>
> > > >>> From: <sip:+972542384166 at 212.179.159.9:5061>;tag=8547
> > > >>>
> > > >>> CSeq: 2 BYE
> > > >>>
> > > >>> Call-ID: 8547-15512 at 212.179.159.9
> > > >>>
> > > >>> Content-Length: 0
> > > >>>
> > > >>> Max-Forwards: 70
> > > >>>
> > > >>> And here’s a wrong one:
> > > >>>
> > > >>> BYE sip:212.179.159.9:7640 SIP/2.0
> > > >>>
> > > >>> Via: SIP/2.0/UDP 212.179.159.18;branch=z9hG4bKc6c7.7ecb1057.0
> > > >>>
> > > >>> To: <sip:+496925420989 at 212.179.159.18:5060>;tag=8547
> > > >>>
> > > >>> From: <sip:+972542384166 at 212.179.159.9:5061>;tag=8546
> > > >>>
> > > >>> CSeq: 2 BYE
> > > >>>
> > > >>> Call-ID: 8546-15512 at 212.179.159.9
> > > >>>
> > > >>> Route: <sip:972123456789 at 212.179.159.9:7640;transport=UDP>
> > > >>>
> > > >>> Content-Length: 0
> > > >>>
> > > >>> Max-Forwards
> > > >>>
> > > >>> In a wrong one there’s Route header inserted (by mistake?) and
> > the
> > > >>> message is cut at Max-Forwards line. It’s missing “:70\r\n”.
> > > >>>
> > > >>> Both of the BYEs above I got just by running test with SIPP.
> This
> > > can
> > > >>> happen even with single call, not related to stress. I.e. one
> > call
> > > it
> > > >>> might send a correct BYE and another call a corrupted BYE,
> > without
> > > >>>
> > > >> any
> > > >>
> > > >>> reason, because calls are exactly the same.
> > > >>>
> > > >>> Another issue is, looks like t_newtran() is unable to handle
> > > >>> retransmissions. In this test UAC and UAS are in the same
> machine
> > > >>> (.9), and you can’t see INVITE from OpenSIPS (.18) to UAS
> because
> > > >>>
> > > >> it’s
> > > >>
> > > >>> fragmented.
> > > >>>
> > > >>> |Time | x.x.x.9 | x.x.x.18 |
> > > >>>
> > > >>> |13.501 | INVITE SDP ( MP4V-ES) |SIP From:
> > > >>> sip:+xxxxxxxxx166 at x.x.x.9:5061
> To:sip:+xxxxxxxxxx82 at x.x.x.18:5060
> > > >>>
> > > >>> | |(5061) ------------------> (5060) |
> > > >>>
> > > >>> |14.003 | INVITE SDP ( MP4V-ES) |SIP From:
> > > >>> sip:+xxxxxxxxx166 at x.x.x.9:5061
> To:sip:+xxxxxxxxxx82 at x.x.x.18:5060
> > > >>>
> > > >>> | |(5061) ------------------> (5060) |
> > > >>>
> > > >>> |15.005 | INVITE SDP ( MP4V-ES) |SIP From:
> > > >>> sip:+xxxxxxxxx166 at x.x.x.9:5061
> To:sip:+xxxxxxxxxx82 at x.x.x.18:5060
> > > >>>
> > > >>> | |(5061) ------------------> (5060) |
> > > >>>
> > > >>> |15.743 | 100 Trying| |SIP Status
> > > >>>
> > > >>> | |(5061) <------------------ (5060) |
> > > >>>
> > > >>> |15.800 | 181 Call is being forwarded |SIP Status
> > > >>>
> > > >>> | |(5061) <------------------ (5060) |
> > > >>>
> > > >>> |15.801 | 100 Trying| |SIP Status
> > > >>>
> > > >>> | |(7640) ------------------> (5060) |
> > > >>>
> > > >>> |15.801 | 180 Ringing |SIP Status
> > > >>>
> > > >>> | |(7640) ------------------> (5060) |
> > > >>>
> > > >>> |15.801 | 200 OK SDP ( G723) |SIP Status
> > > >>>
> > > >>> | |(7640) ------------------> (5060) |
> > > >>>
> > > >>> |15.840 | 181 Call is being forwarded |SIP Status
> > > >>>
> > > >>> | |(5061) <------------------ (5060) |
> > > >>>
> > > >>> |16.041 | 181 Call is being forwarded |SIP Status
> > > >>>
> > > >>> | |(5061) <------------------ (5060) |
> > > >>>
> > > >>> |16.188 | 180 Ringing |SIP Status
> > > >>>
> > > >>> | |(5061) <------------------ (5060) |
> > > >>>
> > > >>> |16.188 | 200 OK SDP ( G723) |SIP Status
> > > >>>
> > > >>> | |(5061) <------------------ (5060) |
> > > >>>
> > > >>> |16.189 | ACK | |SIP Request
> > > >>>
> > > >>> | |(5061) ------------------> (5060) |
> > > >>>
> > > >>> |16.302 | 200 OK SDP ( G723) |SIP Status
> > > >>>
> > > >>> | |(7640) ------------------> (5060) |
> > > >>>
> > > >>> |16.357 | ACK | |SIP Request
> > > >>>
> > > >>> | |(7640) <------------------ (5060) |
> > > >>>
> > > >>> |16.651 | 200 OK SDP ( G723) |SIP Status
> > > >>>
> > > >>> | |(5061) <------------------ (5060) |
> > > >>>
> > > >>> |16.652 | ACK | |SIP Request
> > > >>>
> > > >>> | |(5061) ------------------> (5060) |
> > > >>>
> > > >>> |17.075 | ACK | |SIP Request
> > > >>>
> > > >>> | |(7640) <------------------ (5060) |
> > > >>>
> > > >>> |36.730 | BYE | |SIP Request
> > > >>>
> > > >>> | |(5061) <------------------ (5060) |
> > > >>>
> > > >>> |36.731 | BYE | |SIP Request
> > > >>>
> > > >>> | |(7640) <------------------ (5060) |
> > > >>>
> > > >>> |36.731 | 200 OK | |SIP Status
> > > >>>
> > > >>> | |(5061) ------------------> (5060) |
> > > >>>
> > > >>> |36.731 | 200 OK | |SIP Status
> > > >>>
> > > >>> | |(7640) ------------------> (5060) |
> > > >>>
> > > >>> This issue happens during stress test.
> > > >>>
> > > >>> Any ideas, please? The OpenSIPS 1.6.2 is compiled with system
> > > malloc
> > > >>> and runs over VMware.
> > > >>>
> > > >>> --
> > > >>>
> > > >>> Best Regards,
> > > >>>
> > > >>> Alex Massover
> > > >>>
> > > >>>
> > > >>>
> > > >>> This mail was sent via Mail-SeCure System.
> > > >>> ---------------------------------------------------------------
> --
> > --
> > > --
> > > >>>
> > > >> ---
> > > >>
> > > >>> _______________________________________________
> > > >>> Users mailing list
> > > >>> Users at lists.opensips.org
> > > >>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
> > > >>>
> > > >>>
> > > >> This mail was received via Mail-SeCure System.
> > > >>
> > > >>
> > > >
> > > >
> > > > This mail was sent via Mail-SeCure System.
> > > >
> > > >
> > > >
> > > > _______________________________________________
> > > > Users mailing list
> > > > Users at lists.opensips.org
> > > > http://lists.opensips.org/cgi-bin/mailman/listinfo/users
> > > >
> > >
> > >
> > > --
> > > Bogdan-Andrei Iancu
> > > OpenSIPS Bootcamp
> > > 20 - 24 September 2010, Frankfurt, Germany
> > > www.voice-system.ro
> > >
> > >
> > > _______________________________________________
> > > Users mailing list
> > > Users at lists.opensips.org
> > > http://lists.opensips.org/cgi-bin/mailman/listinfo/users
> > >
> > > This mail was received via Mail-SeCure System.
> >
> > This mail was sent via Mail-SeCure System.
> > _______________________________________________
> > Users mailing list
> > Users at lists.opensips.org
> > http://lists.opensips.org/cgi-bin/mailman/listinfo/users
> >
> > This mail was received via Mail-SeCure System.
> 
> This mail was sent via Mail-SeCure System.
> _______________________________________________
> Users mailing list
> Users at lists.opensips.org
> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
> 
> This mail was received via Mail-SeCure System.

This mail was sent via Mail-SeCure System.

More information about the Users mailing list