[OpenSIPS-Users] 502 Bad Gateway events leads to calls being rejected with 480 Temporarily Unavailable

Mon Jun 8 12:46:21 EST 2020

He is also performing dispatcher failover in these cases using ds_next_domain, which may not be necessary or warranted on an error code relayed from further upstream. Just removing ds_mark_dst will not resolve that. Although some extra unnecessary failover in error cases may not be an issue.

Ben Newlin

From: Users <users-bounces at lists.opensips.org> on behalf of Diptesh Patel <diptesh.patel at ecosmob.com>
Reply-To: OpenSIPS users mailling list <users at lists.opensips.org>
Date: Monday, June 8, 2020 at 8:43 AM
To: OpenSIPS users mailling list <users at lists.opensips.org>
Subject: Re: [OpenSIPS-Users] 502 Bad Gateway events leads to calls being rejected with 480 Temporarily Unavailable

Hello Solarmon,

I think, The  ds_mark_dst("p"); put your destinations on Probing and after a few seconds you will get the reply for OPTIONS and now your destinations are Active. Are you making a second call immediately? If yes then it is clear. Please remove the ds_mark_dst("p"), OpenSIPS automatically change the destination's state using OPTIONS.

Thanks & Regards
Diptesh Patel
Software Developer
Ecosmob Technologies Ltd,
Ahmedabad
Mo:+919898962659

On Mon, Jun 8, 2020 at 4:27 PM solarmon <solarmon at one-n.co.uk<mailto:solarmon at one-n.co.uk>> wrote:
Hi Diptesh ,

Thanks for your reply.

Apologies, I'm using the term 'blacklist' to generally mean that the endpoints are not available.

Also, the 502 Bad Gateway is response to an INVITE, not SIP OPTIONS, returned by the far end and the ITSP is just passing that back to us, because the call has failed. For such call failures, I'm not expecting for the dispatcher endpoints to be marked as unavailable for routing.

I am not using, or have not set, the 'ds_define_blacklist (str)' option in my dispatcher module config.

My probing mode is:

modparam("dispatcher", "ds_probing_mode", 1)

I'm not seeing anything in the logs regarding the dispatcher nodes going into Probing mode - should there be logs for that, or can it be enabled to be logged?

When I check the endpoints with ' opensipsctl dispatcher dump' they always seem to be 'Active' - so it is either they are like that, or they may have only been in 'Probing' mode very briefly. Again, I was hoping to see mode/state change in the historical logs.

In my opensips.cfg (which was created for me) I can see the following code, which looks like this is where it is introducing this behaviour in question:

failure_route[call_failover]
{
        xlog("[$ci] call failed to established with $T_reply_code code\n");

        rtpproxy_unforce("$avp(rtpp_set)");

        if (t_was_cancelled()) {
                t_reply("487","Request cancelled");
                exit;
        }

        # any failure indication ?
        if ( t_check_status("[56][0-9][0-9]")
        || (t_check_status("408") && t_local_replied("all"))
        ) {
                xlog("[$ci] destination $rd failed  with $T_reply_code -> retry\n ");

                ds_mark_dst("p");

                if ( ds_next_domain() ) {
                        xlog("[$ci] using new destination <$rd>\n ");

                        # send it out again
                        t_on_failure("call_failover");
                        t_relay();
                        exit;
                } else {
                        xlog("[$ci] no other destination to retry\n ");
                        t_reply("503","Service not available");
                        exit;
                }
        }

        # if call failure, allow the reply to propagate to caller
        exit;
}

Thank you for the tip about the  'modparam("dispatcher", "options_reply_codes", "502")' option. I will try that if it is not recommend to change the above code.

Thank you.

On Mon, 8 Jun 2020 at 10:47, Diptesh Patel <diptesh.patel at ecosmob.com<mailto:diptesh.patel at ecosmob.com>> wrote:
Hello Solarmon,

Need some clarification on term blacklisting, Are you using the blacklist for Probing Mode of destination? or Are you using the 'ds_define_blacklist (str)' parameter. If you are not using the blacklist parameter then below information help you. It is great if you share your script snippet and output of 'opensipsctl dispatcher dump' which shows you the current status of your destinations.

If you are getting success(200 OK) response on OPTIONS then it is not possible that you got a negative response from a destination and it will not be blacklisted(probing mode) in dispatcher until you blacklist(probing mode) from the script using 'ds_mark_dst()' exported function. I doubt that you are also getting '502 Bad Gateway' on OPTIONS which is sending to the destination to check the availability.  If It is right and you want to add the 502 response as a good response for OPTIONS. you can add the 502 as 'modparam("dispatcher", "options_reply_codes", "502")'.

Thanks & Regards
Diptesh Patel
Software Developer
Ecosmob Technologies Ltd,
Ahmedabad
Mo:+919898962659

On Mon, Jun 8, 2020 at 2:24 PM solarmon <solarmon at one-n.co.uk<mailto:solarmon at one-n.co.uk>> wrote:
Hi,

I'm trying to understand whether this is the correct or expected behaviour.

We have two destinations configured in Dispatcher.

What I am noticing is that when we receive 502 Bad Gateway messages (logged as ("call failed to established with 502 code") from both endpoints. After both endpoints have returned 502 Bad Gateway, opensips pass back 503 Service Unavailable back to the originating endpoint of the call. However, subsequent calls are being immediately rejected with 480 Temporarily Unavailable (logged as "failed to find an available destination, rejecting") for a period of time.

It seems that opensips is blacklisting the Dispatcher endpoints because of receiving the 502 Bad Gateway messages. Is this the correct/expected behaviour? I would have thought the blacklisting should be based on the SIP OPTIONS sent to the Dispatcher endpoints.

I do not currently see any issues with SIP OPTIONS to these endpoints so I'm confused as to why they are seemingly blacklisted.

If this is the correct/expected behaviour, can it be changed to only blacklist based on the SIP OPTIONs pings?

Thank you.
_______________________________________________
Users mailing list
Users at lists.opensips.org<mailto:Users at lists.opensips.org>
http://lists.opensips.org/cgi-bin/mailman/listinfo/users<http://lists.opensips.org/cgi-bin/mailman/listinfo/users>

Disclaimer
In addition to generic Disclaimer which you have agreed on our website, any views or opinions presented in this email are solely those of the originator and do not necessarily represent those of the Company or its sister concerns. Any liability (in negligence, contract or otherwise) arising from any third party taking any action, or refraining from taking any action on the basis of any of the information contained in this email is hereby excluded.

Confidentiality
This communication (including any attachment/s) is intended only for the use of the addressee(s) and contains information that is PRIVILEGED AND CONFIDENTIAL. Unauthorized reading, dissemination, distribution, or copying of this communication is prohibited. Please inform originator if you have received it in error.

Caution for viruses, malware etc.
This communication, including any attachments, may not be free of viruses, trojans, similar or new contaminants/malware, interceptions or interference, and may not be compatible with your systems. You shall carry out virus/malware scanning on your own before opening any attachment to this e-mail. The sender of this e-mail and Company including its sister concerns shall not be liable for any damage that may incur to you as a result of viruses, incompleteness of this message, a delay in receipt of this message or any other computer problems.
_______________________________________________
Users mailing list
Users at lists.opensips.org<mailto:Users at lists.opensips.org>
http://lists.opensips.org/cgi-bin/mailman/listinfo/users<http://lists.opensips.org/cgi-bin/mailman/listinfo/users>
_______________________________________________
Users mailing list
Users at lists.opensips.org<mailto:Users at lists.opensips.org>
http://lists.opensips.org/cgi-bin/mailman/listinfo/users<http://lists.opensips.org/cgi-bin/mailman/listinfo/users>

Disclaimer
In addition to generic Disclaimer which you have agreed on our website, any views or opinions presented in this email are solely those of the originator and do not necessarily represent those of the Company or its sister concerns. Any liability (in negligence, contract or otherwise) arising from any third party taking any action, or refraining from taking any action on the basis of any of the information contained in this email is hereby excluded.

Confidentiality
This communication (including any attachment/s) is intended only for the use of the addressee(s) and contains information that is PRIVILEGED AND CONFIDENTIAL. Unauthorized reading, dissemination, distribution, or copying of this communication is prohibited. Please inform originator if you have received it in error.

Caution for viruses, malware etc.
This communication, including any attachments, may not be free of viruses, trojans, similar or new contaminants/malware, interceptions or interference, and may not be compatible with your systems. You shall carry out virus/malware scanning on your own before opening any attachment to this e-mail. The sender of this e-mail and Company including its sister concerns shall not be liable for any damage that may incur to you as a result of viruses, incompleteness of this message, a delay in receipt of this message or any other computer problems.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opensips.org/pipermail/users/attachments/20200608/6b22e479/attachment-0001.html>