[OpenSIPS-Users] rtpengine errors: can't send command to a RTP proxy (22:Invalid argument)
Ovidiu Sas
osas at voipembedded.com
Sat Jun 27 18:35:42 EST 2020
Fix implemented on trunk, 3.1 and 3.0.
We should update the fix on rtpproxy to match rtpengine and move the
OSIP_IOV_MAX define somewhere upper in the code tree to be visible by
both rtpproxy and rtpengine modules.
For trunk, maybe we should add a new param.
Should we backport this fix to 2.4?
Thanks,
Ovidiu
On Fri, Jun 26, 2020 at 9:35 AM Ovidiu Sas <osas at voipembedded.com> wrote:
>
> Hello Razvan,
>
> On ubuntu we have a failure for more then 1024 buffers. There is something defined in some experimental headers (as __IOV_MAX), but the IOV_MAX define is not visible in the rtpengine module.
>
> And yes, there is an extra buffer. I guess the length of that extra buffer was zero and that’s why it didn’t create issues. But as soon as I concatenated the buffer, the issue showed up and I had an extra trailing string at the end of the rtpengine command that was sent on the wire.
> After reducing the number of buffers by one, all was good on both scenarios: sending with and without concatenated buffers.
> I will prepare a fix for it and push it.
>
> -ovidiu
>
> On Fri, Jun 26, 2020 at 03:55 Răzvan Crainea <razvan at opensips.org> wrote:
>>
>> Hi, Ovidiu!
>>
>> So you're saying that the IOV_MAX is not explicitly defined, but it does
>> fail after 1024 buffers, correct? If so, perhaps we should limit the
>> number of buffers to 1024, if not already defined.
>> I did notice the extra vcnt as well, but I though that was related to
>> the fact that it allocates one extra iovec (at the headr) for the
>> cookie, and sometime uses it, sometime doesn't. Nevertheless, it is
>> consistent on both cases, UNIX & UDP, both have that vcnt + 1. Now I
>> haven't checked whether this is correct or not, could you please confirm?
>>
>> Best regards,
>>
>> Răzvan Crainea
>> OpenSIPS Core Developer
>> http://www.opensips-solutions.com
>>
>> On 6/26/20 2:29 AM, Ovidiu Sas wrote:
>> > It looks like the hidden IOV_MAX is set to 1024 in debian.
>> > I tested a patch and all looks good.
>> > It seems that there is an error in the code: writev has the wrong
>> > number of iovcnt (should be one less).
>> > I tested and all looks ok (the code in rtpproxy.c has the proper iovcnt).
>> >
>> > -ovidiu
>> >
>> > On Thu, Jun 25, 2020 at 1:03 PM Ovidiu Sas <osas at voipembedded.com> wrote:
>> >>
>> >> We will need to add a param to control the max number of buffers.
>> >>
>> >> -ovidiu
>> >>
>> >> On Thu, Jun 25, 2020 at 1:02 PM Ovidiu Sas <osas at voipembedded.com> wrote:
>> >>>
>> >>> It seems that when we have more than roughly 1000 buffers, the send fails.
>> >>>
>> >>> -ovidiu
>> >>>
>> >>> On Wed, Jun 24, 2020 at 8:54 AM Ovidiu Sas <osas at voipembedded.com> wrote:
>> >>>>
>> >>>> Hello Razvan,
>> >>>>
>> >>>> The system is a debian buster one.
>> >>>> I patched the code:
>> >>>> #ifdef IOV_MAX
>> >>>> LM_NOTICE("IOV_MAX=[%d]\n", IOV_MAX);
>> >>>> #else
>> >>>> LM_NOTICE("no IOV_MAX\n");
>> >>>> #endif
>> >>>>
>> >>>> and I get:
>> >>>> NOTICE:rtpengine:send_rtpe_command: no IOV_MAX
>> >>>> in the logs.
>> >>>>
>> >>>> Then I patched the code again to check how many buffers are being used:
>> >>>> The max so far was 73:
>> >>>> LM_NOTICE("writev(rtpe_socks[node->idx], v , %d)\n", vcnt + 1);
>> >>>> got me:
>> >>>> NOTICE:rtpengine:send_rtpe_command: writev(rtpe_socks[node->idx], v , 73)
>> >>>>
>> >>>> I will continue to monitor the system to see if there is a correlation
>> >>>> between the error and the number of buffers.
>> >>>>
>> >>>> Thanks,
>> >>>> Ovidiu
>> >>>>
>> >>>> On Wed, Jun 24, 2020 at 1:59 AM Răzvan Crainea <razvan at opensips.org> wrote:
>> >>>>>
>> >>>>> Hi, Ovidiu!
>> >>>>>
>> >>>>> I doubt this is a problem of OpenSIPS version, but rather of the OS you
>> >>>>> are running on. I suspect that error comes from the fact that the bson
>> >>>>> resulted has more than IOV_MAX elements, which if I recall correctly it
>> >>>>> was 15 on some OSes.
>> >>>>> We had a similar problem in rtpproy[1], where we had merged the buffers
>> >>>>> in a single one just to pass over this limitation. Could you check if
>> >>>>> you are facing a similar issue?
>> >>>>>
>> >>>>> [1]
>> >>>>> https://github.com/OpenSIPS/opensips/blob/master/modules/rtpproxy/rtpproxy.c#L2031
>> >>>>>
>> >>>>> Răzvan Crainea
>> >>>>> OpenSIPS Core Developer
>> >>>>> http://www.opensips-solutions.com
>> >>>>>
>> >>>>> On 6/24/20 7:32 AM, Ovidiu Sas wrote:
>> >>>>>> This is happening also on the latest 3.0.
>> >>>>>> The weird thing is that opensips doesn't send anything to rtpengine.
>> >>>>>> The first opensips/rtpengine exchange on the initial INVITE works ok,
>> >>>>>> but the opensips/rtpengine exchange on the 200ok fails (no command is
>> >>>>>> sent by opensips - confirmed by running ngrep on the loopback
>> >>>>>> interface).
>> >>>>>> On the next call, the initial offer works fine, but the answer fails
>> >>>>>> due to no command issued by opensips.
>> >>>>>>
>> >>>>>> version: opensips 3.0.2 (x86_64/linux)
>> >>>>>> flags: STATS: On, DISABLE_NAGLE, USE_MCAST, SHM_MMAP, PKG_MALLOC,
>> >>>>>> Q_MALLOC, F_MALLOC, HP_MALLOC, DBG_MALLOC, FAST_LOCK-ADAPTIVE_WAIT
>> >>>>>> ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16,
>> >>>>>> MAX_URI_SIZE 1024, BUF_SIZE 65535
>> >>>>>> poll method support: poll, epoll, sigio_rt, select.
>> >>>>>> git revision: 3a8f6f137
>> >>>>>> main.c compiled on 09:36:42 Jun 22 2020 with gcc 9
>> >>>>>>
>> >>>>>> -ovidiu
>> >>>>>>
>> >>>>>>
>> >>>>>> On Wed, Jun 10, 2020 at 2:49 PM Ovidiu Sas <osas at voipembedded.com> wrote:
>> >>>>>>>
>> >>>>>>> Hello all,
>> >>>>>>>
>> >>>>>>> I'm running opensips 3.1.0-beta (latest version) and experiencing
>> >>>>>>> connectivity issues to the rtpengine daemon running on the same host:
>> >>>>>>> ERROR:rtpengine:send_rtpe_command: can't send command to a RTP proxy
>> >>>>>>> (22:Invalid argument)
>> >>>>>>> ERROR:rtpengine:send_rtpe_command: timeout waiting reply from a RTP proxy
>> >>>>>>> ERROR:rtpengine:send_rtpe_command: proxy <udp:127.0.0.1:2223> does not
>> >>>>>>> respond, disable it
>> >>>>>>> ERROR:rtpengine:rtpe_function_call: no available proxies
>> >>>>>>>
>> >>>>>>> After an opensips restart, everything comes to normal.
>> >>>>>>> I was running previously opensips 3.1.0-dev and everything was working fine.
>> >>>>>>>
>> >>>>>>> The issue starts showing up after a few days with very little traffic.
>> >>>>>>> Is there anyone experiencing this issue?
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Thanks,
>> >>>>>>> Ovidiu
>> >>>>>>>
>> >>>>>>> --
>> >>>>>>> VoIP Embedded, Inc.
>> >>>>>>> http://www.voipembedded.com
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>
>> >>>>> _______________________________________________
>> >>>>> Users mailing list
>> >>>>> Users at lists.opensips.org
>> >>>>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> VoIP Embedded, Inc.
>> >>>> http://www.voipembedded.com
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> VoIP Embedded, Inc.
>> >>> http://www.voipembedded.com
>> >>
>> >>
>> >>
>> >> --
>> >> VoIP Embedded, Inc.
>> >> http://www.voipembedded.com
>> >
>> >
>> >
>>
>> _______________________________________________
>> Users mailing list
>> Users at lists.opensips.org
>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>
> --
> VoIP Embedded, Inc.
> http://www.voipembedded.com
--
VoIP Embedded, Inc.
http://www.voipembedded.com
More information about the Users
mailing list