<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<font face="monospace">Better don't :), as we never explored all the
possible side effects of such combination of versions....<br>
<br>
Regards,<br>
</font>
<pre class="moz-signature" cols="72">Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
<a class="moz-txt-link-freetext" href="https://www.opensips-solutions.com">https://www.opensips-solutions.com</a>
OpenSIPS eBootcamp 2021
<a class="moz-txt-link-freetext" href="https://opensips.org/training/OpenSIPS_eBootcamp_2021/">https://opensips.org/training/OpenSIPS_eBootcamp_2021/</a></pre>
<div class="moz-cite-prefix">On 1/19/22 9:45 AM, Schneur Rosenberg
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CANvjR0WvsY0uSBHRKzbenA0KpUHbB_AvsAa37NSk3C_mH2k28w@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="auto">That's a given the question is can it mess up the
active server if the passive one was updated already? I'm not
planning on leaving it like that I just like to do the upgrades
slowly, one at a time and test it and only then to upgrade the
second one.
<div dir="auto"><br>
</div>
<div dir="auto">Scott</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Wed, Jan 19, 2022, 09:26
Bogdan-Andrei Iancu <<a href="mailto:bogdan@opensips.org"
moz-do-not-send="true">bogdan@opensips.org</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Schneur,<br>
<br>
It is strongly recommend that all OpenSIPS nodes in a cluster
to have <br>
the same version.<br>
<br>
Best regards,<br>
<br>
Bogdan-Andrei Iancu<br>
<br>
OpenSIPS Founder and Developer<br>
<a href="https://www.opensips-solutions.com"
rel="noreferrer noreferrer" target="_blank"
moz-do-not-send="true">https://www.opensips-solutions.com</a><br>
OpenSIPS eBootcamp 2021<br>
<a
href="https://opensips.org/training/OpenSIPS_eBootcamp_2021/"
rel="noreferrer noreferrer" target="_blank"
moz-do-not-send="true">https://opensips.org/training/OpenSIPS_eBootcamp_2021/</a><br>
<br>
On 1/18/22 6:08 PM, Schneur Rosenberg wrote:<br>
> Hi, it seems like it was fixed in 3.2, I will have to
migrate all my<br>
> servers, I use binary replication will it break if one
server is<br>
> running 2.4 and the other 3.2? its a active/passive setup
so I will<br>
> take down one at a time and upgrade it, I'm just worried
what will<br>
> happen while one is 3.2 and the second one is still 2.4,
in the past I<br>
> disabled the replication while I was doing the updates
and I'm<br>
> wondering if its necessary.<br>
><br>
> thanks<br>
> Scott (Schneur)<br>
><br>
> On Fri, Dec 17, 2021 at 6:21 PM Bogdan-Andrei Iancu <<a
href="mailto:bogdan@opensips.org" target="_blank"
rel="noreferrer" moz-do-not-send="true">bogdan@opensips.org</a>>
wrote:<br>
>> While trying to reproduce (as I failed to do so), I
noticed you mentioned this is on version 2.4.11, right ? As I
was testing on 3.2 without getting the leak.<br>
>><br>
>> Could you try on 3.2/3.1 ? Keep in mind 2.4 is not
maintained anymore :(<br>
>><br>
>> Regards,<br>
>><br>
>> Bogdan-Andrei Iancu<br>
>><br>
>> OpenSIPS Founder and Developer<br>
>> <a href="https://www.opensips-solutions.com"
rel="noreferrer noreferrer" target="_blank"
moz-do-not-send="true">https://www.opensips-solutions.com</a><br>
>> OpenSIPS eBootcamp 2021<br>
>> <a
href="https://opensips.org/training/OpenSIPS_eBootcamp_2021/"
rel="noreferrer noreferrer" target="_blank"
moz-do-not-send="true">https://opensips.org/training/OpenSIPS_eBootcamp_2021/</a><br>
>><br>
>> On 12/17/21 1:40 PM, Schneur Rosenberg wrote:<br>
>><br>
>> Thanks Bogdan!, this is my entire local_route, all my
dst_uri's are IP only.<br>
>><br>
>> On Fri, Dec 17, 2021, 12:36 Bogdan-Andrei Iancu <<a
href="mailto:bogdan@opensips.org" target="_blank"
rel="noreferrer" moz-do-not-send="true">bogdan@opensips.org</a>>
wrote:<br>
>>> Hi Schneur,<br>
>>><br>
>>> I suspect that the leaking mk_proxy is related to
the changing of the<br>
>>> RURI in local route. Let me test your snippet.
BTW, is that the whole<br>
>>> processing you do in local route? is the $rd
(from LB) a FQDN or<br>
>>> straight IP ?<br>
>>><br>
>>> Regards,<br>
>>><br>
>>> Bogdan-Andrei Iancu<br>
>>><br>
>>> OpenSIPS Founder and Developer<br>
>>> <a href="https://www.opensips-solutions.com"
rel="noreferrer noreferrer" target="_blank"
moz-do-not-send="true">https://www.opensips-solutions.com</a><br>
>>> OpenSIPS eBootcamp 2021<br>
>>> <a
href="https://opensips.org/training/OpenSIPS_eBootcamp_2021/"
rel="noreferrer noreferrer" target="_blank"
moz-do-not-send="true">https://opensips.org/training/OpenSIPS_eBootcamp_2021/</a><br>
>>><br>
>>> On 12/16/21 9:43 AM, Schneur Rosenberg wrote:<br>
>>>> Hi Bogdan<br>
>>>><br>
>>>> I think I found the issue, I recently added
these lines of code,<br>
>>>> because of a probing issue I was having, I
just searched from my<br>
>>>> previous tickets and I see that you have
warned me about the<br>
>>>> implications but for some reason I never read
the message.<br>
>>>><br>
>>>> Here is the old ticket<br>
>>>> <a
href="https://www.mail-archive.com/users@lists.opensips.org/msg43301.html"
rel="noreferrer noreferrer" target="_blank"
moz-do-not-send="true">https://www.mail-archive.com/users@lists.opensips.org/msg43301.html</a><br>
>>>> the reason I'm using INVITE to probe is
because I want the servers<br>
>>>> that were probed not only to respond but also
check if the database is<br>
>>>> working, I did it this way because I had
cases where mysql crashed but<br>
>>>> my asterisk servers were still responding to
the probe but all of the<br>
>>>> calls just hung, so I do a invite and it does
a DB lookup and it will<br>
>>>> only return a positive message if it was able
to query the DB, do you<br>
>>>> have a better solution? at the time I set it
up I couldn't run a query<br>
>>>> on receipt of a OPTIONS but perhaps I didn't
look good enough :-),<br>
>>>> either way can I do anything to make sure
this code doesn't leak<br>
>>>> memory? this probing has worked for years
until I needed the Contact<br>
>>>> header.<br>
>>>><br>
>>>> local_route {<br>
>>>> if (is_method("INVITE")&&
$fU=="pingTest"){<br>
>>>> $ru=<a class="moz-txt-link-rfc2396E" href="sip:s@">"sip:s@"</a>+$rd ;<br>
>>>> append_hf("Contact:
<a class="moz-txt-link-rfc2396E" href="sip:pingTest@$fd:5060"><sip:pingTest@$fd:5060></a>\r\n");<br>
>>>> exit;<br>
>>>> }<br>
>>>> }<br>
>>>><br>
>>>> On Fri, Dec 10, 2021 at 2:16 PM Schneur
Rosenberg<br>
>>>> <<a href="mailto:rosenberg11219@gmail.com"
target="_blank" rel="noreferrer" moz-do-not-send="true">rosenberg11219@gmail.com</a>>
wrote:<br>
>>>>> Hi Bogdan,<br>
>>>>><br>
>>>>> I did it on a backup server, its also
leaking memory but at a slower<br>
>>>>> pace, I'm attaching the logs when running
kill -SIGUSR1 on the pid<br>
>>>>> that's growing in size, it still has
available memory, I hop this will<br>
>>>>> give you a clue.<br>
>>>>><br>
>>>>> Here is a pastbin to the loggs <a
href="https://pastebin.com/KJVb9Y75" rel="noreferrer
noreferrer" target="_blank" moz-do-not-send="true">https://pastebin.com/KJVb9Y75</a><br>
>>>>><br>
>>>>> On Fri, Dec 10, 2021 at 11:00 AM Schneur
Rosenberg<br>
>>>>> <<a
href="mailto:rosenberg11219@gmail.com" target="_blank"
rel="noreferrer" moz-do-not-send="true">rosenberg11219@gmail.com</a>>
wrote:<br>
>>>>>> Thank you, does this reduce
performance? can I leave it enabled on a<br>
>>>>>> production machine? I will wait for
the memory leak to be apparent and<br>
>>>>>> I'll post the result.<br>
>>>>>><br>
>>>>>><br>
>>>>>> On Thu, Dec 9, 2021 at 12:31 PM
Bogdan-Andrei Iancu <<a href="mailto:bogdan@opensips.org"
target="_blank" rel="noreferrer" moz-do-not-send="true">bogdan@opensips.org</a>>
wrote:<br>
>>>>>>> Hi Schneur,<br>
>>>>>>><br>
>>>>>>> Just follow the<br>
>>>>>>> <a
href="https://www.opensips.org/Documentation/TroubleShooting-OutOfMem"
rel="noreferrer noreferrer" target="_blank"
moz-do-not-send="true">https://www.opensips.org/Documentation/TroubleShooting-OutOfMem</a>
and<br>
>>>>>>> provide the dump. This is the
only way to investigate this.<br>
>>>>>>><br>
>>>>>>> Regards,<br>
>>>>>>><br>
>>>>>>> Bogdan-Andrei Iancu<br>
>>>>>>><br>
>>>>>>> OpenSIPS Founder and Developer<br>
>>>>>>> <a
href="https://www.opensips-solutions.com" rel="noreferrer
noreferrer" target="_blank" moz-do-not-send="true">https://www.opensips-solutions.com</a><br>
>>>>>>> OpenSIPS eBootcamp 2021<br>
>>>>>>> <a
href="https://opensips.org/training/OpenSIPS_eBootcamp_2021/"
rel="noreferrer noreferrer" target="_blank"
moz-do-not-send="true">https://opensips.org/training/OpenSIPS_eBootcamp_2021/</a><br>
>>>>>>><br>
>>>>>>> On 12/8/21 12:14 PM, Schneur
Rosenberg wrote:<br>
>>>>>>>> I just noticed that process
88 runs the timer handler, perhaps this<br>
>>>>>>>> might shed light on whats
going on.<br>
>>>>>>>><br>
>>>>>>>> opensipsctl fifo ps<br>
>>>>>>>> Process:: ID=88 PID=5327
Type=Timer handler<br>
>>>>>>>><br>
>>>>>>>> On Wed, Dec 8, 2021 at 10:55
AM Schneur Rosenberg<br>
>>>>>>>> <<a
href="mailto:rosenberg11219@gmail.com" target="_blank"
rel="noreferrer" moz-do-not-send="true">rosenberg11219@gmail.com</a>>
wrote:<br>
>>>>>>>>> Now a few hours later
this is what I'm getting<br>
>>>>>>>>> Dec 8 09:50:13
/sbin/opensips[21699]: ERROR:nathelper:nh_timer: out<br>
>>>>>>>>> of pkg memory<br>
>>>>>>>>> Dec 8 09:50:16
/sbin/opensips[21699]: WARNING:core:fm_malloc: not<br>
>>>>>>>>> enough continuous free
pkg memory (3024 bytes left, need 5128),<br>
>>>>>>>>> attempting
defragmentation... please increase the "-M" command line<br>
>>>>>>>>> parameter!<br>
>>>>>>>>> Dec 8 09:50:16
/sbin/opensips[21699]: ERROR:core:fm_malloc: not<br>
>>>>>>>>> enough free pkg memory
(3024 bytes left, need 5128), please increase<br>
>>>>>>>>> the "-M" command line
parameter!<br>
>>>>>>>>><br>
>>>>>>>>> Here is the last 20
package memory max_used_size<br>
>>>>>>>>> pkmem:70-max_used_size::
1009584<br>
>>>>>>>>> pkmem:71-max_used_size::
1009584<br>
>>>>>>>>> pkmem:72-max_used_size::
1009584<br>
>>>>>>>>> pkmem:73-max_used_size::
1009584<br>
>>>>>>>>> pkmem:74-max_used_size::
1009584<br>
>>>>>>>>> pkmem:75-max_used_size::
1009584<br>
>>>>>>>>> pkmem:76-max_used_size::
1009584<br>
>>>>>>>>> pkmem:77-max_used_size::
1009584<br>
>>>>>>>>> pkmem:78-max_used_size::
1009584<br>
>>>>>>>>> pkmem:79-max_used_size::
1009584<br>
>>>>>>>>> pkmem:80-max_used_size::
1044752<br>
>>>>>>>>> pkmem:81-max_used_size::
1075552<br>
>>>>>>>>> pkmem:82-max_used_size::
1116848<br>
>>>>>>>>> pkmem:83-max_used_size::
1117456<br>
>>>>>>>>> pkmem:84-max_used_size::
1102640<br>
>>>>>>>>> pkmem:85-max_used_size::
1306992<br>
>>>>>>>>> pkmem:86-max_used_size::
1706304<br>
>>>>>>>>> pkmem:87-max_used_size::
2507000<br>
>>>>>>>>> pkmem:88-max_used_size::
4194264<br>
>>>>>>>>> pkmem:89-max_used_size::
1009584<br>
>>>>>>>>><br>
>>>>>>>>> And here is the real used
size, you can see that process 88 maxed out<br>
>>>>>>>>> pkmem:69-real_used_size::
975528<br>
>>>>>>>>> pkmem:70-real_used_size::
978016<br>
>>>>>>>>> pkmem:71-real_used_size::
989592<br>
>>>>>>>>> pkmem:72-real_used_size::
951416<br>
>>>>>>>>> pkmem:73-real_used_size::
982496<br>
>>>>>>>>> pkmem:74-real_used_size::
965744<br>
>>>>>>>>> pkmem:75-real_used_size::
959424<br>
>>>>>>>>> pkmem:76-real_used_size::
949472<br>
>>>>>>>>> pkmem:77-real_used_size::
983080<br>
>>>>>>>>> pkmem:78-real_used_size::
961400<br>
>>>>>>>>> pkmem:79-real_used_size::
977808<br>
>>>>>>>>> pkmem:80-real_used_size::
978928<br>
>>>>>>>>> pkmem:81-real_used_size::
1009936<br>
>>>>>>>>> pkmem:82-real_used_size::
1110760<br>
>>>>>>>>> pkmem:83-real_used_size::
1116720<br>
>>>>>>>>> pkmem:84-real_used_size::
1096568<br>
>>>>>>>>> pkmem:85-real_used_size::
1300592<br>
>>>>>>>>> pkmem:86-real_used_size::
1699648<br>
>>>>>>>>> pkmem:87-real_used_size::
2501096<br>
>>>>>>>>> pkmem:88-real_used_size::
4191280<br>
>>>>>>>>> pkmem:89-real_used_size::
882528<br>
>>>>>>>>><br>
>>>>>>>>> On Tue, Dec 7, 2021 at
7:53 PM Schneur Rosenberg<br>
>>>>>>>>> <<a
href="mailto:rosenberg11219@gmail.com" target="_blank"
rel="noreferrer" moz-do-not-send="true">rosenberg11219@gmail.com</a>>
wrote:<br>
>>>>>>>>>> Hi, lately I'm
getting these errors in my logs.<br>
>>>>>>>>>><br>
>>>>>>>>>> ERROR:core:fm_malloc:
not enough free pkg memory (1792 bytes left,<br>
>>>>>>>>>> need 2184), please
increase the "-M" command line para<br>
>>>>>>>>>> meter!<br>
>>>>>>>>>><br>
>>>>>>>>>>
CRITICAL:core:hostent_cpy: pkg memory allocation failure<br>
>>>>>>>>>><br>
>>>>>>>>>>
ERROR:nathelper:nh_timer: out of pkg memory<br>
>>>>>>>>>><br>
>>>>>>>>>> ERROR:core:fm_malloc:
not enough free pkg memory (5952 bytes left,<br>
>>>>>>>>>> need 5408), please
increase the "-M" command line para<br>
>>>>>>>>>> meter!<br>
>>>>>>>>>><br>
>>>>>>>>>> I was on version
2.4.8 and I upgraded to 2.4.11 and I'm monitoring the<br>
>>>>>>>>>> max_used_size of the
package memory, a few hours later I see that 2<br>
>>>>>>>>>> processes keep on
getting bigger, so far the rest are pretty stable, I<br>
>>>>>>>>>> have 90 processes and
87 and 88 are growing.<br>
>>>>>>>>>><br>
>>>>>>>>>> here you can see the
last few processes, OpenSIPS set aside 4 mb per process.<br>
>>>>>>>>>><br>
>>>>>>>>>>
pkmem:80-max_used_size:: 1009584<br>
>>>>>>>>>>
pkmem:81-max_used_size:: 1009584<br>
>>>>>>>>>>
pkmem:82-max_used_size:: 1009584<br>
>>>>>>>>>>
pkmem:83-max_used_size:: 1009584<br>
>>>>>>>>>>
pkmem:84-max_used_size:: 1009584<br>
>>>>>>>>>>
pkmem:85-max_used_size:: 1009584<br>
>>>>>>>>>>
pkmem:86-max_used_size:: 1143608<br>
>>>>>>>>>>
pkmem:87-max_used_size:: 1323256<br>
>>>>>>>>>>
pkmem:88-max_used_size:: 1831928<br>
>>>>>>>>>>
pkmem:89-max_used_size:: 1009584<br>
>>>>>>>>>><br>
>>>>>>>>>> Any hints where to
start looking besides the solutions fund here.<br>
>>>>>>>>>><br>
>>>>>>>>>> <a
href="https://www.opensips.org/Documentation/TroubleShooting-OutOfMem"
rel="noreferrer noreferrer" target="_blank"
moz-do-not-send="true">https://www.opensips.org/Documentation/TroubleShooting-OutOfMem</a><br>
>>>>>>>>>><br>
>>>>>>>>>> thank you<br>
>>>>>>>>>> Scott<br>
>>>>>>>>
_______________________________________________<br>
>>>>>>>> Users mailing list<br>
>>>>>>>> <a
href="mailto:Users@lists.opensips.org" target="_blank"
rel="noreferrer" moz-do-not-send="true">Users@lists.opensips.org</a><br>
>>>>>>>> <a
href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users"
rel="noreferrer noreferrer" target="_blank"
moz-do-not-send="true">http://lists.opensips.org/cgi-bin/mailman/listinfo/users</a><br>
<br>
</blockquote>
</div>
</blockquote>
<br>
</body>
</html>