[OpenSIPS-Devel] Hang due to "no more nonces can be generated"

[Bogdan-Andrei Iancu]

Hi Alistair,

Thank you for your feedback - I had some investigation with Anca and we 
think we found a bug in the algorithm for monitoring the used nonces. 
I'm aware of the "nonce_reuse" param in kamilio, but I do not like the 
approach of disabling something if it a bug there - we will try to fix 
this issue asap. Nonce reusage is a security risk, a scenario quite easy 
to exploit and this should not be disregarded in a production system.

Btw, you should reduce the registration rate  - make no sense to use 
registration as keep alive mechanism when you have a  proofed  and 
dedicated mechanism for this (try OPTIONS -or sipping). The load on your 
system will dramatically reduce.

Regards,
Bogdan

Alistair Cunningham wrote:
> Bodgen,
>
> Thank you for investigating this. I did try 60 seconds which did not 
> help. I then tried 20 seconds and a nightly restart of OpenSIPS from 
> crontab, and the problem has not occurred since. However, I'm 
> concerned that this system is no-where near the largest we support and 
> so 20 seconds may not suffice for large systems.
>
> I notice that Kamailio suffered from the same problem, and they 
> introduced a "nonce_reuse" modparam:
>
> http://www.mail-archive.com/users@lists.kamailio.org/msg01303.html
>
> Would this be worth implementing for OpenSIPS? Is it safe from a 
> security (e.g. replay attack) point of view?
>
> Alistair Cunningham
> +1 888 468 3111
> +44 20 799 39 799
> http://integrics.com/
>
>
> Bogdan-Andrei Iancu wrote:
>> Hi Alistair,
>>
>> This is related to authentication. OpenSIPS is keeping state for all 
>> the nonces it generates in order to avoid nonce re-usage.  The error 
>> you get means that all the available slots for generating nonces are 
>> used (by default are 100 000 of them), but no nonce was replied 
>> (getting a response).
>>
>> Do you have such a large traffic that you may have more than 100 000 
>> authentication requests at a time (without getting the responses yet) ?
>>
>> You may try to reduce the nonce lifetime and make the un-answered 
>> ones to be released faster . see nonce_expire param:
>>    http://www.opensips.org/html/docs/modules/1.4.x/auth.html#id2526655
>>
>> - try setting this to 30 seconds .
>>
>> I'm trying to figure out if in your case it is a simple problem of 
>> load or it is a bug in the nonce reservation mechanism.
>>
>> Regards,
>> Bogdan
>>
>> Alistair Cunningham wrote:
>>> We've just had OpenSIPS 1.4.2 stop processing SIP packets and 
>>> effectively hang. During this time, it logged the following many 
>>> times to /var/log/daemon.log:
>>>
>>> ERROR:auth:build_auth_hf: no more nonces can be generated
>>> ERROR:auth:challenge: failed to generate nonce
>>>
>>> Restarting OpenSIPS has temporarily cured it, but I expect the 
>>> problem will come back.
>>>
>>> Another problem (probably unrelated) on the same machine was that 
>>> when running "opensipsctl online", no output was produced and the 
>>> following was logged to daemon.log:
>>>
>>> ERROR:core:create_mi_node: no more pkg mem
>>> ERROR:mi_fifo:mi_fifo_server: command (ul_dump) processing failed
>>>
>>> I've since set the following in config.h:
>>>
>>> #define PKG_MEM_POOL_SIZE 10*1024*1024
>>>
>>> and this problem has gone away (opensipsctl online produces 1793 
>>> lines of output), but it's unclear whether this will help with the 
>>> nonce problem (I'm thinking probably not). In any case, may we 
>>> please have either a config file option or a command line option to 
>>> set PKG_MEM_POOL_SIZE without needing to patch the source code?
>>>
>>>   
>>
>>
>>
>