[OpenSIPS-Users] [RFC] Distributed User Location

Bogdan-Andrei Iancu bogdan at opensips.org
Mon Apr 22 14:41:56 CEST 2013 

Hello Adrian,

 From resources point of view, this is how I see things :

1) media - the node which is contacted by user (via DNS) should take 
care of the media (unrelated to registration)

2) registration - you can do it on the access node (used by user to 
access the service), or you can use something like chord (or others) to 
move the registration on a different node . My question is : as 
registration is mainly storing a record in mem, does it pay off to do 
all extra signaling (using PATH to route all calls of the user from the 
node he is registered with via the node he used for accessing service) ? 
(do not forget the NAT restrictions).

3) geo-distribution sometimes comes by itself (as how to do 
distribution) - usually this make sense when having multi-country 
location, and such cases, you will point via DNS/etc users from certain 
countries to certains nodes, so you do not need anything extra in order 
to do the distribution

Best regards,

Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
http://www.opensips-solutions.com


On 04/09/2013 11:51 AM, Adrian Georgescu wrote:
> I am  running for years servers distributed in different countries part of the same service and nobody complained about the latency of signaling but they complained about media. This idea of geo-distribution is more about media path optimization and automatic recovery in case of connectivity failures in one location rather than pointing user from country X to server Y.  Any distribution model that is not done by a formula like Chord does, is not deterministic. This means that mapping must be manually provisioned and changed if nodes come and go. You cannot deterministically  geo-locate to the same server unless you hardwire this as a setting or in a database, is not self-learning and in case of server failure it must be changed manually. Signaling-wise the geo-location is less of a problem. If it takes a fews seconds to make a call setup is not that critical, media relay however need to use shortest path so. An algorithm to allocate the closest relay to the calling party is much more useful. However without an async core, reserving a relay 1 second RTT-wise away is a killer for the overall CPS. Whatever you try to do without an async core any distribution of resources hit the performance problem related to the child being blocked while processing a request and the farther away the database or relay the worse it gets.  I think that by addressing the async issue, will automagically create a multitude of solutions for better distribution and load balancing.
>
> Adrian
>
> On Apr 9, 2013, at 10:17 AM, Bogdan-Andrei Iancu wrote:
>
>> Hi,
>>
>> Putting together what you said and what Adrian and Muhammad said :
>>
>> Actually we may have a distributed USRLOC for 2 purposes: geo distribution and load distribution - how they are approach it is a bit different.
>>
>> But first let's look into the common part (for the 2 cases) : IMHO, in both cases we should have the SIP part (opensips) storing the actual full registration in a certain location (via USRLOC) and an upper layer, distributed, to keep a mapping between users (AORs) and the location(s) they are registered with. So:
>>     - local level - OpenSIPS doing classing registrations (a node)
>>     - distributed level - some other tool to keep (in a distributed fashion) the mapping of AORs on the nodes
>>
>> Now, here comes the difference.
>>
>> If you do geo distribution, you want to keep registration as closes as possible to the user. So the registration will be kept on the OpenSIPS node which was contacted by the user. In this case Chord does not work (at distributed level) as it has its own alg to distributed data across nodes; in our case we want to control the distribution and to say what data/registration stays on what node/opensips.
>>
>> If you do load distribution, you want to balance all received registrations across all existing nodes/opensips - in this case a Chord like approach will help (as it will do the load distribution for you).
>>
>>
>> As I see the solution : have the 2 layers (local and distributed) as built in in OpenSIPS and additionally to be able to use different algorithms to do the mapping between registrations and OpenSIPS nodes.
>>
>>
>> Is the above a good approach ??
>>
>> Regards,
>>
>> Bogdan-Andrei Iancu
>> OpenSIPS Founder and Developer
>> http://www.opensips-solutions.com
>>
>>
>> On 04/05/2013 04:45 PM, Rudy wrote:
>>> Everyone,
>>>
>>>   Before we get too off topic, I think the goal should be to design
>>> something truly distributed. This would be more like what Adrian
>>> suggested and less like a super node / slave node scenario. The nodes
>>> should be able to coordinate amongst themselves, again, similar to the
>>> docs Adrian shared.
>>>
>>>   One thing we will need is a consistent hashing alg. Adrian suggested
>>> Chord, another that works well for us in our implementations is
>>> Ketama. Either way, it needs to be able to have consistent hashing, so
>>> that additions / removals of nodes do not change the location of home
>>> proxy of each registered user.
>>>
>>> http://en.wikipedia.org/wiki/Consistent_hashing
>>>
>>> Thanks in advance,
>>> --Rudy
>>> Dynamic Packet
>>> Toll-Free: 888.929.VOIP ( 8647 )
>>>
>>>
>>> On Fri, Apr 5, 2013 at 9:39 AM, Muhammad Shahzad<shaheryarkh at gmail.com>   wrote:
>>>> Well, i am not much familiar with internals of opensips, i.e. its core and
>>>> modules and how they interact with each other. But as an abstract idea, i
>>>> suggest that both Base Node and Super Node should be opensips modules. No
>>>> change in standard registrar or usrloc modules are actually needed.
>>>>
>>>> In the Super Node module, we will have,
>>>>
>>>> 1. one db table to store base node addresses for monitoring the Event.
>>>> 2. one db table to store data received from the Event, lets call it "Event
>>>> Table".
>>>> 3. one process to manage "Event Table", pretty much the same way location
>>>> table is managed by usrloc module.
>>>> 4. some scripting functions for opensips.cfg, to look up in "Event Table"
>>>> and do SIP redirect.
>>>> 5. some MI functions to manually manage base node table and event table.
>>>>
>>>> In the Base Node module, we will have,
>>>>
>>>> 1. module parameters to define address of Super Node and event advertise
>>>> socket (Super Node will connect to this socket to receive events).
>>>> 2. a process to monitor usrloc table, such that as soon as a new user
>>>> registers, it advertise this to event socket.
>>>> 3. some scripting functions for opensips.cfg, to send call to Super Node if
>>>> lookup function (from registrar module) fails and in reply route to handle
>>>> SIP Redirect to send call to destination base node returned by Super Node.
>>>> 4. some MI functions etc.
>>>>
>>>> Thank you.
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, Apr 5, 2013 at 1:37 PM, Bogdan-Andrei Iancu<bogdan at opensips.org>
>>>> wrote:
>>>>> Hello Muhammad,
>>>>>
>>>>> Your approach is the correct one (from SIP perspective) IMHO. But there
>>>>> are questions on the implementation side too - like the "Super Node" is just
>>>>> a storage or it should have SIP capabilities? How much of this behavior
>>>>> should be hardcoded in the registrar + usrloc module ?
>>>>>
>>>>> Best regards,
>>>>>
>>>>> Bogdan-Andrei Iancu
>>>>> OpenSIPS Founder and Developer
>>>>> http://www.opensips-solutions.com
>>>>>
>>>>>
>>>>> On 04/05/2013 04:57 AM, Muhammad Shahzad wrote:
>>>>>
>>>>> Well at 5 am in the morning while thinking on this topic the only thing
>>>>> ringing in my mind is a mechanism similar to IP to IP Gateway. Here is the
>>>>> main concept.
>>>>>
>>>>> 1. We have number of SIP servers running, say sip1.mydomain.com,
>>>>> sip2.mydomain.com ... sipN.mydomain.com, each serving domain mydomain.com
>>>>> and a SIP client A can select any one of these servers through DNS look-up
>>>>> (or whatever way possible) and registers to that server. Lets call these
>>>>> servers as Base Nodes.
>>>>>
>>>>> 2. Upon successful registration of client A to server sip1.mydomain.com,
>>>>> this Registrar Node fires an Event, which can be subscribed by a back-end
>>>>> SIP server, lets call it Super Node. This event will only contain following
>>>>> things,
>>>>>
>>>>>     a). User part of all Contact URIs of client A with Expiry.
>>>>>     b). Registrar Node information e.g. its IP address + Port.
>>>>>     c). SIP domain of client A. (in case of multi-domain setup)
>>>>>
>>>>> 3. Super Node stores this information in some db back-end (memcache,
>>>>> redis, mysql etc.). This is sort of back-to-back register process but
>>>>> without SIP or authentication, since that has already been handled on Based
>>>>> Node anyway. The Super Node only needs to know which user is registered on
>>>>> which Base Node e.g. user 1001 is registered on node sip1.mydomain.com, user
>>>>> 1203 is registered on sip6.mydomain.com and so on.
>>>>>
>>>>> 4. When a SIP client B tries to send INVITE or MESSAGE or SUBSCRIBE to SIP
>>>>> client A. The SIP request will arrive on Base Node it is currently
>>>>> registered with, say sip2.mydomain.com. This node will first do local
>>>>> look-up for location of client A. Upon failure it will forward request to
>>>>> Super Node, which will do a look-up on Event database and finds that client
>>>>> A is registered on node sip1.mydomain.com, so it will send SIP redirect
>>>>> response 302 to requester Base Node. Now the request source node knows the
>>>>> address of request destination node, where it will send request next and
>>>>> they both, while acting as independent SIP servers, establish SIP session
>>>>> between caller and callee. This should work regardless if both nodes serve
>>>>> same or different SIP domains.
>>>>>
>>>>> 5. The Super Node will also give us global presence of all users currently
>>>>> registered to all Base Nodes, which may be useful for management and
>>>>> monitoring etc.
>>>>>
>>>>> Pros:
>>>>> 1. Completely independent of network topology and SIP.
>>>>> 2. Will work seamlessly for multi and federated domains.
>>>>> 3. Scale-able in every direction.
>>>>> 4. Minimal overhead for session establishment. Once supper node return
>>>>> destination base node address in SIP redirect response, session will
>>>>> establish directly between source and destination base node. Further
>>>>> optimizations are possible, e.g. base node can cache destination base node
>>>>> returned by supper node for any particular user and avoid querying super
>>>>> node for recurring SIP sessions.
>>>>>
>>>>> Cons:
>>>>> 1. Well, the key problem i can guess is of course the Event database size
>>>>> and speed, as it will have information on every user registered to every
>>>>> Base Node. I suggest memory cache db such as Redis would be idle for this
>>>>> storage.
>>>>> 2. Bandwidth consumed in Event transport. We can apply compression and
>>>>> make event queues as optimization.
>>>>>
>>>>> Comments and suggestions are highly welcome.
>>>>>
>>>>> Thank you.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Apr 4, 2013 at 2:05 PM, Vlad Paiu<vladpaiu at opensips.org>   wrote:
>>>>>> Hello all,
>>>>>>
>>>>>> We would like to get suggestions and help on the matter of distributing
>>>>>> the user location information.
>>>>>> Extending the User Location with a built-in distributed support is not
>>>>>> straight forward - it is not about simply sharing data - as it is really SIP
>>>>>> dependent and network limited
>>>>>>
>>>>>> While now, by using the OpenSIPS trunk, it is possible to just share the
>>>>>> actual usrloc info ( by using the db_cachedb module and storing the
>>>>>> information in a MongoDB cluster ), you can encounter real-life scenarios
>>>>>> where just sharing the info is not enough, like :
>>>>>>      - NAT-ed clients, where only the initial server that received the
>>>>>> Register has the pin-hole open, and thus is the only server that can relay
>>>>>> traffic back to the respective client
>>>>>>      - the user has a SIP client that only accepts traffic from the server
>>>>>> IP that it's currently registered against, and thus would reject direct
>>>>>> traffic from other IPs ( due to security reasons )
>>>>>>
>>>>>> We would like to implement a true general solution for this issue, and
>>>>>> would appreciate your feedback on this. Also we'd appreciate if you could
>>>>>> share the needs that you would have from such a distributed user location
>>>>>> feature, and the scenarios that you would use such a feature in real-life
>>>>>> setups.
>>>>>>
>>>>>>
>>>>>> Best Regards,
>>>>>>
>>>>>> --
>>>>>> Vlad Paiu
>>>>>> OpenSIPS Developer
>>>>>> http://www.opensips-solutions.com
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Users mailing list
>>>>>> Users at lists.opensips.org
>>>>>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>>>>>
>>>>>
>>>>> --
>>>>> Mit freundlichen Grüßen
>>>>> Muhammad Shahzad
>>>>> -----------------------------------
>>>>> CISCO Rich Media Communication Specialist (CRMCS)
>>>>> CISCO Certified Network Associate (CCNA)
>>>>> Cell: +49 176 99 83 10 85
>>>>> MSN: shari_786pk at hotmail.com
>>>>> Email: shaheryarkh at googlemail.com
>>>>>
>>>>> _______________________________________________
>>>>> Users mailing list
>>>>> Users at lists.opensips.org
>>>>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>>>>
>>>>
>>>> --
>>>> Mit freundlichen Grüßen
>>>> Muhammad Shahzad
>>>> -----------------------------------
>>>> CISCO Rich Media Communication Specialist (CRMCS)
>>>> CISCO Certified Network Associate (CCNA)
>>>> Cell: +49 176 99 83 10 85
>>>> MSN: shari_786pk at hotmail.com
>>>> Email: shaheryarkh at googlemail.com
>>>>
>>>> _______________________________________________
>>>> Users mailing list
>>>> Users at lists.opensips.org
>>>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>>>>
>>> _______________________________________________
>>> Users mailing list
>>> Users at lists.opensips.org
>>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>>>
>> _______________________________________________
>> Users mailing list
>> Users at lists.opensips.org
>> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>>
>
> _______________________________________________
> Users mailing list
> Users at lists.opensips.org
> http://lists.opensips.org/cgi-bin/mailman/listinfo/users
>

More information about the Users mailing list