[OpenSIPS-Devel] [OpenSIPS-Users] python module - bug and questions

[Mundkowsky, Robert]

Overall, I think trying to use threads, even “Python threads” across multiple processes is a bad idea.  Threads are usually meant to live inside one process.

Robert Mundkowsky

From: Mundkowsky, Robert
Sent: Tuesday, April 25, 2017 1:26 PM
To: 'Maxim Sobolev' <sobomax at sippysoft.com>
Cc: OpenSIPS devel mailling list <devel at lists.opensips.org>; Bogdan-Andrei Iancu <bogdan at opensips.org>
Subject: RE: [OpenSIPS-Users] python module - bug and questions

Thanks for responding.  Thought you would just advise me to use specific compiler flags first.

Do you want me to use your jig (testing setup) to make a crash?
Or do you want me to add python code to your git repo so you can run it using your jig?


What you are doing with code is pretty complex for me and makes my brain want to explode.

My understanding is:

1)      Python threads are not real threads, instead only one python thread runs at a time (per the  global interpreter lock)



2)      You are creating a Python thread (that uses one Python interpreter) in the main openSIPS process that likely loads the openSIPS user’s Python module to be used

3)      Then, I assume, openSIPS main process forks children processes

a.       I assume Python thread and interpreter  is treated as data and code that is copied to all children processes.  Note if it were a real Linux thread then it would not be copied to children processes.

b.      You then use the copied Python thread in all children processes as if it is a single Python thread across the main and child processes or as if it were a single Python thread per each main or child process

I guess this might work, because if the Python thread is truly copied then you really have a different Python thread and interpreter per main and child processes. But Python documentation (https://docs.python.org/2/c-api/init.html#c.PyThreadState) states “When extending or embedding Python, there is no way to inform Python of additional (non-Python) locks that need to be acquired before or reset after a fork”, which makes me think what you are doing will not work.  This is because any Python modules someone might use (e.g. MySQLdb) might use a lock in the main process that can’t get cleared in the child process.

Might be wiser to just start one Python thread and interpreter in each child process I assume is kicked off via child_init.  Then you would still have the module loaded once per child.  Although I am not sure if main process needs a Python thread, but probably not.


Robert Mundkowsky

From: Maxim Sobolev [mailto:sobomax at sippysoft.com]
Sent: Monday, April 24, 2017 10:09 PM
To: Mundkowsky, Robert <rmundkowsky at ets.org<mailto:rmundkowsky at ets.org>>
Cc: OpenSIPS devel mailling list <devel at lists.opensips.org<mailto:devel at lists.opensips.org>>; Bogdan-Andrei Iancu <bogdan at opensips.org<mailto:bogdan at opensips.org>>
Subject: RE: [OpenSIPS-Users] python module - bug and questions

Robert sorry for the delay. What could be useful if you can produce chunk of python routing code (could be NOP on the functional side) that crashes repeatedly on our voiptest jig (https://github.com/sippy/voiptests). The travis is setup to build pull requests, so that you can inject your code and let it rip. It uses similar configiration of linux / python.

-Max

________________________________

This e-mail and any files transmitted with it may contain privileged or confidential information. It is solely for use by the individual for whom it is intended, even if addressed incorrectly. If you received this e-mail in error, please notify the sender; do not disclose, copy, distribute, or take any action in reliance on the contents of this information; and delete it from your system. Any other use of this e-mail is prohibited.


Thank you for your compliance.

________________________________
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opensips.org/pipermail/devel/attachments/20170425/c4f799af/attachment-0001.html>