<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
tt
{mso-style-priority:99;
font-family:"Courier New";}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Consolas;}
p.m7103645024965926472msolistparagraph, li.m7103645024965926472msolistparagraph, div.m7103645024965926472msolistparagraph
{mso-style-name:m_7103645024965926472msolistparagraph;
mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
p.m7103645024965926472m7329958631548415217m-2679145434075060159msolistparagraph, li.m7103645024965926472m7329958631548415217m-2679145434075060159msolistparagraph, div.m7103645024965926472m7329958631548415217m-2679145434075060159msolistparagraph
{mso-style-name:m_7103645024965926472m7329958631548415217m-2679145434075060159msolistparagraph;
mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
span.m7103645024965926472m7329958631548415217m-2679145434075060159pl-c1
{mso-style-name:m_7103645024965926472m7329958631548415217m-2679145434075060159pl-c1;}
span.EmailStyle24
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:#1F497D;}
span.pl-en
{mso-style-name:pl-en;}
span.EmailStyle26
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><a name="_MailEndCompose"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">I don’t have time to setup your test jig right now. But attached are the snippets of code that fail.<o:p></o:p></span></a></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">You will need to the following to use them:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal" style="text-indent:.5in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">I am using Python 2.7.13 (via conda 4.2.13).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> You will need a MySQL database.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal" style="text-indent:.5in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">sudo apt-get install -y libmysqlclient-dev<o:p></o:p></span></p>
<p class="MsoNormal" style="text-indent:.5in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">sudo apt-get install -y mysql-client-core-5.7 python-mysqldb<o:p></o:p></span></p>
<p class="MsoNormal" style="text-indent:.5in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal" style="text-indent:.5in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">conda install MySQL-python OR sudo apt-get -y install python-pip; pip install --upgrade pip<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> You need to make sure opensips uses correct python (it is in PATH) and make sure python modules are reachable (in PYTHONPATH), hence if you use
/etc/init.d/opensips then add those environment variables there.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal" style="text-indent:.5in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Opensips modules:<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-indent:.5in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">db_mysql module<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-indent:.5in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">python module<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> opensips.cfg<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-indent:.5in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">loadmodule "python.so"<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-indent:.5in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">modparam("python", "script_name", "call.py")<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-indent:.5in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">modparam("python", "mod_init_function", "mod_init")<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-indent:.5in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">modparam("python", "child_init_method", "sip_child_init")<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-indent:.5in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-indent:.5in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">#put this anywhere it will get called<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-indent:.5in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">python_exec('incrementMissedCallsCountBackGround');<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-indent:.5in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-indent:.5in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">#or
<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-indent:.5in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">python_exec('incrementMissedCallsCount');<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-indent:.5in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Then just call opensips twice in a row or concurrently. Seemed to fail more than half of the time.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Robert Mundkowsky<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> Mundkowsky, Robert
<br>
<b>Sent:</b> Tuesday, April 25, 2017 4:57 PM<br>
<b>To:</b> 'Maxim Sobolev' <sobomax@sippysoft.com><br>
<b>Cc:</b> OpenSIPS devel mailling list <devel@lists.opensips.org>; Bogdan-Andrei Iancu <bogdan@opensips.org><br>
<b>Subject:</b> RE: [OpenSIPS-Users] python module - bug and questions<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">I am guessing this is not the case. My guess is
</span><span class="pl-en">mod_init</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> is called once by main/master openSIPS process and
</span><span class="pl-en">child_init</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> is called for each child openSIPS process.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Hence you should modify your C code not to initialize the Python stuff in
</span><span class="pl-en">mod_init</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">, but instead in
</span><span class="pl-en">child_init</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">. And your Python code would also follow that change:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal" style="text-indent:.5in">def child_init(rank):<o:p></o:p></p>
<p class="MsoNormal" style="text-indent:.5in"> inst = OpenSIPSHandler()<o:p></o:p></p>
<p class="MsoNormal" style="text-indent:.5in"> return inst<o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">If you do this, then you get a separate Python thread and interpreter per child. Right now, you are getting a Python thread and interpreter that was started
in a different process (the parent openSIPS process) and some of this is copied to the children and it kind of works, but some things like thread locks and real threads started by the Python thread are not copied to the child processes.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Robert Mundkowsky<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> Maxim Sobolev [<a href="mailto:sobomax@sippysoft.com">mailto:sobomax@sippysoft.com</a>]
<br>
<b>Sent:</b> Tuesday, April 25, 2017 3:58 PM<br>
<b>To:</b> Mundkowsky, Robert <<a href="mailto:rmundkowsky@ets.org">rmundkowsky@ets.org</a>><br>
<b>Cc:</b> OpenSIPS devel mailling list <<a href="mailto:devel@lists.opensips.org">devel@lists.opensips.org</a>>; Bogdan-Andrei Iancu <<a href="mailto:bogdan@opensips.org">bogdan@opensips.org</a>><br>
<b>Subject:</b> Re: [OpenSIPS-Users] python module - bug and questions<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">Robert, what you have described is how essentially python module works. We init the interpreter and then the opensips forks, and in each of its children the mod_init() python function exported by your code is called, which returns instance
whose method can be invoked via script. So at the end of the day you end up with as many independent processes as there are opensips children.<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Here for example what we have in our handler:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<div>
<p class="MsoNormal">from OpenSIPS import LM_ERR<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">from ser import SerHandler<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">class OpenSIPSHandler(SerHandler):<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> def www_authenticate(self, msg):<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">...<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> return rval<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">def mod_init():<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> inst = OpenSIPSHandler()<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> return inst<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">-Max<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">On Tue, Apr 25, 2017 at 10:25 AM, Mundkowsky, Robert <<a href="mailto:rmundkowsky@ets.org" target="_blank">rmundkowsky@ets.org</a>> wrote:<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><a name="m_7103645024965926472__MailEndCompose"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Thanks for responding. Thought you would just advise
me to use specific compiler flags first.</span></a><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Do you want me to use your jig (testing setup) to make a crash?
</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-indent:.5in">
<span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Or do you want me to add python code to your git repo so you can run it using your jig?</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">What you are doing with code is pretty complex for me and makes my brain want to explode.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">My understanding is:</span><o:p></o:p></p>
<p class="m7103645024965926472msolistparagraph"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">1)</span><span style="font-size:7.0pt;color:#1F497D">
</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Python threads are not real threads, instead only one python thread runs at a time (per the global interpreter lock)</span><o:p></o:p></p>
<p class="m7103645024965926472msolistparagraph"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
<p class="m7103645024965926472msolistparagraph"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">2)</span><span style="font-size:7.0pt;color:#1F497D">
</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">You are creating a Python thread (that uses one Python interpreter) in the main openSIPS process that likely loads the openSIPS user’s Python module to be used</span><o:p></o:p></p>
<p class="m7103645024965926472msolistparagraph"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">3)</span><span style="font-size:7.0pt;color:#1F497D">
</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Then, I assume, openSIPS main process forks children processes</span><o:p></o:p></p>
<p class="m7103645024965926472msolistparagraph" style="margin-left:1.0in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">a.</span><span style="font-size:7.0pt;color:#1F497D">
</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">I assume Python thread and interpreter is treated as data and code that is copied to all children processes. Note if it were a real Linux thread then it would not be copied
to children processes.</span><o:p></o:p></p>
<p class="m7103645024965926472msolistparagraph" style="margin-left:1.0in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">b.</span><span style="font-size:7.0pt;color:#1F497D">
</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">You then use the copied Python thread in all children processes as if it is a single Python thread across the main and child processes or as if it were a single Python thread
per each main or child process</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">I guess this might work, because if the Python thread is truly copied then you really have a different
Python thread and interpreter per main and child processes. But Python documentation (</span><a href="https://docs.python.org/2/c-api/init.html#c.PyThreadState" target="_blank"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">https://docs.python.org/2/c-api/init.html#c.PyThreadState</span></a><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">)
states “When extending or embedding Python, there is no way to inform Python of additional (non-Python) locks that need to be acquired before or reset after a fork”, which makes me think what you are doing will not work. This is because any Python modules
someone might use (e.g. MySQLdb) might use a lock in the main process that can’t get cleared in the child process.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Might be wiser to just start one Python thread and interpreter in each child process I assume is kicked
off via child_init. Then you would still have the module loaded once per child. Although I am not sure if main process needs a Python thread, but probably not.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Robert Mundkowsky</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> Maxim Sobolev [mailto:<a href="mailto:sobomax@sippysoft.com" target="_blank">sobomax@sippysoft.com</a>]
<br>
<b>Sent:</b> Monday, April 24, 2017 10:09 PM<br>
<b>To:</b> Mundkowsky, Robert <<a href="mailto:rmundkowsky@ets.org" target="_blank">rmundkowsky@ets.org</a>><br>
<b>Cc:</b> OpenSIPS devel mailling list <<a href="mailto:devel@lists.opensips.org" target="_blank">devel@lists.opensips.org</a>>; Bogdan-Andrei Iancu <<a href="mailto:bogdan@opensips.org" target="_blank">bogdan@opensips.org</a>><br>
<b>Subject:</b> RE: [OpenSIPS-Users] python module - bug and questions</span><o:p></o:p></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Robert sorry for the delay. What could be useful if you can produce chunk of python routing code (could be NOP on the functional side) that crashes repeatedly on our voiptest jig
(<a href="https://github.com/sippy/voiptests" target="_blank">https://github.com/sippy/voiptests</a>). The travis is setup to build pull requests, so that you can inject your code and let it rip. It uses similar configiration of linux / python. <o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">-Max<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">On Apr 21, 2017 9:32 AM, "Mundkowsky, Robert" <<a href="mailto:rmundkowsky@ets.org" target="_blank">rmundkowsky@ets.org</a>> wrote:<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><a name="m_7103645024965926472_m_7329958631548415"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Python 2.7.13 on Ubuntu 16.04.2 LTS.</span></a><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">How do you link using pthreads?</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-indent:.5in">
<span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Do you add flags to Makefile, or will setting “USE_PTHREAD_MUTEX” and “USE_POSIX_SEM” via menuconfig do it?</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Since openSIPS is not multithread, seems like it would be a lot cleaner to not use threads at all on
C/C++. Then threads could still be used on Python side if needed.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Robert Mundkowsky</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> Maxim Sobolev [mailto:<a href="mailto:sobomax@sippysoft.com" target="_blank">sobomax@sippysoft.com</a>]
<br>
<b>Sent:</b> Friday, April 21, 2017 11:57 AM<br>
<b>To:</b> Bogdan-Andrei Iancu <<a href="mailto:bogdan@opensips.org" target="_blank">bogdan@opensips.org</a>><br>
<b>Cc:</b> OpenSIPS users mailling list <<a href="mailto:users@lists.opensips.org" target="_blank">users@lists.opensips.org</a>>; Mundkowsky, Robert <<a href="mailto:rmundkowsky@ets.org" target="_blank">rmundkowsky@ets.org</a>><br>
<b>Subject:</b> Re: [OpenSIPS-Users] python module - bug and questions</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Robert, what Bogdan says is essentially correct. The OpenSIPS itself is not using any threads AFAIK, therefore python module code is kept as simple as possible. Now back to the
original question: we use quite a lot of python code in our routing and some of the python modules that are running are actually creating threads on their own and it appears to be pretty stable. What OS / python version are you using? What could be different
in our case is that we link our OpenSIPS binary with pthreads always (which may or may not be the case on your build), so there might be some vital threads runtime infrastructure that is not getting initialized in your case.<o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">-Max<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">On Fri, Apr 21, 2017 at 1:25 AM, Bogdan-Andrei Iancu <<a href="mailto:bogdan@opensips.org" target="_blank">bogdan@opensips.org</a>> wrote:<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;margin-bottom:12.0pt"><tt><span style="font-size:10.0pt">Hi Robert,</span></tt><span style="font-size:10.0pt;font-family:"Courier New""><br>
<br>
<tt>The only question I can answer is 1) - OpenSIPS it is a multi-process application (and not using threads).
</tt><br>
<br>
<tt>How the python module is design (from threading perspective), I do not know - maybe Maxim, the author of this module can help with this.</tt><br>
<br>
<tt>Regards,</tt></span><o:p></o:p></p>
<pre>Bogdan-Andrei Iancu<o:p></o:p></pre>
<pre> OpenSIPS Founder and Developer<o:p></o:p></pre>
<pre> <a href="http://www.opensips-solutions.com" target="_blank">http://www.opensips-solutions.com</a><o:p></o:p></pre>
<pre> <o:p></o:p></pre>
<pre>OpenSIPS Summit May 2017 Amsterdam<o:p></o:p></pre>
<pre> <a href="http://www.opensips.org/events/Summit-2017Amsterdam.html" target="_blank">http://www.opensips.org/events/Summit-2017Amsterdam.html</a><o:p></o:p></pre>
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">On 04/19/2017 08:43 PM, Mundkowsky, Robert wrote:<o:p></o:p></p>
</div>
</div>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Hi,<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">This email should likely go to your other dev forum, but I don’t have access yet.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">I am using openSIPS 2.2.3 on Ubuntu 16.04.2 LTS.
<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">I am using the python module. The python script called works fine when called from outside openSIPS and it works fine when called inside openSIPS, if it is triggered once at a
time.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">But if the python script is triggered twice by two phone calls or more or even one call after another in short order, then there are weird errors which show up in different places
in the code like:<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-indent:.5in">
ERROR:python:python_handle_exception: #011TypeError: an integer is required<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">I am guessing the openSIP python module has some problem/s related to threading.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-indent:.5in">
<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Questions:<o:p></o:p></p>
<p class="m7103645024965926472m7329958631548415217m-2679145434075060159msolistparagraph">
1)<span style="font-size:7.0pt"> </span>I am guessing that OpenSIPS uses a multi-threaded architecture?<o:p></o:p></p>
<p class="m7103645024965926472m7329958631548415217m-2679145434075060159msolistparagraph">
2)<span style="font-size:7.0pt"> </span>Is the python module meant to be a single thread? Or multi-threaded?<o:p></o:p></p>
<p class="m7103645024965926472m7329958631548415217m-2679145434075060159msolistparagraph" style="margin-left:1.0in">
a.<span style="font-size:7.0pt"> </span>If it is meant to be single threaded then why use python library thread functions? Just use python library without multi-threaded stuff.<o:p></o:p></p>
<p class="m7103645024965926472m7329958631548415217m-2679145434075060159msolistparagraph" style="margin-left:1.0in">
<span class="m7103645024965926472m7329958631548415217m-2679145434075060159pl-c1">b.</span><span class="m7103645024965926472m7329958631548415217m-2679145434075060159pl-c1"><span style="font-size:7.0pt">
</span></span>If it is meant to be multi-threaded then why is there only one call to
<span class="m7103645024965926472m7329958631548415217m-2679145434075060159pl-c1">
PyThreadState_New and </span>myThreadState is a global used everywhere?<o:p></o:p></p>
<p class="m7103645024965926472m7329958631548415217m-2679145434075060159msolistparagraph" style="margin-left:1.5in">
<span style="font-size:7.0pt">
</span>i. I would think a thread per call to python_exec would make more sense and make the code easier to understand?<o:p></o:p></p>
<p class="m7103645024965926472m7329958631548415217m-2679145434075060159msolistparagraph">
3)<span style="font-size:7.0pt"> </span>Also why is there no clean up code (PyThreadState_Clear, PyThreadState_Delete, Py_Finalize)?<o:p></o:p></p>
<p class="m7103645024965926472m7329958631548415217m-2679145434075060159msolistparagraph" style="margin-left:1.0in">
a.<span style="font-size:7.0pt"> </span>I am guessing the idea is you do not need clean up, because it only happens when openSIPS is turned off.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Robert<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">For reference:<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-indent:.5in">
<a href="https://www.codeproject.com/articles/11805/embedding-python-in-c-c-part-i" target="_blank">https://www.codeproject.com/articles/11805/embedding-python-in-c-c-part-i</a><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-indent:.5in">
<a href="http://www.awasu.com/weblog/embedding-python/threads/" target="_blank">http://www.awasu.com/weblog/embedding-python/threads/</a><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-indent:.5in">
<a href="http://stackoverflow.com/questions/26061298/python-multi-thread-multi-interpreter-c-api" target="_blank">http://stackoverflow.com/questions/26061298/python-multi-thread-multi-interpreter-c-api</a><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="2" width="100%" align="center">
</div>
<p>This e-mail and any files transmitted with it may contain privileged or confidential information. It is solely for use by the individual for whom it is intended, even if addressed incorrectly. If you received this e-mail in error, please notify the sender;
do not disclose, copy, distribute, or take any action in reliance on the contents of this information; and delete it from your system. Any other use of this e-mail is prohibited.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p>Thank you for your compliance.<o:p></o:p></p>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="2" width="100%" align="center">
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;margin-bottom:12.0pt"> <o:p></o:p></p>
</div>
</div>
<pre>_______________________________________________<o:p></o:p></pre>
<pre>Users mailing list<o:p></o:p></pre>
<pre><a href="mailto:Users@lists.opensips.org" target="_blank">Users@lists.opensips.org</a><o:p></o:p></pre>
<pre><a href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users" target="_blank">http://lists.opensips.org/cgi-bin/mailman/listinfo/users</a><o:p></o:p></pre>
</blockquote>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
</blockquote>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><br>
<br clear="all">
<o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">--
<o:p></o:p></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Maksym Sobolyev<br>
Sippy Software, Inc.<br>
Internet Telephony (VoIP) Experts<br>
Tel (Canada): <a href="tel:(778)%20783-0474" target="_blank">+1-778-783-0474</a><br>
Tel (Toll-Free): <a href="tel:(855)%20747-7779" target="_blank">+1-855-747-7779</a><br>
Fax: <a href="tel:(866)%20857-6942" target="_blank">+1-866-857-6942</a><br>
Web: <a href="http://www.sippysoft.com" target="_blank">http://www.sippysoft.com</a><br>
MSN: <a href="mailto:sales@sippysoft.com" target="_blank">sales@sippysoft.com</a><br>
Skype: SippySoft<o:p></o:p></p>
</div>
</div>
</div>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="2" width="100%" align="center">
</div>
<p>This e-mail and any files transmitted with it may contain privileged or confidential information. It is solely for use by the individual for whom it is intended, even if addressed incorrectly. If you received this e-mail in error, please notify the sender;
do not disclose, copy, distribute, or take any action in reliance on the contents of this information; and delete it from your system. Any other use of this e-mail is prohibited.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p>Thank you for your compliance.<o:p></o:p></p>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="2" width="100%" align="center">
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</div>
<div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="2" width="100%" align="center">
</div>
<p>This e-mail and any files transmitted with it may contain privileged or confidential information. It is solely for use by the individual for whom it is intended, even if addressed incorrectly. If you received this e-mail in error, please notify the sender;
do not disclose, copy, distribute, or take any action in reliance on the contents of this information; and delete it from your system. Any other use of this e-mail is prohibited.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p>Thank you for your compliance.<o:p></o:p></p>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="2" width="100%" align="center">
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><br>
<br clear="all">
<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal">-- <o:p></o:p></p>
<div>
<div>
<p class="MsoNormal">Maksym Sobolyev<br>
Sippy Software, Inc.<br>
Internet Telephony (VoIP) Experts<br>
Tel (Canada): +1-778-783-0474<br>
Tel (Toll-Free): +1-855-747-7779<br>
Fax: +1-866-857-6942<br>
Web: <a href="http://www.sippysoft.com" target="_blank">http://www.sippysoft.com</a><br>
MSN: <a href="mailto:sales@sippysoft.com" target="_blank">sales@sippysoft.com</a><br>
Skype: SippySoft<o:p></o:p></p>
</div>
</div>
</div>
</div>
<br>
<hr>
<p>This e-mail and any files transmitted with it may contain privileged or confidential information. It is solely for use by the individual for whom it is intended, even if addressed incorrectly. If you received this e-mail in error, please notify the sender;
do not disclose, copy, distribute, or take any action in reliance on the contents of this information; and delete it from your system. Any other use of this e-mail is prohibited.</p>
<br>
<p>Thank you for your compliance.</p>
<hr>
</body>
</html>