<div><div>Hi Saúl,</div></div><br><div class="gmail_quote">On Fri, Feb 8, 2013 at 3:24 AM, Saúl Ibarra Corretgé <span dir="ltr"><<a href="mailto:saul@ag-projects.com" target="_blank">saul@ag-projects.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Jeff,<br>
<div><div><br>
On Feb 7, 2013, at 10:52 PM, Jeff Pyle wrote:<br>
<br>
> Hello,<br>
><br>
> On a media relay from Mediaproxy 2.4.4, we've noticed a lot of errors like the following today:<br>
><br>
> media-relay[10351]: Traceback (most recent call last):<br>
> media-relay[10351]: File "/usr/lib/python2.5/site-packages/twisted/internet/udp.py", line 126, in doRead<br>
> media-relay[10351]: self.protocol.datagramReceived(data, addr)<br>
> media-relay[10351]: File "/usr/lib/pymodules/python2.5/mediaproxy/mediacontrol.py", line 130, in datagramReceived<br>
> media-relay[10351]: self.cb_func(host, port, data)<br>
> media-relay[10351]: File "/usr/lib/pymodules/python2.5/mediaproxy/mediacontrol.py", line 226, in got_data<br>
> media-relay[10351]: self.substream.send_data(self, data, is_stun)<br>
> media-relay[10351]: File "/usr/lib/pymodules/python2.5/mediaproxy/mediacontrol.py", line 311, in send_data<br>
> media-relay[10351]: dest.listener.protocol.send(data, is_stun)<br>
> media-relay[10351]: File "/usr/lib/pymodules/python2.5/mediaproxy/mediacontrol.py", line 159, in send<br>
> media-relay[10351]: self.transport.write(data, (ip, port))<br>
> media-relay[10351]: File "/usr/lib/python2.5/site-packages/twisted/internet/udp.py", line 155, in write<br>
> media-relay[10351]: return self.socket.sendto(datagram, addr)<br>
> media-relay[10351]: error: (1, 'Operation not permitted')<br>
><br>
> Is there a limit within the OS that might cause this if media sessions exceeded a certain threshold? It seems to happen around 500 calls.<br>
><br>
<br>
</div></div>I have never seen this. The error you are getting is EPERM, which is kind of odd for UDP. Do you have any firewall on that machine?<br></blockquote><div><br></div><div>Just iptables. The OUTPUT chain's policy on all tables is ACCEPT, and we have no OUTPUT rules (only INPUT). OUTPUT is wide open.</div>
<div><br></div><div>We did find this in /usr/share/pyshared/twisted/internet/process.py:</div><div><br></div><div><div> 601 try:</div><div> 602 import resource</div><div> 603 maxfds = resource.getrlimit(resource.RLIMIT_NOFILE)[1] + 1</div>
<div> 604 # OS-X reports 9223372036854775808. That's a lot of fds to close</div><div> 605 if maxfds > 1024:</div><div> 606 maxfds = 1024</div><div> 607 except:</div>
<div> 608 maxfds = 256</div></div><div><br></div><div>The 1024 was interesting because we seemed to be having the problem around 500 calls, possibly 512? This is from python-twisted 8.1.0. We tried to adjust line 606 to maxfds = 4096, and make what seemed like appropriate updates in /etc/security/limits.conf:</div>
<div><br></div><div><div>root<span class="Apple-tab-span" style="white-space:pre"> </span>soft<span class="Apple-tab-span" style="white-space:pre"> </span>nofile<span class="Apple-tab-span" style="white-space:pre"> </span>1024</div>
<div>root<span class="Apple-tab-span" style="white-space:pre"> </span>hard<span class="Apple-tab-span" style="white-space:pre"> </span>nofile<span class="Apple-tab-span" style="white-space:pre"> </span>16384 # was 1024</div>
</div><div><br></div><div>This is all new to us so it's very possible we're on the wrong track.</div><div><br></div><div>Today we split the traffic between two relays. Each handles around 425 calls at peak usage. We made these twisted+limits updates on one relay only. The quantity of the Traceback errors is lower at 22 so far today on one relay, and 3 on the other. The first apparence was with less than 100 calls on the relay. The two relay machines have the same configurations, except for IP address. I don't have an explanation why they are higher on one relay than the other.</div>
<div><br></div><div><br></div><div>- Jeff</div><div><br></div></div>