<div class="gmail_quote"><div>Hi Bogdan,</div><div>Thanks for this info. I never thought in this direction. My local DNS server IP had indeed changed a few weeks ago and my secondary DNS server is at a different location which might be causing the delay. I had forgot to change the local DNS server address in which Opensips was running.</div>
<div>I've now changed it yesterday and will keep monitoring this week. This was indeed very very helpful. Thanks a lot !!</div><div><br></div><div>--- Jayesh</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Hi Jayesh,<br>
<br>
The trace shows your opensips is blocked in doing a DNS query.<br>
<br>
I guess you have some domain name which get a very long timeout in query<br>
- this long timeout keeps an opensips processes blocked until timeout<br>
fires or gets an answer<br>
<br>
I suggest to set up a local DNS server (as proxy) to force a very low<br>
timeout on queries (to avoid blocking all opensips processes in DNS).<br>
<br>
This majoe problem is to be fixed in new opensips 2.0 version.<br>
<br>
Regards,<br>
Bogdan<br>
<br>
Jayesh Nambiar wrote:<br>
> Hi Bogdan,<br>
> Tried different things on the script but still the script seems to die<br>
> after some time. "Some Time" could be few hours or few days. I was not<br>
> able to do the gdb and get the backtrace as one of my technician would<br>
> always restart it to get the opensips up and working. I was finally<br>
> able do the gdb and get the backtrace today. I have pasted the<br>
> backtrace, Please check and give me some idea on where to check and<br>
> correct for this?<br>
><br>
> #0 0x0000003f83ac8fdf in poll () from /lib64/libc.so.6<br>
> #1 0x0000003f88608e00 in __libc_res_nsend () from /lib64/libresolv.so.2<br>
> #2 0x0000003f88607c26 in __libc_res_nquery () from /lib64/libresolv.so.2<br>
> #3 0x0000003f88607e96 in __libc_res_nquerydomain () from<br>
> /lib64/libresolv.so.2<br>
> #4 0x0000003f886081c0 in __libc_res_nsearch () from /lib64/libresolv.so.2<br>
> #5 0x00002b0de09ca8cf in _nss_dns_gethostbyname3_r () from<br>
> /lib64/libnss_dns.so.2<br>
> #6 0x00002b0de09caabe in _nss_dns_gethostbyname_r () from<br>
> /lib64/libnss_dns.so.2<br>
> #7 0x0000003f83ae7e71 in gethostbyname_r@@GLIBC_2.2.5 () from<br>
> /lib64/libc.so.6<br>
> #8 0x0000003f83ae77a3 in gethostbyname () from /lib64/libc.so.6<br>
> #9 0x0000000000459a29 in sip_resolvehost (name=0x7fff0d2fda80,<br>
> port=0x800b1a, proto=0x800b1c, is_sips=-1, dn=0x800b40) at resolve.h:349<br>
> #10 0x000000000043fe10 in mk_proxy (name=0x7fff0d2fda80, port=0,<br>
> proto=0, is_sips=0) at proxy.c:252<br>
> #11 0x00002b0d9de09588 in t_relay_to (p_msg=0x7f8ae0, proxy=0x0,<br>
> flags=<value optimized out>) at ut.h:111<br>
> #12 0x00002b0d9de1b706 in w_t_relay (p_msg=0x7f8ae0, proxy=0x0,<br>
> flags=0x0) at tm.c:1079<br>
> #13 0x000000000040ea39 in do_action (a=0x790b70, msg=0x7f8ae0) at<br>
> action.c:967<br>
> #14 0x000000000041150c in run_action_list (a=<value optimized out>,<br>
> msg=0x7f8ae0) at action.c:139<br>
> #15 0x0000000000466a34 in eval_expr (e=0x790c48, msg=0x7f8ae0,<br>
> val=0x0) at route.c:1240<br>
> #16 0x000000000046654d in eval_expr (e=0x790c98, msg=0x7f8ae0,<br>
> val=0x0) at route.c:1556<br>
> #17 0x00000000004664f5 in eval_expr (e=0x790ce8, msg=0x7f8ae0,<br>
> val=0x0) at route.c:1561<br>
> #18 0x000000000040da1d in do_action (a=0x7914d8, msg=0x7f8ae0) at<br>
> action.c:689<br>
> #19 0x000000000041150c in run_action_list (a=<value optimized out>,<br>
> msg=0x7f8ae0) at action.c:139<br>
> #20 0x000000000040f845 in do_action (a=0x78e480, msg=0x7f8ae0) at<br>
> action.c:119<br>
> #21 0x000000000041150c in run_action_list (a=<value optimized out>,<br>
> msg=0x7f8ae0) at action.c:139<br>
> #22 0x000000000040fd13 in do_action (a=0x78fb28, msg=0x7f8ae0) at<br>
> action.c:706<br>
> #23 0x000000000041150c in run_action_list (a=<value optimized out>,<br>
> msg=0x7f8ae0) at action.c:139<br>
> #24 0x0000000000410814 in do_action (a=0x78fc00, msg=0x7f8ae0) at<br>
> action.c:712<br>
> #25 0x000000000041150c in run_action_list (a=<value optimized out>,<br>
> msg=0x7f8ae0) at action.c:139<br>
> #26 0x0000000000410814 in do_action (a=0x78fcd8, msg=0x7f8ae0) at<br>
> action.c:712<br>
> #27 0x000000000041150c in run_action_list (a=<value optimized out>,<br>
> msg=0x7f8ae0) at action.c:139<br>
> #28 0x0000000000411869 in run_top_route (a=0x7880a8, msg=0x7f8ae0) at<br>
> action.c:119<br>
> #29 0x0000000000455fa5 in receive_msg (<br>
> buf=0x754e00 "ACK <a href="mailto:sip%3A611@sip.novanetfone.in">sip:611@sip.novanetfone.in</a><br>
> <mailto:<a href="mailto:sip%253A611@sip.novanetfone.in">sip%3A611@sip.novanetfone.in</a>> SIP/2.0\r\nRecord-Route:<br>
> <sip:203.XXX.53.XXX;lr=on;ftag=100ef178-1c01a8c0-13c4-50029-e41ec-12a04394-e41ec>\r\nRecord-Route:<br>
> <sip:203.XXX.53.XXX;lr=on;ftag=100ef178-1c01a8c0-"..., len=4183,<br>
> rcv_info=0x7fff0d2ffac0) at receive.c:162<br>
> #30 0x000000000049a384 in udp_rcv_loop () at udp_server.c:492<br>
> #31 0x0000000000429c3d in main (argc=3, argv=<value optimized out>) at<br>
> main.c:818<br>
><br>
> Thanks for all the help,<br>
><br>
> --- Jayesh<br>
><br>
> Message: 2<br>
> Date: Fri, 02 Apr 2010 14:09:47 +0300<br>
> From: Bogdan-Andrei Iancu <<a href="mailto:bogdan@voice-system.ro">bogdan@voice-system.ro</a><br>
> <mailto:<a href="mailto:bogdan@voice-system.ro">bogdan@voice-system.ro</a>>><br>
> Subject: Re: [OpenSIPS-Users] Opensips just stops responding<br>
> To: OpenSIPS users mailling list <<a href="mailto:users@lists.opensips.org">users@lists.opensips.org</a><br>
> <mailto:<a href="mailto:users@lists.opensips.org">users@lists.opensips.org</a>>><br>
> Message-ID: <<a href="mailto:4BB5D07B.9060803@voice-system.ro">4BB5D07B.9060803@voice-system.ro</a><br>
> <mailto:<a href="mailto:4BB5D07B.9060803@voice-system.ro">4BB5D07B.9060803@voice-system.ro</a>>><br>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed<br>
><br>
> I see, let me know if without the Localcache module works ok or not.<br>
><br>
> Regards,<br>
> Bogdan<br>
><br>
> Jayesh Nambiar wrote:<br>
> > Hi Bogdan,<br>
> > Thanks for your reply. I will try this the next time my config<br>
> behaves<br>
> > like this.<br>
> > By the way the only change that I did in this lastly was, I removed<br>
> > the "LocalCache module" support and have had the opensips to query<br>
> > the DB for Authentication. It's been 18 hours and till now the<br>
> server<br>
> > is up and running fine.<br>
> > I'll keep a watch and follow the steps mentioned by you !!<br>
> ><br>
> > Thanks,<br>
> ><br>
> > --- Jayesh<br>
> ><br>
> ><br>
> > Message: 2<br>
> > Date: Fri, 02 Apr 2010 11:40:32 +0300<br>
> > From: Bogdan-Andrei Iancu <<a href="mailto:bogdan@voice-system.ro">bogdan@voice-system.ro</a><br>
> <mailto:<a href="mailto:bogdan@voice-system.ro">bogdan@voice-system.ro</a>><br>
> > <mailto:<a href="mailto:bogdan@voice-system.ro">bogdan@voice-system.ro</a> <mailto:<a href="mailto:bogdan@voice-system.ro">bogdan@voice-system.ro</a>>>><br>
> > Subject: Re: [OpenSIPS-Users] Opensips just stops responding<br>
> > To: OpenSIPS users mailling list <<a href="mailto:users@lists.opensips.org">users@lists.opensips.org</a><br>
> <mailto:<a href="mailto:users@lists.opensips.org">users@lists.opensips.org</a>><br>
> > <mailto:<a href="mailto:users@lists.opensips.org">users@lists.opensips.org</a><br>
> <mailto:<a href="mailto:users@lists.opensips.org">users@lists.opensips.org</a>>>><br>
> > Message-ID: <<a href="mailto:4BB5AD80.80204@voice-system.ro">4BB5AD80.80204@voice-system.ro</a><br>
> <mailto:<a href="mailto:4BB5AD80.80204@voice-system.ro">4BB5AD80.80204@voice-system.ro</a>><br>
> > <mailto:<a href="mailto:4BB5AD80.80204@voice-system.ro">4BB5AD80.80204@voice-system.ro</a><br>
> <mailto:<a href="mailto:4BB5AD80.80204@voice-system.ro">4BB5AD80.80204@voice-system.ro</a>>>><br>
> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed<br>
> ><br>
> > Hi Jayesh,<br>
> ><br>
> > Could you try to use gdb to attached to the unresponsive<br>
> processes and<br>
> > to see where they are stuck ? Use "gdb /path/opensips PID"<br>
> and do "bt"<br>
> > to get backtrace. Check for processes responsible for the<br>
> network read<br>
> > (do an "opensipsctl fifo ps" to see which are the UDP listener<br>
> > processes)<br>
> ><br>
> > Regards,<br>
> > Bogdan<br>
> ><br>
> > Jayesh Nambiar wrote:<br>
> > > Hi all,<br>
> > > I am running Opensips 1.6.2 and am running a strange<br>
> problem of<br>
> > > Opensips not responding to any SIP messages after a day or<br>
> two. It<br>
> > > needs to be restarted for it to get working again.<br>
> > > The overview of my setup is as follows:<br>
> > > 1) I am using Localcache module to authorize REGISTER and<br>
> INVITE<br>
> > > requests from cache.<br>
> > > 2) Using Mediaproxy for Nated clients with Media-Relay on a<br>
> > different<br>
> > > server<br>
> > > 3) Running basic queries based on INVITE before placing<br>
> the call.<br>
> > ><br>
> > > After running for a day or two, Opensips just stops responding<br>
> > to any<br>
> > > SIP message. I check moni, it shows me the server is alive and<br>
> > in_use<br>
> > > transactions keep on increasing since Opensips is not<br>
> processing<br>
> > > anyone. I checked get_statistics shmem: even that is<br>
> perfectly ok.<br>
> > > I have assigned 1GB to shmem and hardly 30 MB is used. I dont<br>
> > even see<br>
> > > any error logs etc. The top command shows no over<br>
> utilization of CPU<br>
> > > and memory.<br>
> > ><br>
> > > Can someone point me on where and how to check for this<br>
> problem. Has<br>
> > > someone had a similar problem before. It looks similar to this<br>
> > > post<br>
> ><br>
> <a href="http://www.mail-archive.com/users@lists.opensips.org/msg06098.html" target="_blank">http://www.mail-archive.com/users@lists.opensips.org/msg06098.html</a> but<br>
> > > I dont see a solution to this either.<br>
> > ><br>
> > > Thanks for any help.<br>
> > ><br>
> > > --- Jayesh<br>
> > ><br>
> > ><br>
> ><br>
> ------------------------------------------------------------------------<br>
> > ><br>
> > > _______________________________________________<br>
> > > Users mailing list<br>
> > > <a href="mailto:Users@lists.opensips.org">Users@lists.opensips.org</a> <mailto:<a href="mailto:Users@lists.opensips.org">Users@lists.opensips.org</a>><br>
> <mailto:<a href="mailto:Users@lists.opensips.org">Users@lists.opensips.org</a> <mailto:<a href="mailto:Users@lists.opensips.org">Users@lists.opensips.org</a>>><br>
> > > <a href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users" target="_blank">http://lists.opensips.org/cgi-bin/mailman/listinfo/users</a><br>
> ><br>
> ><br>
> ------------------------------------------------------------------------<br>
> ><br>
> > _______________________________________________<br>
> > Users mailing list<br>
> > <a href="mailto:Users@lists.opensips.org">Users@lists.opensips.org</a> <mailto:<a href="mailto:Users@lists.opensips.org">Users@lists.opensips.org</a>><br>
> > <a href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users" target="_blank">http://lists.opensips.org/cgi-bin/mailman/listinfo/users</a><br>
> ><br>
><br>
><br>
> --<br>
> Bogdan-Andrei Iancu<br>
> <a href="http://www.voice-system.ro" target="_blank">www.voice-system.ro</a> <<a href="http://www.voice-system.ro" target="_blank">http://www.voice-system.ro</a>><br>
><br>
> ------------------------------------------------------------------------<br>
><br>
> _______________________________________________<br>
> Users mailing list<br>
> <a href="mailto:Users@lists.opensips.org">Users@lists.opensips.org</a><br>
> <a href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users" target="_blank">http://lists.opensips.org/cgi-bin/mailman/listinfo/users</a><br>
><br>
<br>
<br>
--<br>
Bogdan-Andrei Iancu<br>
<a href="http://www.voice-system.ro" target="_blank">www.voice-system.ro</a><br></blockquote></div>