[OpenSIPS-Users] Opensips 2.2.2 memory issue?

Vasilios Tzanoudakis vasilios.tzanoudakis at voiceland.gr
Wed Mar 29 05:25:35 EDT 2017 

Dear all,

After 11 days os usage Opensips process was not responding at all.

This is the second time I am facing this kind of issue.
I am sending you all the information below:

System has 4 CPU Cores / 4 GB Ram

Opensips version 2.2.2

*Run parameters : *
S_MEMORY=2048
P_MEMORY=1024

*uname -a*
Linux 4.4.0-62-generic #83~14.04.1-Ubuntu SMP Wed Jan 18 18:10:30 UTC 2017
x86_64 x86_64 x86_64 GNU/Linux

*Compiled with the following flags:*

PKG_MALLOC
SHM_MMAP
USE_MCAST
DISABLE_NAGLE
STATISTICS
HAVE_RESOLV_RES
F_MALLOC
F_MALLOC_OPTIMIZATIONS
NO_DEBUG

*and with the following extra modules : *

db_mysql
dialplan
regex
snmpstats

*Loaded modules in opensips.cfg*

loadmodule "proto_udp.so"
loadmodule "mi_datagram.so"
loadmodule "signaling.so"
loadmodule "sl.so"
loadmodule "tm.so"
loadmodule "rr.so"
loadmodule "maxfwd.so"
loadmodule "sipmsgops.so"
loadmodule "mi_fifo.so"
loadmodule "uri.so"
loadmodule "db_mysql.so"
loadmodule "usrloc.so"
loadmodule "registrar.so"
loadmodule "auth.so"
loadmodule "auth_db.so"
loadmodule "dialog.so"
loadmodule "nathelper.so"
loadmodule "rtpproxy.so"
loadmodule "drouting.so"
loadmodule "uac.so"

*/usr/local/opensips2/sbin/opensipsctl fifo ps*

Process::  ID=0 PID=1843 Type=attendant
Process::  ID=1 PID=1845 Type=MI FIFO
Process::  ID=2 PID=1846 Type=MI Datagram
Process::  ID=3 PID=1847 Type=time_keeper
Process::  ID=4 PID=1849 Type=timer
Process::  ID=5 PID=1850 Type=SIP receiver udp:X.X.X.X:65100
Process::  ID=6 PID=1851 Type=SIP receiver udp:X.X.X.X:65100
Process::  ID=7 PID=1852 Type=SIP receiver udp:X.X.X.X:65100
Process::  ID=8 PID=1853 Type=SIP receiver udp:X.X.X.X:65100
Process::  ID=9 PID=1854 Type=SIP receiver udp:X.X.X.X:65100
Process::  ID=10 PID=1855 Type=SIP receiver udp:X.X.X.X:65100
Process::  ID=11 PID=1856 Type=SIP receiver udp:X.X.X.X:65100
Process::  ID=12 PID=1857 Type=SIP receiver udp:X.X.X.X:65100
Process::  ID=13 PID=1859 Type=Timer handler

*gdb /usr/local/opensips2/sbin/opensips 1851*

Reading symbols from /lib/x86_64-linux-gnu/libgcc_s.so.1...(no debugging
symbols found)...done.
Loaded symbols for /lib/x86_64-linux-gnu/libgcc_s.so.1
Reading symbols from /lib/x86_64-linux-gnu/libnss_dns.so.2...Reading
symbols from /usr/lib/debug//lib/x86_64-linux-gnu/libnss_dns-2.19.so...done.
done.
Loaded symbols for /lib/x86_64-linux-gnu/libnss_dns.so.2
0x00007faee52c36cd in parse_backslash (offset=0x7ffc00000000,
words=<optimized out>, max_length=0x7ffc32c06040, word_length=0xd,
word=0x636f732e3379786f) at wordexp.c:222
222 wordexp.c: No such file or directory.


*(gdb) bt*
#0  0x00007faee52c36cd in parse_backslash (offset=0x7ffc00000000,
words=<optimized out>, max_length=0x7ffc32c06040, word_length=0xd,
word=0x636f732e3379786f) at wordexp.c:222
#1  parse_backtick (ifs_white=0x722f7261762f0001 <error: *Cannot access
memory at address* 0x722f7261762f0001>, ifs=0x7fae24ccc960 "",
pwordexp=0x838fd4 <buf+916>, flags=0, offset=0x7ffc00000000,
    words=<optimized out>, max_length=0x1, word_length=0x7ffc32c06060,
word=0x7ffc32c060c0) at wordexp.c:2147
#2  wordexp (words=<optimized out>, pwordexp=<optimized out>, flags=0) at
wordexp.c:2359
#3  0x00007ffc32c06220 in ?? ()
#4  0x0000000000000000 in ?? ()


*(gdb) bt full*
#0  0x00007faee52c36cd in parse_backslash (offset=0x7ffc00000000,
words=<optimized out>, max_length=0x7ffc32c06040, word_length=0xd,
word=0x636f732e3379786f) at wordexp.c:222
No locals.
#1  parse_backtick (ifs_white=0x722f7261762f0001 <error: *Cannot access
memory at address *0x722f7261762f0001>, ifs=0x7fae24ccc960 "",
pwordexp=0x838fd4 <buf+916>, flags=0, offset=0x7ffc00000000,
    words=<optimized out>, max_length=0x1, word_length=0x7ffc32c06060,
word=0x7ffc32c060c0) at wordexp.c:2147
        squoting = <optimized out>
        comm_length = 8622050
        *comm = 0x6 <error: Cannot access memory at address 0x6>*
        error = <optimized out>
        comm_maxlen = 8700952
#2  wordexp (words=<optimized out>, pwordexp=<optimized out>, flags=0) at
wordexp.c:2359
        words_offset = 0
        word_length = 0
        max_length = 140388071391744
        word = 0x7ffc32c065c8 ""
        error = <optimized out>
        ifs = 0x7fae24ccc960 ""
        ifs_white = "\000\000\000"
        old_word = {we_wordc = 0, we_wordv = 0x0, we_offs = 0}
#3  0x00007ffc32c06220 in ?? ()
No symbol table info available.
#4  0x0000000000000000 in ?? ()
No symbol table info available.

*ldd -v /usr/local/opensips2/sbin/opensips*
linux-vdso.so.1 =>  (0x00007fff96f49000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f75c45de000)
libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007f75c43c3000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f75c3ffa000)
/lib64/ld-linux-x86-64.so.2 (0x000055e9cad86000)

Version information:
/usr/local/opensips2/sbin/opensips:
libresolv.so.2 (GLIBC_2.2.5) => /lib/x86_64-linux-gnu/libresolv.so.2
libdl.so.2 (GLIBC_2.2.5) => /lib/x86_64-linux-gnu/libdl.so.2
libc.so.6 (GLIBC_2.3) => /lib/x86_64-linux-gnu/libc.so.6
libc.so.6 (GLIBC_2.3.2) => /lib/x86_64-linux-gnu/libc.so.6
libc.so.6 (GLIBC_2.15) => /lib/x86_64-linux-gnu/libc.so.6
libc.so.6 (GLIBC_2.4) => /lib/x86_64-linux-gnu/libc.so.6
libc.so.6 (GLIBC_2.7) => /lib/x86_64-linux-gnu/libc.so.6
libc.so.6 (GLIBC_2.3.4) => /lib/x86_64-linux-gnu/libc.so.6
libc.so.6 (GLIBC_2.2.5) => /lib/x86_64-linux-gnu/libc.so.6
/lib/x86_64-linux-gnu/libdl.so.2:
ld-linux-x86-64.so.2 (GLIBC_PRIVATE) => /lib64/ld-linux-x86-64.so.2
libc.so.6 (GLIBC_PRIVATE) => /lib/x86_64-linux-gnu/libc.so.6
libc.so.6 (GLIBC_2.2.5) => /lib/x86_64-linux-gnu/libc.so.6
/lib/x86_64-linux-gnu/libresolv.so.2:
libc.so.6 (GLIBC_2.14) => /lib/x86_64-linux-gnu/libc.so.6
libc.so.6 (GLIBC_2.4) => /lib/x86_64-linux-gnu/libc.so.6
libc.so.6 (GLIBC_PRIVATE) => /lib/x86_64-linux-gnu/libc.so.6
libc.so.6 (GLIBC_2.2.5) => /lib/x86_64-linux-gnu/libc.so.6
libc.so.6 (GLIBC_2.3) => /lib/x86_64-linux-gnu/libc.so.6
/lib/x86_64-linux-gnu/libc.so.6:
ld-linux-x86-64.so.2 (GLIBC_2.3) => /lib64/ld-linux-x86-64.so.2
ld-linux-x86-64.so.2 (GLIBC_PRIVATE) => /lib64/ld-linux-x86-64.so.2

*# free -m*
             total       used       free     shared    buffers     cached
Mem:          3951       2842       1109         28         66       2156
-/+ buffers/cache:        619       3332
Swap:         4091         90       4001

*# vmstat *
procs -----------memory---------- ---swap-- -----io---- -system--
------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id
wa st
 2  0  92396 1136328  68576 2207768    0    0     5    14   10    2  1  2
97  0  0

*# w*
 11:43:18 up 11 days, 23:16,  1 user,  load average: 2.46, 2.39, 2.36

*# netstat -ulnp*
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
    PID/Program name
udp   426496      0 X.X.X.X:65100     0.0.0.0:*
1843/opensips

also after restarting opensips process system started to receive traffic
again BUT I got the following errors on syslog regarding rtpproxy

Mar 29 09:57:32 rtp1 /usr/local/opensips2/sbin/opensips[1850]:
ERROR:rtpproxy:force_rtp_proxy_body: incorrect port 0 in reply from rtp
proxy
Mar 29 09:58:09 rtp1 /usr/local/opensips2/sbin/opensips[1850]:
ERROR:rtpproxy:force_rtp_proxy_body: incorrect port 0 in reply from rtp
proxy
Mar 29 09:58:10 rtp1 /usr/local/opensips2/sbin/opensips[1857]:
ERROR:rtpproxy:force_rtp_proxy_body: incorrect port 0 in reply from rtp
proxy
Mar 29 09:58:12 rtp1 /usr/local/opensips2/sbin/opensips[1850]:
ERROR:rtpproxy:force_rtp_proxy_body: incorrect port 0 in reply from rtp
proxy
Mar 29 09:58:26 rtp1 /usr/local/opensips2/sbin/opensips[1850]:
ERROR:rtpproxy:force_rtp_proxy_body: incorrect port 0 in reply from rtp
proxy
Mar 29 09:58:26 rtp1 /usr/local/opensips2/sbin/opensips[1851]:
ERROR:rtpproxy:force_rtp_proxy_body: incorrect port 0 in reply from rtp
proxy
Mar 29 09:58:26 rtp1 /usr/local/opensips2/sbin/opensips[1853]:
ERROR:rtpproxy:force_rtp_proxy_body: incorrect port 0 in reply from rtp
proxy

and I had to reboot rtpproxy processes too for traffic to pass through
without problems.
I don't know if this is another problem with rtpproxy but I am just
mentioning it.

thank you in advance


*Vasilios Tzanoudakis* | Technology Director
t. +30-212-222-8003  |  f. +30-212-222-8001
2, Klisthenous Str., Metamorfosi, 144 52, Attica, Greece
vasilios.tzanoudakis at voiceland.gr  |  www.voiceland.gr
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opensips.org/pipermail/users/attachments/20170329/9b2c302f/attachment-0001.html>

More information about the Users mailing list