<div dir="ltr">Hi Bogdan,<div>Sorry, I just realized that I haven't turned off the auto-scaler for UDP processes so you're right - that error is for them. I've switched TCP auto-scaling off due to all my previous issues that I wasn't able to overcome.</div><div><br></div><div>auto_scaling_profile = PROFILE_UDP<br>     scale up to 32 on 80% for 4 cycles within 5<br>     scale down to 2 on 10% for 5 cycles<br></div><div><br></div><div>udp_workers=1 use_auto_scaling_profile PROFILE_UDP<br></div><div><br></div><div>socket=udp:10.x.x.x:5060<br></div><div>socket=udp:103.x.x.x:7060<br></div><div><br></div><div>In regards to pkg memory - I only have 32MB as -M parameter, maybe that's not enough? I'm not sure how to do 'pkg dump' - never tried to do that before, is this what you're asking about?</div><div><br></div><div>Sep 21 18:52:09 ERROR:core:signal_pkg_status: failed to trigger pkg stats for process 45<br></div><div> opensips-cli -x mi ps<br>{<br>    "Processes": [<br></div><div>...</div><div>        {<br></div><div>            "ID": 45,<br>            "PID": 310115,<br>            "Type": "SIP receiver udp:103.x.x.x:7060"<br>        }<br>    ]<br>}<br>root@osgw1:~# opensips-cli -x mi mem_pkg_dump 310115<br>ERROR: command 'mem_pkg_dump' returned: 500: Internal error<br></div><div><br></div><div>Best regards,</div><div>Yury.</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Sep 21, 2022 at 4:07 PM Bogdan-Andrei Iancu <<a href="mailto:bogdan@opensips.org" target="_blank">bogdan@opensips.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
  
    
  
  <div>
    <font face="monospace">Hi Yury,<br>
      <br>
      By the IDs of the faulty processes, I see they are auto-scaled
      procs.<br>
      <br>
      Could you confirm that the pkg status ever works for such
      processes - with or without below error, is the pkg dump ever done
      by the auto-scaled procs ?<br>
      <br>
      Regards,<br>
    </font>
    <pre cols="72">Bogdan-Andrei Iancu

OpenSIPS Founder and Developer
  <a href="https://www.opensips-solutions.com" target="_blank">https://www.opensips-solutions.com</a>
OpenSIPS Summit 27-30 Sept 2022, Athens
  <a href="https://www.opensips.org/events/Summit-2022Athens/" target="_blank">https://www.opensips.org/events/Summit-2022Athens/</a></pre>
    <div>On 9/20/22 8:23 PM, Yury Kirsanov
      wrote:<br>
    </div>
    <blockquote type="cite">
      
      <div dir="ltr">Hi,
        <div>We're experiencing a lot of messages like this:</div>
        <div><br>
        </div>
        <div>Sep 21 03:18:09 ERROR:core:signal_pkg_status: failed to
          trigger pkg stats for process 42<br>
          Sep 21 03:18:09 ERROR:core:signal_pkg_status: failed to
          trigger pkg stats for process 43<br>
          Sep 21 03:18:09 ERROR:core:signal_pkg_status: failed to
          trigger pkg stats for process 45<br>
          Sep 21 03:18:09 ERROR:core:signal_pkg_status: failed to
          trigger pkg stats for process 42<br>
          Sep 21 03:18:09 ERROR:core:signal_pkg_status: failed to
          trigger pkg stats for process 43<br>
          Sep 21 03:18:09 ERROR:core:signal_pkg_status: failed to
          trigger pkg stats for process 45<br>
          Sep 21 03:18:09 ERROR:core:signal_pkg_status: failed to
          trigger pkg stats for process 42<br>
          Sep 21 03:18:09 ERROR:core:signal_pkg_status: failed to
          trigger pkg stats for process 43<br>
          Sep 21 03:18:09 ERROR:core:signal_pkg_status: failed to
          trigger pkg stats for process 45<br>
          Sep 21 03:18:09 ERROR:core:signal_pkg_status: failed to
          trigger pkg stats for process 42<br>
          Sep 21 03:18:09 ERROR:core:signal_pkg_status: failed to
          trigger pkg stats for process 43<br>
          Sep 21 03:18:09 ERROR:core:signal_pkg_status: failed to
          trigger pkg stats for process 45<br>
          Sep 21 03:18:09 ERROR:core:signal_pkg_status: failed to
          trigger pkg stats for process 42<br>
          Sep 21 03:18:09 ERROR:core:signal_pkg_status: failed to
          trigger pkg stats for process 43<br>
          Sep 21 03:18:09 ERROR:core:signal_pkg_status: failed to
          trigger pkg stats for process 45<br>
          Sep 21 03:18:09 ERROR:core:signal_pkg_status: failed to
          trigger pkg stats for process 42<br>
          Sep 21 03:18:09 ERROR:core:signal_pkg_status: failed to
          trigger pkg stats for process 43<br>
          Sep 21 03:18:09 ERROR:core:signal_pkg_status: failed to
          trigger pkg stats for process 45<br>
        </div>
        <div><br>
        </div>
        <div>Here's a list of all processes:</div>
        <div><br>
        </div>
        <div>opensips-cli -x mi get_statistics load:<br>
          {<br>
              "load:load-proc-1": 0,<br>
              "load:load1m-proc-1": 0,<br>
              "load:load10m-proc-1": 0,<br>
              "load:load-proc-2": 0,<br>
              "load:load1m-proc-2": 0,<br>
              "load:load10m-proc-2": 0,<br>
              "load:load-proc-3": 0,<br>
              "load:load1m-proc-3": 0,<br>
              "load:load10m-proc-3": 0,<br>
              "load:load-proc-4": 0,<br>
              "load:load1m-proc-4": 0,<br>
              "load:load10m-proc-4": 0,<br>
              "load:load-proc-5": 0,<br>
              "load:load1m-proc-5": 0,<br>
              "load:load10m-proc-5": 1,<br>
              "load:load-proc-6": 1,<br>
              "load:load1m-proc-6": 1,<br>
              "load:load10m-proc-6": 1,<br>
              "load:load-proc-7": 0,<br>
              "load:load1m-proc-7": 1,<br>
              "load:load10m-proc-7": 1,<br>
              "load:load-proc-8": 0,<br>
              "load:load1m-proc-8": 1,<br>
              "load:load10m-proc-8": 1,<br>
              "load:load-proc-9": 0,<br>
              "load:load1m-proc-9": 1,<br>
              "load:load10m-proc-9": 1,<br>
              "load:load-proc-10": 0,<br>
              "load:load1m-proc-10": 1,<br>
              "load:load10m-proc-10": 1,<br>
              "load:load-proc-11": 0,<br>
              "load:load1m-proc-11": 1,<br>
              "load:load10m-proc-11": 1,<br>
              "load:load-proc-12": 0,<br>
              "load:load1m-proc-12": 1,<br>
              "load:load10m-proc-12": 1,<br>
              "load:load-proc-13": 0,<br>
              "load:load1m-proc-13": 1,<br>
              "load:load10m-proc-13": 1,<br>
              "load:load-proc-14": 0,<br>
              "load:load1m-proc-14": 1,<br>
              "load:load10m-proc-14": 1,<br>
              "load:load-proc-15": 7,<br>
              "load:load1m-proc-15": 0,<br>
              "load:load10m-proc-15": 0,<br>
              "load:load-proc-16": 0,<br>
              "load:load1m-proc-16": 1,<br>
              "load:load10m-proc-16": 0,<br>
              "load:load-proc-17": 4,<br>
              "load:load1m-proc-17": 0,<br>
              "load:load10m-proc-17": 0,<br>
              "load:load-proc-18": 0,<br>
              "load:load1m-proc-18": 0,<br>
              "load:load10m-proc-18": 0,<br>
              "load:load-proc-19": 0,<br>
              "load:load1m-proc-19": 0,<br>
              "load:load10m-proc-19": 0,<br>
              "load:load-proc-20": 0,<br>
              "load:load1m-proc-20": 0,<br>
              "load:load10m-proc-20": 0,<br>
              "load:load-proc-21": 0,<br>
              "load:load1m-proc-21": 0,<br>
              "load:load10m-proc-21": 0,<br>
              "load:load-proc-22": 0,<br>
              "load:load1m-proc-22": 0,<br>
              "load:load10m-proc-22": 0,<br>
              "load:load-proc-23": 0,<br>
              "load:load1m-proc-23": 0,<br>
              "load:load10m-proc-23": 0,<br>
              "load:load-proc-24": 0,<br>
              "load:load1m-proc-24": 0,<br>
              "load:load10m-proc-24": 0,<br>
              "load:load-proc-25": 0,<br>
              "load:load1m-proc-25": 0,<br>
              "load:load10m-proc-25": 0,<br>
              "load:load-proc-26": 0,<br>
              "load:load1m-proc-26": 0,<br>
              "load:load10m-proc-26": 0,<br>
              "load:load-proc-27": 0,<br>
              "load:load1m-proc-27": 0,<br>
              "load:load10m-proc-27": 0,<br>
              "load:load-proc-28": 0,<br>
              "load:load1m-proc-28": 0,<br>
              "load:load10m-proc-28": 0,<br>
              "load:load-proc-29": 0,<br>
              "load:load1m-proc-29": 0,<br>
              "load:load10m-proc-29": 0,<br>
              "load:load-proc-30": 0,<br>
              "load:load1m-proc-30": 0,<br>
              "load:load10m-proc-30": 0,<br>
              "load:load-proc-31": 0,<br>
              "load:load1m-proc-31": 0,<br>
              "load:load10m-proc-31": 0,<br>
              "load:load-proc-32": 0,<br>
              "load:load1m-proc-32": 0,<br>
              "load:load10m-proc-32": 0,<br>
              "load:load-proc-33": 0,<br>
              "load:load1m-proc-33": 0,<br>
              "load:load10m-proc-33": 0,<br>
              "load:load-proc-34": 0,<br>
              "load:load1m-proc-34": 0,<br>
              "load:load10m-proc-34": 0,<br>
              "load:load-proc-35": 0,<br>
              "load:load1m-proc-35": 0,<br>
              "load:load10m-proc-35": 0,<br>
              "load:load-proc-36": 0,<br>
              "load:load1m-proc-36": 0,<br>
              "load:load10m-proc-36": 0,<br>
              "load:load-proc-37": 0,<br>
              "load:load1m-proc-37": 0,<br>
              "load:load10m-proc-37": 0,<br>
              "load:load-proc-38": 0,<br>
              "load:load1m-proc-38": 0,<br>
              "load:load10m-proc-38": 0,<br>
              "load:load-proc-39": 0,<br>
              "load:load1m-proc-39": 0,<br>
              "load:load10m-proc-39": 0,<br>
              "load:load-proc-40": 9,<br>
              "load:load1m-proc-40": 5,<br>
              "load:load10m-proc-40": 6,<br>
              "load:load-proc-41": 2,<br>
              "load:load1m-proc-41": 0,<br>
              "load:load10m-proc-41": 1,<br>
              "load:load-proc-42": 0,<br>
              "load:load1m-proc-42": 0,<br>
              "load:load10m-proc-42": 0,<br>
              "load:load-proc-43": 0,<br>
              "load:load1m-proc-43": 0,<br>
              "load:load10m-proc-43": 0,<br>
              "load:load-proc-44": 2,<br>
              "load:load1m-proc-44": 1,<br>
              "load:load10m-proc-44": 2,<br>
              "load:load-proc-45": 0,<br>
              "load:load1m-proc-45": 0,<br>
              "load:load10m-proc-45": 0,<br>
              "load:load": 0,<br>
              "load:load1m": 0,<br>
              "load:load10m": 0,<br>
              "load:load-all": 0,<br>
              "load:load1m-all": 0,<br>
              "load:load10m-all": 0,<br>
              "load:processes_number": 46<br>
          }<br>
        </div>
        <div><br>
        </div>
        <div>Here's a list of all running processes:</div>
        <div>opensips-cli -x mi ps<br>
          {<br>
              "Processes": [<br>
                  {<br>
                      "ID": 0,<br>
                      "PID": 226004,<br>
                      "Type": "attendant"<br>
                  },<br>
                  {<br>
                      "ID": 1,<br>
                      "PID": 226010,<br>
                      "Type": "HTTPD 10.x.x.x:8888"<br>
                  },<br>
                  {<br>
                      "ID": 2,<br>
                      "PID": 226011,<br>
                      "Type": "MI FIFO"<br>
                  },<br>
                  {<br>
                      "ID": 3,<br>
                      "PID": 226012,<br>
                      "Type": "time_keeper"<br>
                  },<br>
                  {<br>
                      "ID": 4,<br>
                      "PID": 226013,<br>
                      "Type": "timer"<br>
                  },<br>
                  {<br>
                      "ID": 5,<br>
                      "PID": 226014,<br>
                      "Type": "SIP receiver udp:10.x.x.x:5060"<br>
                  },<br>
                  {<br>
                      "ID": 6,<br>
                      "PID": 226017,<br>
                      "Type": "SIP receiver udp:103.x.x.x:7060"<br>
                  },<br>
                  {<br>
                      "ID": 7,<br>
                      "PID": 226018,<br>
                      "Type": "TCP receiver"<br>
                  },<br>
                  {<br>
                      "ID": 8,<br>
                      "PID": 226019,<br>
                      "Type": "TCP receiver"<br>
                  },<br>
                  {<br>
                      "ID": 9,<br>
                      "PID": 226020,<br>
                      "Type": "TCP receiver"<br>
                  },<br>
                  {<br>
                      "ID": 10,<br>
                      "PID": 226021,<br>
                      "Type": "TCP receiver"<br>
                  },<br>
                  {<br>
                      "ID": 11,<br>
                      "PID": 226022,<br>
                      "Type": "TCP receiver"<br>
                  },<br>
                  {<br>
                      "ID": 12,<br>
                      "PID": 226023,<br>
                      "Type": "TCP receiver"<br>
                  },<br>
                  {<br>
                      "ID": 13,<br>
                      "PID": 226024,<br>
                      "Type": "TCP receiver"<br>
                  },<br>
                  {<br>
                      "ID": 14,<br>
                      "PID": 226025,<br>
                      "Type": "TCP receiver"<br>
                  },<br>
                  {<br>
                      "ID": 15,<br>
                      "PID": 226026,<br>
                      "Type": "TCP receiver"<br>
                  },<br>
                  {<br>
                      "ID": 16,<br>
                      "PID": 226027,<br>
                      "Type": "TCP receiver"<br>
                  },<br>
                  {<br>
                      "ID": 17,<br>
                      "PID": 226028,<br>
                      "Type": "TCP receiver"<br>
                  },<br>
                  {<br>
                      "ID": 18,<br>
                      "PID": 226029,<br>
                      "Type": "TCP receiver"<br>
                  },<br>
                  {<br>
                      "ID": 19,<br>
                      "PID": 226030,<br>
                      "Type": "TCP receiver"<br>
                  },<br>
                  {<br>
                      "ID": 20,<br>
                      "PID": 226031,<br>
                      "Type": "TCP receiver"<br>
                  },<br>
                  {<br>
                      "ID": 21,<br>
                      "PID": 226032,<br>
                      "Type": "TCP receiver"<br>
                  },<br>
                  {<br>
                      "ID": 22,<br>
                      "PID": 226033,<br>
                      "Type": "TCP receiver"<br>
                  },<br>
                  {<br>
                      "ID": 23,<br>
                      "PID": 226034,<br>
                      "Type": "TCP receiver"<br>
                  },<br>
                  {<br>
                      "ID": 24,<br>
                      "PID": 226035,<br>
                      "Type": "TCP receiver"<br>
                  },<br>
                  {<br>
                      "ID": 25,<br>
                      "PID": 226036,<br>
                      "Type": "TCP receiver"<br>
                  },<br>
                  {<br>
                      "ID": 26,<br>
                      "PID": 226038,<br>
                      "Type": "TCP receiver"<br>
                  },<br>
                  {<br>
                      "ID": 27,<br>
                      "PID": 226039,<br>
                      "Type": "TCP receiver"<br>
                  },<br>
                  {<br>
                      "ID": 28,<br>
                      "PID": 226040,<br>
                      "Type": "TCP receiver"<br>
                  },<br>
                  {<br>
                      "ID": 29,<br>
                      "PID": 226041,<br>
                      "Type": "TCP receiver"<br>
                  },<br>
                  {<br>
                      "ID": 30,<br>
                      "PID": 226042,<br>
                      "Type": "TCP receiver"<br>
                  },<br>
                  {<br>
                      "ID": 31,<br>
                      "PID": 226043,<br>
                      "Type": "TCP receiver"<br>
                  },<br>
                  {<br>
                      "ID": 32,<br>
                      "PID": 226044,<br>
                      "Type": "TCP receiver"<br>
                  },<br>
                  {<br>
                      "ID": 33,<br>
                      "PID": 226045,<br>
                      "Type": "TCP receiver"<br>
                  },<br>
                  {<br>
                      "ID": 34,<br>
                      "PID": 226046,<br>
                      "Type": "TCP receiver"<br>
                  },<br>
                  {<br>
                      "ID": 35,<br>
                      "PID": 226047,<br>
                      "Type": "TCP receiver"<br>
                  },<br>
                  {<br>
                      "ID": 36,<br>
                      "PID": 226048,<br>
                      "Type": "TCP receiver"<br>
                  },<br>
                  {<br>
                      "ID": 37,<br>
                      "PID": 226049,<br>
                      "Type": "TCP receiver"<br>
                  },<br>
                  {<br>
                      "ID": 38,<br>
                      "PID": 226050,<br>
                      "Type": "TCP receiver"<br>
                  },<br>
                  {<br>
                      "ID": 39,<br>
                      "PID": 226051,<br>
                      "Type": "Timer handler"<br>
                  },<br>
                  {<br>
                      "ID": 40,<br>
                      "PID": 226052,<br>
                      "Type": "TCP main"<br>
                  },<br>
                  {<br>
                      "ID": 41,<br>
                      "PID": 226069,<br>
                      "Type": "SIP receiver udp:10.x.x.x:5060"<br>
                  },<br>
                  {<br>
                      "ID": 42,<br>
                      "PID": 226077,<br>
                      "Type": "SIP receiver udp:10.x.x.x:5060"<br>
                  },<br>
                  {<br>
                      "ID": 43,<br>
                      "PID": 226650,<br>
                      "Type": "SIP receiver udp:10.x.x.x:5060"<br>
                  },<br>
                  {<br>
                      "ID": 44,<br>
                      "PID": 262593,<br>
                      "Type": "SIP receiver udp:103.x.x.x:7060"<br>
                  },<br>
                  {<br>
                      "ID": 45,<br>
                      "PID": 310115,<br>
                      "Type": "SIP receiver udp:103.x.x.x:7060"<br>
                  }<br>
              ]<br>
          }<br>
        </div>
        <div><br>
        </div>
        <div>OpenSIPS seem to be behaving fine and working but these
          errors are quite confusing...Should I be worried about them?</div>
        <div><br>
        </div>
        <div>OpenSIPS was compiled from sources with some options like
          futexes on:</div>
        <div><br>
        </div>
        <div>version: opensips 3.3.1 (x86_64/linux)<br>
          flags: STATS: On, DISABLE_NAGLE, USE_MCAST, SHM_MMAP,
          PKG_MALLOC, Q_MALLOC, F_MALLOC, HP_MALLOC,
          FAST_LOCK-FUTEX-ADAPTIVE_WAIT<br>
          ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144,
          MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535<br>
          poll method support: poll, epoll, sigio_rt, select.<br>
          git revision: d187a0bd5<br>
          main.c compiled on 06:06:34 Sep 13 2022 with gcc 9<br>
        </div>
        <div><br>
        </div>
        <div>Any ideas?</div>
        <div><br>
        </div>
        <div>Thanks and regards,</div>
        <div>Yury.</div>
        <div><br>
        </div>
      </div>
      <br>
      <fieldset></fieldset>
      <pre>_______________________________________________
Users mailing list
<a href="mailto:Users@lists.opensips.org" target="_blank">Users@lists.opensips.org</a>
<a href="http://lists.opensips.org/cgi-bin/mailman/listinfo/users" target="_blank">http://lists.opensips.org/cgi-bin/mailman/listinfo/users</a>
</pre>
    </blockquote>
    <br>
  </div>

</blockquote></div>