[OpenSIPS-Devel] [OpenSIPS/opensips] ea959e: OPS-6 (Selectable Allocator): Clean up VQ_MALLOC

Fri Mar 15 13:14:11 EDT 2019

  Branch: refs/heads/master
  Home:   https://github.com/OpenSIPS/opensips
  Commit: ea959e19a94b93b75c0871d2c92a25d5e2f101ea
      https://github.com/OpenSIPS/opensips/commit/ea959e19a94b93b75c0871d2c92a25d5e2f101ea
  Author: Liviu Chircu <liviu at opensips.org>
  Date:   2019-03-11 (Mon, 11 Mar 2019)

  Changed paths:
    M Makefile.conf.template
    M Makefile.defs
    M mem/common.h
    M mem/f_malloc.c
    M mem/f_malloc.h
    M mem/hp_malloc.c
    M mem/hp_malloc.h
    M mem/hp_malloc_stats.c
    M mem/mem.c
    M mem/mem.h
    M mem/memtest.c
    M mem/q_malloc.c
    M mem/q_malloc.h
    M mem/shm_mem.c
    M mem/shm_mem.h
    R mem/vq_malloc.c
    R mem/vq_malloc.h
    M packaging/debian/copyright
    M version.h

  Log Message:
  -----------
  OPS-6 (Selectable Allocator): Clean up VQ_MALLOC

This allocator is both unmaintained and non-functional for several
years now.

  Commit: c8804d280d3f603e6846bdeaba501a5ebdcaafab
      https://github.com/OpenSIPS/opensips/commit/c8804d280d3f603e6846bdeaba501a5ebdcaafab
  Author: Liviu Chircu <liviu at opensips.org>
  Date:   2019-03-13 (Wed, 13 Mar 2019)

  Changed paths:
    M Makefile.conf.template
    A mem/common.c
    M mem/common.h
    M mem/f_malloc.c
    M mem/f_malloc.h
    A mem/f_malloc_dyn.h
    M mem/mem.c
    M mem/mem.h
    M mem/shm_mem.c
    M mem/shm_mem.h
    M modules/lua/sipstate.c

  Log Message:
  -----------
  OPS-6: Major allocator rework - F_MALLOC is functional

The purpose of this rework is to be able to do the following:
    * be able to compile an inlined allocator
	(same performance as now, same binary footprint)
    * be able to compile an inlined, dbg allocator
	(same dbg performance as now, same larger binary footprint)
    * be able to compile multiple allocators
	(a bit slower, somewhat flexible, larger+ binary footprint)
    * be able to compile multiple, dbg allocators
	(even slower, most flexible: 3 allocators + 3 dbg allocators,
	    larger++ binary footprint)

This commit creates the framework for the above.  QM_MALLOC and
HP_MALLOC are currently completely broken, but F_MALLOC inlining is
working well, with or w/o DBG_MALLOC.

  Commit: af1f013425ac1cb95e321a17543f9ffd7ceadcf5
      https://github.com/OpenSIPS/opensips/commit/af1f013425ac1cb95e321a17543f9ffd7ceadcf5
  Author: Liviu Chircu <liviu at opensips.org>
  Date:   2019-03-13 (Wed, 13 Mar 2019)

  Changed paths:
    M mem/f_malloc.h
    M mem/f_malloc_dyn.h
    M mem/q_malloc.c
    M mem/q_malloc.h
    A mem/qm_malloc_dyn.h

  Log Message:
  -----------
  OPS-6:  QM_MALLOC is once again functional

  Commit: 67ed0ccbaa83ac91dfad887b39a603db05fb23db
      https://github.com/OpenSIPS/opensips/commit/67ed0ccbaa83ac91dfad887b39a603db05fb23db
  Author: Liviu Chircu <liviu at opensips.org>
  Date:   2019-03-13 (Wed, 13 Mar 2019)

  Changed paths:
    M mem/f_malloc_dyn.h

  Log Message:
  -----------
  OPS-6: Fix some F_MALLOC issues

    * fix missing #ifdef
    * fix badly migrated #ifdef

  Commit: 5e6cf56ff94145b11388c00468e46283bf2ab69e
      https://github.com/OpenSIPS/opensips/commit/5e6cf56ff94145b11388c00468e46283bf2ab69e
  Author: Liviu Chircu <liviu at opensips.org>
  Date:   2019-03-13 (Wed, 13 Mar 2019)

  Changed paths:
    M mem/common.h
    M mem/f_malloc_dyn.h
    M mem/hp_malloc.c
    M mem/hp_malloc.h
    A mem/hp_malloc_dyn.h
    M mem/mem.c
    M mem/mem.h
    M mem/qm_malloc_dyn.h
    M mem/shm_mem.c
    M mem/shm_mem.h

  Log Message:
  -----------
  OPS-6: HP_MALLOC is once again functional

  Commit: e4e9e69412a97bcb520fd0d68c85b83c893b339c
      https://github.com/OpenSIPS/opensips/commit/e4e9e69412a97bcb520fd0d68c85b83c893b339c
  Author: Liviu Chircu <liviu at opensips.org>
  Date:   2019-03-13 (Wed, 13 Mar 2019)

  Changed paths:
    M mem/common.h
    M mem/f_malloc.c
    M mem/f_malloc.h
    M mem/mem.c
    M mem/mem.h
    M mem/q_malloc.c
    M mem/q_malloc.h
    M mem/qm_malloc_dyn.h
    M mem/shm_mem.c
    M mem/shm_mem.h

  Log Message:
  -----------
  OPS-6: F_MALLOC and QM_MALLOC may now compile together

  Commit: 9002881a67d8bf6e8d0ed0938186c047d541761d
      https://github.com/OpenSIPS/opensips/commit/9002881a67d8bf6e8d0ed0938186c047d541761d
  Author: Liviu Chircu <liviu at opensips.org>
  Date:   2019-03-13 (Wed, 13 Mar 2019)

  Changed paths:
    M mem/f_malloc.c
    M mem/f_malloc.h
    M mem/hp_malloc.c
    M mem/hp_malloc.h
    M mem/hp_malloc_dyn.h
    M mem/hp_malloc_stats.c
    M mem/hp_malloc_stats.h

  Log Message:
  -----------
  OPS-6: F_MALLOC and HP_MALLOC may now compile together

  Commit: a58f76c0df6645a9a4fd44144e155becebfd03af
      https://github.com/OpenSIPS/opensips/commit/a58f76c0df6645a9a4fd44144e155becebfd03af
  Author: Liviu Chircu <liviu at opensips.org>
  Date:   2019-03-14 (Thu, 14 Mar 2019)

  Changed paths:
    M main.c
    M mem/common.c
    M mem/common.h
    M mem/hp_malloc.c
    M mem/hp_malloc_dyn.h
    M mem/shm_mem.c
    M mem/shm_mem.h

  Log Message:
  -----------
  OPS-6:  All 3 allocators may now compile together

With or without -DDBG_MALLOC

  Commit: 7a1cf1d923de9f2dbb9d1b5dce1586eaecceae98
      https://github.com/OpenSIPS/opensips/commit/7a1cf1d923de9f2dbb9d1b5dce1586eaecceae98
  Author: Liviu Chircu <liviu at opensips.org>
  Date:   2019-03-14 (Thu, 14 Mar 2019)

  Changed paths:
    M Makefile.conf.template
    M Makefile.defs
    M cfg.y
    M main.c
    M mem/mem.c
    M mem/mem.h
    M reactor.c
    M version.h

  Log Message:
  -----------
  OPS-6: Drop USE_SHM_MEM support

This compile flag has never had a useful application, yet it has
constantly required careful attending / maintenance.

  Commit: c202cfc2a4f0f663cee0585d7bbb687e7fe42b3d
      https://github.com/OpenSIPS/opensips/commit/c202cfc2a4f0f663cee0585d7bbb687e7fe42b3d
  Author: Liviu Chircu <liviu at opensips.org>
  Date:   2019-03-14 (Thu, 14 Mar 2019)

  Changed paths:
    M Makefile.conf.template
    M mem/common.c
    M mem/common.h
    M mem/f_malloc.c
    M mem/f_malloc.h
    M mem/f_malloc_dyn.h
    M mem/hp_malloc.c
    M mem/hp_malloc.h
    M mem/hp_malloc_dyn.h
    M mem/module_info.c
    M mem/q_malloc.c
    M mem/q_malloc.h
    M mem/qm_malloc_dyn.h
    M mem/shm_mem.c
    M mem/shm_mem.h

  Log Message:
  -----------
  OPS-6: SHM_EXTRA_STATS is once again functional

  Commit: 23fea2f8c8d6f6811abebd34c3efdadb3c6e49ec
      https://github.com/OpenSIPS/opensips/commit/23fea2f8c8d6f6811abebd34c3efdadb3c6e49ec
  Author: Liviu Chircu <liviu at opensips.org>
  Date:   2019-03-14 (Thu, 14 Mar 2019)

  Changed paths:
    M Makefile.conf.template

  Log Message:
  -----------
  OPS-6: Update Makefile.conf.template comments

  Commit: 6d7b8ec714edb9b7e25345754ea4ba41f8b566dd
      https://github.com/OpenSIPS/opensips/commit/6d7b8ec714edb9b7e25345754ea4ba41f8b566dd
  Author: Liviu Chircu <liviu at opensips.org>
  Date:   2019-03-14 (Thu, 14 Mar 2019)

  Changed paths:
    M mem/common.c
    M mem/f_malloc.h
    M mem/q_malloc.c
    M mem/q_malloc.h
    M mem/shm_mem.c

  Log Message:
  -----------
  OPS-6: Fix some subtle bugs

    * QM_MALLOC + HP_MALLOC with no DBG_MALLOC failed with QM_MALLOC
      because of incorrectly shared "ROUNDTO" constant between .h files
    * bad copy-pasted functions, leading to linker errors
    * undefined NULL symbol

  Commit: a6a0d812fb5a506324e363e88cd39167e66abe45
      https://github.com/OpenSIPS/opensips/commit/a6a0d812fb5a506324e363e88cd39167e66abe45
  Author: Liviu Chircu <liviu at opensips.org>
  Date:   2019-03-15 (Fri, 15 Mar 2019)

  Changed paths:
    M main.c
    M mem/f_malloc_dyn.h
    M mem/hp_malloc_stats.c
    M mem/shm_mem.c
    M mem/shm_mem.h

  Log Message:
  -----------
  OPS-6: Fix additional incompatibilities

    * fix broken inlined F_MALLOC
    * fix missing symbols when inlining allocators

  Commit: 41ad6727f3d7a17370c38db5d06f0dcdf450b30b
      https://github.com/OpenSIPS/opensips/commit/41ad6727f3d7a17370c38db5d06f0dcdf450b30b
  Author: Liviu Chircu <liviu at opensips.org>
  Date:   2019-03-15 (Fri, 15 Mar 2019)

  Changed paths:
    M mem/f_malloc.c
    M mem/f_malloc.h
    M mem/f_malloc_dyn.h
    M mem/hp_malloc.c
    M mem/hp_malloc.h
    M mem/hp_malloc_dyn.h
    M mem/q_malloc.c
    M mem/q_malloc.h
    M mem/qm_malloc_dyn.h
    M mem/shm_mem.c

  Log Message:
  -----------
  OPS-6: Fix shutdown issues with multi-allocators

The "(fm|qm|hp)_status()" functions also require "_dbg" versions

  Commit: c7d02c0a7e1e29d05b635ea6e0981bfb87330fa3
      https://github.com/OpenSIPS/opensips/commit/c7d02c0a7e1e29d05b635ea6e0981bfb87330fa3
  Author: Liviu Chircu <liviu at opensips.org>
  Date:   2019-03-15 (Fri, 15 Mar 2019)

  Changed paths:
    M mem/common.h
    M mem/mem.c
    M mem/shm_mem.c

  Log Message:
  -----------
  OPS-6: Add a memory manager printing function

  Commit: 4adb1bb5035b422de95dbf587890b9b75efb61ba
      https://github.com/OpenSIPS/opensips/commit/4adb1bb5035b422de95dbf587890b9b75efb61ba
  Author: Liviu Chircu <liviu at opensips.org>
  Date:   2019-03-15 (Fri, 15 Mar 2019)

  Changed paths:
    M mem/f_malloc.h
    M mem/q_malloc.h

  Log Message:
  -----------
  OPS-6: Better organize some SHM_EXTRA_STATS code

  Commit: 572f0df73ea9f70ee9fcb85206312a9be25c7a00
      https://github.com/OpenSIPS/opensips/commit/572f0df73ea9f70ee9fcb85206312a9be25c7a00
  Author: Liviu Chircu <liviu at opensips.org>
  Date:   2019-03-15 (Fri, 15 Mar 2019)

  Changed paths:
    M mem/f_malloc.c
    M mem/f_malloc.h
    M mem/f_malloc_dyn.h
    M mem/hp_malloc.c
    M mem/hp_malloc.h
    M mem/hp_malloc_dyn.h

  Log Message:
  -----------
  OPS-6: HP_MALLOC / F_MALLOC - remove redundant logic

There is no need for a separate frag->is_free field, as this task is
already achieved by the optimally managed frag->prev field in both
allocators.

  Commit: 243fcee7abbfeb7595a77263d9b8fa8a8bde259b
      https://github.com/OpenSIPS/opensips/commit/243fcee7abbfeb7595a77263d9b8fa8a8bde259b
  Author: Liviu Chircu <liviu at opensips.org>
  Date:   2019-03-15 (Fri, 15 Mar 2019)

  Changed paths:
    M mem/hp_malloc.c
    M mem/hp_malloc_dyn.h
    M mem/hp_malloc_stats.c
    M mem/hp_malloc_stats.h

  Log Message:
  -----------
  OPS-6: HP_MALLOC - don't always allocate the stats lock

This lock is only required when HP_MALLOC_FAST_STATS is enabled, which
gives lockless mem stats, but with come at a price of reduced precision.

  Commit: 8b054a173f678541f1066e96c3d44b4e2027c158
      https://github.com/OpenSIPS/opensips/commit/8b054a173f678541f1066e96c3d44b4e2027c158
  Author: Liviu Chircu <liviu at opensips.org>
  Date:   2019-03-15 (Fri, 15 Mar 2019)

  Changed paths:
    M Makefile.conf.template
    M Makefile.defs
    M main.c
    M mem/common.h
    M mem/mem.c
    M mem/mem.h
    M mem/q_malloc.c
    M mem/q_malloc.h
    A mem/q_malloc_dyn.h
    R mem/qm_malloc_dyn.h
    M mem/shm_mem.c
    M mem/shm_mem.h
    M mi/mi_core.c
    M version.h

  Log Message:
  -----------
  OPS-6: Refactor QM_MALLOC into Q_MALLOC

  Commit: 9349a061a2bc21991ddbdb7db5b1fbdaaf18c528
      https://github.com/OpenSIPS/opensips/commit/9349a061a2bc21991ddbdb7db5b1fbdaaf18c528
  Author: Liviu Chircu <liviu at opensips.org>
  Date:   2019-03-15 (Fri, 15 Mar 2019)

  Changed paths:
    M main.c
    M mem/common.c
    M mem/common.h
    M mem/mem.c
    M mem/mem.h
    M mem/shm_mem.c
    M mem/shm_mem.h

  Log Message:
  -----------
  OPS-6: Add new "allocator" cmdline options (-a, -k, -s)

Description:
    * -a, sets the global allocator
    * -k, sets the PKG allocator, overrides -a
    * -s, sets the SHM allocator, overrides -a

The maximally supported set of string values for each of these options:
    * F_MALLOC
    * F_MALLOC_DBG
    * Q_MALLOC
    * Q_MALLOC_DBG
    * HP_MALLOC
    * HP_MALLOC_DBG
            (case-insensitive matching)

Now, depending whether OpenSIPS was compiled in a more optimized manner,
with less allocators, some of these values MAY not be available, case in
which appropriate error logs will be printed at startup.

  Commit: d2f780cb8be6eef5d9cccd219e5516a9792aae69
      https://github.com/OpenSIPS/opensips/commit/d2f780cb8be6eef5d9cccd219e5516a9792aae69
  Author: Liviu Chircu <liviu at opensips.org>
  Date:   2019-03-15 (Fri, 15 Mar 2019)

  Changed paths:
    M Makefile.conf.template
    M mem/f_malloc.c
    M mem/f_malloc.h
    M mem/f_malloc_dyn.h
    M mem/hp_malloc.h
    M mem/hp_malloc_dyn.h
    M mem/q_malloc.c
    M mem/q_malloc.h
    M mem/q_malloc_dyn.h

  Log Message:
  -----------
  OPS-6: Allocator codebase - fix coding style

    * "type *name"
    * "type *(func)"
    * fix "qm" -> "fm" copy-paste leftover
    * consistent func source strings in q_malloc.c

  Commit: 1f821b1ac864eda75b06675ecc3b7af63f4d56c8
      https://github.com/OpenSIPS/opensips/commit/1f821b1ac864eda75b06675ecc3b7af63f4d56c8
  Author: Liviu Chircu <liviu at opensips.org>
  Date:   2019-03-15 (Fri, 15 Mar 2019)

  Changed paths:
    M Makefile.conf.template
    M Makefile.defs
    M cfg.y
    M main.c
    A mem/common.c
    M mem/common.h
    M mem/f_malloc.c
    M mem/f_malloc.h
    A mem/f_malloc_dyn.h
    M mem/hp_malloc.c
    M mem/hp_malloc.h
    A mem/hp_malloc_dyn.h
    M mem/hp_malloc_stats.c
    M mem/hp_malloc_stats.h
    M mem/mem.c
    M mem/mem.h
    M mem/memtest.c
    M mem/module_info.c
    M mem/q_malloc.c
    M mem/q_malloc.h
    A mem/q_malloc_dyn.h
    M mem/shm_mem.c
    M mem/shm_mem.h
    R mem/vq_malloc.c
    R mem/vq_malloc.h
    M mi/mi_core.c
    M modules/lua/sipstate.c
    M packaging/debian/copyright
    M reactor.c
    M version.h

  Log Message:
  -----------
  Merge branch 'feature/selectable-allocator'

Compare: https://github.com/OpenSIPS/opensips/compare/ad60d1f7cdac...1f821b1ac864