[OpenSIPS-Devel] [OpenSIPS/opensips] fc9344: HP_MALLOC: Dramatically reduce memory fragmentation
Liviu Chircu
noreply at github.com
Fri Apr 3 13:44:16 EST 2020
Branch: refs/heads/master
Home: https://github.com/OpenSIPS/opensips
Commit: fc9344a5b22d0da088bfa066cd22f2cbd2450338
https://github.com/OpenSIPS/opensips/commit/fc9344a5b22d0da088bfa066cd22f2cbd2450338
Author: Liviu Chircu <liviu at opensips.org>
Date: 2020-04-03 (Fri, 03 Apr 2020)
Changed paths:
M mem/common.h
M mem/hp_malloc.c
M mem/hp_malloc.h
M mem/hp_malloc_dyn.h
M mem/hp_malloc_stats.c
M mem/hp_malloc_stats.h
M statistics.c
Log Message:
-----------
HP_MALLOC: Dramatically reduce memory fragmentation
For HP_MALLOC to offer optimal parallel allocation performance, it
needs the memory to be fragmented: the more available chunks, the better
its performance. However, there needs to be some coalescing as well,
otherwise variable-length allocations past the 10K range will
quickly deplete (fragment) even SHM pools of 10+ GB.
This patch adds the following heuristics to HP_MALLOC and HP_MALLOC_DBG:
* shm_malloc(): only split a fragment if the resulting chunk is at
least MIN_SPLIT_SIZE bytes (default 256 for PKG, 4096 for SHM)
* shm_free(): try to coalesce the next fragment (enough to fix all
fragmentation issues)
Many thanks to 46Labs for supporting this work!
Commit: 3634b398226306a90999318b2aeb37e99191f484
https://github.com/OpenSIPS/opensips/commit/3634b398226306a90999318b2aeb37e99191f484
Author: Liviu Chircu <liviu at opensips.org>
Date: 2020-04-03 (Fri, 03 Apr 2020)
Changed paths:
M mem/hp_malloc_dyn.h
M mem/hp_malloc_stats.c
M mem/hp_malloc_stats.h
Log Message:
-----------
HP_MALLOC: Fix computation of PKG stats
Many thanks to 46Labs for supporting this work!
Commit: 0d6d33e586f050d7ea5f375cab66fc60c4dbe1a2
https://github.com/OpenSIPS/opensips/commit/0d6d33e586f050d7ea5f375cab66fc60c4dbe1a2
Author: Liviu Chircu <liviu at opensips.org>
Date: 2020-04-03 (Fri, 03 Apr 2020)
Changed paths:
M mem/hp_malloc_dyn.h
Log Message:
-----------
HP_MALLOC: Add "re-scanning" logic
Given that HP_MALLOC has fine-grained locking, if the big fragment
shifts down from hash bucket N to bucket N-1 due to another
process performing the allocation, we may actually "lose" it during
our own scan, since bucket N-1 was empty and we're now waiting
for bucket N to unlock. When it does unlock, it will also be empty.
As a solution: retry the scan up to N times, as long as it's feasible!
Many thanks to 46Labs for supporting this work!
Commit: 54d4af2f34fab12637c06605141fdd008b6054b9
https://github.com/OpenSIPS/opensips/commit/54d4af2f34fab12637c06605141fdd008b6054b9
Author: Liviu Chircu <liviu at opensips.org>
Date: 2020-04-03 (Fri, 03 Apr 2020)
Changed paths:
M mem/hp_malloc_dyn.h
Log Message:
-----------
HP_MALLOC: Aggregate SHM statistic updates
It is better to update the stats with a diff of the
attached/detached/split/merged fragment changes. Updating the stats in
a "step by step" manner has the side-effect of creating temporarily
bogus values. (i.e. negative "used" and/or "real_used")
Many thanks to 46Labs for supporting this work!
Compare: https://github.com/OpenSIPS/opensips/compare/1897599fd8a5...54d4af2f34fa
More information about the Devel
mailing list