server-skynet-source-3rd-je.../include/jemalloc/internal
David Goldblatt 0f6c420f83 HPA: Make purging/hugifying more principled.
Before this change, purge/hugify decisions had several sharp edges that could
lead to pathological behavior if tuning parameters weren't carefully chosen.
It's the first of a series; this introduces basic "make every hugepage with
dirty pages purgeable" functionality, and the next commit expands that
functionality to have a smarter policy for picking hugepages to purge.

Previously, the dehugify logic would *never* dehugify a hugepage unless it was
dirtier than the dehugification threshold.  This can lead to situations in which
these pages (which themselves could never be purged) would push us above the
maximum allowed dirty pages in the shard.  This forces immediate purging of any
pages deallocated in non-hugified hugepages, which in turn places nonobvious
practical limitations on the relationships between various config settings.

Instead, we make our preference not to dehugify to purge a soft one rather than
a hard one.  We'll avoid purging them, but only so long as we can do so by
purging non-hugified pages.  If we need to purge them to satisfy our dirty page
limits, or to hugify other, more worthy candidates, we'll still do so.
2021-02-19 15:10:54 -08:00
..
activity_callback.h Add experimental.thread.activity_callback. 2020-11-05 12:33:25 -08:00
arena_externs.h Arena: share bin offsets in a global. 2021-02-04 14:10:43 -08:00
arena_inlines_a.h Cache an arena's index in the arena. 2021-02-04 14:10:43 -08:00
arena_inlines_b.h Report the correct and wrong sizes on sized dealloc bug detection. 2021-02-08 14:42:27 -08:00
arena_stats.h Minor update to locked int 2020-05-12 08:23:15 -07:00
arena_structs.h Use ticker_geom_t for arena tcache decay. 2021-02-04 14:10:43 -08:00
arena_types.h Use ticker_geom_t for arena tcache decay. 2021-02-04 14:10:43 -08:00
assert.h Disentangle assert and util 2017-03-06 15:08:43 -08:00
atomic_c11.h Introduce a backport of C11 atomics 2017-03-03 13:40:59 -08:00
atomic_gcc_atomic.h Workaround to address g++ unused variable warnings 2019-07-30 11:37:56 -07:00
atomic_gcc_sync.h Restrict lwsync to powerpc64 only 2018-10-24 11:18:55 -07:00
atomic_msvc.h Introduce a backport of C11 atomics 2017-03-03 13:40:59 -08:00
atomic.h atomic: add atomic_load_sub_store 2020-07-09 13:41:04 -07:00
background_thread_externs.h Introduce decay module and put decay objects in PA 2020-04-10 13:12:47 -07:00
background_thread_inlines.h PAC: Move in decay. 2020-07-09 13:41:04 -07:00
background_thread_structs.h Add max_per_bg_thd stats for per background thread mutexes. 2019-09-13 09:23:57 -07:00
base_structs.h Move relevant index into the ehooks_t itself. 2019-12-20 10:18:40 -08:00
base.h Rtree: take the base allocator as a parameter. 2020-02-18 11:22:09 -08:00
bin_info.h Extents: Split out extent struct. 2019-09-23 23:06:27 -07:00
bin_stats.h Reduce footprint of bin_t. 2019-11-21 11:08:36 -08:00
bin_types.h Rename extent_t to edata_t. 2019-12-20 10:18:40 -08:00
bin.h Bin: Move stats closer to the mutex. 2021-02-04 14:10:43 -08:00
bit_util.h bit_util: Guarantee popcount's presence. 2020-12-07 06:21:08 -08:00
bitmap.h bit_util: Change ffs functions indexing. 2020-07-30 15:25:23 -07:00
buf_writer.h Unify write callback signature 2020-05-11 14:51:24 -07:00
cache_bin.h Cache bin: Don't reverse flush order. 2021-02-04 14:10:43 -08:00
ckh.h Header refactoring: tsd - cleanup and dependency breaking. 2017-05-01 10:49:56 -07:00
counter.h Migrate counter to use locked int 2020-05-12 08:23:15 -07:00
ctl.h Optimize stats print using partial name-to-mib 2020-12-18 10:39:58 -08:00
decay.h Decay: Take current time as an argument. 2020-04-10 13:12:47 -07:00
div.h Add the div module, which allows fast division by dynamic values. 2017-12-21 14:25:43 -08:00
ecache.h Rename ecache_grow -> geom_grow. 2020-08-19 16:53:21 -07:00
edata_cache.h Rename edata_tree_t -> edata_avail_t. 2020-11-13 13:42:11 -08:00
edata.h Edata: Move sn into its own field. 2021-02-19 15:10:54 -08:00
ehooks.h Ehooks: Fix a build warning. 2020-02-25 10:21:03 -08:00
emap.h Tcache fluhing: increase cache miss parallelism. 2021-02-04 14:10:43 -08:00
emitter.h Unify write callback signature 2020-05-11 14:51:24 -07:00
eset.h Use the flat bitmap for eset and psset bitmaps. 2021-02-19 15:10:54 -08:00
exp_grow.h Rename geom_grow -> exp_grow. 2020-11-13 13:42:33 -08:00
extent_dss.h Header refactoring: unify and de-catchall extent_dss. 2017-05-31 13:08:45 -07:00
extent_mmap.h Header refactoring: unify and de-catchall extent_mmap module. 2017-05-31 13:08:45 -07:00
extent.h PA->PAC: Make extent.c forget about PA. 2020-07-09 13:41:04 -07:00
flat_bitmap.h flat bitmap: Add a bitwise and/or/not. 2021-02-04 20:58:31 -08:00
fxp.h fxp: Add FXP_INIT_PERCENT 2021-02-04 20:58:31 -08:00
hash.h Fix Undefined Behavior in hash.h 2020-02-13 12:25:26 -08:00
hook.h Hooks: Protect against reentrancy. 2018-05-18 11:43:03 -07:00
hpa_central.h Use the edata_cache_small_t in the HPA. 2020-11-05 12:34:43 -08:00
hpa_opts.h HPA: Make dirty_mult configurable. 2021-02-04 20:58:31 -08:00
hpa.h Pull HPA options into a containing struct. 2021-02-04 20:58:31 -08:00
hpdata.h HPA: Make purging/hugifying more principled. 2021-02-19 15:10:54 -08:00
inspect.h Extents: Split out introspection functionality. 2019-12-20 10:18:40 -08:00
jemalloc_internal_decls.h DragonFlyBSD build support. 2020-10-27 12:35:19 -07:00
jemalloc_internal_defs.h.in Mac OS: Tag mapped pages. 2021-02-03 15:05:53 -08:00
jemalloc_internal_externs.h SEC: Expand option configurability. 2021-02-19 15:10:54 -08:00
jemalloc_internal_includes.h Consolidate prof inline function headers 2020-05-11 12:24:56 -07:00
jemalloc_internal_inlines_a.h Use ticker_geom_t for arena tcache decay. 2021-02-04 14:10:43 -08:00
jemalloc_internal_inlines_b.h Tcache: split up fast and slow path data. 2020-04-14 15:20:19 -07:00
jemalloc_internal_inlines_c.h Inline malloc fastpath into operator new. 2021-02-08 14:17:47 -08:00
jemalloc_internal_macros.h Use __forceinline for JEMALLOC_ALWAYS_INLINE on msvc 2019-11-12 13:50:25 -08:00
jemalloc_internal_types.h Inline malloc fastpath into operator new. 2021-02-08 14:17:47 -08:00
jemalloc_preamble.h.in utrace support with label based signature. 2020-11-30 11:43:00 -08:00
large_externs.h Record request size in prof info 2021-01-07 20:39:49 -08:00
lockedint.h Migrate counter to use locked int 2020-05-12 08:23:15 -07:00
log.h Logging: capitalize log macro. 2017-10-02 20:44:43 -07:00
malloc_io.h Unify write callback signature 2020-05-11 14:51:24 -07:00
mutex_pool.h Header refactoring: unify and de-catchall mutex_pool. 2017-05-31 13:08:45 -07:00
mutex_prof.h Add prof stats mutex stats 2021-01-07 20:39:49 -08:00
mutex.h Add max_per_bg_thd stats for per background thread mutexes. 2019-09-13 09:23:57 -07:00
nstime.h High Resolution Timestamps for Profiling 2020-06-15 12:12:49 -07:00
pa.h SEC: Expand option configurability. 2021-02-19 15:10:54 -08:00
pac.h Add a per-arena oversize_threshold. 2020-11-13 13:45:35 -08:00
pages.h Introduce hpdata_t. 2020-12-07 06:21:08 -08:00
pai.h Add a batch allocation interface to the PAI. 2021-02-19 15:10:54 -08:00
peak_event.h Add thread.peak.[read|reset] mallctls. 2020-06-11 13:54:22 -07:00
peak.h Add peak_t, for tracking allocator net max. 2020-06-11 13:54:22 -07:00
ph.h Add any() and remove_any() to ph. 2017-03-07 10:25:33 -08:00
private_namespace.sh Automatically generate private symbol name mangling macros. 2017-05-11 23:06:54 -07:00
private_symbols.sh Automatically generate private symbol name mangling macros. 2017-05-11 23:06:54 -07:00
prng.h PRNG: Remove atomic functionality. 2020-10-05 19:55:57 -07:00
prof_data.h Do not create size class tables for non-prof builds 2020-08-24 20:10:02 -07:00
prof_externs.h Track per size class internal fragmentation 2021-01-07 20:39:49 -08:00
prof_inlines.h Implement opt.cache_oblivious. 2021-02-11 11:32:01 -08:00
prof_log.h Relocate a few prof utilities to the right modules 2020-06-29 14:27:50 -07:00
prof_recent.h Correct usize in prof last-N record 2020-09-09 13:31:35 -07:00
prof_stats.h Track per size class internal fragmentation 2021-01-07 20:39:49 -08:00
prof_structs.h Record request size in prof info 2021-01-07 20:39:49 -08:00
prof_sys.h Move unwind to prof_sys 2020-06-29 14:27:50 -07:00
prof_types.h Do not create size class tables for non-prof builds 2020-08-24 20:10:02 -07:00
psset.h HPA: Make purging/hugifying more principled. 2021-02-19 15:10:54 -08:00
public_namespace.sh Replace tabs following #define with spaces. 2017-01-20 21:45:53 -08:00
public_unnamespace.sh Fix name mangling for stress tests. 2014-01-16 17:38:01 -08:00
ql.h QL, QR: Add documentation. 2020-04-11 10:32:11 -07:00
qr.h QL, QR: Add documentation. 2020-04-11 10:32:11 -07:00
quantum.h LQ_QUANTUM should be 4 on mips64 hardware. 2020-05-14 10:30:37 -07:00
rb.h Header refactoring: break out rb.h dependencies 2017-04-11 11:52:30 -07:00
rtree_tsd.h Clean compilation -Wextra 2018-07-09 21:40:42 -07:00
rtree.h Rtree: Remove the per-field accessors. 2020-04-10 13:12:47 -07:00
safety_check.h Report the correct and wrong sizes on sized dealloc bug detection. 2021-02-08 14:42:27 -08:00
sc.h Avoid a -Wundef warning on LG_SLAB_MAXREGS. 2020-09-17 10:05:40 -07:00
sec_opts.h SEC: Expand option configurability. 2021-02-19 15:10:54 -08:00
sec.h SEC: Expand option configurability. 2021-02-19 15:10:54 -08:00
seq.h Add the Seq module, a simple seqlock implementation. 2018-05-18 11:43:03 -07:00
slab_data.h Rearrange slab data and constants 2019-09-23 23:06:27 -07:00
smoothstep.h Cleanup smoothstep.sh / .h. 2017-05-25 16:52:10 -07:00
smoothstep.sh Cleanup smoothstep.sh / .h. 2017-05-25 16:52:10 -07:00
spin.h Power: disable the CPU_SPINWAIT macro. 2017-10-04 18:37:23 -07:00
stats.h HPA: Add stats for the hpa_shard. 2020-10-23 11:14:34 -07:00
sz.h Implement opt.cache_oblivious. 2021-02-11 11:32:01 -08:00
tcache_externs.h Switch from opt.lg_tcache_max to opt.tcache_max 2020-10-22 20:40:41 -07:00
tcache_inlines.h Allow opt.tcache_max to accept small size classes. 2020-10-24 20:43:44 -07:00
tcache_structs.h Add a hard limit on tcache max size class. 2020-10-16 13:49:51 -07:00
tcache_types.h Add a hard limit on tcache max size class. 2020-10-16 13:49:51 -07:00
test_hooks.h Rename hooks module to test_hooks. 2018-05-18 11:43:03 -07:00
thread_event.h Add batch allocation API 2020-07-31 09:16:50 -07:00
ticker.h Add ticker_geom_t. 2021-02-04 14:10:43 -08:00
tsd_generic.h Fix recursive malloc during bootstrap on QNX 2020-12-18 10:05:59 -08:00
tsd_malloc_thread_cleanup.h Enforce TLS_MODEL attribute. 2019-04-16 11:07:15 -07:00
tsd_tls.h Enforce TLS_MODEL attribute. 2019-04-16 11:07:15 -07:00
tsd_types.h Header refactoring: tsd - cleanup and dependency breaking. 2017-05-01 10:49:56 -07:00
tsd_win.h Header refactoring: tsd - cleanup and dependency breaking. 2017-05-01 10:49:56 -07:00
tsd.h Use ticker_geom_t for arena tcache decay. 2021-02-04 14:10:43 -08:00
typed_list.h Add empty test and concat for typed list 2020-07-31 09:16:50 -07:00
util.h Tcache flush: prefetch edata contents. 2021-02-04 14:10:43 -08:00
witness.h Track per size class internal fragmentation 2021-01-07 20:39:49 -08:00