project-base/server-skynet-source-3rd-jemalloc
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
120abd703a
server-skynet-source-3rd-je.../include/jemalloc/internal/tcache_structs.h

70 lines
2.2 KiB
C
Raw Normal View History

Break up headers into constituent parts This is part of a broader change to make header files better represent the dependencies between one another (see https://github.com/jemalloc/jemalloc/issues/533). It breaks up component headers into smaller parts that can be made to have a simpler dependency graph. For the autogenerated headers (smoothstep.h and size_classes.h), no splitting was necessary, so I didn't add support to emit multiple headers.
2017-01-11 10:06:31 +08:00
#ifndef JEMALLOC_INTERNAL_TCACHE_STRUCTS_H
#define JEMALLOC_INTERNAL_TCACHE_STRUCTS_H
Make headers self-contained by fixing `#include`s Header files are now self-contained, which makes the relationships between the files clearer, and crucially allows LSP tools like `clangd` to function correctly in all of our header files. I have verified that the headers are self-contained (aside from the various Windows shims) by compiling them as if they were C files – in a follow-up commit I plan to add this to CI to ensure we don't regress on this front.
2023-06-10 08:37:47 +08:00
#include "jemalloc/internal/jemalloc_preamble.h"
Pull out caching for a bin into its own file. This is the first step towards breaking up the tcache and arena (since they interact primarily at the bin level). It should also make a future arena caching implementation more straightforward.
2017-08-11 05:27:58 +08:00
#include "jemalloc/internal/cache_bin.h"
Hide size class computation behind a layer of indirection. This class removes almost all the dependencies on size_classes.h, accessing the data there only via the new module sc.h, which does not depend on any configuration options. In a subsequent commit, we'll remove the configure-time size class computations, doing them at boot time, instead.
2017-12-15 04:46:39 +08:00
#include "jemalloc/internal/ql.h"
#include "jemalloc/internal/sc.h"
Make headers self-contained by fixing `#include`s Header files are now self-contained, which makes the relationships between the files clearer, and crucially allows LSP tools like `clangd` to function correctly in all of our header files. I have verified that the headers are self-contained (aside from the various Windows shims) by compiling them as if they were C files – in a follow-up commit I plan to add this to CI to ensure we don't regress on this front.
2023-06-10 08:37:47 +08:00
#include "jemalloc/internal/tcache_types.h"
Header refactoring: ticker module - remove from the catchall and unify.
2017-04-20 04:39:33 +08:00
#include "jemalloc/internal/ticker.h"
Header refactoring: break out ql.h dependencies
2017-04-11 08:11:33 +08:00
Tcache: split up fast and slow path data.
2020-04-08 08:48:35 +08:00
/*
* The tcache state is split into the slow and hot path data. Each has a
* pointer to the other, and the data always comes in pairs. The layout of each
* of them varies in practice; tcache_slow lives in the TSD for the automatic
* tcache, and as part of a dynamic allocation for manual allocations. Keeping
* a pointer to tcache_slow lets us treat these cases uniformly, rather than
* splitting up the tcache [de]allocation code into those paths called with the
* TSD tcache and those called with a manual tcache.
*/
Make arena stats collection go through cache bins. This eliminates the need for the arena stats code to "know" about tcaches; all that it needs is a cache_bin_array_descriptor_t to tell it where to find cache_bins whose stats it should aggregate.
2017-08-12 08:34:21 +08:00
Tcache: split up fast and slow path data.
2020-04-08 08:48:35 +08:00
struct tcache_slow_s {
Make arena stats collection go through cache bins. This eliminates the need for the arena stats code to "know" about tcaches; all that it needs is a cache_bin_array_descriptor_t to tell it where to find cache_bins whose stats it should aggregate.
2017-08-12 08:34:21 +08:00
/* Lets us track all the tcaches in an arena. */
Tcache: split up fast and slow path data.
2020-04-08 08:48:35 +08:00
ql_elm(tcache_slow_t) link;
Move tsd link and in_hook after tcache. This can lead to better cache utilization down the common paths where we don't touch the link.
2018-06-22 04:02:49 +08:00
Make arena stats collection go through cache bins. This eliminates the need for the arena stats code to "know" about tcaches; all that it needs is a cache_bin_array_descriptor_t to tell it where to find cache_bins whose stats it should aggregate.
2017-08-12 08:34:21 +08:00
/*
* The descriptor lets the arena find our cache bins without seeing the
* tcache definition. This enables arenas to aggregate stats across
* tcaches without having a tcache dependency.
*/
cache_bin_array_descriptor_t cache_bin_array_descriptor;
/* The arena this tcache is associated with. */
arena_t *arena;
/* Next bin to GC. */
szind_t next_gc_bin;
Optimizing TSD and thread cache layout. 1) Re-organize TSD so that frequently accessed fields are closer to the beginning and more compact. Assuming 64-bit, the first 2.5 cachelines now contains everything needed on tcache fast path, expect the tcache struct itself. 2) Re-organize tcache and tbins. Take lg_fill_div out of tbin, and reduce tbin to 24 bytes (down from 32). Split tbins into tbins_small and tbins_large, and place tbins_small close to the beginning.
2017-04-07 03:35:22 +08:00
/* For small bins, fill (ncached_max >> lg_fill_div). */
Hide size class computation behind a layer of indirection. This class removes almost all the dependencies on size_classes.h, accessing the data there only via the new module sc.h, which does not depend on any configuration options. In a subsequent commit, we'll remove the configure-time size class computations, doing them at boot time, instead.
2017-12-15 04:46:39 +08:00
uint8_t lg_fill_div[SC_NBINS];
Track low_water == -1 case explicitly. The -1 value of low_water indicates if the cache has been depleted and refilled. Track the status explicitly in the tcache struct. This allows the fast path to check if (cur_ptr > low_water), instead of >=, which avoids reaching slow path when the last item is allocated.
2019-08-21 09:14:18 +08:00
/* For small bins, whether has been refilled since last GC. */
bool bin_refilled[SC_NBINS];
Tcache: Add tcache gc delay option. This can reduce flushing frequency for small size classes.
2020-05-12 07:24:17 +08:00
/*
* For small bins, the number of items we can pretend to flush before
* actually flushing.
*/
uint8_t bin_flush_delay_items[SC_NBINS];
Tcache: Hold cache bin allocation explicitly.
2020-03-03 10:28:17 +08:00
/*
* The start of the allocation containing the dynamic allocation for
* either the cache bins alone, or the cache bin memory as well as this
Tcache: split up fast and slow path data.
2020-04-08 08:48:35 +08:00
* tcache_slow_t and its associated tcache_t.
Tcache: Hold cache bin allocation explicitly.
2020-03-03 10:28:17 +08:00
*/
void *dyn_alloc;
Tcache: split up fast and slow path data.
2020-04-08 08:48:35 +08:00
/* The associated bins. */
tcache_t *tcache;
};
struct tcache_s {
tcache_slow_t *tcache_slow;
Add a hard limit on tcache max size class. For locality reasons, tcache bins are integrated in TSD. Allowing all size classes to be cached has little benefit, but takes up much thread local storage. In addition, it complicates the layout which we try hard to optimize.
2020-10-16 07:37:16 +08:00
cache_bin_t bins[TCACHE_NBINS_MAX];
Break up headers into constituent parts This is part of a broader change to make header files better represent the dependencies between one another (see https://github.com/jemalloc/jemalloc/issues/533). It breaks up component headers into smaller parts that can be made to have a simpler dependency graph. For the autogenerated headers (smoothstep.h and size_classes.h), no splitting was necessary, so I didn't add support to emit multiple headers.
2017-01-11 10:06:31 +08:00
};
/* Linkage for list of available (previously used) explicit tcache IDs. */
struct tcaches_s {
union {
tcache_t *tcache;
tcaches_t *next;
};
};
#endif /* JEMALLOC_INTERNAL_TCACHE_STRUCTS_H */
Reference in New Issue Copy Permalink