server-skynet-source-3rd-je.../include/jemalloc/internal/huge.h

/******************************************************************************/
#ifdef JEMALLOC_H_TYPES

#endif /* JEMALLOC_H_TYPES */
/******************************************************************************/
#ifdef JEMALLOC_H_STRUCTS

#endif /* JEMALLOC_H_STRUCTS */
/******************************************************************************/
#ifdef JEMALLOC_H_EXTERNS

void	*huge_malloc(tsd_t *tsd, arena_t *arena, size_t size, bool zero);
void	*huge_palloc(tsd_t *tsd, arena_t *arena, size_t usize, size_t alignment,
    bool zero);
bool	huge_ralloc_no_move(void *ptr, size_t oldsize, size_t size,
    size_t extra, bool zero);
void	*huge_ralloc(tsd_t *tsd, arena_t *arena, void *ptr, size_t oldsize,
    size_t size, size_t extra, size_t alignment, bool zero,
    bool try_tcache_dalloc);
#ifdef JEMALLOC_JET
typedef void (huge_dalloc_junk_t)(void *, size_t);
extern huge_dalloc_junk_t *huge_dalloc_junk;
#endif
void	huge_dalloc(tsd_t *tsd, void *ptr);
size_t	huge_salloc(const void *ptr);
prof_tctx_t	*huge_prof_tctx_get(const void *ptr);
void	huge_prof_tctx_set(const void *ptr, prof_tctx_t *tctx);
bool	huge_boot(void);
void	huge_prefork(void);
void	huge_postfork_parent(void);
void	huge_postfork_child(void);

#endif /* JEMALLOC_H_EXTERNS */
/******************************************************************************/
#ifdef JEMALLOC_H_INLINES

#endif /* JEMALLOC_H_INLINES */
/******************************************************************************/
Refactor jemalloc.c into multiple source files. Fix a stats bug in large object curruns accounting. Replace tcache_bin_fill() with arena_tcache_fill(), and fix a bug in an OOM error path. Fix API name mangling to coexist with __attribute__((malloc)). 2010-01-17 01:53:50 +08:00			`/******************************************************************************/`
			`#ifdef JEMALLOC_H_TYPES`

			`#endif /* JEMALLOC_H_TYPES */`
			`/******************************************************************************/`
			`#ifdef JEMALLOC_H_STRUCTS`

			`#endif /* JEMALLOC_H_STRUCTS */`
			`/******************************************************************************/`
			`#ifdef JEMALLOC_H_EXTERNS`

Convert all tsd variables to reside in a single tsd structure. 2014-09-23 12:09:23 +08:00			`void huge_malloc(tsd_t tsd, arena_t *arena, size_t size, bool zero);`
Normalize size classes. Normalize size classes to use the same number of size classes per size doubling (currently hard coded to 4), across the intire range of size classes. Small size classes already used this spacing, but in order to support this change, additional small size classes now fill [4 KiB .. 16 KiB). Large size classes range from [16 KiB .. 4 MiB). Huge size classes now support non-multiples of the chunk size in order to fill (4 MiB .. 16 MiB). 2014-10-06 08:54:10 +08:00			`void huge_palloc(tsd_t tsd, arena_t *arena, size_t usize, size_t alignment,`
Convert all tsd variables to reside in a single tsd structure. 2014-09-23 12:09:23 +08:00			`bool zero);`
Extract profiling code from [re]allocation functions. Extract profiling code from malloc(), imemalign(), calloc(), realloc(), mallocx(), rallocx(), and xallocx(). This slightly reduces the amount of code compiled into the fast paths, but the primary benefit is the combinatorial complexity reduction. Simplify iralloc[t]() by creating a separate ixalloc() that handles the no-move cases. Further simplify [mrxn]allocx() (and by implication [mrn]allocm()) to make request size overflows due to size class and/or alignment constraints trigger undefined behavior (detected by debug-only assertions). Report ENOMEM rather than EINVAL if an OOM occurs during heap profiling backtrace creation in imemalign(). This bug impacted posix_memalign() and aligned_alloc(). 2014-01-13 07:05:44 +08:00			`bool huge_ralloc_no_move(void *ptr, size_t oldsize, size_t size,`
Attempt to expand huge allocations in-place. This adds support for expanding huge allocations in-place by requesting memory at a specific address from the chunk allocator. It's currently only implemented for the chunk recycling path, although in theory it could also be done by optimistically allocating new chunks. On Linux, it could attempt an in-place mremap. However, that won't work in practice since the heap is grown downwards and memory is not unmapped (in a normal build, at least). Repeated vector reallocation micro-benchmark: #include <string.h> #include <stdlib.h> int main(void) { for (size_t i = 0; i < 100; i++) { void ptr = NULL; size_t old_size = 0; for (size_t size = 4; size < (1 << 30); size = 2) { ptr = realloc(ptr, size); if (!ptr) return 1; memset(ptr + old_size, 0xff, size - old_size); old_size = size; } free(ptr); } } The glibc allocator fails to do any in-place reallocations on this benchmark once it passes the M_MMAP_THRESHOLD (default 128k) but it elides the cost of copies via mremap, which is currently not something that jemalloc can use. With this improvement, jemalloc still fails to do any in-place huge reallocations for the first outer loop, but then succeeds 100% of the time for the remaining 99 iterations. The time spent doing allocations and copies drops down to under 5%, with nearly all of it spent doing purging + faulting (when huge pages are disabled) and the array memset. An improved mremap API (MREMAP_RETAIN - #138) would be far more general but this is a portable optimization and would still be useful on Linux for xallocx. Numbers with transparent huge pages enabled: glibc (copies elided via MREMAP_MAYMOVE): 8.471s jemalloc: 17.816s jemalloc + no-op madvise: 13.236s jemalloc + this commit: 6.787s jemalloc + this commit + no-op madvise: 6.144s Numbers with transparent huge pages disabled: glibc (copies elided via MREMAP_MAYMOVE): 15.403s jemalloc: 39.456s jemalloc + no-op madvise: 12.768s jemalloc + this commit: 15.534s jemalloc + this commit + no-op madvise: 6.354s Closes #137 2014-10-04 13:39:32 +08:00			`size_t extra, bool zero);`
Convert all tsd variables to reside in a single tsd structure. 2014-09-23 12:09:23 +08:00			`void huge_ralloc(tsd_t tsd, arena_t arena, void ptr, size_t oldsize,`
			`size_t size, size_t extra, size_t alignment, bool zero,`
			`bool try_tcache_dalloc);`
Add junk/zero filling unit tests, and fix discovered bugs. Fix growing large reallocation to junk fill new space. Fix huge deallocation to junk fill when munmap is disabled. 2014-01-08 08:47:56 +08:00			`#ifdef JEMALLOC_JET`
			`typedef void (huge_dalloc_junk_t)(void *, size_t);`
			`extern huge_dalloc_junk_t *huge_dalloc_junk;`
			`#endif`
Use regular arena allocation for huge tree nodes. This avoids grabbing the base mutex, as a step towards fine-grained locking for huge allocations. The thread cache also provides a tiny (~3%) improvement for serial huge allocations. 2014-10-06 15:42:10 +08:00			`void huge_dalloc(tsd_t tsd, void ptr);`
Refactor jemalloc.c into multiple source files. Fix a stats bug in large object curruns accounting. Replace tcache_bin_fill() with arena_tcache_fill(), and fix a bug in an OOM error path. Fix API name mangling to coexist with __attribute__((malloc)). 2010-01-17 01:53:50 +08:00			`size_t huge_salloc(const void *ptr);`
Implement per thread heap profiling. Rename data structures (prof_thr_cnt_t-->prof_tctx_t, prof_ctx_t-->prof_gctx_t), and convert to storing a prof_tctx_t for sampled objects. Convert PROF_ALLOC_PREP() to prof_alloc_prep(), since precise backtrace depth within jemalloc functions is no longer an issue (pprof prunes irrelevant frames). Implement mallctl's: - prof.reset implements full sample data reset, and optional change of sample interval. - prof.lg_sample reads the current sample interval (opt.lg_prof_sample was the permanent source of truth prior to prof.reset). - thread.prof.name provides naming capability for threads within heap profile dumps. - thread.prof.active makes it possible to activate/deactivate heap profiling for individual threads. Modify the heap dump files to contain per thread heap profile data. This change is incompatible with the existing pprof, which will require enhancements to read and process the enriched data. 2014-08-19 07:22:13 +08:00			`prof_tctx_t huge_prof_tctx_get(const void ptr);`
			`void huge_prof_tctx_set(const void ptr, prof_tctx_t tctx);`
Refactor jemalloc.c into multiple source files. Fix a stats bug in large object curruns accounting. Replace tcache_bin_fill() with arena_tcache_fill(), and fix a bug in an OOM error path. Fix API name mangling to coexist with __attribute__((malloc)). 2010-01-17 01:53:50 +08:00			`bool huge_boot(void);`
Fix fork-related bugs. Acquire/release arena bin locks as part of the prefork/postfork. This bug made deadlock in the child between fork and exec a possibility. Split jemalloc_postfork() into jemalloc_postfork_{parent,child}() so that the child can reinitialize mutexes rather than unlocking them. In practice, this bug tended not to cause problems. 2012-03-14 07:31:41 +08:00			`void huge_prefork(void);`
			`void huge_postfork_parent(void);`
			`void huge_postfork_child(void);`
Refactor jemalloc.c into multiple source files. Fix a stats bug in large object curruns accounting. Replace tcache_bin_fill() with arena_tcache_fill(), and fix a bug in an OOM error path. Fix API name mangling to coexist with __attribute__((malloc)). 2010-01-17 01:53:50 +08:00
			`#endif /* JEMALLOC_H_EXTERNS */`
			`/******************************************************************************/`
			`#ifdef JEMALLOC_H_INLINES`

			`#endif /* JEMALLOC_H_INLINES */`
			`/******************************************************************************/`