Qi Wang 36bd90b962 Optimizing TSD and thread cache layout. ...

1) Re-organize TSD so that frequently accessed fields are closer to the
beginning and more compact.  Assuming 64-bit, the first 2.5 cachelines now
contains everything needed on tcache fast path, expect the tcache struct itself.

2) Re-organize tcache and tbins.  Take lg_fill_div out of tbin, and reduce tbin
to 24 bytes (down from 32). Split tbins into tbins_small and tbins_large, and
place tbins_small close to the beginning.

2017-04-07 14:06:17 -07:00

..

arena_externs.h

Optimizing TSD and thread cache layout.

2017-04-07 14:06:17 -07:00

arena_inlines_a.h

Integrate auto tcache into TSD.

2017-04-07 09:55:14 -07:00

arena_inlines_b.h

Further specialize arena_[s]dalloc() tcache fast path.

2017-03-22 18:33:32 -07:00

arena_structs_a.h

Move arena_slab_data_t's nfree into extent_t's e_bits.

2017-03-27 22:43:39 -07:00

arena_structs_b.h

Transition arena struct fields to C11 atomics

2017-04-05 16:25:37 -07:00

arena_types.h

Implement two-phase decay-based purging.

2017-03-15 13:13:47 -07:00

assert.h

Disentangle assert and util

2017-03-06 15:08:43 -08:00

atomic_c11.h

Introduce a backport of C11 atomics

2017-03-03 13:40:59 -08:00

atomic_gcc_atomic.h

Insert not_reached after an exhaustive switch

2017-03-06 15:08:43 -08:00

atomic_gcc_sync.h

Introduce a backport of C11 atomics

2017-03-03 13:40:59 -08:00

atomic_msvc.h

Introduce a backport of C11 atomics

2017-03-03 13:40:59 -08:00

atomic.h

Remove the pre-C11-atomics API, which is now unused

2017-04-05 16:25:37 -07:00

base_externs.h

Break up headers into constituent parts

2017-01-12 15:43:51 -08:00

base_inlines.h

Remove extraneous parens around return arguments.

2017-01-20 21:43:07 -08:00

base_structs.h

Make base_t's extent_hooks field C11-atomic

2017-04-05 16:25:37 -07:00

base_types.h

Break up headers into constituent parts

2017-01-12 15:43:51 -08:00

bit_util.h

Disentangle assert and util

2017-03-06 15:08:43 -08:00

bitmap_externs.h

Implement bitmap_ffu(), which finds the first unset bit.

2017-03-24 17:52:46 -07:00

bitmap_inlines.h

Remove BITMAP_USE_TREE.

2017-03-27 12:18:40 -07:00

bitmap_structs.h

Remove BITMAP_USE_TREE.

2017-03-27 12:18:40 -07:00

bitmap_types.h

Remove BITMAP_USE_TREE.

2017-03-27 12:18:40 -07:00

ckh_externs.h

Break up headers into constituent parts

2017-01-12 15:43:51 -08:00

ckh_structs.h

Break up headers into constituent parts

2017-01-12 15:43:51 -08:00

ckh_types.h

Replace tabs following #define with spaces.

2017-01-20 21:45:53 -08:00

ctl_externs.h

Refactor mutex profiling code with x-macros.

2017-03-23 00:03:28 -07:00

ctl_structs.h

Refactor mutex profiling code with x-macros.

2017-03-23 00:03:28 -07:00

ctl_types.h

Profile per arena base mutex, instead of just a0.

2017-03-23 00:03:28 -07:00

extent_dss_externs.h

Break up headers into constituent parts

2017-01-12 15:43:51 -08:00

extent_dss_structs.h

Break up headers into constituent parts

2017-01-12 15:43:51 -08:00

extent_dss_types.h

Replace tabs following #define with spaces.

2017-01-20 21:45:53 -08:00

extent_externs.h

Incorporate szind/slab into rtree leaves.

2017-03-22 18:33:32 -07:00

extent_inlines.h

Move arena-tracking atomics in jemalloc.c to C11-style

2017-04-05 16:25:37 -07:00

extent_mmap_externs.h

Break up headers into constituent parts

2017-01-12 15:43:51 -08:00

extent_structs.h

Transition e_prof_tctx in struct extent to C11 atomics

2017-04-04 16:46:04 -07:00

extent_types.h

Disentangle arena and extent locking.

2017-02-01 16:43:46 -08:00

hash_inlines.h

Remove extraneous parens around return arguments.

2017-01-20 21:43:07 -08:00

jemalloc_internal_decls.h

Replace tabs following #define with spaces.

2017-01-20 21:45:53 -08:00

jemalloc_internal_defs.h.in

Fix pages_purge_forced() to discard pages on non-Linux systems.

2017-03-13 18:19:57 -07:00

jemalloc_internal_macros.h

Make type abbreviations consistent: ssize_t is zd everywhere

2017-03-06 18:49:19 -08:00

jemalloc_internal.h.in

Optimizing TSD and thread cache layout.

2017-04-07 14:06:17 -07:00

large_externs.h

Incorporate szind/slab into rtree leaves.

2017-03-22 18:33:32 -07:00

malloc_io.h

Disentangle assert and util

2017-03-06 15:08:43 -08:00

mutex_externs.h

Added "stats.mutexes.reset" mallctl to reset all mutex stats.

2017-03-23 00:03:28 -07:00

mutex_inlines.h

Make the mutex n_waiting_thds field a C11-style atomic

2017-04-05 16:25:37 -07:00

mutex_structs.h

Make the mutex n_waiting_thds field a C11-style atomic

2017-04-05 16:25:37 -07:00

mutex_types.h

Make the mutex n_waiting_thds field a C11-style atomic

2017-04-05 16:25:37 -07:00

nstime_externs.h

Output bin lock profiling results to malloc_stats.

2017-03-23 00:03:28 -07:00

nstime_structs.h

Break up headers into constituent parts

2017-01-12 15:43:51 -08:00

nstime_types.h

First stage of mutex profiling.

2017-03-23 00:03:28 -07:00

pages_externs.h

Break up headers into constituent parts

2017-01-12 15:43:51 -08:00

pages_types.h

Fix pages_purge_forced() to discard pages on non-Linux systems.

2017-03-13 18:19:57 -07:00

ph.h

Add any() and remove_any() to ph.

2017-03-07 10:25:33 -08:00

private_namespace.sh

Replace tabs following #define with spaces.

2017-01-20 21:45:53 -08:00

private_symbols.txt

Integrate auto tcache into TSD.

2017-04-07 09:55:14 -07:00

private_unnamespace.sh

Refactor to support more varied testing.

2013-12-03 22:06:59 -08:00

prng_inlines.h

Convert prng module to use C11-style atomics

2017-04-04 16:45:52 -07:00

prng_types.h

Replace tabs following #define with spaces.

2017-01-20 21:45:53 -08:00

prof_externs.h

Added lock profiling and output for global locks (ctl, prof and base).

2017-03-23 00:03:28 -07:00

prof_inlines_a.h

Convert accumbytes in prof_accum_t to C11 atomics, when possible

2017-04-05 16:25:37 -07:00

prof_inlines_b.h

Push down iealloc() calls.

2017-03-22 18:33:32 -07:00

prof_structs.h

Convert accumbytes in prof_accum_t to C11 atomics, when possible

2017-04-05 16:25:37 -07:00

prof_types.h

Convert arena->prof_accumbytes synchronization to atomics.

2017-02-16 09:39:46 -08:00

public_namespace.sh

Replace tabs following #define with spaces.

2017-01-20 21:45:53 -08:00

public_unnamespace.sh

Fix name mangling for stress tests.

2014-01-16 17:38:01 -08:00

ql.h

Replace tabs following #define with spaces.

2017-01-20 21:45:53 -08:00

qr.h

Replace tabs following #define with spaces.

2017-01-20 21:45:53 -08:00

rb.h

Replace tabs following #define with spaces.

2017-01-20 21:45:53 -08:00

rtree_externs.h

Make the tsd member init functions to take tsd_t * type.

2017-04-04 14:06:07 -07:00

rtree_inlines.h

Integrate auto tcache into TSD.

2017-04-07 09:55:14 -07:00

rtree_structs.h

Optimizing TSD and thread cache layout.

2017-04-07 14:06:17 -07:00

rtree_types.h

Remove the leafkey NULL check in leaf_elm_lookup.

2017-04-04 00:27:35 -07:00

size_classes.sh

Implement compact rtree leaf element representation.

2017-03-22 18:33:32 -07:00

smoothstep.h

Replace tabs following #define with spaces.

2017-01-20 21:45:53 -08:00

smoothstep.sh

Replace tabs following #define with spaces.

2017-01-20 21:45:53 -08:00

spin_inlines.h

Enhance spin_adaptive() to yield after several iterations.

2017-02-08 18:50:03 -08:00

spin_structs.h

Break up headers into constituent parts

2017-01-12 15:43:51 -08:00

spin_types.h

Replace spin_init() with SPIN_INITIALIZER.

2017-02-08 18:50:03 -08:00

stats_externs.h

Break up headers into constituent parts

2017-01-12 15:43:51 -08:00

stats_structs.h

Refactor mutex profiling code with x-macros.

2017-03-23 00:03:28 -07:00

stats_types.h

Implement two-phase decay-based purging.

2017-03-15 13:13:47 -07:00

tcache_externs.h

Integrate auto tcache into TSD.

2017-04-07 09:55:14 -07:00

tcache_inlines.h

Optimizing TSD and thread cache layout.

2017-04-07 14:06:17 -07:00

tcache_structs.h

Optimizing TSD and thread cache layout.

2017-04-07 14:06:17 -07:00

tcache_types.h

Optimizing TSD and thread cache layout.

2017-04-07 14:06:17 -07:00

ticker_inlines.h

Remove extraneous parens around return arguments.

2017-01-20 21:43:07 -08:00

ticker_structs.h

Break up headers into constituent parts

2017-01-12 15:43:51 -08:00

ticker_types.h

Break up headers into constituent parts

2017-01-12 15:43:51 -08:00

tsd_externs.h

Integrate auto tcache into TSD.

2017-04-07 09:55:14 -07:00

tsd_inlines.h

Do proper cleanup for tsd_state_reincarnated.

2017-04-04 00:34:49 -07:00

tsd_structs.h

Optimizing TSD and thread cache layout.

2017-04-07 14:06:17 -07:00

tsd_types.h

Optimizing TSD and thread cache layout.

2017-04-07 14:06:17 -07:00

util.h

Disentangle assert and util

2017-03-06 15:08:43 -08:00

witness_externs.h

Add witness_assert_depth[_to_rank]().

2017-02-01 16:43:46 -08:00

witness_inlines.h

Add witness_assert_depth[_to_rank]().

2017-02-01 16:43:46 -08:00

witness_structs.h

Break up headers into constituent parts

2017-01-12 15:43:51 -08:00

witness_types.h

Update a comment.

2017-02-26 11:05:27 -08:00