Commit Graph

1627 Commits

Author SHA1 Message Date
Jason Evans
f3ff75289b Rewrite red-black trees.
Use left-leaning 2-3 red-black trees instead of left-leaning 2-3-4
red-black trees.  This reduces maximum tree height from (3 lg n) to
(2 lg n).

Do lazy balance fixup, rather than transforming the tree during the down
pass.  This improves insert/remove speed by ~30%.

Use callback-based iteration rather than macros.
2010-02-28 15:00:18 -08:00
Jason Evans
fbb504def6 Don't implicitly enable interval-based profiling. 2010-02-16 15:46:57 -08:00
Jason Evans
f894f74d36 Fix a bug in nmalloc stats. 2010-02-12 14:46:37 -08:00
Jason Evans
65aae2cf57 Fix a man page typo. 2010-02-11 16:46:42 -08:00
Jason Evans
3b5ee5e857 Fix #include ordering for mb.h.
Include mb.h after mutex.h, in case it actually has to use the
mutex-based memory barrier implementation.
2010-02-11 15:56:23 -08:00
Jason Evans
59e9be0f5f Avoid extra dumping for JEMALLOC_OPTIONS=L. 2010-02-11 15:18:17 -08:00
Jason Evans
cd90fca928 Wrap mallctl* references with JEMALLOC_P(). 2010-02-11 14:55:25 -08:00
Jason Evans
376b1529a3 Restructure source tree. 2010-02-11 14:45:59 -08:00
Jason Evans
fe5faa2cc5 Remove tracing (--enable-trace).
Remove all functionality related to tracing.  This functionality was
useful for understanding memory fragmentation during early algorithmic
design of jemalloc, but it had little utility for non-trivial
applications, due to the sheer volume of data written to disk.
2010-02-11 13:38:12 -08:00
Jason Evans
d34f9e7e93 Implement interval-based heap profile dumping.
Add mallctl interfaces for profiling parameters.

Fix a file descriptor leak in heap profile dumping.
2010-02-11 13:19:21 -08:00
Jason Evans
b01a6c2057 Add JEMALLOC_PROF_PREFIX support.
If JEMALLOC_PROF_PREFIX is set in the environment, use it as the
filename prefix when dumping heap profiles, rather than "jeprof".
2010-02-11 10:25:36 -08:00
Jason Evans
c717718115 Dump /proc/<pid>/maps in heap profiles. 2010-02-11 09:25:56 -08:00
Jason Evans
3383af6c2d Fix a profiling bootstrap bug.
Bootstrap profiling in three stages, so that it is usable by the time
the first application allocation occurs.
2010-02-11 08:59:06 -08:00
Jason Evans
b27805b363 Various heap profiling improvements.
Add the --disable-prof-libgcc configure option, and add backtracing
based on libgcc, which is used by default.

Fix a bug in hash().

Fix various configuration-dependent compilation errors.
2010-02-10 18:20:38 -08:00
Jason Evans
6109fe07a1 Implement allocation profiling and leack checking.
Add the --enable-prof and --enable-prof-libunwind configure options.

Add the B/b, F/f, I/i, L/l, and U/u JEMALLOC_OPTIONS.

Interval-based profile dump triggering is not yet implemented.

Add supporting generic code:
* Add memory barriers.
* Add prn (LCG PRNG).
* Add hash (Murmur hash function).
* Add ckh (cuckoo hash tables).
2010-02-10 10:37:57 -08:00
Jason Evans
13668262d1 Fix some comments and whitespace. 2010-01-31 03:57:29 -08:00
Jason Evans
990d10cefb Fix large object stats collection bugs. 2010-01-31 03:49:35 -08:00
Jason Evans
a0bf242230 Fix bootstrapping crash.
If a custom small_size2bin table was required due to non-default size
class settings, memory allocation prior to initializing chunk parameters
would cause a crash due to division by 0.  The fix re-orders the various
*_boot() function calls.

Bootstrapping is simpler now than it was before the base allocator
started just using the chunk allocator directly.  This allows
arena_boot[01]() to be combined.

Add error detection for pthread_atfork() and atexit() function calls.
2010-01-29 14:30:41 -08:00
Jason Evans
d8f565f239 Remove tcache bin sorting during flush.
This feature caused significant performance degradation, and the
fragmentation reduction benefits were difficult to quantify.
2010-01-29 13:37:31 -08:00
Jason Evans
c66aaf1476 Statistics fixes and cleanup.
Fix a type mismatch for "arenas.nlruns" mallctl access.  This bug caused
a crash during statistics printing on 64-bit systems.

Fix the "stats.active" mallctl to include active memory in huge objects.

Report active bytes for the whole application, as well as per arena.

Remove several unused variables.
2010-01-29 11:24:19 -08:00
Jason Evans
4fb7f51337 Fix a chunk leak in chunk_alloc_mmap().
A missing 'else' in chunk_alloc_mmap() caused an extra chunk to be
allocated every time the optimistic alignment path was entered, since
the following block would always be executed immediately afterward.
This chunk leak caused no increase in physical memory usage, but virtual
memory could grow until resource exaustion caused allocation failures.
2010-01-27 18:27:09 -08:00
Jason Evans
95833311f1 madvise(..., MADV_{RANODOM,NOSYNC}) swap files.
Initialize malloc before calling into the ctl_*() functions.
2010-01-27 13:47:28 -08:00
Jason Evans
3c2343518c Implement mallctl{nametomib,bymib}().
Replace chunk stats code that was missing locking; this fixes a race
condition that could corrupt chunk statistics.

Converting malloc_stats_print() to use mallctl*().

Add a missing semicolon in th DSS code.

Convert malloc_tcache_flush() to a mallctl.

Convert malloc_swap_enable() to a set of mallctl's.
2010-01-27 13:10:56 -08:00
Jason Evans
fbbb624fc1 Simplify malloc_{pre,post}fork().
Revert to simpler lock acquistion/release code in
malloc_{pre,post}fork(), since dynamic arena rebalancing is no longer
implemented.
2010-01-24 17:56:48 -08:00
Jason Evans
68ddb6736d Print merged arena stats iff multiple arenas. 2010-01-24 17:21:47 -08:00
Jason Evans
41631d0061 Modify chunk_alloc() to support optional zeroing.
Use optional zeroing in arena_chunk_alloc() to avoid needless zeroing of
chunks.  This is particularly important in the context of swapfile and
DSS allocation, since a long-lived application may commonly recycle
chunks.
2010-01-24 17:13:07 -08:00
Jason Evans
bc25a47ee0 Various minor cleanups.
Clean up whitespace.

Lock access of swap_avail when printing stats.

Use inttypes.h for portable printf() format specifiers, specifically for
uint64_t (PRIu64).

Change highchunks and curchunks stats from (unsigned long) to (size_t).
2010-01-24 16:41:01 -08:00
Jason Evans
4201af0542 Add the --enable-swap configure option.
Add malloc_swap_enable().

Add the O/o JEMALLOC_OPTIONS flags, which control memory overcommit.

Fix mapped memory stats reporting for arenas.
2010-01-24 02:53:40 -08:00
Jason Evans
ed1bf457fb Add the w4opaque argument to numerous functions.
Add the w4opaque argument malloc_message() and malloc_stats_print(), and
propagate the change through all the internal APIs as necessary.
2010-01-19 12:11:25 -08:00
Jason Evans
b34e8684ec Add merged arena stats printing.
Add the "m" and "a" opts flags for malloc_stats_print().
2010-01-17 17:35:19 -08:00
Jason Evans
3363a841ca Merge branch 'disable-tls' 2010-01-17 15:51:35 -08:00
Jason Evans
00b5c93347 Add the write4 parameter to malloc_stats_print().
Add malloc_cprintf() and malloc_vcprintf().
2010-01-17 15:49:25 -08:00
Jason Evans
78d815c9de Add the --disable-tls configure option. 2010-01-17 14:06:20 -08:00
Jason Evans
b0fd5016db Add the --with-install-suffix configure option.
Fix some bugs in the Makefile's install target.
2010-01-17 01:49:20 -08:00
Jason Evans
bbaaecb729 Convert .hgignore to .gitignore. 2010-01-16 22:41:23 -08:00
Jason Evans
e476f8a161 Refactor jemalloc.c into multiple source files.
Fix a stats bug in large object curruns accounting.

Replace tcache_bin_fill() with arena_tcache_fill(), and fix a bug in an OOM
error path.

Fix API name mangling to coexist with __attribute__((malloc)).
2010-01-16 09:53:50 -08:00
Jason Evans
64bd7661a8 Support malloc() even after tcache destruction has occurred, since other tsd
destructors may run after tcache_tsd's.
2010-01-12 18:13:39 -08:00
Jason Evans
79a78691b6 Fix some other minor build system bugs. 2010-01-03 16:17:55 -08:00
Jason Evans
e9db6c92ab Add the S/s JEMALLOC_OPTIONS flags to enable/disable tcache bin sorting. 2010-01-03 16:17:52 -08:00
Jason Evans
279e09d1ff Enhance the H/h MALLOC_OPTIONS flags to control the number of tcache bin slots,
rather than just enabling/disabling the tcache.

Fix an off-by-one bug in large object stats recording.
2010-01-03 16:16:10 -08:00
Jason Evans
3f3ecfb8e8 Enhance the G/g MALLOC_OPTIONS flags to control GC sweep interval, rather than
just enabling/disabling GC.
2010-01-03 14:45:26 -08:00
Jason Evans
952b7d192b Split run deallocation out of arena_dalloc_bin(), into arena_dalloc_bin_run().
Enhance bin run deallocation to avoid marking all pages as dirty, since the
dirty bits are already correct for all but the first page, due to the use of
arena_run_rc_{incr,decr}().  This tends to dramatically reduce the number of
pages that are marked dirty.

Modify arena_bin_run_size_calc() to assure that bin run headers never exceed
one page.  In practice, this can't happen unless hard-coded constants (related
to RUN_MAX_OVRHD) are modified, but the dirty page tracking code assumes bin
run headers never extend past the first page, so it seems worth making this a
universally valid assumption.
2010-01-03 12:10:47 -08:00
Jason Evans
03c2237af3 Make malloc_stats_print() a public symbol, and make it repeatedly callable.
Incrementally merge tcache stats, so that malloc_stats_print() always reports
at least most events that have occurred thus far.
2010-01-03 12:10:42 -08:00
Jason Evans
9ad48230ed Simplify JEMALLOC_ATTR_* macros to become JEMALLOC_ATTR().
Use JEMALLOC_ATTR(tls_model("initial-exec)) instead of -ftls-model=initial-exec
so that libjemalloc_pic.a can be directly linked into another library without
needing linker options changes.

Add attributes to malloc, calloc, and posix_memalign, for compatibility with
glibc's declarations.

Add function prototypes for the standard malloc(3) API.
2010-01-03 11:59:20 -08:00
Jason Evans
bf10ae64a7 Build libjemalloc_pic.a. 2010-01-03 11:59:19 -08:00
Jason Evans
0b2368a6a2 Add the COPYING file, and consolidate copyright/licensing information. 2010-01-03 11:59:14 -08:00
Jason Evans
41e008b87d Fix the 'clean' build target to delete binaries. 2009-12-29 00:09:15 -08:00
Jason Evans
e888879e9c Implement per size class stats collection for large objects. 2009-12-29 00:09:15 -08:00
Jason Evans
45c128d1a9 Base maximum dirty page count on proportion of active memory.
Add dirty page tracking for pages within active small/medium object runs.

Reduce chunks_dirty red-black tree operations via lazy updating.
2009-12-29 00:09:15 -08:00
Jason Evans
6c8b13bf43 Update description of --with-rpath. 2009-12-29 00:09:15 -08:00