Commit Graph

14 Commits

Author SHA1 Message Date
Jason Evans
47e57f9bda Avoid overflow in arena_run_regind().
Fix a regression due to:
    Remove an arena_bin_run_size_calc() constraint.
    2a6f2af6e4
The removed constraint required that small run headers fit in one page,
which indirectly limited runs such that they would not cause overflow in
arena_run_regind().  Add an explicit constraint to
arena_bin_run_size_calc() based on the largest number of regions that
arena_run_regind() can handle (2^11 as currently configured).
2011-03-22 09:00:56 -07:00
Jason Evans
84c8eefeff Use bitmaps to track small regions.
The previous free list implementation, which embedded singly linked
lists in available regions, had the unfortunate side effect of causing
many cache misses during thread cache fills.  Fix this in two places:

  - arena_run_t: Use a new bitmap implementation to track which regions
                 are available.  Furthermore, revert to preferring the
                 lowest available region (as jemalloc did with its old
                 bitmap-based approach).

  - tcache_t: Move read-only tcache_bin_t metadata into
              tcache_bin_info_t, and add a contiguous array of pointers
              to tcache_t in order to track cached objects.  This
              substantially increases the size of tcache_t, but results
              in much higher data locality for common tcache operations.
              As a side benefit, it is again possible to efficiently
              flush the least recently used cached objects, so this
              change changes flushing from MRU to LRU.

The new bitmap implementation uses a multi-level summary approach to
make finding the lowest available region very fast.  In practice,
bitmaps only have one or two levels, though the implementation is
general enough to handle extremely large bitmaps, mainly so that large
page sizes can still be entertained.

Fix tcache_bin_flush_large() to always flush statistics, in the same way
that tcache_bin_flush_small() was recently fixed.

Use JEMALLOC_DEBUG rather than NDEBUG.

Add dassert(), and use it for debug-only asserts.
2011-03-17 16:29:32 -07:00
Jason Evans
a7153a0d7d Fix a "thread.arena" mallctl bug.
Fix a variable reversal bug in mallctl("thread.arena", ...).
2011-03-14 11:43:54 -07:00
Jason Evans
9dcad2dfd1 Fix "thread.{de,}allocatedp" mallctl.
For the non-TLS case (as on OS X), if the "thread.{de,}allocatedp"
mallctl was called before any allocation occurred for that thread, the
TSD was still NULL, thus putting the application at risk of
dereferencing NULL.  Fix this by refactoring the initialization code,
and making it part of the conditional logic for all per thread
allocation counter accesses.
2011-02-13 18:11:54 -08:00
Jason Evans
cfdc8cfbd6 Use mremap(2) for huge realloc().
If mremap(2) is available and supports MREMAP_FIXED, use it for huge
realloc().

Initialize rtree later during bootstrapping, so that --enable-debug
--enable-dss works.

Fix a minor swap_avail stats bug.
2010-11-30 16:50:58 -08:00
Jason Evans
e73397062a Replace JEMALLOC_OPTIONS with MALLOC_CONF.
Replace the single-character run-time flags with key/value pairs, which
can be set via the malloc_conf global, /etc/malloc.conf, and the
MALLOC_CONF environment variable.

Replace the JEMALLOC_PROF_PREFIX environment variable with the
"opt.prof_prefix" option.

Replace umax2s() with u2s().
2010-10-23 18:37:06 -07:00
Jason Evans
93443689a4 Add per thread allocation counters, and enhance heap sampling.
Add the "thread.allocated" and "thread.deallocated" mallctls, which can
be used to query the total number of bytes ever allocated/deallocated by
the calling thread.

Add s2u() and sa2u(), which can be used to compute the usable size that
will result from an allocation request of a particular size/alignment.

Re-factor ipalloc() to use sa2u().

Enhance the heap profiler to trigger samples based on usable size,
rather than request size.  This has a subtle, but important, impact on
the accuracy of heap sampling.  For example, previous to this change,
16- and 17-byte objects were sampled at nearly the same rate, but
17-byte objects actually consume 32 bytes each.  Therefore it was
possible for the sample to be somewhat skewed compared to actual memory
usage of the allocated objects.
2010-10-20 17:39:18 -07:00
Jason Evans
9f3b0a74fd Fix tests build when --with-install-suffix is set.
Add test/jemalloc_test.h.in, which is processed to include
jemalloc/jemalloc@install_suffix@.h, so that test programs can include
it without worrying about the install suffix.
2010-10-07 09:53:26 -07:00
Jason Evans
355b438c85 Fix compiler warnings.
Add --enable-cc-silence, which can be used to silence harmless warnings.

Fix an aliasing bug in ckh_pointer_hash().
2010-09-20 19:20:48 -07:00
Jason Evans
8e3c3c61b5 Add {,r,s,d}allocm().
Add allocm(), rallocm(), sallocm(), and dallocm(), which are a
functional superset of malloc(), calloc(), posix_memalign(),
malloc_usable_size(), and free().
2010-09-17 15:46:18 -07:00
Jason Evans
8d7a94b275 Fix porting regressions.
Fix new build failures and test failures on Linux that were introduced
by the port to OS X.
2010-09-11 23:38:12 -07:00
Jason Evans
58a6f5c9be Add posix_memalign test. 2010-09-11 20:59:16 -07:00
Jason Evans
2dbecf1f62 Port to Mac OS X.
Add Mac OS X support, based in large part on the OS X support in
Mozilla's version of jemalloc.
2010-09-11 18:20:16 -07:00
Jason Evans
b267d0f86a Add the thread.arena mallctl.
Make it possible for each thread to manage which arena it is associated
with.

Implement the 'tests' and 'check' build targets.
2010-08-13 17:36:00 -07:00