Commit Graph

551 Commits

Author SHA1 Message Date
Jason Evans
e24c7af35d Invert NO_TLS to JEMALLOC_TLS. 2012-03-19 10:21:17 -07:00
Jason Evans
e7b8fa18d2 Rename the "tcache.flush" mallctl to "thread.tcache.flush". 2012-03-16 17:09:32 -07:00
Jason Evans
4e2e3dd9cf Fix fork-related bugs.
Acquire/release arena bin locks as part of the prefork/postfork.  This
bug made deadlock in the child between fork and exec a possibility.

Split jemalloc_postfork() into jemalloc_postfork_{parent,child}() so
that the child can reinitialize mutexes rather than unlocking them.  In
practice, this bug tended not to cause problems.
2012-03-13 16:31:41 -07:00
Jason Evans
824d34e5b7 Modify malloc_vsnprintf() validation code.
Modify malloc_vsnprintf() validation code to verify that output is
identical to vsnprintf() output, even if both outputs are truncated due
to buffer exhaustion.
2012-03-13 13:19:04 -07:00
Jason Evans
0a0bbf63e5 Implement aligned_alloc().
Implement aligned_alloc(), which was added in the C11 standard.  The
function is weakly specified to the point that a minimally compliant
implementation would be painful to use (size must be an integral
multiple of alignment!), which in practice makes posix_memalign() a
safer choice.
2012-03-13 12:55:21 -07:00
Jason Evans
4c2faa8a7c Fix a regression in JE_COMPILABLE().
Revert JE_COMPILABLE() so that it detects link errors.  Cross-compiling
should still work as long as a valid configure cache is provided.

Clean up some comments/whitespace.
2012-03-13 11:09:23 -07:00
Jason Evans
eb2398106f Fix malloc_stats_print() option support.
Fix malloc_stats_print() to honor 'b' and 'l' in the opts parameter.
2012-03-13 08:46:12 -07:00
Jason Evans
2bb6c7a632 s/PRIx64/PRIxPTR/ for uintptr_t printf() argument. 2012-03-12 13:38:55 -07:00
Jason Evans
7cca608575 Remove extra '}'. 2012-03-12 11:31:54 -07:00
Jason Evans
d81e4bdd5c Implement malloc_vsnprintf().
Implement malloc_vsnprintf() (a subset of vsnprintf(3)) as well as
several other printing functions based on it, so that formatted printing
can be relied upon without concern for inducing a dependency on floating
point runtime support.  Replace malloc_write() calls with
malloc_*printf() where doing so simplifies the code.

Add name mangling for library-private symbols in the data and BSS
sections.  Adjust CONF_HANDLE_*() macros in malloc_conf_init() to expose
all opt_* variable use to cpp so that proper mangling occurs.
2012-03-07 16:19:19 -08:00
Jason Evans
4507f34628 Remove the lg_tcache_gc_sweep option.
Remove the lg_tcache_gc_sweep option, because it is no longer
very useful.  Prior to the addition of dynamic adjustment of tcache fill
count, it was possible for fill/flush overhead to be a problem, but this
problem no longer occurs.
2012-03-05 14:34:37 -08:00
Jason Evans
b8c8be7f8a Use UINT64_C() rather than LLU for 64-bit constants. 2012-03-05 12:26:26 -08:00
Jason Evans
7e77eaffff Add the --disable-experimental option. 2012-03-02 17:47:37 -08:00
Jason Evans
84f7cdb0c5 Rename prn to prng.
Rename prn to prng so that Windows doesn't choke when trying to create
a file named prn.h.
2012-03-02 15:59:45 -08:00
Jason Evans
0a5489e37d Add --with-mangling.
Add the --with-mangling configure option, which can be used to specify
name mangling on a per public symbol basis that takes precedence over
--with-jemalloc-prefix.

Expose the memalign() and valloc() overrides even if
--with-jemalloc-prefix is specified.  This change does no real harm, and
simplifies the code.
2012-03-01 17:19:20 -08:00
Jason Evans
166a745b39 Simplify zone_good_size().
Simplify zone_good_size() to avoid memory allocation.

Submitted by Mike Hommey.
2012-02-29 13:03:29 -08:00
Jason Evans
7e15dab94d Add nallocm().
Add nallocm(), which computes the real allocation size that would result
from the corresponding allocm() call.  nallocm() is a functional
superset of OS X's malloc_good_size(), in that it takes alignment
constraints into account.
2012-02-29 12:56:37 -08:00
Jason Evans
4bb0983013 Use glibc allocator hooks.
When jemalloc is used as a libc malloc replacement (i.e. not prefixed),
some particular setups may end up inconsistently calling malloc from
libc and free from jemalloc, or the other way around.

glibc provides hooks to make its functions use alternative
implementations.  Use them.

Submitted by Karl Tomlinson and Mike Hommey.
2012-02-29 10:37:27 -08:00
Jason Evans
5965631636 Do not enforce minimum alignment in memalign().
Do not enforce minimum alignment in memalign().  This is a non-standard
function, and there is disagreement over whether to enforce minimum
alignment.  Solaris documentation (whence memalign() originated) says
that minimum alignment is required:

  The value of alignment must be a power of two and must be greater than
  or equal  to  the size of a word.

However, Linux's manual page says in its NOTES section:

  memalign() may not check that the boundary parameter is correct.

This is descriptive rather than prescriptive, but applications with
bad assumptions about memalign() exist, so be as forgiving as possible.

Reported by Mike Hommey.
2012-02-28 21:37:38 -08:00
Jason Evans
93c023d181 Remove unused variables in stats_print().
Submitted by Mike Hommey.
2012-02-28 21:13:12 -08:00
Jason Evans
bdcadf41e9 Remove unused variable in arena_run_split().
Submitted by Mike Hommey.
2012-02-28 21:11:03 -08:00
Jason Evans
d073a32109 Enable the stats configuration option by default. 2012-02-28 20:41:16 -08:00
Jason Evans
c90ad71237 Remove the sysv option. 2012-02-28 20:31:37 -08:00
Jason Evans
f081b88dfb Fix realloc(p, 0) to act like free(p).
Reported by Yoni Londer.
2012-02-28 20:24:05 -08:00
Jason Evans
b172610317 Simplify small size class infrastructure.
Program-generate small size class tables for all valid combinations of
LG_TINY_MIN, LG_QUANTUM, and PAGE_SHIFT.  Use the appropriate table to generate
all relevant data structures, and remove the distinction between
tiny/quantum/cacheline/subpage bins.

Remove --enable-dynamic-page-shift.  This option didn't prove useful in
practice, and it prevented optimizations.

Add Tilera architecture support.
2012-02-28 16:50:47 -08:00
Jason Evans
5389146191 Remove the opt.lg_prof_bt_max option.
Remove opt.lg_prof_bt_max, and hard code it to 7.  The original
intention of this option was to enable faster backtracing by limiting
backtrace depth.  However, this makes graphical pprof output very
difficult to interpret.  In practice, decreasing sampling frequency is a
better mechanism for limiting profiling overhead.
2012-02-13 18:41:36 -08:00
Jason Evans
0b526ff94d Remove the opt.lg_prof_tcmax option.
Remove the opt.lg_prof_tcmax option and hard-code a cache size of 1024.
This setting is something that users just shouldn't have to worry about.
If lock contention actually ends up being a problem, the simple solution
available to the user is to reduce sampling frequency.
2012-02-13 18:04:26 -08:00
Jason Evans
e7a1058aaa Fix bin->runcur management.
Fix an interaction between arena_dissociate_bin_run() and
arena_bin_lower_run() that made it possible for bin->runcur to point to
a run other than the lowest non-full run.  This bug violated jemalloc's
layout policy, but did not affect correctness.
2012-02-13 17:36:52 -08:00
Jason Evans
746868929a Remove highruns statistics. 2012-02-13 15:18:19 -08:00
Jason Evans
ef8897b4b9 Make 8-byte tiny size class non-optional.
When tiny size class support was first added, it was intended to support
truly tiny size classes (even 2 bytes).  However, this wasn't very
useful in practice, so the minimum tiny size class has been limited to
sizeof(void *) for a long time now.  This is too small to be standards
compliant, but other commonly used malloc implementations do not even
bother using a 16-byte quantum  on systems with vector units (SSE2+,
AltiVEC, etc.).  As such, it is safe in practice to support an 8-byte
tiny size class on 64-bit systems that support 16-byte types.
2012-02-13 15:03:59 -08:00
Jason Evans
6ffbbeb5d6 Silence compiler warnings. 2012-02-13 12:31:30 -08:00
Jason Evans
962463d9b5 Streamline tcache-related malloc/free fast paths.
tcache_get() is inlined, so do the config_tcache check inside
tcache_get() and simplify its callers.

Make arena_malloc() an inline function, since it is part of the malloc()
fast path.

Remove conditional logic that cause build issues if --disable-tcache was
specified.
2012-02-13 12:29:49 -08:00
Jason Evans
4162627757 Remove the swap feature.
Remove the swap feature, which enabled per application swap files.  In
practice this feature has not proven itself useful to users.
2012-02-13 10:56:17 -08:00
Jason Evans
fd56043c53 Remove magic.
Remove structure magic, because 1) it is no longer conditional, and 2)
it stopped being very effective at detecting memory corruption several
years ago.
2012-02-13 10:24:43 -08:00
Jason Evans
7372b15a31 Reduce cpp conditional logic complexity.
Convert configuration-related cpp conditional logic to use static
constant variables, e.g.:

  #ifdef JEMALLOC_DEBUG
    [...]
  #endif

becomes:

  if (config_debug) {
    [...]
  }

The advantage is clearer, more concise code.  The main disadvantage is
that data structures no longer have conditionally defined fields, so
they pay the cost of all fields regardless of whether they are used.  In
practice, this is only a minor concern; config_stats will go away in an
upcoming change, and config_prof is the only other major feature that
depends on more than a few special-purpose fields.
2012-02-10 20:22:09 -08:00
Jason Evans
334cc02142 Fix malloc_stats_print(..., "a") output.
Fix the logic in stats_print() such that if the "a" flag is passed in
without the "m" flag, merged statistics will be printed even if only one
arena is initialized.
2011-11-11 14:46:04 -08:00
Jason Evans
12a4887826 Fix huge_ralloc to maintain chunk statistics.
Fix huge_ralloc() to properly maintain chunk statistics when using
mremap(2).
2011-11-11 14:41:59 -08:00
Jason Evans
fa351d9fdc Fix huge_ralloc() race when using mremap(2).
Fix huge_ralloc() to remove the old memory region from tree of huge
allocations *before* calling mremap(2), in order to make sure that no
other thread acquires the old memory region via mmap() and encounters
stale metadata in the tree.

Reported by: Rich Prohaska
2011-11-09 11:55:19 -08:00
Jason Evans
30fbef8aea Fix rallocm() test to support >4KiB pages. 2011-11-05 21:06:55 -07:00
Jason Evans
8e6f8b490d Initialize arenas_tsd before setting it.
Reported by: Ethan Burns, Rich Prohaska, Tudor Bosman
2011-11-03 18:40:03 -07:00
Jason Evans
a9076c9483 Fix a prof-related race condition.
Fix prof_lookup() to artificially raise curobjs for all paths through
the code that creates a new entry in the per thread bt2cnt hash table.
This fixes a race condition that could corrupt memory if prof_accum were
false, and a non-default lg_prof_tcmax were used and/or threads were
destroyed.
2011-08-30 23:40:11 -07:00
Jason Evans
46405e670f Fix a prof-related bug in realloc().
Fix realloc() such that it only records the object passed in as freed if
no OOM error occurs.
2011-08-30 23:37:29 -07:00
Jason Evans
749c2a0ab6 Add missing prof_malloc() call in allocm().
Add a missing prof_malloc() call in allocm().  Before this fix, negative
object/byte counts could be observed in heap profiles for applications
that use allocm().
2011-08-12 18:37:54 -07:00
Jason Evans
a507004d29 Fix off-by-one backtracing issues.
Rewrite prof_alloc_prep() as a cpp macro, PROF_ALLOC_PREP(), in order to
remove any doubt as to whether an additional stack frame is created.
Prior to this change, it was assumed that inlining would reduce the
total number of frames in the backtrace, but in practice behavior wasn't
completely predictable.

Create imemalign() and call it from posix_memalign(), memalign(), and
valloc(), so that all entry points require the same number of stack
frames to be ignored during backtracing.
2011-08-12 13:48:27 -07:00
Jason Evans
b493ce22a4 Conditionalize an isalloc() call in rallocm().
Conditionalize an isalloc() call in rallocm() that be unnecessary.
2011-08-12 11:28:47 -07:00
Jason Evans
183ba50c19 Fix two prof-related bugs in rallocm().
Properly handle boundary conditions for sampled region promotion in
rallocm().  Prior to this fix, some combinations of 'size' and 'extra'
values could cause erroneous behavior.  Additionally, size class
recording for promoted regions was incorrect.
2011-08-11 23:00:25 -07:00
Jason Evans
0cdd42eb32 Clean up prof-related comments.
Clean up some prof-related comments to more accurately reflect how the
code works.

Simplify OOM handling code in a couple of prof-related error paths.
2011-08-09 19:06:06 -07:00
Jason Evans
41b954ed36 Use prof_tdata_cleanup() argument.
Use the argument to prof_tdata_cleanup(), rather than calling
PROF_TCACHE_GET().  This fixes a bug in the NO_TLS case.
2011-08-08 17:10:07 -07:00
Jason Evans
f9a8edbb50 Fix assertions in arena_purge().
Fix assertions in arena_purge() to accurately reflect the constraints in
arena_maybe_purge().  There were two bugs here, one of which merely
weakened the assertion, and the other of which referred to an
uninitialized variable (typo; used npurgatory instead of
arena->npurgatory).
2011-06-12 17:13:39 -07:00
Jason Evans
f0b22cf932 Use LLU suffix for all 64-bit constants.
Add the LLU suffix for all 0x... 64-bit constants.

Reported by Jakob Blomer.
2011-05-22 10:49:44 -07:00
Jason Evans
7427525c28 Move repo contents in jemalloc/ to top level. 2011-03-31 20:36:17 -07:00