Commit Graph

1121 Commits

Author SHA1 Message Date
Jason Evans
cec0d63d8b Make one call to prof_active_get_unlocked() per allocation event.
Make one call to prof_active_get_unlocked() per allocation event, and
use the result throughout the relevant functions that handle an
allocation event.  Also add a missing check in prof_realloc().  These
fixes protect allocation events against concurrent prof_active changes.
2015-09-14 23:55:48 -07:00
Jason Evans
ef363de701 Fix irealloc_prof() to prof_alloc_rollback() on OOM. 2015-09-14 23:54:42 -07:00
Jason Evans
46ff049128 Optimize irallocx_prof() to optimistically update the sampler state. 2015-09-14 22:47:18 -07:00
Jason Evans
4acb6c7ff3 Fix ixallocx_prof() size+extra overflow.
Fix ixallocx_prof() to clamp the extra parameter if size+extra would
overflow HUGE_MAXCLASS.
2015-09-14 22:47:12 -07:00
Jason Evans
8f57e3f1ae Remove check_stress from check target's dependencies.
Prior to this change the debug build/test command needed to look like:

  make all tests && make check_unit && make check_integration && \
    make check_integration_prof

This is now simply:

  make check

Rename the check_stress target to stress.
2015-09-11 21:05:10 -07:00
Jason Evans
676df88e48 Rename arena_maxclass to large_maxclass.
arena_maxclass is no longer an appropriate name, because arenas also
manage huge allocations.
2015-09-11 20:50:20 -07:00
Jason Evans
560a4e1e01 Fix xallocx() bugs.
Fix xallocx() bugs related to the 'extra' parameter when specified as
non-zero.
2015-09-11 20:40:34 -07:00
Jason Evans
a00b10735a Fix "prof.reset" mallctl-related corruption.
Fix heap profiling to distinguish among otherwise identical sample sites
with interposed resets (triggered via the "prof.reset" mallctl).  This
bug could cause data structure corruption that would most likely result
in a segfault.
2015-09-09 23:16:10 -07:00
Dmitry-Me
a306a60651 Reduce variables scope 2015-09-04 10:42:33 -07:00
Mike Hommey
0a116faf95 Force initialization of the init_lock in malloc_init_hard on Windows XP
This resolves #269.
2015-09-04 10:35:20 -07:00
Jason Evans
b4330b02a8 Fix pointer comparision with undefined behavior.
This didn't cause bad code generation in the one case spot-checked (gcc
4.8.1), but had the potential to to so.  This bug was introduced by
594c759f37 (Optimize
arena_prof_tctx_set().).
2015-09-04 10:31:41 -07:00
Jason Evans
594c759f37 Optimize arena_prof_tctx_set().
Optimize arena_prof_tctx_set() to avoid reading run metadata when
deciding whether it's actually necessary to write.
2015-09-02 14:52:24 -07:00
Jason Evans
c0f43b6550 Fix TLS configuration.
Fix TLS configuration such that it is enabled by default for platforms
on which it works correctly.  This regression was introduced by
ac5db02034 (Make --enable-tls and
--enable-lazy-lock take precedence over configure.ac-hardcoded
defaults).
2015-09-02 12:46:35 -07:00
Mike Hommey
4a2a3c9a6e Don't purge junk filled chunks when shrinking huge allocations
When junk filling is enabled, shrinking an allocation fills the bytes
that were previously allocated but now aren't. Purging the chunk before
doing that is just a waste of time.

This resolves #260.
2015-08-27 22:00:09 -07:00
Mike Hommey
6d8075f1e6 Fix chunk purge hook calls for in-place huge shrinking reallocation.
Fix chunk purge hook calls for in-place huge shrinking reallocation to
specify the old chunk size rather than the new chunk size.  This bug
caused no correctness issues for the default chunk purge function, but
was visible to custom functions set via the "arena.<i>.chunk_hooks"
mallctl.

This resolves #264.
2015-08-27 20:32:57 -07:00
Jason Evans
30949da601 Fix arenas_cache_cleanup() and arena_get_hard().
Fix arenas_cache_cleanup() and arena_get_hard() to handle
allocation/deallocation within the application's thread-specific data
cleanup functions even after arenas_cache is torn down.

This is a more general fix that complements
45e9f66c28 (Fix arenas_cache_cleanup().).
2015-08-27 20:32:35 -07:00
Jason Evans
5d2e875ac9 Add JEMALLOC_CXX_THROW to the memalign() function prototype.
Add JEMALLOC_CXX_THROW to the memalign() function prototype, in order to
match glibc and avoid compilation errors when including both
jemalloc/jemalloc.h and malloc.h in C++ code.

This change was unintentionally omitted from
ae93d6bf36 (Avoid function prototype
incompatibilities.).
2015-08-26 13:47:20 -07:00
Christopher Ferris
45e9f66c28 Fix arenas_cache_cleanup().
Fix arenas_cache_cleanup() to handle allocation/deallocation within the
application's thread-specific data cleanup functions even after
arenas_cache is torn down.
2015-08-21 12:33:17 -07:00
Jason Evans
b5c2a347d7 Silence compiler warnings for unreachable code.
Reported by Ingvar Hagelund.
2015-08-19 23:28:34 -07:00
Jason Evans
d01fd19755 Rename index_t to szind_t to avoid an existing type on Solaris.
This resolves #256.
2015-08-19 15:21:32 -07:00
Jason Evans
5ef33a9f2b Don't bitshift by negative amounts.
Don't bitshift by negative amounts when encoding/decoding run sizes in
chunk header maps.  This affected systems with page sizes greater than 8
KiB.

Reported by Ingvar Hagelund <ingvar@redpill-linpro.com>.
2015-08-19 14:16:30 -07:00
Jason Evans
6e98caf8f0 Merge branch 'dev' 2015-08-17 13:23:29 -07:00
Jason Evans
9b68f67223 Update ChangeLog for 4.0.0. 2015-08-17 13:21:08 -07:00
Jason Evans
92e96e3bfc Improve arena.<i>.chunk_hooks documentation formatting. 2015-08-14 13:46:08 -07:00
Jason Evans
38f864947b Update in-place reallocation documentation. 2015-08-14 00:55:44 -07:00
Jason Evans
a5dbaef08f Update large/huge size class cutoff documentation. 2015-08-14 00:35:11 -07:00
Jason Evans
85ae064e96 Fix a comment. 2015-08-13 14:54:06 -07:00
Jason Evans
fead75fd52 Fix gcc build failure (define __has_builtin). 2015-08-12 16:46:09 -07:00
Jason Evans
7928f62273 Check whether gcc version supports __builtin_unreachable(). 2015-08-12 16:38:39 -07:00
Jason Evans
56af64dc19 Fix a strict aliasing violation. 2015-08-12 16:38:20 -07:00
Jason Evans
828d919b5e Fix test for MinGW. 2015-08-12 15:21:07 -07:00
Jason Evans
6ed18cb348 Fix chunk_dalloc_arena() re: zeroing due to purge. 2015-08-12 15:20:34 -07:00
Jason Evans
694d0829c0 Update list of private symbols. 2015-08-12 13:03:43 -07:00
Jason Evans
38d8f50144 Fix assertion in test. 2015-08-12 12:06:38 -07:00
Jason Evans
e1ed698973 Remove obsolete entry. 2015-08-12 11:34:48 -07:00
Jason Evans
2662ba5449 Stop forcing --enable-munmap on MinGW.
This is no longer necessary because of the more general chunk
merge/split approach to dealing with map coalescing.
2015-08-12 11:10:42 -07:00
Jason Evans
03bf5b67be Try to decommit new chunks.
Always leave decommit disabled on non-Windows systems.
2015-08-12 10:26:54 -07:00
Jason Evans
1f27abc1b1 Refactor arena_mapbits_{small,large}_set() to not preserve unzeroed.
Fix arena_run_split_large_helper() to treat newly committed memory as
zeroed.
2015-08-11 16:45:47 -07:00
Jason Evans
6bdeddb697 Fix build failure.
This regression was introduced by
de249c8679 (Arena chunk decommit cleanups
and fixes.).

This resolves #254.
2015-08-10 23:42:33 -07:00
Mike Hommey
ac5db02034 Make --enable-tls and --enable-lazy-lock take precedence over configure.ac-hardcoded defaults 2015-08-10 23:36:12 -07:00
Jason Evans
45186f0c07 Refactor arena_mapbits unzeroed flag management.
Only set the unzeroed flag when initializing the entire mapbits entry,
rather than mutating just the unzeroed bit.  This simplifies the
possible mapbits state transitions.
2015-08-10 23:03:34 -07:00
Jason Evans
de249c8679 Arena chunk decommit cleanups and fixes.
Decommit arena chunk header during chunk deallocation if the rest of the
chunk is decommitted.
2015-08-10 17:13:59 -07:00
Jason Evans
4be0c3ca42 Add no-OOM assertions to test. 2015-08-07 00:51:11 -07:00
Jason Evans
8fadb1a8c2 Implement chunk hook support for page run commit/decommit.
Cascade from decommit to purge when purging unused dirty pages, so that
it is possible to decommit cleaned memory rather than just purging.  For
non-Windows debug builds, decommit runs rather than purging them, since
this causes access of deallocated runs to segfault.

This resolves #251.
2015-08-07 00:50:58 -07:00
Jason Evans
5716d97f75 Fix an in-place growing large reallocation regression.
Fix arena_ralloc_large_grow() to properly account for large_pad, so that
in-place large reallocation succeeds when possible, rather than always
failing.  This regression was introduced by
8a03cf039c (Implement cache index
randomization for large allocations.)
2015-08-06 23:45:45 -07:00
Daniel Micay
67c46a9e53 work around _FORTIFY_SOURCE false positive
In builds with profiling disabled (default), the opt_prof_prefix array
has a one byte length as a micro-optimization. This will cause the usage
of write in the unused profiling code to be statically detected as a
buffer overflow by Bionic's _FORTIFY_SOURCE implementation as it tries
to detect read overflows in addition to write overflows.

This works around the problem by informing the compiler that
not_reached() means code in unreachable in release builds.
2015-08-04 17:09:43 -04:00
Matthijs
c1a6a51e40 MSVC compatibility changes
- Decorate public function with __declspec(allocator) and __declspec(restrict), just like MSVC 1900
- Support JEMALLOC_HAS_RESTRICT by defining the restrict keyword
- Move __declspec(nothrow) between 'void' and '*' so it compiles once more
2015-08-04 09:01:48 -07:00
Jason Evans
b49a334a64 Generalize chunk management hooks.
Add the "arena.<i>.chunk_hooks" mallctl, which replaces and expands on
the "arena.<i>.chunk.{alloc,dalloc,purge}" mallctls.  The chunk hooks
allow control over chunk allocation/deallocation, decommit/commit,
purging, and splitting/merging, such that the application can rely on
jemalloc's internal chunk caching and retaining functionality, yet
implement a variety of chunk management mechanisms and policies.

Merge the chunks_[sz]ad_{mmap,dss} red-black trees into
chunks_[sz]ad_retained.  This slightly reduces how hard jemalloc tries
to honor the dss precedence setting; prior to this change the precedence
setting was also consulted when recycling chunks.

Fix chunk purging.  Don't purge chunks in arena_purge_stashed(); instead
deallocate them in arena_unstash_purged(), so that the dirty memory
linkage remains valid until after the last time it is used.

This resolves #176 and #201.
2015-08-03 21:49:02 -07:00
Jason Evans
d059b9d6a1 Implement support for non-coalescing maps on MinGW.
- Do not reallocate huge objects in place if the number of backing
  chunks would change.
- Do not cache multi-chunk mappings.

This resolves #213.
2015-07-24 18:39:14 -07:00
Jason Evans
40cbd30d50 Fix huge_ralloc_no_move() to succeed more often.
Fix huge_ralloc_no_move() to succeed if an allocation request results in
the same usable size as the existing allocation, even if the request
size is smaller than the usable size.  This bug did not cause
correctness issues, but it could cause unnecessary moves during
reallocation.
2015-07-24 18:20:48 -07:00