project-base/server-skynet-source-3rd-jemalloc
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
a0dd3a4483
server-skynet-source-3rd-je.../test/unit/bitmap.c

351 lines
7.6 KiB
C
Raw Normal View History

Refactor to support more varied testing. Refactor the test harness to support three types of tests: - unit: White box unit tests. These tests have full access to all internal jemalloc library symbols. Though in actuality all symbols are prefixed by jet_, macro-based name mangling abstracts this away from test code. - integration: Black box integration tests. These tests link with the installable shared jemalloc library, and with the exception of some utility code and configure-generated macro definitions, they have no access to jemalloc internals. - stress: Black box stress tests. These tests link with the installable shared jemalloc library, as well as with an internal allocator with symbols prefixed by jet_ (same as for unit tests) that can be used to allocate data structures that are internal to the test code. Move existing tests into test/{unit,integration}/ as appropriate. Split out internal parts of jemalloc_defs.h.in and put them in jemalloc_internal_defs.h.in. This reduces internals exposure to applications that #include <jemalloc/jemalloc.h>. Refactor jemalloc.h header generation so that a single header file results, and the prototypes can be used to generate jet_ prototypes for tests. Split jemalloc.h.in into multiple parts (jemalloc_defs.h.in, jemalloc_macros.h.in, jemalloc_protos.h.in, jemalloc_mangle.h.in) and use a shell script to generate a unified jemalloc.h at configure time. Change the default private namespace prefix from "" to "je_". Add missing private namespace mangling. Remove hard-coded private_namespace.h. Instead generate it and private_unnamespace.h from private_symbols.txt. Use similar logic for public symbols, which aids in name mangling for jet_ symbols. Add test_warn() and test_fail(). Replace existing exit(1) calls with test_fail() calls.
2013-12-01 07:25:42 +08:00
#include "test/jemalloc_test.h"
Use bitmaps to track small regions. The previous free list implementation, which embedded singly linked lists in available regions, had the unfortunate side effect of causing many cache misses during thread cache fills. Fix this in two places: - arena_run_t: Use a new bitmap implementation to track which regions are available. Furthermore, revert to preferring the lowest available region (as jemalloc did with its old bitmap-based approach). - tcache_t: Move read-only tcache_bin_t metadata into tcache_bin_info_t, and add a contiguous array of pointers to tcache_t in order to track cached objects. This substantially increases the size of tcache_t, but results in much higher data locality for common tcache operations. As a side benefit, it is again possible to efficiently flush the least recently used cached objects, so this change changes flushing from MRU to LRU. The new bitmap implementation uses a multi-level summary approach to make finding the lowest available region very fast. In practice, bitmaps only have one or two levels, though the implementation is general enough to handle extremely large bitmaps, mainly so that large page sizes can still be entertained. Fix tcache_bin_flush_large() to always flush statistics, in the same way that tcache_bin_flush_small() was recently fixed. Use JEMALLOC_DEBUG rather than NDEBUG. Add dassert(), and use it for debug-only asserts.
2011-03-17 01:30:13 +08:00
Implement BITMAP_INFO_INITIALIZER(nbits). This allows static initialization of bitmap_info_t structures.
2016-04-07 09:30:02 +08:00
#define NBITS_TAB \
NB( 1) \
NB( 2) \
NB( 3) \
NB( 4) \
NB( 5) \
NB( 6) \
NB( 7) \
NB( 8) \
NB( 9) \
NB(10) \
NB(11) \
NB(12) \
NB(13) \
NB(14) \
NB(15) \
NB(16) \
NB(17) \
NB(18) \
NB(19) \
NB(20) \
NB(21) \
NB(22) \
NB(23) \
NB(24) \
NB(25) \
NB(26) \
NB(27) \
NB(28) \
NB(29) \
NB(30) \
NB(31) \
NB(32) \
\
NB(33) \
NB(34) \
NB(35) \
NB(36) \
NB(37) \
NB(38) \
NB(39) \
NB(40) \
NB(41) \
NB(42) \
NB(43) \
NB(44) \
NB(45) \
NB(46) \
NB(47) \
NB(48) \
NB(49) \
NB(50) \
NB(51) \
NB(52) \
NB(53) \
NB(54) \
NB(55) \
NB(56) \
NB(57) \
NB(58) \
NB(59) \
NB(60) \
NB(61) \
NB(62) \
NB(63) \
NB(64) \
NB(65) \
\
NB(126) \
NB(127) \
NB(128) \
NB(129) \
NB(130) \
\
NB(254) \
NB(255) \
NB(256) \
NB(257) \
NB(258) \
\
NB(510) \
NB(511) \
NB(512) \
NB(513) \
NB(514) \
\
NB(1024) \
NB(2048) \
NB(4096) \
NB(8192) \
NB(16384) \
static void
test_bitmap_initializer_body(const bitmap_info_t *binfo, size_t nbits)
{
bitmap_info_t binfo_dyn;
bitmap_info_init(&binfo_dyn, nbits);
assert_zu_eq(bitmap_size(binfo), bitmap_size(&binfo_dyn),
"Unexpected difference between static and dynamic initialization, "
"nbits=%zu", nbits);
assert_zu_eq(binfo->nbits, binfo_dyn.nbits,
"Unexpected difference between static and dynamic initialization, "
"nbits=%zu", nbits);
#ifdef BITMAP_USE_TREE
assert_u_eq(binfo->nlevels, binfo_dyn.nlevels,
"Unexpected difference between static and dynamic initialization, "
"nbits=%zu", nbits);
{
unsigned i;
for (i = 0; i < binfo->nlevels; i++) {
assert_zu_eq(binfo->levels[i].group_offset,
binfo_dyn.levels[i].group_offset,
"Unexpected difference between static and dynamic "
"initialization, nbits=%zu, level=%u", nbits, i);
}
}
#else
assert_zu_eq(binfo->ngroups, binfo_dyn.ngroups,
"Unexpected difference between static and dynamic initialization");
#endif
}
TEST_BEGIN(test_bitmap_initializer)
{
#define NB(nbits) { \
if (nbits <= BITMAP_MAXBITS) { \
bitmap_info_t binfo = \
BITMAP_INFO_INITIALIZER(nbits); \
test_bitmap_initializer_body(&binfo, nbits); \
} \
}
NBITS_TAB
#undef NB
}
TEST_END
static size_t
test_bitmap_size_body(const bitmap_info_t *binfo, size_t nbits,
size_t prev_size)
{
size_t size = bitmap_size(binfo);
assert_zu_ge(size, (nbits >> 3),
"Bitmap size is smaller than expected");
assert_zu_ge(size, prev_size, "Bitmap size is smaller than expected");
return (size);
}
Refactor tests. Refactor tests to use explicit testing assertions, rather than diff'ing test output. This makes the test code a bit shorter, more explicitly encodes testing intent, and makes test failure diagnosis more straightforward.
2013-12-09 12:52:21 +08:00
TEST_BEGIN(test_bitmap_size)
Use bitmaps to track small regions. The previous free list implementation, which embedded singly linked lists in available regions, had the unfortunate side effect of causing many cache misses during thread cache fills. Fix this in two places: - arena_run_t: Use a new bitmap implementation to track which regions are available. Furthermore, revert to preferring the lowest available region (as jemalloc did with its old bitmap-based approach). - tcache_t: Move read-only tcache_bin_t metadata into tcache_bin_info_t, and add a contiguous array of pointers to tcache_t in order to track cached objects. This substantially increases the size of tcache_t, but results in much higher data locality for common tcache operations. As a side benefit, it is again possible to efficiently flush the least recently used cached objects, so this change changes flushing from MRU to LRU. The new bitmap implementation uses a multi-level summary approach to make finding the lowest available region very fast. In practice, bitmaps only have one or two levels, though the implementation is general enough to handle extremely large bitmaps, mainly so that large page sizes can still be entertained. Fix tcache_bin_flush_large() to always flush statistics, in the same way that tcache_bin_flush_small() was recently fixed. Use JEMALLOC_DEBUG rather than NDEBUG. Add dassert(), and use it for debug-only asserts.
2011-03-17 01:30:13 +08:00
{
Implement BITMAP_INFO_INITIALIZER(nbits). This allows static initialization of bitmap_info_t structures.
2016-04-07 09:30:02 +08:00
size_t nbits, prev_size;
Use bitmaps to track small regions. The previous free list implementation, which embedded singly linked lists in available regions, had the unfortunate side effect of causing many cache misses during thread cache fills. Fix this in two places: - arena_run_t: Use a new bitmap implementation to track which regions are available. Furthermore, revert to preferring the lowest available region (as jemalloc did with its old bitmap-based approach). - tcache_t: Move read-only tcache_bin_t metadata into tcache_bin_info_t, and add a contiguous array of pointers to tcache_t in order to track cached objects. This substantially increases the size of tcache_t, but results in much higher data locality for common tcache operations. As a side benefit, it is again possible to efficiently flush the least recently used cached objects, so this change changes flushing from MRU to LRU. The new bitmap implementation uses a multi-level summary approach to make finding the lowest available region very fast. In practice, bitmaps only have one or two levels, though the implementation is general enough to handle extremely large bitmaps, mainly so that large page sizes can still be entertained. Fix tcache_bin_flush_large() to always flush statistics, in the same way that tcache_bin_flush_small() was recently fixed. Use JEMALLOC_DEBUG rather than NDEBUG. Add dassert(), and use it for debug-only asserts.
2011-03-17 01:30:13 +08:00
prev_size = 0;
Implement BITMAP_INFO_INITIALIZER(nbits). This allows static initialization of bitmap_info_t structures.
2016-04-07 09:30:02 +08:00
for (nbits = 1; nbits <= BITMAP_MAXBITS; nbits++) {
Miscellaneous bitmap refactoring.
2016-02-27 05:59:41 +08:00
bitmap_info_t binfo;
Implement BITMAP_INFO_INITIALIZER(nbits). This allows static initialization of bitmap_info_t structures.
2016-04-07 09:30:02 +08:00
bitmap_info_init(&binfo, nbits);
prev_size = test_bitmap_size_body(&binfo, nbits, prev_size);
}
#define NB(nbits) { \
bitmap_info_t binfo = BITMAP_INFO_INITIALIZER(nbits); \
prev_size = test_bitmap_size_body(&binfo, nbits, \
prev_size); \
Use bitmaps to track small regions. The previous free list implementation, which embedded singly linked lists in available regions, had the unfortunate side effect of causing many cache misses during thread cache fills. Fix this in two places: - arena_run_t: Use a new bitmap implementation to track which regions are available. Furthermore, revert to preferring the lowest available region (as jemalloc did with its old bitmap-based approach). - tcache_t: Move read-only tcache_bin_t metadata into tcache_bin_info_t, and add a contiguous array of pointers to tcache_t in order to track cached objects. This substantially increases the size of tcache_t, but results in much higher data locality for common tcache operations. As a side benefit, it is again possible to efficiently flush the least recently used cached objects, so this change changes flushing from MRU to LRU. The new bitmap implementation uses a multi-level summary approach to make finding the lowest available region very fast. In practice, bitmaps only have one or two levels, though the implementation is general enough to handle extremely large bitmaps, mainly so that large page sizes can still be entertained. Fix tcache_bin_flush_large() to always flush statistics, in the same way that tcache_bin_flush_small() was recently fixed. Use JEMALLOC_DEBUG rather than NDEBUG. Add dassert(), and use it for debug-only asserts.
2011-03-17 01:30:13 +08:00
}
Implement BITMAP_INFO_INITIALIZER(nbits). This allows static initialization of bitmap_info_t structures.
2016-04-07 09:30:02 +08:00
prev_size = 0;
NBITS_TAB
#undef NB
Use bitmaps to track small regions. The previous free list implementation, which embedded singly linked lists in available regions, had the unfortunate side effect of causing many cache misses during thread cache fills. Fix this in two places: - arena_run_t: Use a new bitmap implementation to track which regions are available. Furthermore, revert to preferring the lowest available region (as jemalloc did with its old bitmap-based approach). - tcache_t: Move read-only tcache_bin_t metadata into tcache_bin_info_t, and add a contiguous array of pointers to tcache_t in order to track cached objects. This substantially increases the size of tcache_t, but results in much higher data locality for common tcache operations. As a side benefit, it is again possible to efficiently flush the least recently used cached objects, so this change changes flushing from MRU to LRU. The new bitmap implementation uses a multi-level summary approach to make finding the lowest available region very fast. In practice, bitmaps only have one or two levels, though the implementation is general enough to handle extremely large bitmaps, mainly so that large page sizes can still be entertained. Fix tcache_bin_flush_large() to always flush statistics, in the same way that tcache_bin_flush_small() was recently fixed. Use JEMALLOC_DEBUG rather than NDEBUG. Add dassert(), and use it for debug-only asserts.
2011-03-17 01:30:13 +08:00
}
Refactor tests. Refactor tests to use explicit testing assertions, rather than diff'ing test output. This makes the test code a bit shorter, more explicitly encodes testing intent, and makes test failure diagnosis more straightforward.
2013-12-09 12:52:21 +08:00
TEST_END
Use bitmaps to track small regions. The previous free list implementation, which embedded singly linked lists in available regions, had the unfortunate side effect of causing many cache misses during thread cache fills. Fix this in two places: - arena_run_t: Use a new bitmap implementation to track which regions are available. Furthermore, revert to preferring the lowest available region (as jemalloc did with its old bitmap-based approach). - tcache_t: Move read-only tcache_bin_t metadata into tcache_bin_info_t, and add a contiguous array of pointers to tcache_t in order to track cached objects. This substantially increases the size of tcache_t, but results in much higher data locality for common tcache operations. As a side benefit, it is again possible to efficiently flush the least recently used cached objects, so this change changes flushing from MRU to LRU. The new bitmap implementation uses a multi-level summary approach to make finding the lowest available region very fast. In practice, bitmaps only have one or two levels, though the implementation is general enough to handle extremely large bitmaps, mainly so that large page sizes can still be entertained. Fix tcache_bin_flush_large() to always flush statistics, in the same way that tcache_bin_flush_small() was recently fixed. Use JEMALLOC_DEBUG rather than NDEBUG. Add dassert(), and use it for debug-only asserts.
2011-03-17 01:30:13 +08:00
Implement BITMAP_INFO_INITIALIZER(nbits). This allows static initialization of bitmap_info_t structures.
2016-04-07 09:30:02 +08:00
static void
test_bitmap_init_body(const bitmap_info_t *binfo, size_t nbits)
Use bitmaps to track small regions. The previous free list implementation, which embedded singly linked lists in available regions, had the unfortunate side effect of causing many cache misses during thread cache fills. Fix this in two places: - arena_run_t: Use a new bitmap implementation to track which regions are available. Furthermore, revert to preferring the lowest available region (as jemalloc did with its old bitmap-based approach). - tcache_t: Move read-only tcache_bin_t metadata into tcache_bin_info_t, and add a contiguous array of pointers to tcache_t in order to track cached objects. This substantially increases the size of tcache_t, but results in much higher data locality for common tcache operations. As a side benefit, it is again possible to efficiently flush the least recently used cached objects, so this change changes flushing from MRU to LRU. The new bitmap implementation uses a multi-level summary approach to make finding the lowest available region very fast. In practice, bitmaps only have one or two levels, though the implementation is general enough to handle extremely large bitmaps, mainly so that large page sizes can still be entertained. Fix tcache_bin_flush_large() to always flush statistics, in the same way that tcache_bin_flush_small() was recently fixed. Use JEMALLOC_DEBUG rather than NDEBUG. Add dassert(), and use it for debug-only asserts.
2011-03-17 01:30:13 +08:00
{
size_t i;
Implement BITMAP_INFO_INITIALIZER(nbits). This allows static initialization of bitmap_info_t structures.
2016-04-07 09:30:02 +08:00
bitmap_t *bitmap = (bitmap_t *)malloc(bitmap_size(binfo));
assert_ptr_not_null(bitmap, "Unexpected malloc() failure");
bitmap_init(bitmap, binfo);
for (i = 0; i < nbits; i++) {
assert_false(bitmap_get(bitmap, binfo, i),
"Bit should be unset");
}
free(bitmap);
}
Use bitmaps to track small regions. The previous free list implementation, which embedded singly linked lists in available regions, had the unfortunate side effect of causing many cache misses during thread cache fills. Fix this in two places: - arena_run_t: Use a new bitmap implementation to track which regions are available. Furthermore, revert to preferring the lowest available region (as jemalloc did with its old bitmap-based approach). - tcache_t: Move read-only tcache_bin_t metadata into tcache_bin_info_t, and add a contiguous array of pointers to tcache_t in order to track cached objects. This substantially increases the size of tcache_t, but results in much higher data locality for common tcache operations. As a side benefit, it is again possible to efficiently flush the least recently used cached objects, so this change changes flushing from MRU to LRU. The new bitmap implementation uses a multi-level summary approach to make finding the lowest available region very fast. In practice, bitmaps only have one or two levels, though the implementation is general enough to handle extremely large bitmaps, mainly so that large page sizes can still be entertained. Fix tcache_bin_flush_large() to always flush statistics, in the same way that tcache_bin_flush_small() was recently fixed. Use JEMALLOC_DEBUG rather than NDEBUG. Add dassert(), and use it for debug-only asserts.
2011-03-17 01:30:13 +08:00
Implement BITMAP_INFO_INITIALIZER(nbits). This allows static initialization of bitmap_info_t structures.
2016-04-07 09:30:02 +08:00
TEST_BEGIN(test_bitmap_init)
{
size_t nbits;
for (nbits = 1; nbits <= BITMAP_MAXBITS; nbits++) {
Use bitmaps to track small regions. The previous free list implementation, which embedded singly linked lists in available regions, had the unfortunate side effect of causing many cache misses during thread cache fills. Fix this in two places: - arena_run_t: Use a new bitmap implementation to track which regions are available. Furthermore, revert to preferring the lowest available region (as jemalloc did with its old bitmap-based approach). - tcache_t: Move read-only tcache_bin_t metadata into tcache_bin_info_t, and add a contiguous array of pointers to tcache_t in order to track cached objects. This substantially increases the size of tcache_t, but results in much higher data locality for common tcache operations. As a side benefit, it is again possible to efficiently flush the least recently used cached objects, so this change changes flushing from MRU to LRU. The new bitmap implementation uses a multi-level summary approach to make finding the lowest available region very fast. In practice, bitmaps only have one or two levels, though the implementation is general enough to handle extremely large bitmaps, mainly so that large page sizes can still be entertained. Fix tcache_bin_flush_large() to always flush statistics, in the same way that tcache_bin_flush_small() was recently fixed. Use JEMALLOC_DEBUG rather than NDEBUG. Add dassert(), and use it for debug-only asserts.
2011-03-17 01:30:13 +08:00
bitmap_info_t binfo;
Implement BITMAP_INFO_INITIALIZER(nbits). This allows static initialization of bitmap_info_t structures.
2016-04-07 09:30:02 +08:00
bitmap_info_init(&binfo, nbits);
test_bitmap_init_body(&binfo, nbits);
}
#define NB(nbits) { \
bitmap_info_t binfo = BITMAP_INFO_INITIALIZER(nbits); \
test_bitmap_init_body(&binfo, nbits); \
Use bitmaps to track small regions. The previous free list implementation, which embedded singly linked lists in available regions, had the unfortunate side effect of causing many cache misses during thread cache fills. Fix this in two places: - arena_run_t: Use a new bitmap implementation to track which regions are available. Furthermore, revert to preferring the lowest available region (as jemalloc did with its old bitmap-based approach). - tcache_t: Move read-only tcache_bin_t metadata into tcache_bin_info_t, and add a contiguous array of pointers to tcache_t in order to track cached objects. This substantially increases the size of tcache_t, but results in much higher data locality for common tcache operations. As a side benefit, it is again possible to efficiently flush the least recently used cached objects, so this change changes flushing from MRU to LRU. The new bitmap implementation uses a multi-level summary approach to make finding the lowest available region very fast. In practice, bitmaps only have one or two levels, though the implementation is general enough to handle extremely large bitmaps, mainly so that large page sizes can still be entertained. Fix tcache_bin_flush_large() to always flush statistics, in the same way that tcache_bin_flush_small() was recently fixed. Use JEMALLOC_DEBUG rather than NDEBUG. Add dassert(), and use it for debug-only asserts.
2011-03-17 01:30:13 +08:00
}
Implement BITMAP_INFO_INITIALIZER(nbits). This allows static initialization of bitmap_info_t structures.
2016-04-07 09:30:02 +08:00
NBITS_TAB
#undef NB
Use bitmaps to track small regions. The previous free list implementation, which embedded singly linked lists in available regions, had the unfortunate side effect of causing many cache misses during thread cache fills. Fix this in two places: - arena_run_t: Use a new bitmap implementation to track which regions are available. Furthermore, revert to preferring the lowest available region (as jemalloc did with its old bitmap-based approach). - tcache_t: Move read-only tcache_bin_t metadata into tcache_bin_info_t, and add a contiguous array of pointers to tcache_t in order to track cached objects. This substantially increases the size of tcache_t, but results in much higher data locality for common tcache operations. As a side benefit, it is again possible to efficiently flush the least recently used cached objects, so this change changes flushing from MRU to LRU. The new bitmap implementation uses a multi-level summary approach to make finding the lowest available region very fast. In practice, bitmaps only have one or two levels, though the implementation is general enough to handle extremely large bitmaps, mainly so that large page sizes can still be entertained. Fix tcache_bin_flush_large() to always flush statistics, in the same way that tcache_bin_flush_small() was recently fixed. Use JEMALLOC_DEBUG rather than NDEBUG. Add dassert(), and use it for debug-only asserts.
2011-03-17 01:30:13 +08:00
}
Refactor tests. Refactor tests to use explicit testing assertions, rather than diff'ing test output. This makes the test code a bit shorter, more explicitly encodes testing intent, and makes test failure diagnosis more straightforward.
2013-12-09 12:52:21 +08:00
TEST_END
Use bitmaps to track small regions. The previous free list implementation, which embedded singly linked lists in available regions, had the unfortunate side effect of causing many cache misses during thread cache fills. Fix this in two places: - arena_run_t: Use a new bitmap implementation to track which regions are available. Furthermore, revert to preferring the lowest available region (as jemalloc did with its old bitmap-based approach). - tcache_t: Move read-only tcache_bin_t metadata into tcache_bin_info_t, and add a contiguous array of pointers to tcache_t in order to track cached objects. This substantially increases the size of tcache_t, but results in much higher data locality for common tcache operations. As a side benefit, it is again possible to efficiently flush the least recently used cached objects, so this change changes flushing from MRU to LRU. The new bitmap implementation uses a multi-level summary approach to make finding the lowest available region very fast. In practice, bitmaps only have one or two levels, though the implementation is general enough to handle extremely large bitmaps, mainly so that large page sizes can still be entertained. Fix tcache_bin_flush_large() to always flush statistics, in the same way that tcache_bin_flush_small() was recently fixed. Use JEMALLOC_DEBUG rather than NDEBUG. Add dassert(), and use it for debug-only asserts.
2011-03-17 01:30:13 +08:00
Implement BITMAP_INFO_INITIALIZER(nbits). This allows static initialization of bitmap_info_t structures.
2016-04-07 09:30:02 +08:00
static void
test_bitmap_set_body(const bitmap_info_t *binfo, size_t nbits)
Use bitmaps to track small regions. The previous free list implementation, which embedded singly linked lists in available regions, had the unfortunate side effect of causing many cache misses during thread cache fills. Fix this in two places: - arena_run_t: Use a new bitmap implementation to track which regions are available. Furthermore, revert to preferring the lowest available region (as jemalloc did with its old bitmap-based approach). - tcache_t: Move read-only tcache_bin_t metadata into tcache_bin_info_t, and add a contiguous array of pointers to tcache_t in order to track cached objects. This substantially increases the size of tcache_t, but results in much higher data locality for common tcache operations. As a side benefit, it is again possible to efficiently flush the least recently used cached objects, so this change changes flushing from MRU to LRU. The new bitmap implementation uses a multi-level summary approach to make finding the lowest available region very fast. In practice, bitmaps only have one or two levels, though the implementation is general enough to handle extremely large bitmaps, mainly so that large page sizes can still be entertained. Fix tcache_bin_flush_large() to always flush statistics, in the same way that tcache_bin_flush_small() was recently fixed. Use JEMALLOC_DEBUG rather than NDEBUG. Add dassert(), and use it for debug-only asserts.
2011-03-17 01:30:13 +08:00
{
size_t i;
Implement BITMAP_INFO_INITIALIZER(nbits). This allows static initialization of bitmap_info_t structures.
2016-04-07 09:30:02 +08:00
bitmap_t *bitmap = (bitmap_t *)malloc(bitmap_size(binfo));
assert_ptr_not_null(bitmap, "Unexpected malloc() failure");
bitmap_init(bitmap, binfo);
for (i = 0; i < nbits; i++)
bitmap_set(bitmap, binfo, i);
assert_true(bitmap_full(bitmap, binfo), "All bits should be set");
free(bitmap);
}
TEST_BEGIN(test_bitmap_set)
{
size_t nbits;
Use bitmaps to track small regions. The previous free list implementation, which embedded singly linked lists in available regions, had the unfortunate side effect of causing many cache misses during thread cache fills. Fix this in two places: - arena_run_t: Use a new bitmap implementation to track which regions are available. Furthermore, revert to preferring the lowest available region (as jemalloc did with its old bitmap-based approach). - tcache_t: Move read-only tcache_bin_t metadata into tcache_bin_info_t, and add a contiguous array of pointers to tcache_t in order to track cached objects. This substantially increases the size of tcache_t, but results in much higher data locality for common tcache operations. As a side benefit, it is again possible to efficiently flush the least recently used cached objects, so this change changes flushing from MRU to LRU. The new bitmap implementation uses a multi-level summary approach to make finding the lowest available region very fast. In practice, bitmaps only have one or two levels, though the implementation is general enough to handle extremely large bitmaps, mainly so that large page sizes can still be entertained. Fix tcache_bin_flush_large() to always flush statistics, in the same way that tcache_bin_flush_small() was recently fixed. Use JEMALLOC_DEBUG rather than NDEBUG. Add dassert(), and use it for debug-only asserts.
2011-03-17 01:30:13 +08:00
Implement BITMAP_INFO_INITIALIZER(nbits). This allows static initialization of bitmap_info_t structures.
2016-04-07 09:30:02 +08:00
for (nbits = 1; nbits <= BITMAP_MAXBITS; nbits++) {
Use bitmaps to track small regions. The previous free list implementation, which embedded singly linked lists in available regions, had the unfortunate side effect of causing many cache misses during thread cache fills. Fix this in two places: - arena_run_t: Use a new bitmap implementation to track which regions are available. Furthermore, revert to preferring the lowest available region (as jemalloc did with its old bitmap-based approach). - tcache_t: Move read-only tcache_bin_t metadata into tcache_bin_info_t, and add a contiguous array of pointers to tcache_t in order to track cached objects. This substantially increases the size of tcache_t, but results in much higher data locality for common tcache operations. As a side benefit, it is again possible to efficiently flush the least recently used cached objects, so this change changes flushing from MRU to LRU. The new bitmap implementation uses a multi-level summary approach to make finding the lowest available region very fast. In practice, bitmaps only have one or two levels, though the implementation is general enough to handle extremely large bitmaps, mainly so that large page sizes can still be entertained. Fix tcache_bin_flush_large() to always flush statistics, in the same way that tcache_bin_flush_small() was recently fixed. Use JEMALLOC_DEBUG rather than NDEBUG. Add dassert(), and use it for debug-only asserts.
2011-03-17 01:30:13 +08:00
bitmap_info_t binfo;
Implement BITMAP_INFO_INITIALIZER(nbits). This allows static initialization of bitmap_info_t structures.
2016-04-07 09:30:02 +08:00
bitmap_info_init(&binfo, nbits);
test_bitmap_set_body(&binfo, nbits);
}
#define NB(nbits) { \
bitmap_info_t binfo = BITMAP_INFO_INITIALIZER(nbits); \
test_bitmap_set_body(&binfo, nbits); \
Use bitmaps to track small regions. The previous free list implementation, which embedded singly linked lists in available regions, had the unfortunate side effect of causing many cache misses during thread cache fills. Fix this in two places: - arena_run_t: Use a new bitmap implementation to track which regions are available. Furthermore, revert to preferring the lowest available region (as jemalloc did with its old bitmap-based approach). - tcache_t: Move read-only tcache_bin_t metadata into tcache_bin_info_t, and add a contiguous array of pointers to tcache_t in order to track cached objects. This substantially increases the size of tcache_t, but results in much higher data locality for common tcache operations. As a side benefit, it is again possible to efficiently flush the least recently used cached objects, so this change changes flushing from MRU to LRU. The new bitmap implementation uses a multi-level summary approach to make finding the lowest available region very fast. In practice, bitmaps only have one or two levels, though the implementation is general enough to handle extremely large bitmaps, mainly so that large page sizes can still be entertained. Fix tcache_bin_flush_large() to always flush statistics, in the same way that tcache_bin_flush_small() was recently fixed. Use JEMALLOC_DEBUG rather than NDEBUG. Add dassert(), and use it for debug-only asserts.
2011-03-17 01:30:13 +08:00
}
Implement BITMAP_INFO_INITIALIZER(nbits). This allows static initialization of bitmap_info_t structures.
2016-04-07 09:30:02 +08:00
NBITS_TAB
#undef NB
Use bitmaps to track small regions. The previous free list implementation, which embedded singly linked lists in available regions, had the unfortunate side effect of causing many cache misses during thread cache fills. Fix this in two places: - arena_run_t: Use a new bitmap implementation to track which regions are available. Furthermore, revert to preferring the lowest available region (as jemalloc did with its old bitmap-based approach). - tcache_t: Move read-only tcache_bin_t metadata into tcache_bin_info_t, and add a contiguous array of pointers to tcache_t in order to track cached objects. This substantially increases the size of tcache_t, but results in much higher data locality for common tcache operations. As a side benefit, it is again possible to efficiently flush the least recently used cached objects, so this change changes flushing from MRU to LRU. The new bitmap implementation uses a multi-level summary approach to make finding the lowest available region very fast. In practice, bitmaps only have one or two levels, though the implementation is general enough to handle extremely large bitmaps, mainly so that large page sizes can still be entertained. Fix tcache_bin_flush_large() to always flush statistics, in the same way that tcache_bin_flush_small() was recently fixed. Use JEMALLOC_DEBUG rather than NDEBUG. Add dassert(), and use it for debug-only asserts.
2011-03-17 01:30:13 +08:00
}
Refactor tests. Refactor tests to use explicit testing assertions, rather than diff'ing test output. This makes the test code a bit shorter, more explicitly encodes testing intent, and makes test failure diagnosis more straightforward.
2013-12-09 12:52:21 +08:00
TEST_END
Use bitmaps to track small regions. The previous free list implementation, which embedded singly linked lists in available regions, had the unfortunate side effect of causing many cache misses during thread cache fills. Fix this in two places: - arena_run_t: Use a new bitmap implementation to track which regions are available. Furthermore, revert to preferring the lowest available region (as jemalloc did with its old bitmap-based approach). - tcache_t: Move read-only tcache_bin_t metadata into tcache_bin_info_t, and add a contiguous array of pointers to tcache_t in order to track cached objects. This substantially increases the size of tcache_t, but results in much higher data locality for common tcache operations. As a side benefit, it is again possible to efficiently flush the least recently used cached objects, so this change changes flushing from MRU to LRU. The new bitmap implementation uses a multi-level summary approach to make finding the lowest available region very fast. In practice, bitmaps only have one or two levels, though the implementation is general enough to handle extremely large bitmaps, mainly so that large page sizes can still be entertained. Fix tcache_bin_flush_large() to always flush statistics, in the same way that tcache_bin_flush_small() was recently fixed. Use JEMALLOC_DEBUG rather than NDEBUG. Add dassert(), and use it for debug-only asserts.
2011-03-17 01:30:13 +08:00
Implement BITMAP_INFO_INITIALIZER(nbits). This allows static initialization of bitmap_info_t structures.
2016-04-07 09:30:02 +08:00
static void
test_bitmap_unset_body(const bitmap_info_t *binfo, size_t nbits)
Use bitmaps to track small regions. The previous free list implementation, which embedded singly linked lists in available regions, had the unfortunate side effect of causing many cache misses during thread cache fills. Fix this in two places: - arena_run_t: Use a new bitmap implementation to track which regions are available. Furthermore, revert to preferring the lowest available region (as jemalloc did with its old bitmap-based approach). - tcache_t: Move read-only tcache_bin_t metadata into tcache_bin_info_t, and add a contiguous array of pointers to tcache_t in order to track cached objects. This substantially increases the size of tcache_t, but results in much higher data locality for common tcache operations. As a side benefit, it is again possible to efficiently flush the least recently used cached objects, so this change changes flushing from MRU to LRU. The new bitmap implementation uses a multi-level summary approach to make finding the lowest available region very fast. In practice, bitmaps only have one or two levels, though the implementation is general enough to handle extremely large bitmaps, mainly so that large page sizes can still be entertained. Fix tcache_bin_flush_large() to always flush statistics, in the same way that tcache_bin_flush_small() was recently fixed. Use JEMALLOC_DEBUG rather than NDEBUG. Add dassert(), and use it for debug-only asserts.
2011-03-17 01:30:13 +08:00
{
size_t i;
Implement BITMAP_INFO_INITIALIZER(nbits). This allows static initialization of bitmap_info_t structures.
2016-04-07 09:30:02 +08:00
bitmap_t *bitmap = (bitmap_t *)malloc(bitmap_size(binfo));
assert_ptr_not_null(bitmap, "Unexpected malloc() failure");
bitmap_init(bitmap, binfo);
for (i = 0; i < nbits; i++)
bitmap_set(bitmap, binfo, i);
assert_true(bitmap_full(bitmap, binfo), "All bits should be set");
for (i = 0; i < nbits; i++)
bitmap_unset(bitmap, binfo, i);
for (i = 0; i < nbits; i++)
bitmap_set(bitmap, binfo, i);
assert_true(bitmap_full(bitmap, binfo), "All bits should be set");
free(bitmap);
}
Use bitmaps to track small regions. The previous free list implementation, which embedded singly linked lists in available regions, had the unfortunate side effect of causing many cache misses during thread cache fills. Fix this in two places: - arena_run_t: Use a new bitmap implementation to track which regions are available. Furthermore, revert to preferring the lowest available region (as jemalloc did with its old bitmap-based approach). - tcache_t: Move read-only tcache_bin_t metadata into tcache_bin_info_t, and add a contiguous array of pointers to tcache_t in order to track cached objects. This substantially increases the size of tcache_t, but results in much higher data locality for common tcache operations. As a side benefit, it is again possible to efficiently flush the least recently used cached objects, so this change changes flushing from MRU to LRU. The new bitmap implementation uses a multi-level summary approach to make finding the lowest available region very fast. In practice, bitmaps only have one or two levels, though the implementation is general enough to handle extremely large bitmaps, mainly so that large page sizes can still be entertained. Fix tcache_bin_flush_large() to always flush statistics, in the same way that tcache_bin_flush_small() was recently fixed. Use JEMALLOC_DEBUG rather than NDEBUG. Add dassert(), and use it for debug-only asserts.
2011-03-17 01:30:13 +08:00
Implement BITMAP_INFO_INITIALIZER(nbits). This allows static initialization of bitmap_info_t structures.
2016-04-07 09:30:02 +08:00
TEST_BEGIN(test_bitmap_unset)
{
size_t nbits;
for (nbits = 1; nbits <= BITMAP_MAXBITS; nbits++) {
Use bitmaps to track small regions. The previous free list implementation, which embedded singly linked lists in available regions, had the unfortunate side effect of causing many cache misses during thread cache fills. Fix this in two places: - arena_run_t: Use a new bitmap implementation to track which regions are available. Furthermore, revert to preferring the lowest available region (as jemalloc did with its old bitmap-based approach). - tcache_t: Move read-only tcache_bin_t metadata into tcache_bin_info_t, and add a contiguous array of pointers to tcache_t in order to track cached objects. This substantially increases the size of tcache_t, but results in much higher data locality for common tcache operations. As a side benefit, it is again possible to efficiently flush the least recently used cached objects, so this change changes flushing from MRU to LRU. The new bitmap implementation uses a multi-level summary approach to make finding the lowest available region very fast. In practice, bitmaps only have one or two levels, though the implementation is general enough to handle extremely large bitmaps, mainly so that large page sizes can still be entertained. Fix tcache_bin_flush_large() to always flush statistics, in the same way that tcache_bin_flush_small() was recently fixed. Use JEMALLOC_DEBUG rather than NDEBUG. Add dassert(), and use it for debug-only asserts.
2011-03-17 01:30:13 +08:00
bitmap_info_t binfo;
Implement BITMAP_INFO_INITIALIZER(nbits). This allows static initialization of bitmap_info_t structures.
2016-04-07 09:30:02 +08:00
bitmap_info_init(&binfo, nbits);
test_bitmap_unset_body(&binfo, nbits);
Use bitmaps to track small regions. The previous free list implementation, which embedded singly linked lists in available regions, had the unfortunate side effect of causing many cache misses during thread cache fills. Fix this in two places: - arena_run_t: Use a new bitmap implementation to track which regions are available. Furthermore, revert to preferring the lowest available region (as jemalloc did with its old bitmap-based approach). - tcache_t: Move read-only tcache_bin_t metadata into tcache_bin_info_t, and add a contiguous array of pointers to tcache_t in order to track cached objects. This substantially increases the size of tcache_t, but results in much higher data locality for common tcache operations. As a side benefit, it is again possible to efficiently flush the least recently used cached objects, so this change changes flushing from MRU to LRU. The new bitmap implementation uses a multi-level summary approach to make finding the lowest available region very fast. In practice, bitmaps only have one or two levels, though the implementation is general enough to handle extremely large bitmaps, mainly so that large page sizes can still be entertained. Fix tcache_bin_flush_large() to always flush statistics, in the same way that tcache_bin_flush_small() was recently fixed. Use JEMALLOC_DEBUG rather than NDEBUG. Add dassert(), and use it for debug-only asserts.
2011-03-17 01:30:13 +08:00
}
Implement BITMAP_INFO_INITIALIZER(nbits). This allows static initialization of bitmap_info_t structures.
2016-04-07 09:30:02 +08:00
#define NB(nbits) { \
bitmap_info_t binfo = BITMAP_INFO_INITIALIZER(nbits); \
test_bitmap_unset_body(&binfo, nbits); \
}
NBITS_TAB
#undef NB
Use bitmaps to track small regions. The previous free list implementation, which embedded singly linked lists in available regions, had the unfortunate side effect of causing many cache misses during thread cache fills. Fix this in two places: - arena_run_t: Use a new bitmap implementation to track which regions are available. Furthermore, revert to preferring the lowest available region (as jemalloc did with its old bitmap-based approach). - tcache_t: Move read-only tcache_bin_t metadata into tcache_bin_info_t, and add a contiguous array of pointers to tcache_t in order to track cached objects. This substantially increases the size of tcache_t, but results in much higher data locality for common tcache operations. As a side benefit, it is again possible to efficiently flush the least recently used cached objects, so this change changes flushing from MRU to LRU. The new bitmap implementation uses a multi-level summary approach to make finding the lowest available region very fast. In practice, bitmaps only have one or two levels, though the implementation is general enough to handle extremely large bitmaps, mainly so that large page sizes can still be entertained. Fix tcache_bin_flush_large() to always flush statistics, in the same way that tcache_bin_flush_small() was recently fixed. Use JEMALLOC_DEBUG rather than NDEBUG. Add dassert(), and use it for debug-only asserts.
2011-03-17 01:30:13 +08:00
}
Refactor tests. Refactor tests to use explicit testing assertions, rather than diff'ing test output. This makes the test code a bit shorter, more explicitly encodes testing intent, and makes test failure diagnosis more straightforward.
2013-12-09 12:52:21 +08:00
TEST_END
Use bitmaps to track small regions. The previous free list implementation, which embedded singly linked lists in available regions, had the unfortunate side effect of causing many cache misses during thread cache fills. Fix this in two places: - arena_run_t: Use a new bitmap implementation to track which regions are available. Furthermore, revert to preferring the lowest available region (as jemalloc did with its old bitmap-based approach). - tcache_t: Move read-only tcache_bin_t metadata into tcache_bin_info_t, and add a contiguous array of pointers to tcache_t in order to track cached objects. This substantially increases the size of tcache_t, but results in much higher data locality for common tcache operations. As a side benefit, it is again possible to efficiently flush the least recently used cached objects, so this change changes flushing from MRU to LRU. The new bitmap implementation uses a multi-level summary approach to make finding the lowest available region very fast. In practice, bitmaps only have one or two levels, though the implementation is general enough to handle extremely large bitmaps, mainly so that large page sizes can still be entertained. Fix tcache_bin_flush_large() to always flush statistics, in the same way that tcache_bin_flush_small() was recently fixed. Use JEMALLOC_DEBUG rather than NDEBUG. Add dassert(), and use it for debug-only asserts.
2011-03-17 01:30:13 +08:00
Implement BITMAP_INFO_INITIALIZER(nbits). This allows static initialization of bitmap_info_t structures.
2016-04-07 09:30:02 +08:00
static void
test_bitmap_sfu_body(const bitmap_info_t *binfo, size_t nbits)
Use bitmaps to track small regions. The previous free list implementation, which embedded singly linked lists in available regions, had the unfortunate side effect of causing many cache misses during thread cache fills. Fix this in two places: - arena_run_t: Use a new bitmap implementation to track which regions are available. Furthermore, revert to preferring the lowest available region (as jemalloc did with its old bitmap-based approach). - tcache_t: Move read-only tcache_bin_t metadata into tcache_bin_info_t, and add a contiguous array of pointers to tcache_t in order to track cached objects. This substantially increases the size of tcache_t, but results in much higher data locality for common tcache operations. As a side benefit, it is again possible to efficiently flush the least recently used cached objects, so this change changes flushing from MRU to LRU. The new bitmap implementation uses a multi-level summary approach to make finding the lowest available region very fast. In practice, bitmaps only have one or two levels, though the implementation is general enough to handle extremely large bitmaps, mainly so that large page sizes can still be entertained. Fix tcache_bin_flush_large() to always flush statistics, in the same way that tcache_bin_flush_small() was recently fixed. Use JEMALLOC_DEBUG rather than NDEBUG. Add dassert(), and use it for debug-only asserts.
2011-03-17 01:30:13 +08:00
{
size_t i;
Implement BITMAP_INFO_INITIALIZER(nbits). This allows static initialization of bitmap_info_t structures.
2016-04-07 09:30:02 +08:00
bitmap_t *bitmap = (bitmap_t *)malloc(bitmap_size(binfo));
assert_ptr_not_null(bitmap, "Unexpected malloc() failure");
bitmap_init(bitmap, binfo);
/* Iteratively set bits starting at the beginning. */
for (i = 0; i < nbits; i++) {
assert_zd_eq(bitmap_sfu(bitmap, binfo), i,
"First unset bit should be just after previous first unset "
"bit");
}
assert_true(bitmap_full(bitmap, binfo), "All bits should be set");
/*
* Iteratively unset bits starting at the end, and verify that
* bitmap_sfu() reaches the unset bits.
*/
for (i = nbits - 1; i < nbits; i--) { /* (nbits..0] */
bitmap_unset(bitmap, binfo, i);
assert_zd_eq(bitmap_sfu(bitmap, binfo), i,
"First unset bit should the bit previously unset");
bitmap_unset(bitmap, binfo, i);
}
assert_false(bitmap_get(bitmap, binfo, 0), "Bit should be unset");
/*
* Iteratively set bits starting at the beginning, and verify that
* bitmap_sfu() looks past them.
*/
for (i = 1; i < nbits; i++) {
bitmap_set(bitmap, binfo, i - 1);
assert_zd_eq(bitmap_sfu(bitmap, binfo), i,
"First unset bit should be just after the bit previously "
"set");
bitmap_unset(bitmap, binfo, i);
}
assert_zd_eq(bitmap_sfu(bitmap, binfo), nbits - 1,
"First unset bit should be the last bit");
assert_true(bitmap_full(bitmap, binfo), "All bits should be set");
free(bitmap);
}
Use bitmaps to track small regions. The previous free list implementation, which embedded singly linked lists in available regions, had the unfortunate side effect of causing many cache misses during thread cache fills. Fix this in two places: - arena_run_t: Use a new bitmap implementation to track which regions are available. Furthermore, revert to preferring the lowest available region (as jemalloc did with its old bitmap-based approach). - tcache_t: Move read-only tcache_bin_t metadata into tcache_bin_info_t, and add a contiguous array of pointers to tcache_t in order to track cached objects. This substantially increases the size of tcache_t, but results in much higher data locality for common tcache operations. As a side benefit, it is again possible to efficiently flush the least recently used cached objects, so this change changes flushing from MRU to LRU. The new bitmap implementation uses a multi-level summary approach to make finding the lowest available region very fast. In practice, bitmaps only have one or two levels, though the implementation is general enough to handle extremely large bitmaps, mainly so that large page sizes can still be entertained. Fix tcache_bin_flush_large() to always flush statistics, in the same way that tcache_bin_flush_small() was recently fixed. Use JEMALLOC_DEBUG rather than NDEBUG. Add dassert(), and use it for debug-only asserts.
2011-03-17 01:30:13 +08:00
Implement BITMAP_INFO_INITIALIZER(nbits). This allows static initialization of bitmap_info_t structures.
2016-04-07 09:30:02 +08:00
TEST_BEGIN(test_bitmap_sfu)
{
size_t nbits;
for (nbits = 1; nbits <= BITMAP_MAXBITS; nbits++) {
Use bitmaps to track small regions. The previous free list implementation, which embedded singly linked lists in available regions, had the unfortunate side effect of causing many cache misses during thread cache fills. Fix this in two places: - arena_run_t: Use a new bitmap implementation to track which regions are available. Furthermore, revert to preferring the lowest available region (as jemalloc did with its old bitmap-based approach). - tcache_t: Move read-only tcache_bin_t metadata into tcache_bin_info_t, and add a contiguous array of pointers to tcache_t in order to track cached objects. This substantially increases the size of tcache_t, but results in much higher data locality for common tcache operations. As a side benefit, it is again possible to efficiently flush the least recently used cached objects, so this change changes flushing from MRU to LRU. The new bitmap implementation uses a multi-level summary approach to make finding the lowest available region very fast. In practice, bitmaps only have one or two levels, though the implementation is general enough to handle extremely large bitmaps, mainly so that large page sizes can still be entertained. Fix tcache_bin_flush_large() to always flush statistics, in the same way that tcache_bin_flush_small() was recently fixed. Use JEMALLOC_DEBUG rather than NDEBUG. Add dassert(), and use it for debug-only asserts.
2011-03-17 01:30:13 +08:00
bitmap_info_t binfo;
Implement BITMAP_INFO_INITIALIZER(nbits). This allows static initialization of bitmap_info_t structures.
2016-04-07 09:30:02 +08:00
bitmap_info_init(&binfo, nbits);
test_bitmap_sfu_body(&binfo, nbits);
}
#define NB(nbits) { \
bitmap_info_t binfo = BITMAP_INFO_INITIALIZER(nbits); \
test_bitmap_sfu_body(&binfo, nbits); \
Use bitmaps to track small regions. The previous free list implementation, which embedded singly linked lists in available regions, had the unfortunate side effect of causing many cache misses during thread cache fills. Fix this in two places: - arena_run_t: Use a new bitmap implementation to track which regions are available. Furthermore, revert to preferring the lowest available region (as jemalloc did with its old bitmap-based approach). - tcache_t: Move read-only tcache_bin_t metadata into tcache_bin_info_t, and add a contiguous array of pointers to tcache_t in order to track cached objects. This substantially increases the size of tcache_t, but results in much higher data locality for common tcache operations. As a side benefit, it is again possible to efficiently flush the least recently used cached objects, so this change changes flushing from MRU to LRU. The new bitmap implementation uses a multi-level summary approach to make finding the lowest available region very fast. In practice, bitmaps only have one or two levels, though the implementation is general enough to handle extremely large bitmaps, mainly so that large page sizes can still be entertained. Fix tcache_bin_flush_large() to always flush statistics, in the same way that tcache_bin_flush_small() was recently fixed. Use JEMALLOC_DEBUG rather than NDEBUG. Add dassert(), and use it for debug-only asserts.
2011-03-17 01:30:13 +08:00
}
Implement BITMAP_INFO_INITIALIZER(nbits). This allows static initialization of bitmap_info_t structures.
2016-04-07 09:30:02 +08:00
NBITS_TAB
#undef NB
Use bitmaps to track small regions. The previous free list implementation, which embedded singly linked lists in available regions, had the unfortunate side effect of causing many cache misses during thread cache fills. Fix this in two places: - arena_run_t: Use a new bitmap implementation to track which regions are available. Furthermore, revert to preferring the lowest available region (as jemalloc did with its old bitmap-based approach). - tcache_t: Move read-only tcache_bin_t metadata into tcache_bin_info_t, and add a contiguous array of pointers to tcache_t in order to track cached objects. This substantially increases the size of tcache_t, but results in much higher data locality for common tcache operations. As a side benefit, it is again possible to efficiently flush the least recently used cached objects, so this change changes flushing from MRU to LRU. The new bitmap implementation uses a multi-level summary approach to make finding the lowest available region very fast. In practice, bitmaps only have one or two levels, though the implementation is general enough to handle extremely large bitmaps, mainly so that large page sizes can still be entertained. Fix tcache_bin_flush_large() to always flush statistics, in the same way that tcache_bin_flush_small() was recently fixed. Use JEMALLOC_DEBUG rather than NDEBUG. Add dassert(), and use it for debug-only asserts.
2011-03-17 01:30:13 +08:00
}
Refactor tests. Refactor tests to use explicit testing assertions, rather than diff'ing test output. This makes the test code a bit shorter, more explicitly encodes testing intent, and makes test failure diagnosis more straightforward.
2013-12-09 12:52:21 +08:00
TEST_END
Use bitmaps to track small regions. The previous free list implementation, which embedded singly linked lists in available regions, had the unfortunate side effect of causing many cache misses during thread cache fills. Fix this in two places: - arena_run_t: Use a new bitmap implementation to track which regions are available. Furthermore, revert to preferring the lowest available region (as jemalloc did with its old bitmap-based approach). - tcache_t: Move read-only tcache_bin_t metadata into tcache_bin_info_t, and add a contiguous array of pointers to tcache_t in order to track cached objects. This substantially increases the size of tcache_t, but results in much higher data locality for common tcache operations. As a side benefit, it is again possible to efficiently flush the least recently used cached objects, so this change changes flushing from MRU to LRU. The new bitmap implementation uses a multi-level summary approach to make finding the lowest available region very fast. In practice, bitmaps only have one or two levels, though the implementation is general enough to handle extremely large bitmaps, mainly so that large page sizes can still be entertained. Fix tcache_bin_flush_large() to always flush statistics, in the same way that tcache_bin_flush_small() was recently fixed. Use JEMALLOC_DEBUG rather than NDEBUG. Add dassert(), and use it for debug-only asserts.
2011-03-17 01:30:13 +08:00
int
main(void)
{
Refactor tests. Refactor tests to use explicit testing assertions, rather than diff'ing test output. This makes the test code a bit shorter, more explicitly encodes testing intent, and makes test failure diagnosis more straightforward.
2013-12-09 12:52:21 +08:00
return (test(
Implement BITMAP_INFO_INITIALIZER(nbits). This allows static initialization of bitmap_info_t structures.
2016-04-07 09:30:02 +08:00
test_bitmap_initializer,
Refactor tests. Refactor tests to use explicit testing assertions, rather than diff'ing test output. This makes the test code a bit shorter, more explicitly encodes testing intent, and makes test failure diagnosis more straightforward.
2013-12-09 12:52:21 +08:00
test_bitmap_size,
test_bitmap_init,
test_bitmap_set,
test_bitmap_unset,
test_bitmap_sfu));
Use bitmaps to track small regions. The previous free list implementation, which embedded singly linked lists in available regions, had the unfortunate side effect of causing many cache misses during thread cache fills. Fix this in two places: - arena_run_t: Use a new bitmap implementation to track which regions are available. Furthermore, revert to preferring the lowest available region (as jemalloc did with its old bitmap-based approach). - tcache_t: Move read-only tcache_bin_t metadata into tcache_bin_info_t, and add a contiguous array of pointers to tcache_t in order to track cached objects. This substantially increases the size of tcache_t, but results in much higher data locality for common tcache operations. As a side benefit, it is again possible to efficiently flush the least recently used cached objects, so this change changes flushing from MRU to LRU. The new bitmap implementation uses a multi-level summary approach to make finding the lowest available region very fast. In practice, bitmaps only have one or two levels, though the implementation is general enough to handle extremely large bitmaps, mainly so that large page sizes can still be entertained. Fix tcache_bin_flush_large() to always flush statistics, in the same way that tcache_bin_flush_small() was recently fixed. Use JEMALLOC_DEBUG rather than NDEBUG. Add dassert(), and use it for debug-only asserts.
2011-03-17 01:30:13 +08:00
}
Reference in New Issue Copy Permalink