project-base/server-skynet-source-3rd-jemalloc
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
b2c08ef2e6
server-skynet-source-3rd-je.../test/unit/arena_decay.c

597 lines
18 KiB
C
Raw Normal View History

Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
#include "test/jemalloc_test.h"
Header refactoring: ticker module - remove from the catchall and unify.
2017-04-20 04:39:33 +08:00
#include "jemalloc/internal/ticker.h"
Do not advance decay epoch when time goes backwards. Instead, move the epoch backward in time. Additionally, add nstime_monotonic() and use it in debug builds to assert that time only goes backward if nstime_update() is using a non-monotonic time source.
2016-10-11 13:15:10 +08:00
static nstime_monotonic_t *nstime_monotonic_orig;
Refactor time_* into nstime_*. Use a single uint64_t in nstime_t to store nanoseconds rather than using struct timespec. This reduces fragility around conversions between long and uint64_t, especially missing casts that only cause problems on 32-bit platforms.
2016-02-22 03:25:02 +08:00
static nstime_update_t *nstime_update_orig;
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
static unsigned nupdates_mock;
Refactor time_* into nstime_*. Use a single uint64_t in nstime_t to store nanoseconds rather than using struct timespec. This reduces fragility around conversions between long and uint64_t, especially missing casts that only cause problems on 32-bit platforms.
2016-02-22 03:25:02 +08:00
static nstime_t time_mock;
Fix decay tests to all adapt to nstime_monotonic().
2016-10-12 06:28:43 +08:00
static bool monotonic_mock;
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
Implementing opt.background_thread. Added opt.background_thread to enable background threads, which handles purging currently. When enabled, decay ticks will not trigger purging (which will be left to the background threads). We limit the max number of threads to NCPUs. When percpu arena is enabled, set CPU affinity for the background threads as well. The sleep interval of background threads is dynamic and determined by computing number of pages to purge in the future (based on backlog).
2017-03-18 03:42:33 +08:00
static bool
check_background_thread_enabled(void) {
bool enabled;
size_t sz = sizeof(bool);
int ret = mallctl("background_thread", (void *)&enabled, &sz, NULL,0);
if (ret == ENOENT) {
return false;
}
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_d_eq(ret, 0, "Unexpected mallctl error");
Implementing opt.background_thread. Added opt.background_thread to enable background threads, which handles purging currently. When enabled, decay ticks will not trigger purging (which will be left to the background threads). We limit the max number of threads to NCPUs. When percpu arena is enabled, set CPU affinity for the background threads as well. The sleep interval of background threads is dynamic and determined by computing number of pages to purge in the future (based on backlog).
2017-03-18 03:42:33 +08:00
return enabled;
}
Do not advance decay epoch when time goes backwards. Instead, move the epoch backward in time. Additionally, add nstime_monotonic() and use it in debug builds to assert that time only goes backward if nstime_update() is using a non-monotonic time source.
2016-10-11 13:15:10 +08:00
static bool
Update brace style. Add braces around single-line blocks, and remove line breaks before function-opening braces. This resolves #537.
2017-01-16 08:56:30 +08:00
nstime_monotonic_mock(void) {
Remove extraneous parens around return arguments. This resolves #540.
2017-01-20 10:15:45 +08:00
return monotonic_mock;
Do not advance decay epoch when time goes backwards. Instead, move the epoch backward in time. Additionally, add nstime_monotonic() and use it in debug builds to assert that time only goes backward if nstime_update() is using a non-monotonic time source.
2016-10-11 13:15:10 +08:00
}
High Resolution Timestamps for Profiling
2020-06-02 21:42:44 +08:00
static void
Update brace style. Add braces around single-line blocks, and remove line breaks before function-opening braces. This resolves #537.
2017-01-16 08:56:30 +08:00
nstime_update_mock(nstime_t *time) {
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
nupdates_mock++;
Update brace style. Add braces around single-line blocks, and remove line breaks before function-opening braces. This resolves #537.
2017-01-16 08:56:30 +08:00
if (monotonic_mock) {
Refactor time_* into nstime_*. Use a single uint64_t in nstime_t to store nanoseconds rather than using struct timespec. This reduces fragility around conversions between long and uint64_t, especially missing casts that only cause problems on 32-bit platforms.
2016-02-22 03:25:02 +08:00
nstime_copy(time, &time_mock);
Update brace style. Add braces around single-line blocks, and remove line breaks before function-opening braces. This resolves #537.
2017-01-16 08:56:30 +08:00
}
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
}
Fix flakiness in test_decay_ticker. Fix the test_decay_ticker test to carefully control slab creation/destruction such that the decay backlog reliably reaches zero. Use an isolated arena so that no extraneous allocation can confuse the situation. Speed up time during the latter part of the test so that the entire decay time can expire in a reasonable amount of wall time.
2017-03-03 15:32:42 +08:00
static unsigned
Refactor *decay_time into *decay_ms. Support millisecond resolution for decay times. Among other use cases this makes it possible to specify a short initial dirty-->muzzy decay phase, followed by a longer muzzy-->clean decay phase. This resolves #812.
2017-05-18 01:47:00 +08:00
do_arena_create(ssize_t dirty_decay_ms, ssize_t muzzy_decay_ms) {
Fix flakiness in test_decay_ticker. Fix the test_decay_ticker test to carefully control slab creation/destruction such that the decay backlog reliably reaches zero. Use an isolated arena so that no extraneous allocation can confuse the situation. Speed up time during the latter part of the test so that the entire decay time can expire in a reasonable amount of wall time.
2017-03-03 15:32:42 +08:00
unsigned arena_ind;
size_t sz = sizeof(unsigned);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_d_eq(mallctl("arenas.create", (void *)&arena_ind, &sz, NULL, 0),
Fix flakiness in test_decay_ticker. Fix the test_decay_ticker test to carefully control slab creation/destruction such that the decay backlog reliably reaches zero. Use an isolated arena so that no extraneous allocation can confuse the situation. Speed up time during the latter part of the test so that the entire decay time can expire in a reasonable amount of wall time.
2017-03-03 15:32:42 +08:00
0, "Unexpected mallctl() failure");
size_t mib[3];
size_t miblen = sizeof(mib)/sizeof(size_t);
Implement two-phase decay-based purging. Split decay-based purging into two phases, the first of which uses lazy purging to convert dirty pages to "muzzy", and the second of which uses forced purging, decommit, or unmapping to convert pages to clean or destroy them altogether. Not all operating systems support lazy purging, yet the application may provide extent hooks that implement lazy purging, so care must be taken to dynamically omit the first phase when necessary. The mallctl interfaces change as follows: - opt.decay_time --> opt.{dirty,muzzy}_decay_time - arena.<i>.decay_time --> arena.<i>.{dirty,muzzy}_decay_time - arenas.decay_time --> arenas.{dirty,muzzy}_decay_time - stats.arenas.<i>.pdirty --> stats.arenas.<i>.p{dirty,muzzy} - stats.arenas.<i>.{npurge,nmadvise,purged} --> stats.arenas.<i>.{dirty,muzzy}_{npurge,nmadvise,purged} This resolves #521.
2017-03-09 14:42:57 +08:00
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_d_eq(mallctlnametomib("arena.0.dirty_decay_ms", mib, &miblen),
Implement two-phase decay-based purging. Split decay-based purging into two phases, the first of which uses lazy purging to convert dirty pages to "muzzy", and the second of which uses forced purging, decommit, or unmapping to convert pages to clean or destroy them altogether. Not all operating systems support lazy purging, yet the application may provide extent hooks that implement lazy purging, so care must be taken to dynamically omit the first phase when necessary. The mallctl interfaces change as follows: - opt.decay_time --> opt.{dirty,muzzy}_decay_time - arena.<i>.decay_time --> arena.<i>.{dirty,muzzy}_decay_time - arenas.decay_time --> arenas.{dirty,muzzy}_decay_time - stats.arenas.<i>.pdirty --> stats.arenas.<i>.p{dirty,muzzy} - stats.arenas.<i>.{npurge,nmadvise,purged} --> stats.arenas.<i>.{dirty,muzzy}_{npurge,nmadvise,purged} This resolves #521.
2017-03-09 14:42:57 +08:00
0, "Unexpected mallctlnametomib() failure");
mib[1] = (size_t)arena_ind;
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_d_eq(mallctlbymib(mib, miblen, NULL, NULL,
Refactor *decay_time into *decay_ms. Support millisecond resolution for decay times. Among other use cases this makes it possible to specify a short initial dirty-->muzzy decay phase, followed by a longer muzzy-->clean decay phase. This resolves #812.
2017-05-18 01:47:00 +08:00
(void *)&dirty_decay_ms, sizeof(dirty_decay_ms)), 0,
"Unexpected mallctlbymib() failure");
Implement two-phase decay-based purging. Split decay-based purging into two phases, the first of which uses lazy purging to convert dirty pages to "muzzy", and the second of which uses forced purging, decommit, or unmapping to convert pages to clean or destroy them altogether. Not all operating systems support lazy purging, yet the application may provide extent hooks that implement lazy purging, so care must be taken to dynamically omit the first phase when necessary. The mallctl interfaces change as follows: - opt.decay_time --> opt.{dirty,muzzy}_decay_time - arena.<i>.decay_time --> arena.<i>.{dirty,muzzy}_decay_time - arenas.decay_time --> arenas.{dirty,muzzy}_decay_time - stats.arenas.<i>.pdirty --> stats.arenas.<i>.p{dirty,muzzy} - stats.arenas.<i>.{npurge,nmadvise,purged} --> stats.arenas.<i>.{dirty,muzzy}_{npurge,nmadvise,purged} This resolves #521.
2017-03-09 14:42:57 +08:00
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_d_eq(mallctlnametomib("arena.0.muzzy_decay_ms", mib, &miblen),
Implement two-phase decay-based purging. Split decay-based purging into two phases, the first of which uses lazy purging to convert dirty pages to "muzzy", and the second of which uses forced purging, decommit, or unmapping to convert pages to clean or destroy them altogether. Not all operating systems support lazy purging, yet the application may provide extent hooks that implement lazy purging, so care must be taken to dynamically omit the first phase when necessary. The mallctl interfaces change as follows: - opt.decay_time --> opt.{dirty,muzzy}_decay_time - arena.<i>.decay_time --> arena.<i>.{dirty,muzzy}_decay_time - arenas.decay_time --> arenas.{dirty,muzzy}_decay_time - stats.arenas.<i>.pdirty --> stats.arenas.<i>.p{dirty,muzzy} - stats.arenas.<i>.{npurge,nmadvise,purged} --> stats.arenas.<i>.{dirty,muzzy}_{npurge,nmadvise,purged} This resolves #521.
2017-03-09 14:42:57 +08:00
0, "Unexpected mallctlnametomib() failure");
Fix flakiness in test_decay_ticker. Fix the test_decay_ticker test to carefully control slab creation/destruction such that the decay backlog reliably reaches zero. Use an isolated arena so that no extraneous allocation can confuse the situation. Speed up time during the latter part of the test so that the entire decay time can expire in a reasonable amount of wall time.
2017-03-03 15:32:42 +08:00
mib[1] = (size_t)arena_ind;
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_d_eq(mallctlbymib(mib, miblen, NULL, NULL,
Refactor *decay_time into *decay_ms. Support millisecond resolution for decay times. Among other use cases this makes it possible to specify a short initial dirty-->muzzy decay phase, followed by a longer muzzy-->clean decay phase. This resolves #812.
2017-05-18 01:47:00 +08:00
(void *)&muzzy_decay_ms, sizeof(muzzy_decay_ms)), 0,
"Unexpected mallctlbymib() failure");
Implement two-phase decay-based purging. Split decay-based purging into two phases, the first of which uses lazy purging to convert dirty pages to "muzzy", and the second of which uses forced purging, decommit, or unmapping to convert pages to clean or destroy them altogether. Not all operating systems support lazy purging, yet the application may provide extent hooks that implement lazy purging, so care must be taken to dynamically omit the first phase when necessary. The mallctl interfaces change as follows: - opt.decay_time --> opt.{dirty,muzzy}_decay_time - arena.<i>.decay_time --> arena.<i>.{dirty,muzzy}_decay_time - arenas.decay_time --> arenas.{dirty,muzzy}_decay_time - stats.arenas.<i>.pdirty --> stats.arenas.<i>.p{dirty,muzzy} - stats.arenas.<i>.{npurge,nmadvise,purged} --> stats.arenas.<i>.{dirty,muzzy}_{npurge,nmadvise,purged} This resolves #521.
2017-03-09 14:42:57 +08:00
Fix flakiness in test_decay_ticker. Fix the test_decay_ticker test to carefully control slab creation/destruction such that the decay backlog reliably reaches zero. Use an isolated arena so that no extraneous allocation can confuse the situation. Speed up time during the latter part of the test so that the entire decay time can expire in a reasonable amount of wall time.
2017-03-03 15:32:42 +08:00
return arena_ind;
}
static void
do_arena_destroy(unsigned arena_ind) {
size_t mib[3];
size_t miblen = sizeof(mib)/sizeof(size_t);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_d_eq(mallctlnametomib("arena.0.destroy", mib, &miblen), 0,
Fix flakiness in test_decay_ticker. Fix the test_decay_ticker test to carefully control slab creation/destruction such that the decay backlog reliably reaches zero. Use an isolated arena so that no extraneous allocation can confuse the situation. Speed up time during the latter part of the test so that the entire decay time can expire in a reasonable amount of wall time.
2017-03-03 15:32:42 +08:00
"Unexpected mallctlnametomib() failure");
mib[1] = (size_t)arena_ind;
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_d_eq(mallctlbymib(mib, miblen, NULL, NULL, NULL, 0), 0,
Fix flakiness in test_decay_ticker. Fix the test_decay_ticker test to carefully control slab creation/destruction such that the decay backlog reliably reaches zero. Use an isolated arena so that no extraneous allocation can confuse the situation. Speed up time during the latter part of the test so that the entire decay time can expire in a reasonable amount of wall time.
2017-03-03 15:32:42 +08:00
"Unexpected mallctlbymib() failure");
}
void
do_epoch(void) {
uint64_t epoch = 1;
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_d_eq(mallctl("epoch", NULL, NULL, (void *)&epoch, sizeof(epoch)),
Fix flakiness in test_decay_ticker. Fix the test_decay_ticker test to carefully control slab creation/destruction such that the decay backlog reliably reaches zero. Use an isolated arena so that no extraneous allocation can confuse the situation. Speed up time during the latter part of the test so that the entire decay time can expire in a reasonable amount of wall time.
2017-03-03 15:32:42 +08:00
0, "Unexpected mallctl() failure");
}
void
do_purge(unsigned arena_ind) {
size_t mib[3];
size_t miblen = sizeof(mib)/sizeof(size_t);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_d_eq(mallctlnametomib("arena.0.purge", mib, &miblen), 0,
Fix flakiness in test_decay_ticker. Fix the test_decay_ticker test to carefully control slab creation/destruction such that the decay backlog reliably reaches zero. Use an isolated arena so that no extraneous allocation can confuse the situation. Speed up time during the latter part of the test so that the entire decay time can expire in a reasonable amount of wall time.
2017-03-03 15:32:42 +08:00
"Unexpected mallctlnametomib() failure");
mib[1] = (size_t)arena_ind;
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_d_eq(mallctlbymib(mib, miblen, NULL, NULL, NULL, 0), 0,
Fix flakiness in test_decay_ticker. Fix the test_decay_ticker test to carefully control slab creation/destruction such that the decay backlog reliably reaches zero. Use an isolated arena so that no extraneous allocation can confuse the situation. Speed up time during the latter part of the test so that the entire decay time can expire in a reasonable amount of wall time.
2017-03-03 15:32:42 +08:00
"Unexpected mallctlbymib() failure");
}
void
do_decay(unsigned arena_ind) {
size_t mib[3];
size_t miblen = sizeof(mib)/sizeof(size_t);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_d_eq(mallctlnametomib("arena.0.decay", mib, &miblen), 0,
Fix flakiness in test_decay_ticker. Fix the test_decay_ticker test to carefully control slab creation/destruction such that the decay backlog reliably reaches zero. Use an isolated arena so that no extraneous allocation can confuse the situation. Speed up time during the latter part of the test so that the entire decay time can expire in a reasonable amount of wall time.
2017-03-03 15:32:42 +08:00
"Unexpected mallctlnametomib() failure");
mib[1] = (size_t)arena_ind;
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_d_eq(mallctlbymib(mib, miblen, NULL, NULL, NULL, 0), 0,
Fix flakiness in test_decay_ticker. Fix the test_decay_ticker test to carefully control slab creation/destruction such that the decay backlog reliably reaches zero. Use an isolated arena so that no extraneous allocation can confuse the situation. Speed up time during the latter part of the test so that the entire decay time can expire in a reasonable amount of wall time.
2017-03-03 15:32:42 +08:00
"Unexpected mallctlbymib() failure");
}
static uint64_t
Implement two-phase decay-based purging. Split decay-based purging into two phases, the first of which uses lazy purging to convert dirty pages to "muzzy", and the second of which uses forced purging, decommit, or unmapping to convert pages to clean or destroy them altogether. Not all operating systems support lazy purging, yet the application may provide extent hooks that implement lazy purging, so care must be taken to dynamically omit the first phase when necessary. The mallctl interfaces change as follows: - opt.decay_time --> opt.{dirty,muzzy}_decay_time - arena.<i>.decay_time --> arena.<i>.{dirty,muzzy}_decay_time - arenas.decay_time --> arenas.{dirty,muzzy}_decay_time - stats.arenas.<i>.pdirty --> stats.arenas.<i>.p{dirty,muzzy} - stats.arenas.<i>.{npurge,nmadvise,purged} --> stats.arenas.<i>.{dirty,muzzy}_{npurge,nmadvise,purged} This resolves #521.
2017-03-09 14:42:57 +08:00
get_arena_npurge_impl(const char *mibname, unsigned arena_ind) {
Fix flakiness in test_decay_ticker. Fix the test_decay_ticker test to carefully control slab creation/destruction such that the decay backlog reliably reaches zero. Use an isolated arena so that no extraneous allocation can confuse the situation. Speed up time during the latter part of the test so that the entire decay time can expire in a reasonable amount of wall time.
2017-03-03 15:32:42 +08:00
size_t mib[4];
size_t miblen = sizeof(mib)/sizeof(size_t);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_d_eq(mallctlnametomib(mibname, mib, &miblen), 0,
Fix flakiness in test_decay_ticker. Fix the test_decay_ticker test to carefully control slab creation/destruction such that the decay backlog reliably reaches zero. Use an isolated arena so that no extraneous allocation can confuse the situation. Speed up time during the latter part of the test so that the entire decay time can expire in a reasonable amount of wall time.
2017-03-03 15:32:42 +08:00
"Unexpected mallctlnametomib() failure");
mib[2] = (size_t)arena_ind;
uint64_t npurge = 0;
size_t sz = sizeof(npurge);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_d_eq(mallctlbymib(mib, miblen, (void *)&npurge, &sz, NULL, 0),
Fix flakiness in test_decay_ticker. Fix the test_decay_ticker test to carefully control slab creation/destruction such that the decay backlog reliably reaches zero. Use an isolated arena so that no extraneous allocation can confuse the situation. Speed up time during the latter part of the test so that the entire decay time can expire in a reasonable amount of wall time.
2017-03-03 15:32:42 +08:00
config_stats ? 0 : ENOENT, "Unexpected mallctlbymib() failure");
return npurge;
}
Implement two-phase decay-based purging. Split decay-based purging into two phases, the first of which uses lazy purging to convert dirty pages to "muzzy", and the second of which uses forced purging, decommit, or unmapping to convert pages to clean or destroy them altogether. Not all operating systems support lazy purging, yet the application may provide extent hooks that implement lazy purging, so care must be taken to dynamically omit the first phase when necessary. The mallctl interfaces change as follows: - opt.decay_time --> opt.{dirty,muzzy}_decay_time - arena.<i>.decay_time --> arena.<i>.{dirty,muzzy}_decay_time - arenas.decay_time --> arenas.{dirty,muzzy}_decay_time - stats.arenas.<i>.pdirty --> stats.arenas.<i>.p{dirty,muzzy} - stats.arenas.<i>.{npurge,nmadvise,purged} --> stats.arenas.<i>.{dirty,muzzy}_{npurge,nmadvise,purged} This resolves #521.
2017-03-09 14:42:57 +08:00
static uint64_t
get_arena_dirty_npurge(unsigned arena_ind) {
do_epoch();
return get_arena_npurge_impl("stats.arenas.0.dirty_npurge", arena_ind);
}
Eagerly purge oversized merged extents. This change improves memory usage slightly, at virtually no CPU cost.
2019-03-13 06:02:41 +08:00
static uint64_t
get_arena_dirty_purged(unsigned arena_ind) {
do_epoch();
return get_arena_npurge_impl("stats.arenas.0.dirty_purged", arena_ind);
}
Implement two-phase decay-based purging. Split decay-based purging into two phases, the first of which uses lazy purging to convert dirty pages to "muzzy", and the second of which uses forced purging, decommit, or unmapping to convert pages to clean or destroy them altogether. Not all operating systems support lazy purging, yet the application may provide extent hooks that implement lazy purging, so care must be taken to dynamically omit the first phase when necessary. The mallctl interfaces change as follows: - opt.decay_time --> opt.{dirty,muzzy}_decay_time - arena.<i>.decay_time --> arena.<i>.{dirty,muzzy}_decay_time - arenas.decay_time --> arenas.{dirty,muzzy}_decay_time - stats.arenas.<i>.pdirty --> stats.arenas.<i>.p{dirty,muzzy} - stats.arenas.<i>.{npurge,nmadvise,purged} --> stats.arenas.<i>.{dirty,muzzy}_{npurge,nmadvise,purged} This resolves #521.
2017-03-09 14:42:57 +08:00
static uint64_t
get_arena_muzzy_npurge(unsigned arena_ind) {
do_epoch();
return get_arena_npurge_impl("stats.arenas.0.muzzy_npurge", arena_ind);
}
static uint64_t
get_arena_npurge(unsigned arena_ind) {
do_epoch();
return get_arena_npurge_impl("stats.arenas.0.dirty_npurge", arena_ind) +
get_arena_npurge_impl("stats.arenas.0.muzzy_npurge", arena_ind);
}
Fix flakiness in test_decay_ticker. Fix the test_decay_ticker test to carefully control slab creation/destruction such that the decay backlog reliably reaches zero. Use an isolated arena so that no extraneous allocation can confuse the situation. Speed up time during the latter part of the test so that the entire decay time can expire in a reasonable amount of wall time.
2017-03-03 15:32:42 +08:00
static size_t
get_arena_pdirty(unsigned arena_ind) {
do_epoch();
size_t mib[4];
size_t miblen = sizeof(mib)/sizeof(size_t);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_d_eq(mallctlnametomib("stats.arenas.0.pdirty", mib, &miblen), 0,
Fix flakiness in test_decay_ticker. Fix the test_decay_ticker test to carefully control slab creation/destruction such that the decay backlog reliably reaches zero. Use an isolated arena so that no extraneous allocation can confuse the situation. Speed up time during the latter part of the test so that the entire decay time can expire in a reasonable amount of wall time.
2017-03-03 15:32:42 +08:00
"Unexpected mallctlnametomib() failure");
mib[2] = (size_t)arena_ind;
size_t pdirty;
size_t sz = sizeof(pdirty);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_d_eq(mallctlbymib(mib, miblen, (void *)&pdirty, &sz, NULL, 0), 0,
Fix flakiness in test_decay_ticker. Fix the test_decay_ticker test to carefully control slab creation/destruction such that the decay backlog reliably reaches zero. Use an isolated arena so that no extraneous allocation can confuse the situation. Speed up time during the latter part of the test so that the entire decay time can expire in a reasonable amount of wall time.
2017-03-03 15:32:42 +08:00
"Unexpected mallctlbymib() failure");
return pdirty;
}
Implement two-phase decay-based purging. Split decay-based purging into two phases, the first of which uses lazy purging to convert dirty pages to "muzzy", and the second of which uses forced purging, decommit, or unmapping to convert pages to clean or destroy them altogether. Not all operating systems support lazy purging, yet the application may provide extent hooks that implement lazy purging, so care must be taken to dynamically omit the first phase when necessary. The mallctl interfaces change as follows: - opt.decay_time --> opt.{dirty,muzzy}_decay_time - arena.<i>.decay_time --> arena.<i>.{dirty,muzzy}_decay_time - arenas.decay_time --> arenas.{dirty,muzzy}_decay_time - stats.arenas.<i>.pdirty --> stats.arenas.<i>.p{dirty,muzzy} - stats.arenas.<i>.{npurge,nmadvise,purged} --> stats.arenas.<i>.{dirty,muzzy}_{npurge,nmadvise,purged} This resolves #521.
2017-03-09 14:42:57 +08:00
static size_t
get_arena_pmuzzy(unsigned arena_ind) {
do_epoch();
size_t mib[4];
size_t miblen = sizeof(mib)/sizeof(size_t);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_d_eq(mallctlnametomib("stats.arenas.0.pmuzzy", mib, &miblen), 0,
Implement two-phase decay-based purging. Split decay-based purging into two phases, the first of which uses lazy purging to convert dirty pages to "muzzy", and the second of which uses forced purging, decommit, or unmapping to convert pages to clean or destroy them altogether. Not all operating systems support lazy purging, yet the application may provide extent hooks that implement lazy purging, so care must be taken to dynamically omit the first phase when necessary. The mallctl interfaces change as follows: - opt.decay_time --> opt.{dirty,muzzy}_decay_time - arena.<i>.decay_time --> arena.<i>.{dirty,muzzy}_decay_time - arenas.decay_time --> arenas.{dirty,muzzy}_decay_time - stats.arenas.<i>.pdirty --> stats.arenas.<i>.p{dirty,muzzy} - stats.arenas.<i>.{npurge,nmadvise,purged} --> stats.arenas.<i>.{dirty,muzzy}_{npurge,nmadvise,purged} This resolves #521.
2017-03-09 14:42:57 +08:00
"Unexpected mallctlnametomib() failure");
mib[2] = (size_t)arena_ind;
size_t pmuzzy;
size_t sz = sizeof(pmuzzy);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_d_eq(mallctlbymib(mib, miblen, (void *)&pmuzzy, &sz, NULL, 0), 0,
Implement two-phase decay-based purging. Split decay-based purging into two phases, the first of which uses lazy purging to convert dirty pages to "muzzy", and the second of which uses forced purging, decommit, or unmapping to convert pages to clean or destroy them altogether. Not all operating systems support lazy purging, yet the application may provide extent hooks that implement lazy purging, so care must be taken to dynamically omit the first phase when necessary. The mallctl interfaces change as follows: - opt.decay_time --> opt.{dirty,muzzy}_decay_time - arena.<i>.decay_time --> arena.<i>.{dirty,muzzy}_decay_time - arenas.decay_time --> arenas.{dirty,muzzy}_decay_time - stats.arenas.<i>.pdirty --> stats.arenas.<i>.p{dirty,muzzy} - stats.arenas.<i>.{npurge,nmadvise,purged} --> stats.arenas.<i>.{dirty,muzzy}_{npurge,nmadvise,purged} This resolves #521.
2017-03-09 14:42:57 +08:00
"Unexpected mallctlbymib() failure");
return pmuzzy;
}
Fix flakiness in test_decay_ticker. Fix the test_decay_ticker test to carefully control slab creation/destruction such that the decay backlog reliably reaches zero. Use an isolated arena so that no extraneous allocation can confuse the situation. Speed up time during the latter part of the test so that the entire decay time can expire in a reasonable amount of wall time.
2017-03-03 15:32:42 +08:00
static void *
do_mallocx(size_t size, int flags) {
void *p = mallocx(size, flags);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_ptr_not_null(p, "Unexpected mallocx() failure");
Fix flakiness in test_decay_ticker. Fix the test_decay_ticker test to carefully control slab creation/destruction such that the decay backlog reliably reaches zero. Use an isolated arena so that no extraneous allocation can confuse the situation. Speed up time during the latter part of the test so that the entire decay time can expire in a reasonable amount of wall time.
2017-03-03 15:32:42 +08:00
return p;
}
static void
generate_dirty(unsigned arena_ind, size_t size) {
int flags = MALLOCX_ARENA(arena_ind) | MALLOCX_TCACHE_NONE;
void *p = do_mallocx(size, flags);
dallocx(p, flags);
}
Update brace style. Add braces around single-line blocks, and remove line breaks before function-opening braces. This resolves #537.
2017-01-16 08:56:30 +08:00
TEST_BEGIN(test_decay_ticks) {
Implementing opt.background_thread. Added opt.background_thread to enable background threads, which handles purging currently. When enabled, decay ticks will not trigger purging (which will be left to the background threads). We limit the max number of threads to NCPUs. When percpu arena is enabled, set CPU affinity for the background threads as well. The sleep interval of background threads is dynamic and determined by computing number of pages to purge in the future (based on backlog).
2017-03-18 03:42:33 +08:00
test_skip_if(check_background_thread_enabled());
HPA: Tie components into a PAI implementation.
2020-08-15 04:36:41 +08:00
test_skip_if(opt_hpa);
Implementing opt.background_thread. Added opt.background_thread to enable background threads, which handles purging currently. When enabled, decay ticks will not trigger purging (which will be left to the background threads). We limit the max number of threads to NCPUs. When percpu arena is enabled, set CPU affinity for the background threads as well. The sleep interval of background threads is dynamic and determined by computing number of pages to purge in the future (based on backlog).
2017-03-18 03:42:33 +08:00
Use ticker_geom_t for arena tcache decay.
2021-02-01 04:50:55 +08:00
ticker_geom_t *decay_ticker;
Store associated arena in tcache. This fixes tcache_flush for manual tcaches, which wasn't able to find the correct arena it associated with. Also changed the decay test to cover this case (by using manually created arenas).
2017-03-07 04:51:41 +08:00
unsigned tick0, tick1, arena_ind;
Rename huge to large.
2016-06-01 05:50:21 +08:00
size_t sz, large0;
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
void *p;
sz = sizeof(size_t);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_d_eq(mallctl("arenas.lextent.0.size", (void *)&large0, &sz, NULL,
Uniformly cast mallctl[bymib]() oldp/newp arguments to (void *). This avoids warnings in some cases, and is otherwise generally good hygiene.
2016-10-28 12:31:25 +08:00
0), 0, "Unexpected mallctl failure");
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
Store associated arena in tcache. This fixes tcache_flush for manual tcaches, which wasn't able to find the correct arena it associated with. Also changed the decay test to cover this case (by using manually created arenas).
2017-03-07 04:51:41 +08:00
/* Set up a manually managed arena for test. */
Implement two-phase decay-based purging. Split decay-based purging into two phases, the first of which uses lazy purging to convert dirty pages to "muzzy", and the second of which uses forced purging, decommit, or unmapping to convert pages to clean or destroy them altogether. Not all operating systems support lazy purging, yet the application may provide extent hooks that implement lazy purging, so care must be taken to dynamically omit the first phase when necessary. The mallctl interfaces change as follows: - opt.decay_time --> opt.{dirty,muzzy}_decay_time - arena.<i>.decay_time --> arena.<i>.{dirty,muzzy}_decay_time - arenas.decay_time --> arenas.{dirty,muzzy}_decay_time - stats.arenas.<i>.pdirty --> stats.arenas.<i>.p{dirty,muzzy} - stats.arenas.<i>.{npurge,nmadvise,purged} --> stats.arenas.<i>.{dirty,muzzy}_{npurge,nmadvise,purged} This resolves #521.
2017-03-09 14:42:57 +08:00
arena_ind = do_arena_create(0, 0);
Store associated arena in tcache. This fixes tcache_flush for manual tcaches, which wasn't able to find the correct arena it associated with. Also changed the decay test to cover this case (by using manually created arenas).
2017-03-07 04:51:41 +08:00
/* Migrate to the new arena, and get the ticker. */
unsigned old_arena_ind;
size_t sz_arena_ind = sizeof(old_arena_ind);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_d_eq(mallctl("thread.arena", (void *)&old_arena_ind,
Remove --disable-tcache. Simplify configuration by removing the --disable-tcache option, but replace the testing for that configuration with --with-malloc-conf=tcache:false. Fix the thread.arena and thread.tcache.flush mallctls to work correctly if tcache is disabled. This partially resolves #580.
2017-04-21 08:21:37 +08:00
&sz_arena_ind, (void *)&arena_ind, sizeof(arena_ind)), 0,
"Unexpected mallctl() failure");
Use ticker_geom_t for arena tcache decay.
2021-02-01 04:50:55 +08:00
decay_ticker = tsd_arena_decay_tickerp_get(tsd_fetch());
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_ptr_not_null(decay_ticker,
Store associated arena in tcache. This fixes tcache_flush for manual tcaches, which wasn't able to find the correct arena it associated with. Also changed the decay test to cover this case (by using manually created arenas).
2017-03-07 04:51:41 +08:00
"Unexpected failure getting decay ticker");
Increase test coverage in test_decay_ticks.
2016-02-21 01:02:49 +08:00
/*
Rename huge to large.
2016-06-01 05:50:21 +08:00
* Test the standard APIs using a large size class, since we can't
Use huge size class infrastructure for large size classes.
2016-05-28 15:17:28 +08:00
* control tcache interactions for small size classes (except by
* completely disabling tcache for the entire test program).
Increase test coverage in test_decay_ticks.
2016-02-21 01:02:49 +08:00
*/
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
/* malloc(). */
Use ticker_geom_t for arena tcache decay.
2021-02-01 04:50:55 +08:00
tick0 = ticker_geom_read(decay_ticker);
Rename huge to large.
2016-06-01 05:50:21 +08:00
p = malloc(large0);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_ptr_not_null(p, "Unexpected malloc() failure");
Use ticker_geom_t for arena tcache decay.
2021-02-01 04:50:55 +08:00
tick1 = ticker_geom_read(decay_ticker);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_u32_ne(tick1, tick0, "Expected ticker to tick during malloc()");
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
/* free(). */
Use ticker_geom_t for arena tcache decay.
2021-02-01 04:50:55 +08:00
tick0 = ticker_geom_read(decay_ticker);
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
free(p);
Use ticker_geom_t for arena tcache decay.
2021-02-01 04:50:55 +08:00
tick1 = ticker_geom_read(decay_ticker);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_u32_ne(tick1, tick0, "Expected ticker to tick during free()");
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
/* calloc(). */
Use ticker_geom_t for arena tcache decay.
2021-02-01 04:50:55 +08:00
tick0 = ticker_geom_read(decay_ticker);
Rename huge to large.
2016-06-01 05:50:21 +08:00
p = calloc(1, large0);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_ptr_not_null(p, "Unexpected calloc() failure");
Use ticker_geom_t for arena tcache decay.
2021-02-01 04:50:55 +08:00
tick1 = ticker_geom_read(decay_ticker);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_u32_ne(tick1, tick0, "Expected ticker to tick during calloc()");
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
free(p);
/* posix_memalign(). */
Use ticker_geom_t for arena tcache decay.
2021-02-01 04:50:55 +08:00
tick0 = ticker_geom_read(decay_ticker);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_d_eq(posix_memalign(&p, sizeof(size_t), large0), 0,
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
"Unexpected posix_memalign() failure");
Use ticker_geom_t for arena tcache decay.
2021-02-01 04:50:55 +08:00
tick1 = ticker_geom_read(decay_ticker);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_u32_ne(tick1, tick0,
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
"Expected ticker to tick during posix_memalign()");
free(p);
/* aligned_alloc(). */
Use ticker_geom_t for arena tcache decay.
2021-02-01 04:50:55 +08:00
tick0 = ticker_geom_read(decay_ticker);
Rename huge to large.
2016-06-01 05:50:21 +08:00
p = aligned_alloc(sizeof(size_t), large0);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_ptr_not_null(p, "Unexpected aligned_alloc() failure");
Use ticker_geom_t for arena tcache decay.
2021-02-01 04:50:55 +08:00
tick1 = ticker_geom_read(decay_ticker);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_u32_ne(tick1, tick0,
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
"Expected ticker to tick during aligned_alloc()");
free(p);
/* realloc(). */
/* Allocate. */
Use ticker_geom_t for arena tcache decay.
2021-02-01 04:50:55 +08:00
tick0 = ticker_geom_read(decay_ticker);
Rename huge to large.
2016-06-01 05:50:21 +08:00
p = realloc(NULL, large0);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_ptr_not_null(p, "Unexpected realloc() failure");
Use ticker_geom_t for arena tcache decay.
2021-02-01 04:50:55 +08:00
tick1 = ticker_geom_read(decay_ticker);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_u32_ne(tick1, tick0, "Expected ticker to tick during realloc()");
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
/* Reallocate. */
Use ticker_geom_t for arena tcache decay.
2021-02-01 04:50:55 +08:00
tick0 = ticker_geom_read(decay_ticker);
Rename huge to large.
2016-06-01 05:50:21 +08:00
p = realloc(p, large0);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_ptr_not_null(p, "Unexpected realloc() failure");
Use ticker_geom_t for arena tcache decay.
2021-02-01 04:50:55 +08:00
tick1 = ticker_geom_read(decay_ticker);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_u32_ne(tick1, tick0, "Expected ticker to tick during realloc()");
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
/* Deallocate. */
Use ticker_geom_t for arena tcache decay.
2021-02-01 04:50:55 +08:00
tick0 = ticker_geom_read(decay_ticker);
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
realloc(p, 0);
Use ticker_geom_t for arena tcache decay.
2021-02-01 04:50:55 +08:00
tick1 = ticker_geom_read(decay_ticker);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_u32_ne(tick1, tick0, "Expected ticker to tick during realloc()");
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
Increase test coverage in test_decay_ticks.
2016-02-21 01:02:49 +08:00
/*
Rename huge to large.
2016-06-01 05:50:21 +08:00
* Test the *allocx() APIs using large and small size classes, with
Use huge size class infrastructure for large size classes.
2016-05-28 15:17:28 +08:00
* tcache explicitly disabled.
Increase test coverage in test_decay_ticks.
2016-02-21 01:02:49 +08:00
*/
{
unsigned i;
Use huge size class infrastructure for large size classes.
2016-05-28 15:17:28 +08:00
size_t allocx_sizes[2];
Rename huge to large.
2016-06-01 05:50:21 +08:00
allocx_sizes[0] = large0;
Use huge size class infrastructure for large size classes.
2016-05-28 15:17:28 +08:00
allocx_sizes[1] = 1;
Increase test coverage in test_decay_ticks.
2016-02-21 01:02:49 +08:00
for (i = 0; i < sizeof(allocx_sizes) / sizeof(size_t); i++) {
sz = allocx_sizes[i];
/* mallocx(). */
Use ticker_geom_t for arena tcache decay.
2021-02-01 04:50:55 +08:00
tick0 = ticker_geom_read(decay_ticker);
Increase test coverage in test_decay_ticks.
2016-02-21 01:02:49 +08:00
p = mallocx(sz, MALLOCX_TCACHE_NONE);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_ptr_not_null(p, "Unexpected mallocx() failure");
Use ticker_geom_t for arena tcache decay.
2021-02-01 04:50:55 +08:00
tick1 = ticker_geom_read(decay_ticker);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_u32_ne(tick1, tick0,
Increase test coverage in test_decay_ticks.
2016-02-21 01:02:49 +08:00
"Expected ticker to tick during mallocx() (sz=%zu)",
sz);
/* rallocx(). */
Use ticker_geom_t for arena tcache decay.
2021-02-01 04:50:55 +08:00
tick0 = ticker_geom_read(decay_ticker);
Increase test coverage in test_decay_ticks.
2016-02-21 01:02:49 +08:00
p = rallocx(p, sz, MALLOCX_TCACHE_NONE);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_ptr_not_null(p, "Unexpected rallocx() failure");
Use ticker_geom_t for arena tcache decay.
2021-02-01 04:50:55 +08:00
tick1 = ticker_geom_read(decay_ticker);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_u32_ne(tick1, tick0,
Increase test coverage in test_decay_ticks.
2016-02-21 01:02:49 +08:00
"Expected ticker to tick during rallocx() (sz=%zu)",
sz);
/* xallocx(). */
Use ticker_geom_t for arena tcache decay.
2021-02-01 04:50:55 +08:00
tick0 = ticker_geom_read(decay_ticker);
Increase test coverage in test_decay_ticks.
2016-02-21 01:02:49 +08:00
xallocx(p, sz, 0, MALLOCX_TCACHE_NONE);
Use ticker_geom_t for arena tcache decay.
2021-02-01 04:50:55 +08:00
tick1 = ticker_geom_read(decay_ticker);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_u32_ne(tick1, tick0,
Increase test coverage in test_decay_ticks.
2016-02-21 01:02:49 +08:00
"Expected ticker to tick during xallocx() (sz=%zu)",
sz);
/* dallocx(). */
Use ticker_geom_t for arena tcache decay.
2021-02-01 04:50:55 +08:00
tick0 = ticker_geom_read(decay_ticker);
Increase test coverage in test_decay_ticks.
2016-02-21 01:02:49 +08:00
dallocx(p, MALLOCX_TCACHE_NONE);
Use ticker_geom_t for arena tcache decay.
2021-02-01 04:50:55 +08:00
tick1 = ticker_geom_read(decay_ticker);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_u32_ne(tick1, tick0,
Increase test coverage in test_decay_ticks.
2016-02-21 01:02:49 +08:00
"Expected ticker to tick during dallocx() (sz=%zu)",
sz);
/* sdallocx(). */
p = mallocx(sz, MALLOCX_TCACHE_NONE);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_ptr_not_null(p, "Unexpected mallocx() failure");
Use ticker_geom_t for arena tcache decay.
2021-02-01 04:50:55 +08:00
tick0 = ticker_geom_read(decay_ticker);
Increase test coverage in test_decay_ticks.
2016-02-21 01:02:49 +08:00
sdallocx(p, sz, MALLOCX_TCACHE_NONE);
Use ticker_geom_t for arena tcache decay.
2021-02-01 04:50:55 +08:00
tick1 = ticker_geom_read(decay_ticker);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_u32_ne(tick1, tick0,
Increase test coverage in test_decay_ticks.
2016-02-21 01:02:49 +08:00
"Expected ticker to tick during sdallocx() "
"(sz=%zu)", sz);
}
}
/*
Rename huge to large.
2016-06-01 05:50:21 +08:00
* Test tcache fill/flush interactions for large and small size classes,
Increase test coverage in test_decay_ticks.
2016-02-21 01:02:49 +08:00
* using an explicit tcache.
*/
Remove --disable-tcache. Simplify configuration by removing the --disable-tcache option, but replace the testing for that configuration with --with-malloc-conf=tcache:false. Fix the thread.arena and thread.tcache.flush mallctls to work correctly if tcache is disabled. This partially resolves #580.
2017-04-21 08:21:37 +08:00
unsigned tcache_ind, i;
size_t tcache_sizes[2];
tcache_sizes[0] = large0;
tcache_sizes[1] = 1;
size_t tcache_max, sz_tcache_max;
sz_tcache_max = sizeof(tcache_max);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_d_eq(mallctl("arenas.tcache_max", (void *)&tcache_max,
Remove --disable-tcache. Simplify configuration by removing the --disable-tcache option, but replace the testing for that configuration with --with-malloc-conf=tcache:false. Fix the thread.arena and thread.tcache.flush mallctls to work correctly if tcache is disabled. This partially resolves #580.
2017-04-21 08:21:37 +08:00
&sz_tcache_max, NULL, 0), 0, "Unexpected mallctl() failure");
sz = sizeof(unsigned);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_d_eq(mallctl("tcache.create", (void *)&tcache_ind, &sz,
Remove --disable-tcache. Simplify configuration by removing the --disable-tcache option, but replace the testing for that configuration with --with-malloc-conf=tcache:false. Fix the thread.arena and thread.tcache.flush mallctls to work correctly if tcache is disabled. This partially resolves #580.
2017-04-21 08:21:37 +08:00
NULL, 0), 0, "Unexpected mallctl failure");
for (i = 0; i < sizeof(tcache_sizes) / sizeof(size_t); i++) {
sz = tcache_sizes[i];
/* tcache fill. */
Use ticker_geom_t for arena tcache decay.
2021-02-01 04:50:55 +08:00
tick0 = ticker_geom_read(decay_ticker);
Remove --disable-tcache. Simplify configuration by removing the --disable-tcache option, but replace the testing for that configuration with --with-malloc-conf=tcache:false. Fix the thread.arena and thread.tcache.flush mallctls to work correctly if tcache is disabled. This partially resolves #580.
2017-04-21 08:21:37 +08:00
p = mallocx(sz, MALLOCX_TCACHE(tcache_ind));
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_ptr_not_null(p, "Unexpected mallocx() failure");
Use ticker_geom_t for arena tcache decay.
2021-02-01 04:50:55 +08:00
tick1 = ticker_geom_read(decay_ticker);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_u32_ne(tick1, tick0,
Remove --disable-tcache. Simplify configuration by removing the --disable-tcache option, but replace the testing for that configuration with --with-malloc-conf=tcache:false. Fix the thread.arena and thread.tcache.flush mallctls to work correctly if tcache is disabled. This partially resolves #580.
2017-04-21 08:21:37 +08:00
"Expected ticker to tick during tcache fill "
"(sz=%zu)", sz);
/* tcache flush. */
dallocx(p, MALLOCX_TCACHE(tcache_ind));
Use ticker_geom_t for arena tcache decay.
2021-02-01 04:50:55 +08:00
tick0 = ticker_geom_read(decay_ticker);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_d_eq(mallctl("tcache.flush", NULL, NULL,
Remove --disable-tcache. Simplify configuration by removing the --disable-tcache option, but replace the testing for that configuration with --with-malloc-conf=tcache:false. Fix the thread.arena and thread.tcache.flush mallctls to work correctly if tcache is disabled. This partially resolves #580.
2017-04-21 08:21:37 +08:00
(void *)&tcache_ind, sizeof(unsigned)), 0,
"Unexpected mallctl failure");
Use ticker_geom_t for arena tcache decay.
2021-02-01 04:50:55 +08:00
tick1 = ticker_geom_read(decay_ticker);
Remove --disable-tcache. Simplify configuration by removing the --disable-tcache option, but replace the testing for that configuration with --with-malloc-conf=tcache:false. Fix the thread.arena and thread.tcache.flush mallctls to work correctly if tcache is disabled. This partially resolves #580.
2017-04-21 08:21:37 +08:00
/* Will only tick if it's in tcache. */
Use ticker_geom_t for arena tcache decay.
2021-02-01 04:50:55 +08:00
expect_u32_ne(tick1, tick0,
"Expected ticker to tick during tcache flush (sz=%zu)", sz);
Increase test coverage in test_decay_ticks.
2016-02-21 01:02:49 +08:00
}
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
}
TEST_END
Implement two-phase decay-based purging. Split decay-based purging into two phases, the first of which uses lazy purging to convert dirty pages to "muzzy", and the second of which uses forced purging, decommit, or unmapping to convert pages to clean or destroy them altogether. Not all operating systems support lazy purging, yet the application may provide extent hooks that implement lazy purging, so care must be taken to dynamically omit the first phase when necessary. The mallctl interfaces change as follows: - opt.decay_time --> opt.{dirty,muzzy}_decay_time - arena.<i>.decay_time --> arena.<i>.{dirty,muzzy}_decay_time - arenas.decay_time --> arenas.{dirty,muzzy}_decay_time - stats.arenas.<i>.pdirty --> stats.arenas.<i>.p{dirty,muzzy} - stats.arenas.<i>.{npurge,nmadvise,purged} --> stats.arenas.<i>.{dirty,muzzy}_{npurge,nmadvise,purged} This resolves #521.
2017-03-09 14:42:57 +08:00
static void
decay_ticker_helper(unsigned arena_ind, int flags, bool dirty, ssize_t dt,
uint64_t dirty_npurge0, uint64_t muzzy_npurge0, bool terminate_asap) {
Fix flakiness in test_decay_ticker. Fix the test_decay_ticker test to carefully control slab creation/destruction such that the decay backlog reliably reaches zero. Use an isolated arena so that no extraneous allocation can confuse the situation. Speed up time during the latter part of the test so that the entire decay time can expire in a reasonable amount of wall time.
2017-03-03 15:32:42 +08:00
#define NINTERVALS 101
Refactor *decay_time into *decay_ms. Support millisecond resolution for decay times. Among other use cases this makes it possible to specify a short initial dirty-->muzzy decay phase, followed by a longer muzzy-->clean decay phase. This resolves #812.
2017-05-18 01:47:00 +08:00
nstime_t time, update_interval, decay_ms, deadline;
Implement two-phase decay-based purging. Split decay-based purging into two phases, the first of which uses lazy purging to convert dirty pages to "muzzy", and the second of which uses forced purging, decommit, or unmapping to convert pages to clean or destroy them altogether. Not all operating systems support lazy purging, yet the application may provide extent hooks that implement lazy purging, so care must be taken to dynamically omit the first phase when necessary. The mallctl interfaces change as follows: - opt.decay_time --> opt.{dirty,muzzy}_decay_time - arena.<i>.decay_time --> arena.<i>.{dirty,muzzy}_decay_time - arenas.decay_time --> arenas.{dirty,muzzy}_decay_time - stats.arenas.<i>.pdirty --> stats.arenas.<i>.p{dirty,muzzy} - stats.arenas.<i>.{npurge,nmadvise,purged} --> stats.arenas.<i>.{dirty,muzzy}_{npurge,nmadvise,purged} This resolves #521.
2017-03-09 14:42:57 +08:00
Initialization utilities for nstime
2019-12-17 04:41:06 +08:00
nstime_init_update(&time);
Implement two-phase decay-based purging. Split decay-based purging into two phases, the first of which uses lazy purging to convert dirty pages to "muzzy", and the second of which uses forced purging, decommit, or unmapping to convert pages to clean or destroy them altogether. Not all operating systems support lazy purging, yet the application may provide extent hooks that implement lazy purging, so care must be taken to dynamically omit the first phase when necessary. The mallctl interfaces change as follows: - opt.decay_time --> opt.{dirty,muzzy}_decay_time - arena.<i>.decay_time --> arena.<i>.{dirty,muzzy}_decay_time - arenas.decay_time --> arenas.{dirty,muzzy}_decay_time - stats.arenas.<i>.pdirty --> stats.arenas.<i>.p{dirty,muzzy} - stats.arenas.<i>.{npurge,nmadvise,purged} --> stats.arenas.<i>.{dirty,muzzy}_{npurge,nmadvise,purged} This resolves #521.
2017-03-09 14:42:57 +08:00
Refactor *decay_time into *decay_ms. Support millisecond resolution for decay times. Among other use cases this makes it possible to specify a short initial dirty-->muzzy decay phase, followed by a longer muzzy-->clean decay phase. This resolves #812.
2017-05-18 01:47:00 +08:00
nstime_init2(&decay_ms, dt, 0);
Implement two-phase decay-based purging. Split decay-based purging into two phases, the first of which uses lazy purging to convert dirty pages to "muzzy", and the second of which uses forced purging, decommit, or unmapping to convert pages to clean or destroy them altogether. Not all operating systems support lazy purging, yet the application may provide extent hooks that implement lazy purging, so care must be taken to dynamically omit the first phase when necessary. The mallctl interfaces change as follows: - opt.decay_time --> opt.{dirty,muzzy}_decay_time - arena.<i>.decay_time --> arena.<i>.{dirty,muzzy}_decay_time - arenas.decay_time --> arenas.{dirty,muzzy}_decay_time - stats.arenas.<i>.pdirty --> stats.arenas.<i>.p{dirty,muzzy} - stats.arenas.<i>.{npurge,nmadvise,purged} --> stats.arenas.<i>.{dirty,muzzy}_{npurge,nmadvise,purged} This resolves #521.
2017-03-09 14:42:57 +08:00
nstime_copy(&deadline, &time);
Refactor *decay_time into *decay_ms. Support millisecond resolution for decay times. Among other use cases this makes it possible to specify a short initial dirty-->muzzy decay phase, followed by a longer muzzy-->clean decay phase. This resolves #812.
2017-05-18 01:47:00 +08:00
nstime_add(&deadline, &decay_ms);
Implement two-phase decay-based purging. Split decay-based purging into two phases, the first of which uses lazy purging to convert dirty pages to "muzzy", and the second of which uses forced purging, decommit, or unmapping to convert pages to clean or destroy them altogether. Not all operating systems support lazy purging, yet the application may provide extent hooks that implement lazy purging, so care must be taken to dynamically omit the first phase when necessary. The mallctl interfaces change as follows: - opt.decay_time --> opt.{dirty,muzzy}_decay_time - arena.<i>.decay_time --> arena.<i>.{dirty,muzzy}_decay_time - arenas.decay_time --> arenas.{dirty,muzzy}_decay_time - stats.arenas.<i>.pdirty --> stats.arenas.<i>.p{dirty,muzzy} - stats.arenas.<i>.{npurge,nmadvise,purged} --> stats.arenas.<i>.{dirty,muzzy}_{npurge,nmadvise,purged} This resolves #521.
2017-03-09 14:42:57 +08:00
nstime_init2(&update_interval, dt, 0);
nstime_idivide(&update_interval, NINTERVALS);
/*
* Keep q's slab from being deallocated during the looping below. If a
* cached slab were to repeatedly come and go during looping, it could
* prevent the decay backlog ever becoming empty.
*/
void *p = do_mallocx(1, flags);
uint64_t dirty_npurge1, muzzy_npurge1;
do {
Use ticker_geom_t for arena tcache decay.
2021-02-01 04:50:55 +08:00
for (unsigned i = 0; i < ARENA_DECAY_NTICKS_PER_UPDATE / 2;
Implement two-phase decay-based purging. Split decay-based purging into two phases, the first of which uses lazy purging to convert dirty pages to "muzzy", and the second of which uses forced purging, decommit, or unmapping to convert pages to clean or destroy them altogether. Not all operating systems support lazy purging, yet the application may provide extent hooks that implement lazy purging, so care must be taken to dynamically omit the first phase when necessary. The mallctl interfaces change as follows: - opt.decay_time --> opt.{dirty,muzzy}_decay_time - arena.<i>.decay_time --> arena.<i>.{dirty,muzzy}_decay_time - arenas.decay_time --> arenas.{dirty,muzzy}_decay_time - stats.arenas.<i>.pdirty --> stats.arenas.<i>.p{dirty,muzzy} - stats.arenas.<i>.{npurge,nmadvise,purged} --> stats.arenas.<i>.{dirty,muzzy}_{npurge,nmadvise,purged} This resolves #521.
2017-03-09 14:42:57 +08:00
i++) {
void *q = do_mallocx(1, flags);
dallocx(q, flags);
}
dirty_npurge1 = get_arena_dirty_npurge(arena_ind);
muzzy_npurge1 = get_arena_muzzy_npurge(arena_ind);
nstime_add(&time_mock, &update_interval);
nstime_update(&time);
} while (nstime_compare(&time, &deadline) <= 0 && ((dirty_npurge1 ==
dirty_npurge0 && muzzy_npurge1 == muzzy_npurge0) ||
!terminate_asap));
dallocx(p, flags);
if (config_stats) {
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_u64_gt(dirty_npurge1 + muzzy_npurge1, dirty_npurge0 +
Implement two-phase decay-based purging. Split decay-based purging into two phases, the first of which uses lazy purging to convert dirty pages to "muzzy", and the second of which uses forced purging, decommit, or unmapping to convert pages to clean or destroy them altogether. Not all operating systems support lazy purging, yet the application may provide extent hooks that implement lazy purging, so care must be taken to dynamically omit the first phase when necessary. The mallctl interfaces change as follows: - opt.decay_time --> opt.{dirty,muzzy}_decay_time - arena.<i>.decay_time --> arena.<i>.{dirty,muzzy}_decay_time - arenas.decay_time --> arenas.{dirty,muzzy}_decay_time - stats.arenas.<i>.pdirty --> stats.arenas.<i>.p{dirty,muzzy} - stats.arenas.<i>.{npurge,nmadvise,purged} --> stats.arenas.<i>.{dirty,muzzy}_{npurge,nmadvise,purged} This resolves #521.
2017-03-09 14:42:57 +08:00
muzzy_npurge0, "Expected purging to occur");
}
#undef NINTERVALS
}
TEST_BEGIN(test_decay_ticker) {
Implementing opt.background_thread. Added opt.background_thread to enable background threads, which handles purging currently. When enabled, decay ticks will not trigger purging (which will be left to the background threads). We limit the max number of threads to NCPUs. When percpu arena is enabled, set CPU affinity for the background threads as well. The sleep interval of background threads is dynamic and determined by computing number of pages to purge in the future (based on backlog).
2017-03-18 03:42:33 +08:00
test_skip_if(check_background_thread_enabled());
HPA: Tie components into a PAI implementation.
2020-08-15 04:36:41 +08:00
test_skip_if(opt_hpa);
Implement two-phase decay-based purging. Split decay-based purging into two phases, the first of which uses lazy purging to convert dirty pages to "muzzy", and the second of which uses forced purging, decommit, or unmapping to convert pages to clean or destroy them altogether. Not all operating systems support lazy purging, yet the application may provide extent hooks that implement lazy purging, so care must be taken to dynamically omit the first phase when necessary. The mallctl interfaces change as follows: - opt.decay_time --> opt.{dirty,muzzy}_decay_time - arena.<i>.decay_time --> arena.<i>.{dirty,muzzy}_decay_time - arenas.decay_time --> arenas.{dirty,muzzy}_decay_time - stats.arenas.<i>.pdirty --> stats.arenas.<i>.p{dirty,muzzy} - stats.arenas.<i>.{npurge,nmadvise,purged} --> stats.arenas.<i>.{dirty,muzzy}_{npurge,nmadvise,purged} This resolves #521.
2017-03-09 14:42:57 +08:00
#define NPS 2048
Refactor *decay_time into *decay_ms. Support millisecond resolution for decay times. Among other use cases this makes it possible to specify a short initial dirty-->muzzy decay phase, followed by a longer muzzy-->clean decay phase. This resolves #812.
2017-05-18 01:47:00 +08:00
ssize_t ddt = opt_dirty_decay_ms;
ssize_t mdt = opt_muzzy_decay_ms;
Implement two-phase decay-based purging. Split decay-based purging into two phases, the first of which uses lazy purging to convert dirty pages to "muzzy", and the second of which uses forced purging, decommit, or unmapping to convert pages to clean or destroy them altogether. Not all operating systems support lazy purging, yet the application may provide extent hooks that implement lazy purging, so care must be taken to dynamically omit the first phase when necessary. The mallctl interfaces change as follows: - opt.decay_time --> opt.{dirty,muzzy}_decay_time - arena.<i>.decay_time --> arena.<i>.{dirty,muzzy}_decay_time - arenas.decay_time --> arenas.{dirty,muzzy}_decay_time - stats.arenas.<i>.pdirty --> stats.arenas.<i>.p{dirty,muzzy} - stats.arenas.<i>.{npurge,nmadvise,purged} --> stats.arenas.<i>.{dirty,muzzy}_{npurge,nmadvise,purged} This resolves #521.
2017-03-09 14:42:57 +08:00
unsigned arena_ind = do_arena_create(ddt, mdt);
Fix flakiness in test_decay_ticker. Fix the test_decay_ticker test to carefully control slab creation/destruction such that the decay backlog reliably reaches zero. Use an isolated arena so that no extraneous allocation can confuse the situation. Speed up time during the latter part of the test so that the entire decay time can expire in a reasonable amount of wall time.
2017-03-03 15:32:42 +08:00
int flags = (MALLOCX_ARENA(arena_ind) | MALLOCX_TCACHE_NONE);
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
void *ps[NPS];
/*
Implement two-phase decay-based purging. Split decay-based purging into two phases, the first of which uses lazy purging to convert dirty pages to "muzzy", and the second of which uses forced purging, decommit, or unmapping to convert pages to clean or destroy them altogether. Not all operating systems support lazy purging, yet the application may provide extent hooks that implement lazy purging, so care must be taken to dynamically omit the first phase when necessary. The mallctl interfaces change as follows: - opt.decay_time --> opt.{dirty,muzzy}_decay_time - arena.<i>.decay_time --> arena.<i>.{dirty,muzzy}_decay_time - arenas.decay_time --> arenas.{dirty,muzzy}_decay_time - stats.arenas.<i>.pdirty --> stats.arenas.<i>.p{dirty,muzzy} - stats.arenas.<i>.{npurge,nmadvise,purged} --> stats.arenas.<i>.{dirty,muzzy}_{npurge,nmadvise,purged} This resolves #521.
2017-03-09 14:42:57 +08:00
* Allocate a bunch of large objects, pause the clock, deallocate every
* other object (to fragment virtual memory), restore the clock, then
* [md]allocx() in a tight loop while advancing time rapidly to verify
* the ticker triggers purging.
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
*/
Allow opt.tcache_max to accept small size classes. Previously all the small size classes were cached. However this has downsides -- particularly when page size is greater than 4K (e.g. iOS), which will result in much higher SMALL_MAXCLASS. This change allows tcache_max to be set to lower values, to better control resources taken by tcache.
2020-10-22 10:47:57 +08:00
size_t large;
Remove --disable-tcache. Simplify configuration by removing the --disable-tcache option, but replace the testing for that configuration with --with-malloc-conf=tcache:false. Fix the thread.arena and thread.tcache.flush mallctls to work correctly if tcache is disabled. This partially resolves #580.
2017-04-21 08:21:37 +08:00
size_t sz = sizeof(size_t);
Allow opt.tcache_max to accept small size classes. Previously all the small size classes were cached. However this has downsides -- particularly when page size is greater than 4K (e.g. iOS), which will result in much higher SMALL_MAXCLASS. This change allows tcache_max to be set to lower values, to better control resources taken by tcache.
2020-10-22 10:47:57 +08:00
expect_d_eq(mallctl("arenas.lextent.0.size", (void *)&large, &sz, NULL,
Remove --disable-tcache. Simplify configuration by removing the --disable-tcache option, but replace the testing for that configuration with --with-malloc-conf=tcache:false. Fix the thread.arena and thread.tcache.flush mallctls to work correctly if tcache is disabled. This partially resolves #580.
2017-04-21 08:21:37 +08:00
0), 0, "Unexpected mallctl failure");
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
Fix flakiness in test_decay_ticker. Fix the test_decay_ticker test to carefully control slab creation/destruction such that the decay backlog reliably reaches zero. Use an isolated arena so that no extraneous allocation can confuse the situation. Speed up time during the latter part of the test so that the entire decay time can expire in a reasonable amount of wall time.
2017-03-03 15:32:42 +08:00
do_purge(arena_ind);
Implement two-phase decay-based purging. Split decay-based purging into two phases, the first of which uses lazy purging to convert dirty pages to "muzzy", and the second of which uses forced purging, decommit, or unmapping to convert pages to clean or destroy them altogether. Not all operating systems support lazy purging, yet the application may provide extent hooks that implement lazy purging, so care must be taken to dynamically omit the first phase when necessary. The mallctl interfaces change as follows: - opt.decay_time --> opt.{dirty,muzzy}_decay_time - arena.<i>.decay_time --> arena.<i>.{dirty,muzzy}_decay_time - arenas.decay_time --> arenas.{dirty,muzzy}_decay_time - stats.arenas.<i>.pdirty --> stats.arenas.<i>.p{dirty,muzzy} - stats.arenas.<i>.{npurge,nmadvise,purged} --> stats.arenas.<i>.{dirty,muzzy}_{npurge,nmadvise,purged} This resolves #521.
2017-03-09 14:42:57 +08:00
uint64_t dirty_npurge0 = get_arena_dirty_npurge(arena_ind);
uint64_t muzzy_npurge0 = get_arena_muzzy_npurge(arena_ind);
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
Fix flakiness in test_decay_ticker. Fix the test_decay_ticker test to carefully control slab creation/destruction such that the decay backlog reliably reaches zero. Use an isolated arena so that no extraneous allocation can confuse the situation. Speed up time during the latter part of the test so that the entire decay time can expire in a reasonable amount of wall time.
2017-03-03 15:32:42 +08:00
for (unsigned i = 0; i < NPS; i++) {
ps[i] = do_mallocx(large, flags);
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
}
nupdates_mock = 0;
Initialization utilities for nstime
2019-12-17 04:41:06 +08:00
nstime_init_update(&time_mock);
Fix decay tests to all adapt to nstime_monotonic().
2016-10-12 06:28:43 +08:00
monotonic_mock = true;
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
Fix decay tests to all adapt to nstime_monotonic().
2016-10-12 06:28:43 +08:00
nstime_monotonic_orig = nstime_monotonic;
Refactor time_* into nstime_*. Use a single uint64_t in nstime_t to store nanoseconds rather than using struct timespec. This reduces fragility around conversions between long and uint64_t, especially missing casts that only cause problems on 32-bit platforms.
2016-02-22 03:25:02 +08:00
nstime_update_orig = nstime_update;
Fix decay tests to all adapt to nstime_monotonic().
2016-10-12 06:28:43 +08:00
nstime_monotonic = nstime_monotonic_mock;
Refactor time_* into nstime_*. Use a single uint64_t in nstime_t to store nanoseconds rather than using struct timespec. This reduces fragility around conversions between long and uint64_t, especially missing casts that only cause problems on 32-bit platforms.
2016-02-22 03:25:02 +08:00
nstime_update = nstime_update_mock;
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
Implement two-phase decay-based purging. Split decay-based purging into two phases, the first of which uses lazy purging to convert dirty pages to "muzzy", and the second of which uses forced purging, decommit, or unmapping to convert pages to clean or destroy them altogether. Not all operating systems support lazy purging, yet the application may provide extent hooks that implement lazy purging, so care must be taken to dynamically omit the first phase when necessary. The mallctl interfaces change as follows: - opt.decay_time --> opt.{dirty,muzzy}_decay_time - arena.<i>.decay_time --> arena.<i>.{dirty,muzzy}_decay_time - arenas.decay_time --> arenas.{dirty,muzzy}_decay_time - stats.arenas.<i>.pdirty --> stats.arenas.<i>.p{dirty,muzzy} - stats.arenas.<i>.{npurge,nmadvise,purged} --> stats.arenas.<i>.{dirty,muzzy}_{npurge,nmadvise,purged} This resolves #521.
2017-03-09 14:42:57 +08:00
for (unsigned i = 0; i < NPS; i += 2) {
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
dallocx(ps[i], flags);
Fix flakiness in test_decay_ticker. Fix the test_decay_ticker test to carefully control slab creation/destruction such that the decay backlog reliably reaches zero. Use an isolated arena so that no extraneous allocation can confuse the situation. Speed up time during the latter part of the test so that the entire decay time can expire in a reasonable amount of wall time.
2017-03-03 15:32:42 +08:00
unsigned nupdates0 = nupdates_mock;
do_decay(arena_ind);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_u_gt(nupdates_mock, nupdates0,
Refactor time_* into nstime_*. Use a single uint64_t in nstime_t to store nanoseconds rather than using struct timespec. This reduces fragility around conversions between long and uint64_t, especially missing casts that only cause problems on 32-bit platforms.
2016-02-22 03:25:02 +08:00
"Expected nstime_update() to be called");
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
}
Implement two-phase decay-based purging. Split decay-based purging into two phases, the first of which uses lazy purging to convert dirty pages to "muzzy", and the second of which uses forced purging, decommit, or unmapping to convert pages to clean or destroy them altogether. Not all operating systems support lazy purging, yet the application may provide extent hooks that implement lazy purging, so care must be taken to dynamically omit the first phase when necessary. The mallctl interfaces change as follows: - opt.decay_time --> opt.{dirty,muzzy}_decay_time - arena.<i>.decay_time --> arena.<i>.{dirty,muzzy}_decay_time - arenas.decay_time --> arenas.{dirty,muzzy}_decay_time - stats.arenas.<i>.pdirty --> stats.arenas.<i>.p{dirty,muzzy} - stats.arenas.<i>.{npurge,nmadvise,purged} --> stats.arenas.<i>.{dirty,muzzy}_{npurge,nmadvise,purged} This resolves #521.
2017-03-09 14:42:57 +08:00
decay_ticker_helper(arena_ind, flags, true, ddt, dirty_npurge0,
muzzy_npurge0, true);
decay_ticker_helper(arena_ind, flags, false, ddt+mdt, dirty_npurge0,
muzzy_npurge0, false);
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
Implement two-phase decay-based purging. Split decay-based purging into two phases, the first of which uses lazy purging to convert dirty pages to "muzzy", and the second of which uses forced purging, decommit, or unmapping to convert pages to clean or destroy them altogether. Not all operating systems support lazy purging, yet the application may provide extent hooks that implement lazy purging, so care must be taken to dynamically omit the first phase when necessary. The mallctl interfaces change as follows: - opt.decay_time --> opt.{dirty,muzzy}_decay_time - arena.<i>.decay_time --> arena.<i>.{dirty,muzzy}_decay_time - arenas.decay_time --> arenas.{dirty,muzzy}_decay_time - stats.arenas.<i>.pdirty --> stats.arenas.<i>.p{dirty,muzzy} - stats.arenas.<i>.{npurge,nmadvise,purged} --> stats.arenas.<i>.{dirty,muzzy}_{npurge,nmadvise,purged} This resolves #521.
2017-03-09 14:42:57 +08:00
do_arena_destroy(arena_ind);
Fix flakiness in test_decay_ticker. Fix the test_decay_ticker test to carefully control slab creation/destruction such that the decay backlog reliably reaches zero. Use an isolated arena so that no extraneous allocation can confuse the situation. Speed up time during the latter part of the test so that the entire decay time can expire in a reasonable amount of wall time.
2017-03-03 15:32:42 +08:00
nstime_monotonic = nstime_monotonic_orig;
nstime_update = nstime_update_orig;
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
#undef NPS
}
TEST_END
Update brace style. Add braces around single-line blocks, and remove line breaks before function-opening braces. This resolves #537.
2017-01-16 08:56:30 +08:00
TEST_BEGIN(test_decay_nonmonotonic) {
Implementing opt.background_thread. Added opt.background_thread to enable background threads, which handles purging currently. When enabled, decay ticks will not trigger purging (which will be left to the background threads). We limit the max number of threads to NCPUs. When percpu arena is enabled, set CPU affinity for the background threads as well. The sleep interval of background threads is dynamic and determined by computing number of pages to purge in the future (based on backlog).
2017-03-18 03:42:33 +08:00
test_skip_if(check_background_thread_enabled());
HPA: Tie components into a PAI implementation.
2020-08-15 04:36:41 +08:00
test_skip_if(opt_hpa);
Replace tabs following #define with spaces. This resolves #564.
2017-01-20 13:41:41 +08:00
#define NPS (SMOOTHSTEP_NSTEPS + 1)
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
int flags = (MALLOCX_ARENA(0) | MALLOCX_TCACHE_NONE);
void *ps[NPS];
Fix decay tests for --disable-stats case.
2016-02-28 12:38:29 +08:00
uint64_t npurge0 = 0;
uint64_t npurge1 = 0;
Rename huge to large.
2016-06-01 05:50:21 +08:00
size_t sz, large0;
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
unsigned i, nupdates0;
sz = sizeof(size_t);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_d_eq(mallctl("arenas.lextent.0.size", (void *)&large0, &sz, NULL,
Uniformly cast mallctl[bymib]() oldp/newp arguments to (void *). This avoids warnings in some cases, and is otherwise generally good hygiene.
2016-10-28 12:31:25 +08:00
0), 0, "Unexpected mallctl failure");
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_d_eq(mallctl("arena.0.purge", NULL, NULL, NULL, 0), 0,
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
"Unexpected mallctl failure");
Fix flakiness in test_decay_ticker. Fix the test_decay_ticker test to carefully control slab creation/destruction such that the decay backlog reliably reaches zero. Use an isolated arena so that no extraneous allocation can confuse the situation. Speed up time during the latter part of the test so that the entire decay time can expire in a reasonable amount of wall time.
2017-03-03 15:32:42 +08:00
do_epoch();
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
sz = sizeof(uint64_t);
Implement two-phase decay-based purging. Split decay-based purging into two phases, the first of which uses lazy purging to convert dirty pages to "muzzy", and the second of which uses forced purging, decommit, or unmapping to convert pages to clean or destroy them altogether. Not all operating systems support lazy purging, yet the application may provide extent hooks that implement lazy purging, so care must be taken to dynamically omit the first phase when necessary. The mallctl interfaces change as follows: - opt.decay_time --> opt.{dirty,muzzy}_decay_time - arena.<i>.decay_time --> arena.<i>.{dirty,muzzy}_decay_time - arenas.decay_time --> arenas.{dirty,muzzy}_decay_time - stats.arenas.<i>.pdirty --> stats.arenas.<i>.p{dirty,muzzy} - stats.arenas.<i>.{npurge,nmadvise,purged} --> stats.arenas.<i>.{dirty,muzzy}_{npurge,nmadvise,purged} This resolves #521.
2017-03-09 14:42:57 +08:00
npurge0 = get_arena_npurge(0);
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
nupdates_mock = 0;
Initialization utilities for nstime
2019-12-17 04:41:06 +08:00
nstime_init_update(&time_mock);
Fix decay tests to all adapt to nstime_monotonic().
2016-10-12 06:28:43 +08:00
monotonic_mock = false;
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
Do not advance decay epoch when time goes backwards. Instead, move the epoch backward in time. Additionally, add nstime_monotonic() and use it in debug builds to assert that time only goes backward if nstime_update() is using a non-monotonic time source.
2016-10-11 13:15:10 +08:00
nstime_monotonic_orig = nstime_monotonic;
Refactor time_* into nstime_*. Use a single uint64_t in nstime_t to store nanoseconds rather than using struct timespec. This reduces fragility around conversions between long and uint64_t, especially missing casts that only cause problems on 32-bit platforms.
2016-02-22 03:25:02 +08:00
nstime_update_orig = nstime_update;
Do not advance decay epoch when time goes backwards. Instead, move the epoch backward in time. Additionally, add nstime_monotonic() and use it in debug builds to assert that time only goes backward if nstime_update() is using a non-monotonic time source.
2016-10-11 13:15:10 +08:00
nstime_monotonic = nstime_monotonic_mock;
Refactor time_* into nstime_*. Use a single uint64_t in nstime_t to store nanoseconds rather than using struct timespec. This reduces fragility around conversions between long and uint64_t, especially missing casts that only cause problems on 32-bit platforms.
2016-02-22 03:25:02 +08:00
nstime_update = nstime_update_mock;
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
for (i = 0; i < NPS; i++) {
Rename huge to large.
2016-06-01 05:50:21 +08:00
ps[i] = mallocx(large0, flags);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_ptr_not_null(ps[i], "Unexpected mallocx() failure");
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
}
for (i = 0; i < NPS; i++) {
dallocx(ps[i], flags);
nupdates0 = nupdates_mock;
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_d_eq(mallctl("arena.0.decay", NULL, NULL, NULL, 0), 0,
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
"Unexpected arena.0.decay failure");
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_u_gt(nupdates_mock, nupdates0,
Refactor time_* into nstime_*. Use a single uint64_t in nstime_t to store nanoseconds rather than using struct timespec. This reduces fragility around conversions between long and uint64_t, especially missing casts that only cause problems on 32-bit platforms.
2016-02-22 03:25:02 +08:00
"Expected nstime_update() to be called");
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
}
Fix flakiness in test_decay_ticker. Fix the test_decay_ticker test to carefully control slab creation/destruction such that the decay backlog reliably reaches zero. Use an isolated arena so that no extraneous allocation can confuse the situation. Speed up time during the latter part of the test so that the entire decay time can expire in a reasonable amount of wall time.
2017-03-03 15:32:42 +08:00
do_epoch();
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
sz = sizeof(uint64_t);
Implement two-phase decay-based purging. Split decay-based purging into two phases, the first of which uses lazy purging to convert dirty pages to "muzzy", and the second of which uses forced purging, decommit, or unmapping to convert pages to clean or destroy them altogether. Not all operating systems support lazy purging, yet the application may provide extent hooks that implement lazy purging, so care must be taken to dynamically omit the first phase when necessary. The mallctl interfaces change as follows: - opt.decay_time --> opt.{dirty,muzzy}_decay_time - arena.<i>.decay_time --> arena.<i>.{dirty,muzzy}_decay_time - arenas.decay_time --> arenas.{dirty,muzzy}_decay_time - stats.arenas.<i>.pdirty --> stats.arenas.<i>.p{dirty,muzzy} - stats.arenas.<i>.{npurge,nmadvise,purged} --> stats.arenas.<i>.{dirty,muzzy}_{npurge,nmadvise,purged} This resolves #521.
2017-03-09 14:42:57 +08:00
npurge1 = get_arena_npurge(0);
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
Update brace style. Add braces around single-line blocks, and remove line breaks before function-opening braces. This resolves #537.
2017-01-16 08:56:30 +08:00
if (config_stats) {
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_u64_eq(npurge0, npurge1, "Unexpected purging occurred");
Update brace style. Add braces around single-line blocks, and remove line breaks before function-opening braces. This resolves #537.
2017-01-16 08:56:30 +08:00
}
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
Do not advance decay epoch when time goes backwards. Instead, move the epoch backward in time. Additionally, add nstime_monotonic() and use it in debug builds to assert that time only goes backward if nstime_update() is using a non-monotonic time source.
2016-10-11 13:15:10 +08:00
nstime_monotonic = nstime_monotonic_orig;
Refactor time_* into nstime_*. Use a single uint64_t in nstime_t to store nanoseconds rather than using struct timespec. This reduces fragility around conversions between long and uint64_t, especially missing casts that only cause problems on 32-bit platforms.
2016-02-22 03:25:02 +08:00
nstime_update = nstime_update_orig;
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
#undef NPS
}
TEST_END
Immediately purge cached extents if decay_time is 0. This fixes a regression caused by 54269dc0ed3e4d04b2539016431de3cfe8330719 (Remove obsolete arena_maybe_purge() call.), as well as providing a general fix. This resolves #665.
2017-03-02 07:25:48 +08:00
TEST_BEGIN(test_decay_now) {
Implementing opt.background_thread. Added opt.background_thread to enable background threads, which handles purging currently. When enabled, decay ticks will not trigger purging (which will be left to the background threads). We limit the max number of threads to NCPUs. When percpu arena is enabled, set CPU affinity for the background threads as well. The sleep interval of background threads is dynamic and determined by computing number of pages to purge in the future (based on backlog).
2017-03-18 03:42:33 +08:00
test_skip_if(check_background_thread_enabled());
HPA: Tie components into a PAI implementation.
2020-08-15 04:36:41 +08:00
test_skip_if(opt_hpa);
Implementing opt.background_thread. Added opt.background_thread to enable background threads, which handles purging currently. When enabled, decay ticks will not trigger purging (which will be left to the background threads). We limit the max number of threads to NCPUs. When percpu arena is enabled, set CPU affinity for the background threads as well. The sleep interval of background threads is dynamic and determined by computing number of pages to purge in the future (based on backlog).
2017-03-18 03:42:33 +08:00
Implement two-phase decay-based purging. Split decay-based purging into two phases, the first of which uses lazy purging to convert dirty pages to "muzzy", and the second of which uses forced purging, decommit, or unmapping to convert pages to clean or destroy them altogether. Not all operating systems support lazy purging, yet the application may provide extent hooks that implement lazy purging, so care must be taken to dynamically omit the first phase when necessary. The mallctl interfaces change as follows: - opt.decay_time --> opt.{dirty,muzzy}_decay_time - arena.<i>.decay_time --> arena.<i>.{dirty,muzzy}_decay_time - arenas.decay_time --> arenas.{dirty,muzzy}_decay_time - stats.arenas.<i>.pdirty --> stats.arenas.<i>.p{dirty,muzzy} - stats.arenas.<i>.{npurge,nmadvise,purged} --> stats.arenas.<i>.{dirty,muzzy}_{npurge,nmadvise,purged} This resolves #521.
2017-03-09 14:42:57 +08:00
unsigned arena_ind = do_arena_create(0, 0);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_zu_eq(get_arena_pdirty(arena_ind), 0, "Unexpected dirty pages");
expect_zu_eq(get_arena_pmuzzy(arena_ind), 0, "Unexpected muzzy pages");
Immediately purge cached extents if decay_time is 0. This fixes a regression caused by 54269dc0ed3e4d04b2539016431de3cfe8330719 (Remove obsolete arena_maybe_purge() call.), as well as providing a general fix. This resolves #665.
2017-03-02 07:25:48 +08:00
size_t sizes[] = {16, PAGE<<2, HUGEPAGE<<2};
Implement two-phase decay-based purging. Split decay-based purging into two phases, the first of which uses lazy purging to convert dirty pages to "muzzy", and the second of which uses forced purging, decommit, or unmapping to convert pages to clean or destroy them altogether. Not all operating systems support lazy purging, yet the application may provide extent hooks that implement lazy purging, so care must be taken to dynamically omit the first phase when necessary. The mallctl interfaces change as follows: - opt.decay_time --> opt.{dirty,muzzy}_decay_time - arena.<i>.decay_time --> arena.<i>.{dirty,muzzy}_decay_time - arenas.decay_time --> arenas.{dirty,muzzy}_decay_time - stats.arenas.<i>.pdirty --> stats.arenas.<i>.p{dirty,muzzy} - stats.arenas.<i>.{npurge,nmadvise,purged} --> stats.arenas.<i>.{dirty,muzzy}_{npurge,nmadvise,purged} This resolves #521.
2017-03-09 14:42:57 +08:00
/* Verify that dirty/muzzy pages never linger after deallocation. */
Immediately purge cached extents if decay_time is 0. This fixes a regression caused by 54269dc0ed3e4d04b2539016431de3cfe8330719 (Remove obsolete arena_maybe_purge() call.), as well as providing a general fix. This resolves #665.
2017-03-02 07:25:48 +08:00
for (unsigned i = 0; i < sizeof(sizes)/sizeof(size_t); i++) {
size_t size = sizes[i];
generate_dirty(arena_ind, size);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_zu_eq(get_arena_pdirty(arena_ind), 0,
Immediately purge cached extents if decay_time is 0. This fixes a regression caused by 54269dc0ed3e4d04b2539016431de3cfe8330719 (Remove obsolete arena_maybe_purge() call.), as well as providing a general fix. This resolves #665.
2017-03-02 07:25:48 +08:00
"Unexpected dirty pages");
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_zu_eq(get_arena_pmuzzy(arena_ind), 0,
Implement two-phase decay-based purging. Split decay-based purging into two phases, the first of which uses lazy purging to convert dirty pages to "muzzy", and the second of which uses forced purging, decommit, or unmapping to convert pages to clean or destroy them altogether. Not all operating systems support lazy purging, yet the application may provide extent hooks that implement lazy purging, so care must be taken to dynamically omit the first phase when necessary. The mallctl interfaces change as follows: - opt.decay_time --> opt.{dirty,muzzy}_decay_time - arena.<i>.decay_time --> arena.<i>.{dirty,muzzy}_decay_time - arenas.decay_time --> arenas.{dirty,muzzy}_decay_time - stats.arenas.<i>.pdirty --> stats.arenas.<i>.p{dirty,muzzy} - stats.arenas.<i>.{npurge,nmadvise,purged} --> stats.arenas.<i>.{dirty,muzzy}_{npurge,nmadvise,purged} This resolves #521.
2017-03-09 14:42:57 +08:00
"Unexpected muzzy pages");
Immediately purge cached extents if decay_time is 0. This fixes a regression caused by 54269dc0ed3e4d04b2539016431de3cfe8330719 (Remove obsolete arena_maybe_purge() call.), as well as providing a general fix. This resolves #665.
2017-03-02 07:25:48 +08:00
}
do_arena_destroy(arena_ind);
}
TEST_END
TEST_BEGIN(test_decay_never) {
Eagerly purge oversized merged extents. This change improves memory usage slightly, at virtually no CPU cost.
2019-03-13 06:02:41 +08:00
test_skip_if(check_background_thread_enabled() || !config_stats);
HPA: Tie components into a PAI implementation.
2020-08-15 04:36:41 +08:00
test_skip_if(opt_hpa);
Implementing opt.background_thread. Added opt.background_thread to enable background threads, which handles purging currently. When enabled, decay ticks will not trigger purging (which will be left to the background threads). We limit the max number of threads to NCPUs. When percpu arena is enabled, set CPU affinity for the background threads as well. The sleep interval of background threads is dynamic and determined by computing number of pages to purge in the future (based on backlog).
2017-03-18 03:42:33 +08:00
Implement two-phase decay-based purging. Split decay-based purging into two phases, the first of which uses lazy purging to convert dirty pages to "muzzy", and the second of which uses forced purging, decommit, or unmapping to convert pages to clean or destroy them altogether. Not all operating systems support lazy purging, yet the application may provide extent hooks that implement lazy purging, so care must be taken to dynamically omit the first phase when necessary. The mallctl interfaces change as follows: - opt.decay_time --> opt.{dirty,muzzy}_decay_time - arena.<i>.decay_time --> arena.<i>.{dirty,muzzy}_decay_time - arenas.decay_time --> arenas.{dirty,muzzy}_decay_time - stats.arenas.<i>.pdirty --> stats.arenas.<i>.p{dirty,muzzy} - stats.arenas.<i>.{npurge,nmadvise,purged} --> stats.arenas.<i>.{dirty,muzzy}_{npurge,nmadvise,purged} This resolves #521.
2017-03-09 14:42:57 +08:00
unsigned arena_ind = do_arena_create(-1, -1);
Immediately purge cached extents if decay_time is 0. This fixes a regression caused by 54269dc0ed3e4d04b2539016431de3cfe8330719 (Remove obsolete arena_maybe_purge() call.), as well as providing a general fix. This resolves #665.
2017-03-02 07:25:48 +08:00
int flags = MALLOCX_ARENA(arena_ind) | MALLOCX_TCACHE_NONE;
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_zu_eq(get_arena_pdirty(arena_ind), 0, "Unexpected dirty pages");
expect_zu_eq(get_arena_pmuzzy(arena_ind), 0, "Unexpected muzzy pages");
Immediately purge cached extents if decay_time is 0. This fixes a regression caused by 54269dc0ed3e4d04b2539016431de3cfe8330719 (Remove obsolete arena_maybe_purge() call.), as well as providing a general fix. This resolves #665.
2017-03-02 07:25:48 +08:00
size_t sizes[] = {16, PAGE<<2, HUGEPAGE<<2};
void *ptrs[sizeof(sizes)/sizeof(size_t)];
for (unsigned i = 0; i < sizeof(sizes)/sizeof(size_t); i++) {
ptrs[i] = do_mallocx(sizes[i], flags);
}
/* Verify that each deallocation generates additional dirty pages. */
size_t pdirty_prev = get_arena_pdirty(arena_ind);
Implement two-phase decay-based purging. Split decay-based purging into two phases, the first of which uses lazy purging to convert dirty pages to "muzzy", and the second of which uses forced purging, decommit, or unmapping to convert pages to clean or destroy them altogether. Not all operating systems support lazy purging, yet the application may provide extent hooks that implement lazy purging, so care must be taken to dynamically omit the first phase when necessary. The mallctl interfaces change as follows: - opt.decay_time --> opt.{dirty,muzzy}_decay_time - arena.<i>.decay_time --> arena.<i>.{dirty,muzzy}_decay_time - arenas.decay_time --> arenas.{dirty,muzzy}_decay_time - stats.arenas.<i>.pdirty --> stats.arenas.<i>.p{dirty,muzzy} - stats.arenas.<i>.{npurge,nmadvise,purged} --> stats.arenas.<i>.{dirty,muzzy}_{npurge,nmadvise,purged} This resolves #521.
2017-03-09 14:42:57 +08:00
size_t pmuzzy_prev = get_arena_pmuzzy(arena_ind);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_zu_eq(pdirty_prev, 0, "Unexpected dirty pages");
expect_zu_eq(pmuzzy_prev, 0, "Unexpected muzzy pages");
Immediately purge cached extents if decay_time is 0. This fixes a regression caused by 54269dc0ed3e4d04b2539016431de3cfe8330719 (Remove obsolete arena_maybe_purge() call.), as well as providing a general fix. This resolves #665.
2017-03-02 07:25:48 +08:00
for (unsigned i = 0; i < sizeof(sizes)/sizeof(size_t); i++) {
dallocx(ptrs[i], flags);
size_t pdirty = get_arena_pdirty(arena_ind);
Implement two-phase decay-based purging. Split decay-based purging into two phases, the first of which uses lazy purging to convert dirty pages to "muzzy", and the second of which uses forced purging, decommit, or unmapping to convert pages to clean or destroy them altogether. Not all operating systems support lazy purging, yet the application may provide extent hooks that implement lazy purging, so care must be taken to dynamically omit the first phase when necessary. The mallctl interfaces change as follows: - opt.decay_time --> opt.{dirty,muzzy}_decay_time - arena.<i>.decay_time --> arena.<i>.{dirty,muzzy}_decay_time - arenas.decay_time --> arenas.{dirty,muzzy}_decay_time - stats.arenas.<i>.pdirty --> stats.arenas.<i>.p{dirty,muzzy} - stats.arenas.<i>.{npurge,nmadvise,purged} --> stats.arenas.<i>.{dirty,muzzy}_{npurge,nmadvise,purged} This resolves #521.
2017-03-09 14:42:57 +08:00
size_t pmuzzy = get_arena_pmuzzy(arena_ind);
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_zu_gt(pdirty + (size_t)get_arena_dirty_purged(arena_ind),
Eagerly purge oversized merged extents. This change improves memory usage slightly, at virtually no CPU cost.
2019-03-13 06:02:41 +08:00
pdirty_prev, "Expected dirty pages to increase.");
Change assert_* to expect_* in tests ``` grep -Irl assert_ test/ | xargs sed -i \ 's/witness_assert/witness_do_not_replace/g'; grep -Irl assert_ test/ | xargs sed -i \ 's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g'; grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \ grep -v "^assert_"; # confirm no output grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g'; grep -Irl witness_do_not_replace test/ | xargs sed -i \ 's/witness_do_not_replace/witness_assert/g'; grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \ 's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g'; ```
2020-02-19 06:39:06 +08:00
expect_zu_eq(pmuzzy, 0, "Unexpected muzzy pages");
Immediately purge cached extents if decay_time is 0. This fixes a regression caused by 54269dc0ed3e4d04b2539016431de3cfe8330719 (Remove obsolete arena_maybe_purge() call.), as well as providing a general fix. This resolves #665.
2017-03-02 07:25:48 +08:00
pdirty_prev = pdirty;
}
do_arena_destroy(arena_ind);
}
TEST_END
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
int
Update brace style. Add braces around single-line blocks, and remove line breaks before function-opening braces. This resolves #537.
2017-01-16 08:56:30 +08:00
main(void) {
Remove extraneous parens around return arguments. This resolves #540.
2017-01-20 10:15:45 +08:00
return test(
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
test_decay_ticks,
test_decay_ticker,
Immediately purge cached extents if decay_time is 0. This fixes a regression caused by 54269dc0ed3e4d04b2539016431de3cfe8330719 (Remove obsolete arena_maybe_purge() call.), as well as providing a general fix. This resolves #665.
2017-03-02 07:25:48 +08:00
test_decay_nonmonotonic,
test_decay_now,
test_decay_never);
Implement decay-based unused dirty page purging. This is an alternative to the existing ratio-based unused dirty page purging, and is intended to eventually become the sole purging mechanism. Add mallctls: - opt.purge - opt.decay_time - arena.<i>.decay - arena.<i>.decay_time - arenas.decay_time - stats.arenas.<i>.decay_time This resolves #325.
2016-02-20 12:09:31 +08:00
}
Reference in New Issue Copy Permalink