project-base/server-skynet-source-3rd-jemalloc
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
3,006 Commits 2 Branches 0 Tags 13 MiB
734e72ce8f
Commit Graph

3006 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
David Goldblatt
d701a085c2 Fast path: allow low-water mark changes. 
This lets us put more allocations on an "almost as fast" path after a flush.
This results in around a 4% reduction in malloc cycles in prod workloads
(corresponding to about a 0.1% reduction in overall cycles).
 2020-03-12 11:54:19 -07:00
David Goldblatt
397da03865 Cache bin: rewrite to track more state. 
With this, we track all of the empty, full, and low water states together.  This
simplifies a lot of the tracking logic, since we now don't need the
cache_bin_info_t for state queries (except for some debugging).
 2020-03-12 11:54:19 -07:00
David Goldblatt
fef0b1ffe4 Cache bin: Remove last internals accesses. 2020-03-12 11:54:19 -07:00
David Goldblatt
0a2fcfac01 Tcache: Hold cache bin allocation explicitly. 2020-03-12 11:54:19 -07:00
David Goldblatt
d498a4bb08 Cache bin: Add an emptiness assertion. 2020-03-12 11:54:19 -07:00
David Goldblatt
6a7aa46ef7 Cache bin: Add a debug method for init checking. 2020-03-12 11:54:19 -07:00
David Goldblatt
370c1ea007 Cache bin: Write the unit test in terms of the API 
I.e. stop allowing the unit test to have secret access to implementation
internals.
 2020-03-12 11:54:19 -07:00
David Goldblatt
7f5ebd211c Cache bin: set low-water internally. 2020-03-12 11:54:19 -07:00
David Goldblatt
60113dfe3b Cache bin: Move in initialization code. 2020-03-12 11:54:19 -07:00
David Goldblatt
44529da852 Cache-bin: Make flush modifications internal 
I.e. the tcache code just calls a cache-bin function to finish flush (and move
pointers around, etc.).  It doesn't directly access the cache-bin's owned memory
any more.
 2020-03-12 11:54:19 -07:00
David Goldblatt
ff6acc6ed5 Cache bin: simplify names and argument ordering. 
We always start with the cache bin, then its info (if necessary).
 2020-03-12 11:54:19 -07:00
David Goldblatt
e1dcc557d6 Cache bin: Only take the relevant cache_bin_info_t 
Previously, we took an array of cache_bin_info_ts and an index, and dereferenced
ourselves.  But infos for other cache_bins aren't relevant to any particular
cache bin, so that should be the caller's job.
 2020-03-12 11:54:19 -07:00
David Goldblatt
1b00d808d7 cache_bin: Don't let arena see empty position. 2020-03-12 11:54:19 -07:00
David Goldblatt
d303f30796 cache_bin nflush -> n. 
We're going to use it on the fill pathway as well.
 2020-03-12 11:54:19 -07:00
David Goldblatt
74d36d78ef Cache bin: Make ncached_max a query on the info_t. 2020-03-12 11:54:19 -07:00
David Goldblatt
b66c0973cc cache_bin: Don't allow direct internals access. 2020-03-12 11:54:19 -07:00
David Goldblatt
da68f73296 Move percpu_arena_update. 
It's not really part of the API of the arena; it changes which arena we're using
that API on.
 2020-03-12 11:54:19 -07:00
David Goldblatt
909c501b07 Cache_bin: Shouldn't know about tcache. 
Instead, have it take the cache_bin_info_ts to use by pointer.  While we're
here, add a src file for the cache bin.
 2020-03-12 11:54:19 -07:00
David Goldblatt
79f1ee2fc0 Move junking out of arena/tcache code. 
This is debug only and we keep it off the fast path.  Moving it here simplifies
the internal logic.

This never tries to junk on regions that were shrunk via xallocx.  I think this
is fine for two reasons:
- The shrunk-with-xallocx case is rare.
- We don't always do that anyway before this diff (it depends on the opt
  settings and extent hooks in effect).
 2020-03-12 11:54:19 -07:00
David Goldblatt
b428dceeaf Config: Warn on void * pointer arithmetic. 
This is handy while developing, but not portable.
 2020-03-12 11:54:19 -07:00
David Goldblatt
22657a5e65 Extents: Silence the "potentially unused" warning. 2020-03-12 11:54:19 -07:00
Yinan Zhang
4a78c6d81b Correct thread event unit test 2020-03-10 09:31:55 -07:00
Yinan Zhang
305b1f6d96 Correction on geometric sampling 2020-03-04 13:55:21 -08:00
David T. Goldblatt
6c3491ad31 Tcache: Unify bin flush logic. 
The small and large pathways share most of their logic, even if some of the
individual operations are different.  We pull out the common logic into a
force-inlined function, and then specialize twice, once for each value of
"small".
 2020-02-25 10:21:03 -08:00
David T. Goldblatt
9f4fc27389 Ehooks: Fix a build warning. 
We wrote `return some_void_func()` in a function returning void, which is
confusing and triggers warnings on MSVC.
 2020-02-25 10:21:03 -08:00
Qi Wang
bc31041edb Cirrus-CI: test on new freebsd releases. 2020-02-23 20:43:38 -08:00
Yinan Zhang
51bd147422 Make use of assert_* in test/unit/thread_event.c 2020-02-19 16:03:16 -08:00
Yinan Zhang
9d2cc3b0fa Make use of assert_* in test/unit/prof_recent.c 2020-02-19 16:03:16 -08:00
Yinan Zhang
a88d22ea11 Make use of assert_* in test/unit/inspect.c 2020-02-19 16:03:16 -08:00
Yinan Zhang
0ceb31184d Make use of assert_* in test/unit/buf_writer.c 2020-02-19 16:03:16 -08:00
Yinan Zhang
fa61579382 Add assert_* functionality to tests 2020-02-19 16:03:16 -08:00
Yinan Zhang
21dfa4300d Change assert_* to expect_* in tests 
```
grep -Irl assert_ test/ | xargs sed -i \
    's/witness_assert/witness_do_not_replace/g';
grep -Irl assert_ test/ | xargs sed -i \
    's/malloc_mutex_assert_owner/malloc_mutex_do_not_replace_owner/g';

grep -Ir assert_ test/ | grep -o "[_a-zA-Z]*assert_[_a-zA-Z]*" | \
    grep -v "^assert_"; # confirm no output
grep -Irl assert_ test/ | xargs sed -i 's/assert_/expect_/g';

grep -Irl witness_do_not_replace test/ | xargs sed -i \
    's/witness_do_not_replace/witness_assert/g';
grep -Irl malloc_mutex_do_not_replace_owner test/ | xargs sed -i \
    's/malloc_mutex_do_not_replace_owner/malloc_mutex_assert_owner/g';
```
 2020-02-19 16:03:16 -08:00
David T. Goldblatt
162c2bcf31 Background thread: take base as a parameter. 2020-02-18 11:22:09 -08:00
David T. Goldblatt
29436fa056 Break prof and tcache knowledge of b0. 2020-02-18 11:22:09 -08:00
David T. Goldblatt
a0c1f4ac57 Rtree: take the base allocator as a parameter. 
This facilitates better testing by avoiding mixing of the "real" base with the
base used by the rtree under test.
 2020-02-18 11:22:09 -08:00
David T. Goldblatt
7013716aaa Emap: Take (and propagate) a zeroed parameter. 
Rtree needs this, and we should really treat them similarly.
 2020-02-18 11:22:09 -08:00
David T. Goldblatt
182192f83c Base: Pull into a single header. 2020-02-18 11:22:09 -08:00
David T. Goldblatt
34b7165fde Put szind_t, pszind_t in sz.h. 2020-02-18 11:22:09 -08:00
David Goldblatt
7e6c8a7286 Emap: Standardize naming. 
Namespace everything under emap_, always specify what it is we're looking up
(emap_lookup -> emap_edata_lookup), and use "ctx" over "info".
 2020-02-17 10:50:51 -08:00
David Goldblatt
ac50c1e44b Emap: Remove direct access to emap internals. 
In the process, we do a few local cleanups and optimizations.  In particular,
the size safety check on tcache flush no longer does a redundant load.
 2020-02-17 10:50:51 -08:00
David Goldblatt
06e42090f7 Make jemalloc.c use the emap interface. 
While we're here, we'll also clean up some style nits.
 2020-02-17 10:50:51 -08:00
David Goldblatt
f7d9c6c42d Emap: Move in alloc_ctx lookup functionality. 2020-02-17 10:50:51 -08:00
David Goldblatt
65a54d7714 Emap: Move in szind and slab modifications. 2020-02-17 10:50:51 -08:00
David Goldblatt
9b5d105fc3 Emap: Move in iealloc. 
This is logically scoped to the emap.
 2020-02-17 10:50:51 -08:00
David Goldblatt
1d449bd9a6 Emap: Internal rtree context setting. 
The only time sharing an rtree context saves across extent operations isn't a
no-op is when tsd is unavailable.  But this happens only in situations like
thread death or initialization, and we don't care about shaving off every
possible cycle in such scenarios.
 2020-02-17 10:50:51 -08:00
David Goldblatt
08eb1e6c31 Emap: Comments and cleanup 
Document some of the public interface, and hide the functions that are no longer
used outside of the emap module.
 2020-02-17 10:50:51 -08:00
David Goldblatt
231d1477e5 Rename emap_split_prepare_t -> emap_prepare_t. 
Both the split and merge functions use it.
 2020-02-17 10:50:51 -08:00
David Goldblatt
0586a56f39 Emap: Move in merge functionality. 2020-02-17 10:50:51 -08:00
David Goldblatt
040eac77cc Tell edatas their creation arena immediately. 
This avoids having to pass it in anywhere else.
 2020-02-17 10:50:51 -08:00
David Goldblatt
7c7b702064 Emap: Move over metadata splitting logic. 2020-02-17 10:50:51 -08:00