Abael He
56ddbea270
error: implicit declaration of function 'pthread_create_fptr_init' is invalid in C99
...
./autogen.sh \
&& ./configure --prefix=/usr/local --enable-static --enable-autogen --enable-xmalloc --with-static-libunwind=/usr/local/lib/libunwind.a --enable-lazy-lock --with-jemalloc-prefix='' \
&& make -j16
...
gcc -std=gnu11 -Werror=unknown-warning-option -Wall -Wextra -Wshorten-64-to-32 -Wsign-compare -Wundef -Wno-format-zero-length -Wpointer-arith -Wno-missing-braces -Wno-missing-field-initializers -pipe -g3 -Wimplicit-fallthrough -O3 -funroll-loops -fPIC -DPIC -c -D_REENTRANT -Iinclude -Iinclude -DJEMALLOC_NO_PRIVATE_NAMESPACE -o src/edata_cache.sym.o src/edata_cache.c
src/background_thread.c:768:6: error: implicit declaration of function 'pthread_create_fptr_init' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
pthread_create_fptr_init()) {
^
src/background_thread.c:768:6: note: did you mean 'pthread_create_wrapper_init'?
src/background_thread.c:34:1: note: 'pthread_create_wrapper_init' declared here
pthread_create_wrapper_init(void) {
^
1 error generated.
make: *** [src/background_thread.sym.o] Error 1
make: *** Waiting for unfinished jobs....
2022-09-07 11:56:41 -07:00
Guangli Dai
ce29b4c3d9
Refactor the remote / cross thread cache bin stats reading
...
Refactored cache_bin.h so that only one function is racy.
2022-09-06 19:41:19 -07:00
Guangli Dai
42daa1ac44
Add double free detection using slab bitmap for debug build
...
Add a sanity check for double free issue in the arena in case that the tcache has been flushed.
2022-09-06 12:54:21 -07:00
Ivan Zaitsev
36366f3c4c
Add double free detection in thread cache for debug build
...
Add new runtime option `debug_double_free_max_scan` that specifies the max
number of stack entries to scan in the cache bit when trying to detect the
double free bug (currently debug build only).
2022-08-04 16:58:22 -07:00
David CARLIER
adc70c0511
update travis
2022-07-19 13:23:08 -07:00
David CARLIER
4e12d21c8d
enabled percpu_arena settings on macOs.
...
follow-up on #2280
2022-07-19 13:23:08 -07:00
David Carlier
58478412be
OpenBSD build fix. still no cpu affinity.
...
- enabling pthread_get/pthread_set_name_np api.
- disabling per thread cpu affinity handling, unsupported on this platform.
2022-07-19 13:20:11 -07:00
Qi Wang
a1c7d9c046
Add the missing opt.cache_oblivious handling.
2022-07-14 22:41:27 -07:00
Jasmin Parent
41a859ef73
Remove duplicated words in documentation
2022-07-11 15:30:16 -07:00
Azat Khuzhin
cb578bbe01
Fix possible "nmalloc >= ndalloc" assertion
...
In arena_stats_merge() first nmalloc was read, and after ndalloc.
However with this order, it is possible for some thread to incement
ndalloc in between, and then nmalloc < ndalloc, and assertion will fail,
like again found by ClickHouse CI [1] (even after #2234 ).
[1]: https://github.com/ClickHouse/ClickHouse/issues/31531
Swap the order to avoid possible assertion.
Cc: @interwq
Follow-up for: #2234
2022-07-11 15:27:51 -07:00
David CARLIER
a9215bf18a
CI update FreeBSD version.
2022-06-28 11:48:23 -07:00
Alex Lapenkou
3713932836
Update building for Windows instructions
...
Explain how to build for Windows in INSTALL.md and remove another readme.txt in
an obscure location.
2022-06-14 14:04:48 -07:00
David Carlier
4fc5c4fbac
New configure option '--enable-pageid' for Linux
...
The option makes jemalloc use prctl with PR_SET_VMA to tag memory mappings with
"jemalloc_pg" or "jemalloc_pg_overcommit". This allows to easily identify
jemalloc's mappings in /proc/<pid>/maps. PR_SET_VMA is only available in Linux
5.17 and above.
2022-06-09 18:54:08 -07:00
Qi Wang
b950934916
Enable retain by default on macOS.
...
High number of mappings result in unusually high fork() cost on macOS. Retain
fixes the issue, at a small cost of extra VM space reserved.
2022-06-09 11:37:44 -07:00
David Carlier
df8f7d10af
Implement malloc_getcpu for amd64 and arm64 macOS
...
This enables per CPU arena on MacOS
2022-06-08 15:13:55 -07:00
Alex Lapenkou
df7ad8a9b6
Revert "Echo installed files via verbose 'install' command"
...
This reverts commit f15d8f3b41
. "install -v"
turned out to be not portable and not work on NetBSD.
2022-06-07 12:28:45 -07:00
barracuda156
70e3735f3a
jemalloc: fix PowerPC definitions in quantum.h
2022-05-26 10:51:10 -07:00
Alex Lapenkou
5b1f2cc5d7
Implement pvalloc replacement
...
Despite being an obsolete function, pvalloc is still present in GLIBC and should
work correctly when jemalloc replaces libc allocator.
2022-05-18 17:01:09 -07:00
Qi Wang
cd5aaf308a
Improve the failure message upon opt_experimental_infallible_new.
2022-05-17 16:07:40 -07:00
Yuriy Chernyshov
70d4102f48
Fix compiling edata.h with MSVC
...
At the time an attempt to compile jemalloc 5.3.0 with MSVC 2019 results in the followin error message:
> jemalloc/include/jemalloc/internal/edata.h:660: error C4576: a parenthesized type followed by an initializer list is a non-standard explicit type conversion syntax
2022-05-09 14:51:07 -07:00
Qi Wang
54eaed1d8b
Merge branch 'dev'
2022-05-06 11:28:25 -07:00
Qi Wang
304c919829
Update ChangeLog for 5.3.0.
2022-05-06 11:24:21 -07:00
Qi Wang
8cb814629a
Make the default option of zero realloc match the system allocator.
2022-05-05 17:11:18 -07:00
Qi Wang
66c889500a
Make test/unit/background_thread_enable more conservative.
...
To avoid resource exhaustion on 32-bit platforms.
2022-05-04 15:32:57 -07:00
Qi Wang
a7d73dd4c9
Update TUNING.md to include the new tcache_max option.
2022-05-04 10:59:40 -07:00
Qi Wang
254b011915
Small doc tweak of opt.trust_madvise.
...
Avoid quoted enabled and disabled because it's a bool type instead of char *.
2022-04-28 21:16:25 -07:00
Qi Wang
f5e840bbf0
Minor typo fix in doc.
2022-04-27 20:25:29 -07:00
Qi Wang
ceca07d2ca
Correct the name of stats.mutexes.prof_thds_data in doc.
2022-04-25 20:12:58 -07:00
Qi Wang
391bad4b95
Avoid abort() in test/integration/cpp/infallible_new_true.
...
Allow setting the safety check abort hook through mallctl, which avoids abort()
and core dumps.
2022-04-25 11:29:32 -07:00
cuishuang
9a242f16d9
fix some typos
...
Signed-off-by: cuishuang <imcusg@gmail.com>
2022-04-25 11:29:00 -07:00
Qi Wang
0e29ad4efa
Rename zero_realloc option "strict" to "alloc".
...
With realloc(ptr, 0) being UB per C23, the option name "strict" makes less sense
now. Rename to "alloc" which describes the behavior.
2022-04-20 10:27:25 -07:00
Qi Wang
5841b6dbe7
Update FreeBSD image to 12.3 for cirrus ci.
2022-04-19 15:29:30 -07:00
Qi Wang
ed5fc14b28
Use volatile to workaround buffer overflow false positives.
...
In test/integration/rallocx, full usable size is checked which may confuse
overflow detection.
2022-04-04 12:16:46 -07:00
Alex Lapenkou
25517b852e
Reoreder TravisCI jobs to optimize CI time
...
Sorting jobs by descending expected runtime helps to utilize concurrency
better.
2022-03-29 11:58:27 -07:00
Alex Lapenkou
8a49b62e78
Enable TravisCI for Windows
2022-03-29 11:58:27 -07:00
Alex Lapenkou
fdb6c10162
Add FreeBSD to TravisCI
...
Implement the generation of Travis jobs for FreeBSD. The generated jobs
replicate the existing CirrusCI config.
2022-03-29 11:58:27 -07:00
Alex Lapenkou
a93931537e
Do not disable SEC by default for 64k pages platforms
...
Default SEC max_alloc option value was 32k, disabling SEC for platforms with
lg-page=16. This change enables SEC for all platforms, making minimum max_alloc
value equal to PAGE.
2022-03-24 22:05:35 -07:00
Charles
eaaa368bab
Add comments and use meaningful vars in sz_psz2ind.
2022-03-24 16:56:59 -07:00
Alex Lapenkou
5bf03f8ce5
Implement PAGE_FLOOR macro
2022-03-22 17:45:55 -07:00
Alex Lapenkou
52631c90f6
Fix size class calculation for sec
...
Due to a bug in sec initialization, the number of cached size classes
was equal to 198. The bug caused the creation of more than a hundred of
unused bins, although it didn't affect the caching logic.
2022-03-22 17:45:55 -07:00
Qi Wang
7ae0f15c59
Add a default page size when cross-compile for Apple M1.
...
When cross-compile for M1 and no page size specified, use the default 16K and
skip detecting the page size (which is likely incorrect).
2022-03-21 14:30:48 -07:00
Alex Lapenkov
eb65d1b078
Fix FreeBSD system jemalloc TSD cleanup
...
Before this commit, in case FreeBSD libc jemalloc was overridden by another
jemalloc, proper thread shutdown callback was involved only for the overriding
jemalloc. A call to _malloc_thread_cleanup from libthr would be redirected to
user jemalloc, leaving data about dead threads hanging in system jemalloc. This
change tackles the issue in two ways. First, for current and old system
jemallocs, which we can not modify, the overriding jemalloc would locate and
invoke system cleanup routine. For upcoming jemalloc integrations, the cleanup
registering function will also be redirected to user jemalloc, which means that
system jemalloc's cleanup routine will be registered in user's jemalloc and a
single call to _malloc_thread_cleanup will be sufficient to invoke both
callbacks.
2022-03-02 10:10:27 -08:00
Azat Khuzhin
78b58379c8
Fix possible "nmalloc >= ndalloc" assertion.
...
It is possible that ndalloc will be updated before nmalloc, in
arena_large_ralloc_stats_update(), fix this by reorder those calls.
It was found by ClickHouse CI, that periodically hits this assertion [1].
[1]: https://github.com/ClickHouse/ClickHouse/issues/31531
That issue contains lots of examples, with core dump and some gdb output [2].
[2]: https://s3.amazonaws.com/clickhouse-test-reports/34951/96390a9263cb5af3d6e42a84988239c9ae87ce32/stress_test__debug__actions_.html
Here you can find binaries for that particular report [3] you need
clickhouse debug build [4].
[3]: https://s3.amazonaws.com/clickhouse-builds/34951/96390a9263cb5af3d6e42a84988239c9ae87ce32/clickhouse_build_check_(actions)/report.html
[4]: https://s3.amazonaws.com/clickhouse-builds/34951/96390a9263cb5af3d6e42a84988239c9ae87ce32/package_debug/clickhouse
Brief info from that report:
2 0x000000002ad6dbfe in arena_stats_merge (tsdn=0x7f2399abdd20, arena=0x7f241ce01080, nthreads=0x7f24e4360958, dss=0x7f24e4360960, dirty_decay_ms=0x7f24e4360968, muzzy_decay_ms=0x7f24e4360970, nactive=0x7f24e4360978, ndirty=0x7f24e43
e4360988, astats=0x7f24e4360998, bstats=0x7f24e4363310, lstats=0x7f24e4364990, estats=0x7f24e4366e50, hpastats=0x7f24e43693a0, secstats=0x7f24e436a020) at ../contrib/jemalloc/src/arena.c:138
ndalloc = 226
nflush = 0
curlextents = 0
nmalloc = 225
nrequests = 0
Here you can see that they differs only by 1.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-01 12:28:28 -08:00
Alex Lapenkou
ca709c3139
Fix failed assertion due to racy memory access
...
While calculating the number of stashed pointers, multiple variables
potentially modified by a concurrent thread were used for the
calculation. This led to some inconsistencies, correctly detected by
the assertions. The change eliminates some possible inconsistencies by
using unmodified variables and only once a concurrently modified one.
The assertions are omitted for the cases where we acknowledge potential
inconsistencies too.
2022-02-17 09:35:52 -08:00
Qi Wang
063d134aeb
Properly detect background thread support on Darwin.
...
When cross-compile, the host type / abi should be checked to determine
background thread compatibility.
2022-02-15 10:10:11 -08:00
Alex Lapenkou
a4e81221cc
Document 'make uninstall'
...
Update INSTALL.md, reflecting the addition of 'uninstall' target.
2022-01-31 14:55:00 -08:00
Qi Wang
20f9802e4f
Avoid overflow warnings in test/unit/safety_check.
2022-01-27 10:29:54 -08:00
Qi Wang
8c59c44ffa
Add a dependency checking step at the end of malloc_conf_init.
...
Currently only prof_leak_error and prof_final are checked.
2022-01-26 17:17:48 -08:00
Qi Wang
efc539c040
Initialize prof_leak during prof init.
...
Otherwise, prof_leak may get set after prof_leak_error, and disagree with each
other.
2022-01-26 17:17:48 -08:00
Alex Lapenkou
002f0e9397
Disable TravisCI jobs generation for Windows
...
These jobs take about 20 minutes to complete. We don't want to enable
them until we switch to unlimited concurrency plan, otherwise the builds
will take way too long.
2022-01-26 10:16:57 -08:00