3713932836
Explain how to build for Windows in INSTALL.md and remove another readme.txt in an obscure location.
504 lines
18 KiB
Markdown
504 lines
18 KiB
Markdown
Building and installing a packaged release of jemalloc can be as simple as
|
|
typing the following while in the root directory of the source tree:
|
|
|
|
./configure
|
|
make
|
|
make install
|
|
|
|
If building from unpackaged developer sources, the simplest command sequence
|
|
that might work is:
|
|
|
|
./autogen.sh
|
|
make
|
|
make install
|
|
|
|
You can uninstall the installed build artifacts like this:
|
|
|
|
make uninstall
|
|
|
|
Notes:
|
|
- "autoconf" needs to be installed
|
|
- Documentation is built by the default target only when xsltproc is
|
|
available. Build will warn but not stop if the dependency is missing.
|
|
|
|
|
|
## Advanced configuration
|
|
|
|
The 'configure' script supports numerous options that allow control of which
|
|
functionality is enabled, where jemalloc is installed, etc. Optionally, pass
|
|
any of the following arguments (not a definitive list) to 'configure':
|
|
|
|
* `--help`
|
|
|
|
Print a definitive list of options.
|
|
|
|
* `--prefix=<install-root-dir>`
|
|
|
|
Set the base directory in which to install. For example:
|
|
|
|
./configure --prefix=/usr/local
|
|
|
|
will cause files to be installed into /usr/local/include, /usr/local/lib,
|
|
and /usr/local/man.
|
|
|
|
* `--with-version=(<major>.<minor>.<bugfix>-<nrev>-g<gid>|VERSION)`
|
|
|
|
The VERSION file is mandatory for successful configuration, and the
|
|
following steps are taken to assure its presence:
|
|
1) If --with-version=<major>.<minor>.<bugfix>-<nrev>-g<gid> is specified,
|
|
generate VERSION using the specified value.
|
|
2) If --with-version is not specified in either form and the source
|
|
directory is inside a git repository, try to generate VERSION via 'git
|
|
describe' invocations that pattern-match release tags.
|
|
3) If VERSION is missing, generate it with a bogus version:
|
|
0.0.0-0-g0000000000000000000000000000000000000000
|
|
|
|
Note that --with-version=VERSION bypasses (1) and (2), which simplifies
|
|
VERSION configuration when embedding a jemalloc release into another
|
|
project's git repository.
|
|
|
|
* `--with-rpath=<colon-separated-rpath>`
|
|
|
|
Embed one or more library paths, so that libjemalloc can find the libraries
|
|
it is linked to. This works only on ELF-based systems.
|
|
|
|
* `--with-mangling=<map>`
|
|
|
|
Mangle public symbols specified in <map> which is a comma-separated list of
|
|
name:mangled pairs.
|
|
|
|
For example, to use ld's --wrap option as an alternative method for
|
|
overriding libc's malloc implementation, specify something like:
|
|
|
|
--with-mangling=malloc:__wrap_malloc,free:__wrap_free[...]
|
|
|
|
Note that mangling happens prior to application of the prefix specified by
|
|
--with-jemalloc-prefix, and mangled symbols are then ignored when applying
|
|
the prefix.
|
|
|
|
* `--with-jemalloc-prefix=<prefix>`
|
|
|
|
Prefix all public APIs with <prefix>. For example, if <prefix> is
|
|
"prefix_", API changes like the following occur:
|
|
|
|
malloc() --> prefix_malloc()
|
|
malloc_conf --> prefix_malloc_conf
|
|
/etc/malloc.conf --> /etc/prefix_malloc.conf
|
|
MALLOC_CONF --> PREFIX_MALLOC_CONF
|
|
|
|
This makes it possible to use jemalloc at the same time as the system
|
|
allocator, or even to use multiple copies of jemalloc simultaneously.
|
|
|
|
By default, the prefix is "", except on OS X, where it is "je_". On OS X,
|
|
jemalloc overlays the default malloc zone, but makes no attempt to actually
|
|
replace the "malloc", "calloc", etc. symbols.
|
|
|
|
* `--without-export`
|
|
|
|
Don't export public APIs. This can be useful when building jemalloc as a
|
|
static library, or to avoid exporting public APIs when using the zone
|
|
allocator on OSX.
|
|
|
|
* `--with-private-namespace=<prefix>`
|
|
|
|
Prefix all library-private APIs with <prefix>je_. For shared libraries,
|
|
symbol visibility mechanisms prevent these symbols from being exported, but
|
|
for static libraries, naming collisions are a real possibility. By
|
|
default, <prefix> is empty, which results in a symbol prefix of je_ .
|
|
|
|
* `--with-install-suffix=<suffix>`
|
|
|
|
Append <suffix> to the base name of all installed files, such that multiple
|
|
versions of jemalloc can coexist in the same installation directory. For
|
|
example, libjemalloc.so.0 becomes libjemalloc<suffix>.so.0.
|
|
|
|
* `--with-malloc-conf=<malloc_conf>`
|
|
|
|
Embed `<malloc_conf>` as a run-time options string that is processed prior to
|
|
the malloc_conf global variable, the /etc/malloc.conf symlink, and the
|
|
MALLOC_CONF environment variable. For example, to change the default decay
|
|
time to 30 seconds:
|
|
|
|
--with-malloc-conf=decay_ms:30000
|
|
|
|
* `--enable-debug`
|
|
|
|
Enable assertions and validation code. This incurs a substantial
|
|
performance hit, but is very useful during application development.
|
|
|
|
* `--disable-stats`
|
|
|
|
Disable statistics gathering functionality. See the "opt.stats_print"
|
|
option documentation for usage details.
|
|
|
|
* `--enable-prof`
|
|
|
|
Enable heap profiling and leak detection functionality. See the "opt.prof"
|
|
option documentation for usage details. When enabled, there are several
|
|
approaches to backtracing, and the configure script chooses the first one
|
|
in the following list that appears to function correctly:
|
|
|
|
+ libunwind (requires --enable-prof-libunwind)
|
|
+ libgcc (unless --disable-prof-libgcc)
|
|
+ gcc intrinsics (unless --disable-prof-gcc)
|
|
|
|
* `--enable-prof-libunwind`
|
|
|
|
Use the libunwind library (http://www.nongnu.org/libunwind/) for stack
|
|
backtracing.
|
|
|
|
* `--disable-prof-libgcc`
|
|
|
|
Disable the use of libgcc's backtracing functionality.
|
|
|
|
* `--disable-prof-gcc`
|
|
|
|
Disable the use of gcc intrinsics for backtracing.
|
|
|
|
* `--with-static-libunwind=<libunwind.a>`
|
|
|
|
Statically link against the specified libunwind.a rather than dynamically
|
|
linking with -lunwind.
|
|
|
|
* `--disable-fill`
|
|
|
|
Disable support for junk/zero filling of memory. See the "opt.junk" and
|
|
"opt.zero" option documentation for usage details.
|
|
|
|
* `--disable-zone-allocator`
|
|
|
|
Disable zone allocator for Darwin. This means jemalloc won't be hooked as
|
|
the default allocator on OSX/iOS.
|
|
|
|
* `--enable-utrace`
|
|
|
|
Enable utrace(2)-based allocation tracing. This feature is not broadly
|
|
portable (FreeBSD has it, but Linux and OS X do not).
|
|
|
|
* `--enable-xmalloc`
|
|
|
|
Enable support for optional immediate termination due to out-of-memory
|
|
errors, as is commonly implemented by "xmalloc" wrapper function for malloc.
|
|
See the "opt.xmalloc" option documentation for usage details.
|
|
|
|
* `--enable-lazy-lock`
|
|
|
|
Enable code that wraps pthread_create() to detect when an application
|
|
switches from single-threaded to multi-threaded mode, so that it can avoid
|
|
mutex locking/unlocking operations while in single-threaded mode. In
|
|
practice, this feature usually has little impact on performance unless
|
|
thread-specific caching is disabled.
|
|
|
|
* `--disable-cache-oblivious`
|
|
|
|
Disable cache-oblivious large allocation alignment by default, for large
|
|
allocation requests with no alignment constraints. If this feature is
|
|
disabled, all large allocations are page-aligned as an implementation
|
|
artifact, which can severely harm CPU cache utilization. However, the
|
|
cache-oblivious layout comes at the cost of one extra page per large
|
|
allocation, which in the most extreme case increases physical memory usage
|
|
for the 16 KiB size class to 20 KiB.
|
|
|
|
* `--disable-syscall`
|
|
|
|
Disable use of syscall(2) rather than {open,read,write,close}(2). This is
|
|
intended as a workaround for systems that place security limitations on
|
|
syscall(2).
|
|
|
|
* `--disable-cxx`
|
|
|
|
Disable C++ integration. This will cause new and delete operator
|
|
implementations to be omitted.
|
|
|
|
* `--with-xslroot=<path>`
|
|
|
|
Specify where to find DocBook XSL stylesheets when building the
|
|
documentation.
|
|
|
|
* `--with-lg-page=<lg-page>`
|
|
|
|
Specify the base 2 log of the allocator page size, which must in turn be at
|
|
least as large as the system page size. By default the configure script
|
|
determines the host's page size and sets the allocator page size equal to
|
|
the system page size, so this option need not be specified unless the
|
|
system page size may change between configuration and execution, e.g. when
|
|
cross compiling.
|
|
|
|
* `--with-lg-hugepage=<lg-hugepage>`
|
|
|
|
Specify the base 2 log of the system huge page size. This option is useful
|
|
when cross compiling, or when overriding the default for systems that do
|
|
not explicitly support huge pages.
|
|
|
|
* `--with-lg-quantum=<lg-quantum>`
|
|
|
|
Specify the base 2 log of the minimum allocation alignment. jemalloc needs
|
|
to know the minimum alignment that meets the following C standard
|
|
requirement (quoted from the April 12, 2011 draft of the C11 standard):
|
|
|
|
> The pointer returned if the allocation succeeds is suitably aligned so
|
|
that it may be assigned to a pointer to any type of object with a
|
|
fundamental alignment requirement and then used to access such an object
|
|
or an array of such objects in the space allocated [...]
|
|
|
|
This setting is architecture-specific, and although jemalloc includes known
|
|
safe values for the most commonly used modern architectures, there is a
|
|
wrinkle related to GNU libc (glibc) that may impact your choice of
|
|
<lg-quantum>. On most modern architectures, this mandates 16-byte
|
|
alignment (<lg-quantum>=4), but the glibc developers chose not to meet this
|
|
requirement for performance reasons. An old discussion can be found at
|
|
<https://sourceware.org/bugzilla/show_bug.cgi?id=206> . Unlike glibc,
|
|
jemalloc does follow the C standard by default (caveat: jemalloc
|
|
technically cheats for size classes smaller than the quantum), but the fact
|
|
that Linux systems already work around this allocator noncompliance means
|
|
that it is generally safe in practice to let jemalloc's minimum alignment
|
|
follow glibc's lead. If you specify `--with-lg-quantum=3` during
|
|
configuration, jemalloc will provide additional size classes that are not
|
|
16-byte-aligned (24, 40, and 56).
|
|
|
|
* `--with-lg-vaddr=<lg-vaddr>`
|
|
|
|
Specify the number of significant virtual address bits. By default, the
|
|
configure script attempts to detect virtual address size on those platforms
|
|
where it knows how, and picks a default otherwise. This option may be
|
|
useful when cross-compiling.
|
|
|
|
* `--disable-initial-exec-tls`
|
|
|
|
Disable the initial-exec TLS model for jemalloc's internal thread-local
|
|
storage (on those platforms that support explicit settings). This can allow
|
|
jemalloc to be dynamically loaded after program startup (e.g. using dlopen).
|
|
Note that in this case, there will be two malloc implementations operating
|
|
in the same process, which will almost certainly result in confusing runtime
|
|
crashes if pointers leak from one implementation to the other.
|
|
|
|
* `--disable-libdl`
|
|
|
|
Disable the usage of libdl, namely dlsym(3) which is required by the lazy
|
|
lock option. This can allow building static binaries.
|
|
|
|
The following environment variables (not a definitive list) impact configure's
|
|
behavior:
|
|
|
|
* `CFLAGS="?"`
|
|
* `CXXFLAGS="?"`
|
|
|
|
Pass these flags to the C/C++ compiler. Any flags set by the configure
|
|
script are prepended, which means explicitly set flags generally take
|
|
precedence. Take care when specifying flags such as -Werror, because
|
|
configure tests may be affected in undesirable ways.
|
|
|
|
* `EXTRA_CFLAGS="?"`
|
|
* `EXTRA_CXXFLAGS="?"`
|
|
|
|
Append these flags to CFLAGS/CXXFLAGS, without passing them to the
|
|
compiler(s) during configuration. This makes it possible to add flags such
|
|
as -Werror, while allowing the configure script to determine what other
|
|
flags are appropriate for the specified configuration.
|
|
|
|
* `CPPFLAGS="?"`
|
|
|
|
Pass these flags to the C preprocessor. Note that CFLAGS is not passed to
|
|
'cpp' when 'configure' is looking for include files, so you must use
|
|
CPPFLAGS instead if you need to help 'configure' find header files.
|
|
|
|
* `LD_LIBRARY_PATH="?"`
|
|
|
|
'ld' uses this colon-separated list to find libraries.
|
|
|
|
* `LDFLAGS="?"`
|
|
|
|
Pass these flags when linking.
|
|
|
|
* `PATH="?"`
|
|
|
|
'configure' uses this to find programs.
|
|
|
|
In some cases it may be necessary to work around configuration results that do
|
|
not match reality. For example, Linux 4.5 added support for the MADV_FREE flag
|
|
to madvise(2), which can cause problems if building on a host with MADV_FREE
|
|
support and deploying to a target without. To work around this, use a cache
|
|
file to override the relevant configuration variable defined in configure.ac,
|
|
e.g.:
|
|
|
|
echo "je_cv_madv_free=no" > config.cache && ./configure -C
|
|
|
|
|
|
## Advanced compilation
|
|
|
|
To build only parts of jemalloc, use the following targets:
|
|
|
|
build_lib_shared
|
|
build_lib_static
|
|
build_lib
|
|
build_doc_html
|
|
build_doc_man
|
|
build_doc
|
|
|
|
To install only parts of jemalloc, use the following targets:
|
|
|
|
install_bin
|
|
install_include
|
|
install_lib_shared
|
|
install_lib_static
|
|
install_lib_pc
|
|
install_lib
|
|
install_doc_html
|
|
install_doc_man
|
|
install_doc
|
|
|
|
To clean up build results to varying degrees, use the following make targets:
|
|
|
|
clean
|
|
distclean
|
|
relclean
|
|
|
|
|
|
## Advanced installation
|
|
|
|
Optionally, define make variables when invoking make, including (not
|
|
exclusively):
|
|
|
|
* `INCLUDEDIR="?"`
|
|
|
|
Use this as the installation prefix for header files.
|
|
|
|
* `LIBDIR="?"`
|
|
|
|
Use this as the installation prefix for libraries.
|
|
|
|
* `MANDIR="?"`
|
|
|
|
Use this as the installation prefix for man pages.
|
|
|
|
* `DESTDIR="?"`
|
|
|
|
Prepend DESTDIR to INCLUDEDIR, LIBDIR, DATADIR, and MANDIR. This is useful
|
|
when installing to a different path than was specified via --prefix.
|
|
|
|
* `CC="?"`
|
|
|
|
Use this to invoke the C compiler.
|
|
|
|
* `CFLAGS="?"`
|
|
|
|
Pass these flags to the compiler.
|
|
|
|
* `CPPFLAGS="?"`
|
|
|
|
Pass these flags to the C preprocessor.
|
|
|
|
* `LDFLAGS="?"`
|
|
|
|
Pass these flags when linking.
|
|
|
|
* `PATH="?"`
|
|
|
|
Use this to search for programs used during configuration and building.
|
|
|
|
## Building for Windows
|
|
|
|
There are at least two ways to build jemalloc's libraries for Windows. They
|
|
differ in their ease of use and flexibility.
|
|
|
|
### With MSVC solutions
|
|
This is the easy, but less flexible approach. It doesn't let you specify
|
|
arguments to the `configure` script.
|
|
|
|
1. Install Cygwin with at least the following packages:
|
|
* autoconf
|
|
* autogen
|
|
* gawk
|
|
* grep
|
|
* sed
|
|
|
|
2. Install Visual Studio 2015 or 2017 with Visual C++
|
|
|
|
3. Add Cygwin\bin to the PATH environment variable
|
|
|
|
4. Open "x64 Native Tools Command Prompt for VS 2017"
|
|
(note: x86/x64 doesn't matter at this point)
|
|
|
|
5. Generate header files:
|
|
sh -c "CC=cl ./autogen.sh"
|
|
|
|
6. Now the project can be opened and built in Visual Studio:
|
|
msvc\jemalloc_vc2017.sln
|
|
|
|
### With MSYS
|
|
This is a more involved approach that offers the same configuration flexibility
|
|
as Linux builds. We use it for our CI workflow to test different jemalloc
|
|
configurations on Windows.
|
|
|
|
1. Install the prerequisites
|
|
1. MSYS2
|
|
2. Chocolatey
|
|
3. Visual Studio if you want to compile with MSVC compiler
|
|
|
|
2. Run your bash emulation. It could be MSYS2 or Git Bash (this manual was
|
|
tested on both)
|
|
3. Manually and selectively follow
|
|
[before_install.sh](https://github.com/jemalloc/jemalloc/blob/dev/scripts/windows/before_install.sh)
|
|
script.
|
|
1. Skip the `TRAVIS_OS_NAME` check, `rm -rf C:/tools/msys64` and `choco
|
|
uninstall/upgrade` part.
|
|
2. If using `msys2` shell, add path to `RefreshEnv.cmd` to `PATH`:
|
|
`PATH="$PATH:/c/ProgramData/chocolatey/bin"`
|
|
3. Assign `msys_shell_cmd`, `msys2`, `mingw32` and `mingw64` as in the
|
|
script.
|
|
4. Pick `CROSS_COMPILE_32BIT` , `CC` and `USE_MSVC` values depending on
|
|
your needs. For instance, if you'd like to build for x86_64 Windows
|
|
with `gcc`, then `CROSS_COMPILE_32BIT="no"`, `CC="gcc"` and
|
|
`USE_MSVC=""`. If you'd like to build for x86 Windows with `cl.exe`,
|
|
then `CROSS_COMPILE_32BIT="yes"`, `CC="cl.exe"`, `USE_MSVC="x86"`.
|
|
For x86_64 builds with `cl.exe`, assign `USE_MSVC="amd64"` and
|
|
`CROSS_COMPILE_32BIT="no"`.
|
|
5. Replace the path to `vcvarsall.bat` with the path on your system. For
|
|
instance, on my Windows PC with Visual Studio 17, the path is
|
|
`C:\Program Files (x86)\Microsoft Visual
|
|
Studio\2017\BuildTools\VC\Auxiliary\Build\vcvarsall.bat`.
|
|
6. Execute the rest of the script. It will install the required
|
|
dependencies and assign the variable `build_env`, which is a function
|
|
that executes following commands with the correct environment
|
|
variables set.
|
|
4. Use `$build_env <command>` as you would in a Linux shell:
|
|
1. `$build_env autoconf`
|
|
2. `$build_env ./configure CC="<desired compiler>" <configuration flags>`
|
|
3. `$build_env mingw32-make`
|
|
|
|
If you're having any issues with the above, ensure the following:
|
|
|
|
5. When you run `cmd //C RefreshEnv.cmd`, you get an output line starting with
|
|
`Refreshing` . If it errors saying `RefreshEnv.cmd` is not found, then you
|
|
need to add it to your `PATH` as described above in item 3.2
|
|
|
|
6. When you run `cmd //C $vcvarsall`, it prints a bunch of environment
|
|
variables. Otherwise, check the path to the `vcvarsall.bat` in `$vcvarsall`
|
|
script and fix it.
|
|
|
|
## Development
|
|
|
|
If you intend to make non-trivial changes to jemalloc, use the 'autogen.sh'
|
|
script rather than 'configure'. This re-generates 'configure', enables
|
|
configuration dependency rules, and enables re-generation of automatically
|
|
generated source files.
|
|
|
|
The build system supports using an object directory separate from the source
|
|
tree. For example, you can create an 'obj' directory, and from within that
|
|
directory, issue configuration and build commands:
|
|
|
|
autoconf
|
|
mkdir obj
|
|
cd obj
|
|
../configure --enable-autogen
|
|
make
|
|
|
|
|
|
## Documentation
|
|
|
|
The manual page is generated in both html and roff formats. Any web browser
|
|
can be used to view the html manual. The roff manual page can be formatted
|
|
prior to installation via the following command:
|
|
|
|
nroff -man -t doc/jemalloc.3
|