project-base/server-skynet-source-3rd-jemalloc
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
115ce93562
server-skynet-source-3rd-je.../scripts/gen_travis.py

141 lines
4.2 KiB
Python
Raw Normal View History

Beef up travis CI integration testing Introduces gen_travis.py, which generates .travis.yml, and updates .travis.yml to be the generated version. The travis build matrix approach doesn't play well with mixing and matching various different environment settings, so we generate every build explicitly, rather than letting them do it for us. To avoid abusing travis resources (and save us time waiting for CI results), we don't test every possible combination of options; we only check up to 2 unusual settings at a time.
2017-01-26 04:58:50 +08:00
#!/usr/bin/env python
from itertools import combinations
travis_template = """\
language: generic
Simplify output of gen_travis.py script This commit simplifies the output of the `gen_travis.py` script by reusing addons. The `.travis.yml` script is updated to reflect these changes.
2018-07-10 20:41:20 +08:00
dist: precise
Beef up travis CI integration testing Introduces gen_travis.py, which generates .travis.yml, and updates .travis.yml to be the generated version. The travis build matrix approach doesn't play well with mixing and matching various different environment settings, so we generate every build explicitly, rather than letting them do it for us. To avoid abusing travis resources (and save us time waiting for CI results), we don't test every possible combination of options; we only check up to 2 unusual settings at a time.
2017-01-26 04:58:50 +08:00
matrix:
include:
%s
before_script:
- autoconf
Test that .travis.yml has been produced by gen_travis.py on CI This commits checks on Travis-CI that the current `.travis.yml` file equals the output of the `gen_travis.py` script, and updated the `.travis.yml` file accordingly.
2018-07-11 21:18:40 +08:00
- scripts/gen_travis.py > travis_script && diff .travis.yml travis_script
Beef up travis CI integration testing Introduces gen_travis.py, which generates .travis.yml, and updates .travis.yml to be the generated version. The travis build matrix approach doesn't play well with mixing and matching various different environment settings, so we generate every build explicitly, rather than letting them do it for us. To avoid abusing travis resources (and save us time waiting for CI results), we don't test every possible combination of options; we only check up to 2 unusual settings at a time.
2017-01-26 04:58:50 +08:00
- ./configure ${COMPILER_FLAGS:+ \
CC="$CC $COMPILER_FLAGS" \
CXX="$CXX $COMPILER_FLAGS" } \
$CONFIGURE_FLAGS
- make -j3
- make -j3 tests
script:
- make check
"""
# The 'default' configuration is gcc, on linux, with no compiler or configure
# flags. We also test with clang, -m32, --enable-debug, --enable-prof,
Remove --disable-tcache. Simplify configuration by removing the --disable-tcache option, but replace the testing for that configuration with --with-malloc-conf=tcache:false. Fix the thread.arena and thread.tcache.flush mallctls to work correctly if tcache is disabled. This partially resolves #580.
2017-04-21 08:21:37 +08:00
# --disable-stats, and --with-malloc-conf=tcache:false. To avoid abusing
# travis though, we don't test all 2**7 = 128 possible combinations of these;
# instead, we only test combinations of up to 2 'unusual' settings, under the
# hope that bugs involving interactions of such settings are rare.
# Things at once, for C(7, 0) + C(7, 1) + C(7, 2) = 29
Beef up travis CI integration testing Introduces gen_travis.py, which generates .travis.yml, and updates .travis.yml to be the generated version. The travis build matrix approach doesn't play well with mixing and matching various different environment settings, so we generate every build explicitly, rather than letting them do it for us. To avoid abusing travis resources (and save us time waiting for CI results), we don't test every possible combination of options; we only check up to 2 unusual settings at a time.
2017-01-26 04:58:50 +08:00
MAX_UNUSUAL_OPTIONS = 2
os_default = 'linux'
os_unusual = 'osx'
compilers_default = 'CC=gcc CXX=g++'
compilers_unusual = 'CC=clang CXX=clang++'
compiler_flag_unusuals = ['-m32']
configure_flag_unusuals = [
Remove --disable-tcache. Simplify configuration by removing the --disable-tcache option, but replace the testing for that configuration with --with-malloc-conf=tcache:false. Fix the thread.arena and thread.tcache.flush mallctls to work correctly if tcache is disabled. This partially resolves #580.
2017-04-21 08:21:37 +08:00
'--enable-debug',
'--enable-prof',
'--disable-stats',
Add dss:primary testing. Generalize the run_tests.sh and .travis.yml test generation to handle combinations of arguments to the --with-malloc-conf configure option, and merge "dss:primary" into the existing "tcache:false" testing.
2017-05-23 08:15:57 +08:00
]
malloc_conf_unusuals = [
'tcache:false',
'dss:primary',
Disable percpu_arena by default.
2017-05-24 01:53:36 +08:00
'percpu_arena:percpu',
Test with background_thread:true. Add testing for background_thread:true, and condition a xallocx() --> rallocx() escalation assertion to allow for spurious in-place rallocx() following xallocx() failure.
2017-06-01 02:56:31 +08:00
'background_thread:true',
Beef up travis CI integration testing Introduces gen_travis.py, which generates .travis.yml, and updates .travis.yml to be the generated version. The travis build matrix approach doesn't play well with mixing and matching various different environment settings, so we generate every build explicitly, rather than letting them do it for us. To avoid abusing travis resources (and save us time waiting for CI results), we don't test every possible combination of options; we only check up to 2 unusual settings at a time.
2017-01-26 04:58:50 +08:00
]
all_unusuals = (
[os_unusual] + [compilers_unusual] + compiler_flag_unusuals
Add dss:primary testing. Generalize the run_tests.sh and .travis.yml test generation to handle combinations of arguments to the --with-malloc-conf configure option, and merge "dss:primary" into the existing "tcache:false" testing.
2017-05-23 08:15:57 +08:00
+ configure_flag_unusuals + malloc_conf_unusuals
Beef up travis CI integration testing Introduces gen_travis.py, which generates .travis.yml, and updates .travis.yml to be the generated version. The travis build matrix approach doesn't play well with mixing and matching various different environment settings, so we generate every build explicitly, rather than letting them do it for us. To avoid abusing travis resources (and save us time waiting for CI results), we don't test every possible combination of options; we only check up to 2 unusual settings at a time.
2017-01-26 04:58:50 +08:00
)
unusual_combinations_to_test = []
for i in xrange(MAX_UNUSUAL_OPTIONS + 1):
unusual_combinations_to_test += combinations(all_unusuals, i)
Simplify output of gen_travis.py script This commit simplifies the output of the `gen_travis.py` script by reusing addons. The `.travis.yml` script is updated to reflect these changes.
2018-07-10 20:41:20 +08:00
gcc_multilib_set = False
# Formats a job from a combination of flags
def format_job(combination):
global gcc_multilib_set
Beef up travis CI integration testing Introduces gen_travis.py, which generates .travis.yml, and updates .travis.yml to be the generated version. The travis build matrix approach doesn't play well with mixing and matching various different environment settings, so we generate every build explicitly, rather than letting them do it for us. To avoid abusing travis resources (and save us time waiting for CI results), we don't test every possible combination of options; we only check up to 2 unusual settings at a time.
2017-01-26 04:58:50 +08:00
Simplify output of gen_travis.py script This commit simplifies the output of the `gen_travis.py` script by reusing addons. The `.travis.yml` script is updated to reflect these changes.
2018-07-10 20:41:20 +08:00
os = os_unusual if os_unusual in combination else os_default
compilers = compilers_unusual if compilers_unusual in combination else compilers_default
Beef up travis CI integration testing Introduces gen_travis.py, which generates .travis.yml, and updates .travis.yml to be the generated version. The travis build matrix approach doesn't play well with mixing and matching various different environment settings, so we generate every build explicitly, rather than letting them do it for us. To avoid abusing travis resources (and save us time waiting for CI results), we don't test every possible combination of options; we only check up to 2 unusual settings at a time.
2017-01-26 04:58:50 +08:00
Simplify output of gen_travis.py script This commit simplifies the output of the `gen_travis.py` script by reusing addons. The `.travis.yml` script is updated to reflect these changes.
2018-07-10 20:41:20 +08:00
compiler_flags = [x for x in combination if x in compiler_flag_unusuals]
configure_flags = [x for x in combination if x in configure_flag_unusuals]
malloc_conf = [x for x in combination if x in malloc_conf_unusuals]
Beef up travis CI integration testing Introduces gen_travis.py, which generates .travis.yml, and updates .travis.yml to be the generated version. The travis build matrix approach doesn't play well with mixing and matching various different environment settings, so we generate every build explicitly, rather than letting them do it for us. To avoid abusing travis resources (and save us time waiting for CI results), we don't test every possible combination of options; we only check up to 2 unusual settings at a time.
2017-01-26 04:58:50 +08:00
Disable percpu_arena by default.
2017-05-24 01:53:36 +08:00
# Filter out unsupported configurations on OS X.
if os == 'osx' and ('dss:primary' in malloc_conf or \
Test with background_thread:true. Add testing for background_thread:true, and condition a xallocx() --> rallocx() escalation assertion to allow for spurious in-place rallocx() following xallocx() failure.
2017-06-01 02:56:31 +08:00
'percpu_arena:percpu' in malloc_conf or 'background_thread:true' \
in malloc_conf):
Simplify output of gen_travis.py script This commit simplifies the output of the `gen_travis.py` script by reusing addons. The `.travis.yml` script is updated to reflect these changes.
2018-07-10 20:41:20 +08:00
return ""
Add dss:primary testing. Generalize the run_tests.sh and .travis.yml test generation to handle combinations of arguments to the --with-malloc-conf configure option, and merge "dss:primary" into the existing "tcache:false" testing.
2017-05-23 08:15:57 +08:00
if len(malloc_conf) > 0:
configure_flags.append('--with-malloc-conf=' + ",".join(malloc_conf))
Beef up travis CI integration testing Introduces gen_travis.py, which generates .travis.yml, and updates .travis.yml to be the generated version. The travis build matrix approach doesn't play well with mixing and matching various different environment settings, so we generate every build explicitly, rather than letting them do it for us. To avoid abusing travis resources (and save us time waiting for CI results), we don't test every possible combination of options; we only check up to 2 unusual settings at a time.
2017-01-26 04:58:50 +08:00
# Filter out an unsupported configuration - heap profiling on OS X.
if os == 'osx' and '--enable-prof' in configure_flags:
Simplify output of gen_travis.py script This commit simplifies the output of the `gen_travis.py` script by reusing addons. The `.travis.yml` script is updated to reflect these changes.
2018-07-10 20:41:20 +08:00
return ""
Beef up travis CI integration testing Introduces gen_travis.py, which generates .travis.yml, and updates .travis.yml to be the generated version. The travis build matrix approach doesn't play well with mixing and matching various different environment settings, so we generate every build explicitly, rather than letting them do it for us. To avoid abusing travis resources (and save us time waiting for CI results), we don't test every possible combination of options; we only check up to 2 unusual settings at a time.
2017-01-26 04:58:50 +08:00
Turn on -Werror for travis CI builds
2017-04-11 04:26:21 +08:00
# We get some spurious errors when -Warray-bounds is enabled.
env_string = ('{} COMPILER_FLAGS="{}" CONFIGURE_FLAGS="{}" '
'EXTRA_CFLAGS="-Werror -Wno-array-bounds"').format(
Beef up travis CI integration testing Introduces gen_travis.py, which generates .travis.yml, and updates .travis.yml to be the generated version. The travis build matrix approach doesn't play well with mixing and matching various different environment settings, so we generate every build explicitly, rather than letting them do it for us. To avoid abusing travis resources (and save us time waiting for CI results), we don't test every possible combination of options; we only check up to 2 unusual settings at a time.
2017-01-26 04:58:50 +08:00
compilers, " ".join(compiler_flags), " ".join(configure_flags))
Simplify output of gen_travis.py script This commit simplifies the output of the `gen_travis.py` script by reusing addons. The `.travis.yml` script is updated to reflect these changes.
2018-07-10 20:41:20 +08:00
job = ""
job += ' - os: %s\n' % os
job += ' env: %s\n' % env_string
if '-m32' in combination and os == 'linux':
job += ' addons:'
if gcc_multilib_set:
job += ' *gcc_multilib\n'
else:
job += ' &gcc_multilib\n'
job += ' apt:\n'
job += ' packages:\n'
job += ' - gcc-multilib\n'
gcc_multilib_set = True
return job
include_rows = ""
for combination in unusual_combinations_to_test:
include_rows += format_job(combination)
# Development build
include_rows += '''\
Add valgrind build bots to CI This commit adds two build-bots to CI that test the release builds of jemalloc on linux and macOS under valgrind. The macOS build is not enabled because valgrind reports errors about reads of uninitialized memory in some tests and segfaults in others.
2018-07-10 19:58:37 +08:00
# Development build
Simplify output of gen_travis.py script This commit simplifies the output of the `gen_travis.py` script by reusing addons. The `.travis.yml` script is updated to reflect these changes.
2018-07-10 20:41:20 +08:00
- os: linux
env: CC=gcc CXX=g++ COMPILER_FLAGS="" CONFIGURE_FLAGS="--enable-debug --disable-cache-oblivious --enable-stats --enable-log --enable-prof" EXTRA_CFLAGS="-Werror -Wno-array-bounds"
'''
Beef up travis CI integration testing Introduces gen_travis.py, which generates .travis.yml, and updates .travis.yml to be the generated version. The travis build matrix approach doesn't play well with mixing and matching various different environment settings, so we generate every build explicitly, rather than letting them do it for us. To avoid abusing travis resources (and save us time waiting for CI results), we don't test every possible combination of options; we only check up to 2 unusual settings at a time.
2017-01-26 04:58:50 +08:00
Add valgrind build bots to CI This commit adds two build-bots to CI that test the release builds of jemalloc on linux and macOS under valgrind. The macOS build is not enabled because valgrind reports errors about reads of uninitialized memory in some tests and segfaults in others.
2018-07-10 19:58:37 +08:00
# Valgrind build bots
include_rows += '''
# Valgrind
- os: linux
env: CC=gcc CXX=g++ COMPILER_FLAGS="" CONFIGURE_FLAGS="" EXTRA_CFLAGS="-Werror -Wno-array-bounds" JEMALLOC_TEST_PREFIX="valgrind"
addons:
apt:
packages:
- valgrind
'''
# To enable valgrind on macosx add:
#
# - os: osx
# env: CC=gcc CXX=g++ COMPILER_FLAGS="" CONFIGURE_FLAGS="" EXTRA_CFLAGS="-Werror -Wno-array-bounds" JEMALLOC_TEST_PREFIX="valgrind"
# install: brew install valgrind
#
# It currently fails due to: https://github.com/jemalloc/jemalloc/issues/1274
Beef up travis CI integration testing Introduces gen_travis.py, which generates .travis.yml, and updates .travis.yml to be the generated version. The travis build matrix approach doesn't play well with mixing and matching various different environment settings, so we generate every build explicitly, rather than letting them do it for us. To avoid abusing travis resources (and save us time waiting for CI results), we don't test every possible combination of options; we only check up to 2 unusual settings at a time.
2017-01-26 04:58:50 +08:00
print travis_template % include_rows
Reference in New Issue Copy Permalink