project-base/server-skynet-source-3rd-jemalloc
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
7dcf77809c
server-skynet-source-3rd-je.../scripts/gen_travis.py

184 lines
5.6 KiB
Python
Raw Normal View History

Travis: Unbreak the builds. In the hopes of future-proofing as much as possible, jump to the latest distribution Travis supports.
2021-06-22 05:07:10 +08:00
#!/usr/bin/env python3
Beef up travis CI integration testing Introduces gen_travis.py, which generates .travis.yml, and updates .travis.yml to be the generated version. The travis build matrix approach doesn't play well with mixing and matching various different environment settings, so we generate every build explicitly, rather than letting them do it for us. To avoid abusing travis resources (and save us time waiting for CI results), we don't test every possible combination of options; we only check up to 2 unusual settings at a time.
2017-01-26 04:58:50 +08:00
from itertools import combinations
travis_template = """\
language: generic
Travis: Unbreak the builds. In the hopes of future-proofing as much as possible, jump to the latest distribution Travis supports.
2021-06-22 05:07:10 +08:00
dist: focal
Beef up travis CI integration testing Introduces gen_travis.py, which generates .travis.yml, and updates .travis.yml to be the generated version. The travis build matrix approach doesn't play well with mixing and matching various different environment settings, so we generate every build explicitly, rather than letting them do it for us. To avoid abusing travis resources (and save us time waiting for CI results), we don't test every possible combination of options; we only check up to 2 unusual settings at a time.
2017-01-26 04:58:50 +08:00
matrix:
include:
%s
before_script:
- autoconf
Test that .travis.yml has been produced by gen_travis.py on CI This commits checks on Travis-CI that the current `.travis.yml` file equals the output of the `gen_travis.py` script, and updated the `.travis.yml` file accordingly.
2018-07-11 21:18:40 +08:00
- scripts/gen_travis.py > travis_script && diff .travis.yml travis_script
Beef up travis CI integration testing Introduces gen_travis.py, which generates .travis.yml, and updates .travis.yml to be the generated version. The travis build matrix approach doesn't play well with mixing and matching various different environment settings, so we generate every build explicitly, rather than letting them do it for us. To avoid abusing travis resources (and save us time waiting for CI results), we don't test every possible combination of options; we only check up to 2 unusual settings at a time.
2017-01-26 04:58:50 +08:00
- ./configure ${COMPILER_FLAGS:+ \
CC="$CC $COMPILER_FLAGS" \
CXX="$CXX $COMPILER_FLAGS" } \
$CONFIGURE_FLAGS
- make -j3
- make -j3 tests
script:
- make check
"""
# The 'default' configuration is gcc, on linux, with no compiler or configure
# flags. We also test with clang, -m32, --enable-debug, --enable-prof,
Remove --disable-tcache. Simplify configuration by removing the --disable-tcache option, but replace the testing for that configuration with --with-malloc-conf=tcache:false. Fix the thread.arena and thread.tcache.flush mallctls to work correctly if tcache is disabled. This partially resolves #580.
2017-04-21 08:21:37 +08:00
# --disable-stats, and --with-malloc-conf=tcache:false. To avoid abusing
# travis though, we don't test all 2**7 = 128 possible combinations of these;
# instead, we only test combinations of up to 2 'unusual' settings, under the
# hope that bugs involving interactions of such settings are rare.
Beef up travis CI integration testing Introduces gen_travis.py, which generates .travis.yml, and updates .travis.yml to be the generated version. The travis build matrix approach doesn't play well with mixing and matching various different environment settings, so we generate every build explicitly, rather than letting them do it for us. To avoid abusing travis resources (and save us time waiting for CI results), we don't test every possible combination of options; we only check up to 2 unusual settings at a time.
2017-01-26 04:58:50 +08:00
MAX_UNUSUAL_OPTIONS = 2
os_default = 'linux'
os_unusual = 'osx'
CI support for PPC64LE architecture
2020-09-11 16:07:10 +08:00
arch_default = 'amd64'
arch_unusual = 'ppc64le'
Beef up travis CI integration testing Introduces gen_travis.py, which generates .travis.yml, and updates .travis.yml to be the generated version. The travis build matrix approach doesn't play well with mixing and matching various different environment settings, so we generate every build explicitly, rather than letting them do it for us. To avoid abusing travis resources (and save us time waiting for CI results), we don't test every possible combination of options; we only check up to 2 unusual settings at a time.
2017-01-26 04:58:50 +08:00
compilers_default = 'CC=gcc CXX=g++'
compilers_unusual = 'CC=clang CXX=clang++'
compiler_flag_unusuals = ['-m32']
configure_flag_unusuals = [
Remove --disable-tcache. Simplify configuration by removing the --disable-tcache option, but replace the testing for that configuration with --with-malloc-conf=tcache:false. Fix the thread.arena and thread.tcache.flush mallctls to work correctly if tcache is disabled. This partially resolves #580.
2017-04-21 08:21:37 +08:00
'--enable-debug',
'--enable-prof',
'--disable-stats',
Add --disable-libdl to travis.
2019-02-07 05:36:56 +08:00
'--disable-libdl',
Safety checks: Run tests by default
2019-03-23 08:08:53 +08:00
'--enable-opt-safety-checks',
Add --with-lg-page=16 to CI.
2019-09-12 01:31:30 +08:00
'--with-lg-page=16',
Add dss:primary testing. Generalize the run_tests.sh and .travis.yml test generation to handle combinations of arguments to the --with-malloc-conf configure option, and merge "dss:primary" into the existing "tcache:false" testing.
2017-05-23 08:15:57 +08:00
]
malloc_conf_unusuals = [
'tcache:false',
'dss:primary',
Disable percpu_arena by default.
2017-05-24 01:53:36 +08:00
'percpu_arena:percpu',
Test with background_thread:true. Add testing for background_thread:true, and condition a xallocx() --> rallocx() escalation assertion to allow for spurious in-place rallocx() following xallocx() failure.
2017-06-01 02:56:31 +08:00
'background_thread:true',
Beef up travis CI integration testing Introduces gen_travis.py, which generates .travis.yml, and updates .travis.yml to be the generated version. The travis build matrix approach doesn't play well with mixing and matching various different environment settings, so we generate every build explicitly, rather than letting them do it for us. To avoid abusing travis resources (and save us time waiting for CI results), we don't test every possible combination of options; we only check up to 2 unusual settings at a time.
2017-01-26 04:58:50 +08:00
]
all_unusuals = (
CI support for PPC64LE architecture
2020-09-11 16:07:10 +08:00
[os_unusual] + [arch_unusual] + [compilers_unusual] + compiler_flag_unusuals
Add dss:primary testing. Generalize the run_tests.sh and .travis.yml test generation to handle combinations of arguments to the --with-malloc-conf configure option, and merge "dss:primary" into the existing "tcache:false" testing.
2017-05-23 08:15:57 +08:00
+ configure_flag_unusuals + malloc_conf_unusuals
Beef up travis CI integration testing Introduces gen_travis.py, which generates .travis.yml, and updates .travis.yml to be the generated version. The travis build matrix approach doesn't play well with mixing and matching various different environment settings, so we generate every build explicitly, rather than letting them do it for us. To avoid abusing travis resources (and save us time waiting for CI results), we don't test every possible combination of options; we only check up to 2 unusual settings at a time.
2017-01-26 04:58:50 +08:00
)
unusual_combinations_to_test = []
Travis: Unbreak the builds. In the hopes of future-proofing as much as possible, jump to the latest distribution Travis supports.
2021-06-22 05:07:10 +08:00
for i in range(MAX_UNUSUAL_OPTIONS + 1):
Beef up travis CI integration testing Introduces gen_travis.py, which generates .travis.yml, and updates .travis.yml to be the generated version. The travis build matrix approach doesn't play well with mixing and matching various different environment settings, so we generate every build explicitly, rather than letting them do it for us. To avoid abusing travis resources (and save us time waiting for CI results), we don't test every possible combination of options; we only check up to 2 unusual settings at a time.
2017-01-26 04:58:50 +08:00
unusual_combinations_to_test += combinations(all_unusuals, i)
Simplify output of gen_travis.py script This commit simplifies the output of the `gen_travis.py` script by reusing addons. The `.travis.yml` script is updated to reflect these changes.
2018-07-10 20:41:20 +08:00
gcc_multilib_set = False
CI support for PPC64LE architecture
2020-09-11 16:07:10 +08:00
gcc_ppc_set = False
Simplify output of gen_travis.py script This commit simplifies the output of the `gen_travis.py` script by reusing addons. The `.travis.yml` script is updated to reflect these changes.
2018-07-10 20:41:20 +08:00
# Formats a job from a combination of flags
def format_job(combination):
global gcc_multilib_set
CI support for PPC64LE architecture
2020-09-11 16:07:10 +08:00
global gcc_ppc_set
Beef up travis CI integration testing Introduces gen_travis.py, which generates .travis.yml, and updates .travis.yml to be the generated version. The travis build matrix approach doesn't play well with mixing and matching various different environment settings, so we generate every build explicitly, rather than letting them do it for us. To avoid abusing travis resources (and save us time waiting for CI results), we don't test every possible combination of options; we only check up to 2 unusual settings at a time.
2017-01-26 04:58:50 +08:00
Simplify output of gen_travis.py script This commit simplifies the output of the `gen_travis.py` script by reusing addons. The `.travis.yml` script is updated to reflect these changes.
2018-07-10 20:41:20 +08:00
os = os_unusual if os_unusual in combination else os_default
compilers = compilers_unusual if compilers_unusual in combination else compilers_default
CI support for PPC64LE architecture
2020-09-11 16:07:10 +08:00
arch = arch_unusual if arch_unusual in combination else arch_default
Simplify output of gen_travis.py script This commit simplifies the output of the `gen_travis.py` script by reusing addons. The `.travis.yml` script is updated to reflect these changes.
2018-07-10 20:41:20 +08:00
compiler_flags = [x for x in combination if x in compiler_flag_unusuals]
configure_flags = [x for x in combination if x in configure_flag_unusuals]
malloc_conf = [x for x in combination if x in malloc_conf_unusuals]
Beef up travis CI integration testing Introduces gen_travis.py, which generates .travis.yml, and updates .travis.yml to be the generated version. The travis build matrix approach doesn't play well with mixing and matching various different environment settings, so we generate every build explicitly, rather than letting them do it for us. To avoid abusing travis resources (and save us time waiting for CI results), we don't test every possible combination of options; we only check up to 2 unusual settings at a time.
2017-01-26 04:58:50 +08:00
Disable percpu_arena by default.
2017-05-24 01:53:36 +08:00
# Filter out unsupported configurations on OS X.
if os == 'osx' and ('dss:primary' in malloc_conf or \
Test with background_thread:true. Add testing for background_thread:true, and condition a xallocx() --> rallocx() escalation assertion to allow for spurious in-place rallocx() following xallocx() failure.
2017-06-01 02:56:31 +08:00
'percpu_arena:percpu' in malloc_conf or 'background_thread:true' \
in malloc_conf):
Simplify output of gen_travis.py script This commit simplifies the output of the `gen_travis.py` script by reusing addons. The `.travis.yml` script is updated to reflect these changes.
2018-07-10 20:41:20 +08:00
return ""
Travis: Don't test "clang" on OS X. On OS X, "gcc" is really just clang anyways, so this combination gets tested by the gcc test. This is purely redundant, and (since it runs early in the output) increases time to signal for real breakages further down in the list.
2021-07-08 06:16:38 +08:00
# gcc is just a redirect to clang on OS X. No need to test both.
if os == 'osx' and compilers_unusual in combination:
return ""
Add dss:primary testing. Generalize the run_tests.sh and .travis.yml test generation to handle combinations of arguments to the --with-malloc-conf configure option, and merge "dss:primary" into the existing "tcache:false" testing.
2017-05-23 08:15:57 +08:00
if len(malloc_conf) > 0:
configure_flags.append('--with-malloc-conf=' + ",".join(malloc_conf))
Beef up travis CI integration testing Introduces gen_travis.py, which generates .travis.yml, and updates .travis.yml to be the generated version. The travis build matrix approach doesn't play well with mixing and matching various different environment settings, so we generate every build explicitly, rather than letting them do it for us. To avoid abusing travis resources (and save us time waiting for CI results), we don't test every possible combination of options; we only check up to 2 unusual settings at a time.
2017-01-26 04:58:50 +08:00
# Filter out an unsupported configuration - heap profiling on OS X.
if os == 'osx' and '--enable-prof' in configure_flags:
Simplify output of gen_travis.py script This commit simplifies the output of the `gen_travis.py` script by reusing addons. The `.travis.yml` script is updated to reflect these changes.
2018-07-10 20:41:20 +08:00
return ""
Beef up travis CI integration testing Introduces gen_travis.py, which generates .travis.yml, and updates .travis.yml to be the generated version. The travis build matrix approach doesn't play well with mixing and matching various different environment settings, so we generate every build explicitly, rather than letting them do it for us. To avoid abusing travis resources (and save us time waiting for CI results), we don't test every possible combination of options; we only check up to 2 unusual settings at a time.
2017-01-26 04:58:50 +08:00
CI support for PPC64LE architecture
2020-09-11 16:07:10 +08:00
# Filter out unsupported OSX configuration on PPC64LE
if arch == 'ppc64le' and (
os == 'osx'
or '-m32' in combination
or compilers_unusual in combination
):
return ""
Beef up travis CI integration testing Introduces gen_travis.py, which generates .travis.yml, and updates .travis.yml to be the generated version. The travis build matrix approach doesn't play well with mixing and matching various different environment settings, so we generate every build explicitly, rather than letting them do it for us. To avoid abusing travis resources (and save us time waiting for CI results), we don't test every possible combination of options; we only check up to 2 unusual settings at a time.
2017-01-26 04:58:50 +08:00
Simplify output of gen_travis.py script This commit simplifies the output of the `gen_travis.py` script by reusing addons. The `.travis.yml` script is updated to reflect these changes.
2018-07-10 20:41:20 +08:00
job = ""
job += ' - os: %s\n' % os
CI support for PPC64LE architecture
2020-09-11 16:07:10 +08:00
job += ' arch: %s\n' % arch
Simplify output of gen_travis.py script This commit simplifies the output of the `gen_travis.py` script by reusing addons. The `.travis.yml` script is updated to reflect these changes.
2018-07-10 20:41:20 +08:00
if '-m32' in combination and os == 'linux':
job += ' addons:'
if gcc_multilib_set:
job += ' *gcc_multilib\n'
else:
job += ' &gcc_multilib\n'
job += ' apt:\n'
job += ' packages:\n'
job += ' - gcc-multilib\n'
Travis: Unbreak the builds. In the hopes of future-proofing as much as possible, jump to the latest distribution Travis supports.
2021-06-22 05:07:10 +08:00
job += ' - g++-multilib\n'
Simplify output of gen_travis.py script This commit simplifies the output of the `gen_travis.py` script by reusing addons. The `.travis.yml` script is updated to reflect these changes.
2018-07-10 20:41:20 +08:00
gcc_multilib_set = True
CI support for PPC64LE architecture
2020-09-11 16:07:10 +08:00
# We get some spurious errors when -Warray-bounds is enabled.
Travis: Unbreak the builds. In the hopes of future-proofing as much as possible, jump to the latest distribution Travis supports.
2021-06-22 05:07:10 +08:00
extra_cflags = ['-Werror', '-Wno-array-bounds']
if 'clang' in compilers or os == 'osx':
extra_cflags += [
'-Wno-unknown-warning-option',
'-Wno-ignored-attributes'
]
if os == 'osx':
extra_cflags += [
'-Wno-deprecated-declarations',
]
CI support for PPC64LE architecture
2020-09-11 16:07:10 +08:00
env_string = ('{} COMPILER_FLAGS="{}" CONFIGURE_FLAGS="{}" '
Travis: Unbreak the builds. In the hopes of future-proofing as much as possible, jump to the latest distribution Travis supports.
2021-06-22 05:07:10 +08:00
'EXTRA_CFLAGS="{}"'.format(
compilers, ' '.join(compiler_flags), ' '.join(configure_flags),
' '.join(extra_cflags)))
CI support for PPC64LE architecture
2020-09-11 16:07:10 +08:00
job += ' env: %s\n' % env_string
Simplify output of gen_travis.py script This commit simplifies the output of the `gen_travis.py` script by reusing addons. The `.travis.yml` script is updated to reflect these changes.
2018-07-10 20:41:20 +08:00
return job
include_rows = ""
for combination in unusual_combinations_to_test:
include_rows += format_job(combination)
# Development build
include_rows += '''\
Add valgrind build bots to CI This commit adds two build-bots to CI that test the release builds of jemalloc on linux and macOS under valgrind. The macOS build is not enabled because valgrind reports errors about reads of uninitialized memory in some tests and segfaults in others.
2018-07-10 19:58:37 +08:00
# Development build
Simplify output of gen_travis.py script This commit simplifies the output of the `gen_travis.py` script by reusing addons. The `.travis.yml` script is updated to reflect these changes.
2018-07-10 20:41:20 +08:00
- os: linux
env: CC=gcc CXX=g++ COMPILER_FLAGS="" CONFIGURE_FLAGS="--enable-debug --disable-cache-oblivious --enable-stats --enable-log --enable-prof" EXTRA_CFLAGS="-Werror -Wno-array-bounds"
'''
Beef up travis CI integration testing Introduces gen_travis.py, which generates .travis.yml, and updates .travis.yml to be the generated version. The travis build matrix approach doesn't play well with mixing and matching various different environment settings, so we generate every build explicitly, rather than letting them do it for us. To avoid abusing travis resources (and save us time waiting for CI results), we don't test every possible combination of options; we only check up to 2 unusual settings at a time.
2017-01-26 04:58:50 +08:00
Test smallocx on Travis-CI This commit updates the gen_travis script with a new build bot that covers the experimental `smallocx` API and updates the travis CI script to test this API under travis.
2018-07-11 21:11:53 +08:00
# Enable-expermental-smallocx
include_rows += '''\
# --enable-expermental-smallocx:
- os: linux
env: CC=gcc CXX=g++ COMPILER_FLAGS="" CONFIGURE_FLAGS="--enable-debug --enable-experimental-smallocx --enable-stats --enable-prof" EXTRA_CFLAGS="-Werror -Wno-array-bounds"
'''
Travis: Unbreak the builds. In the hopes of future-proofing as much as possible, jump to the latest distribution Travis supports.
2021-06-22 05:07:10 +08:00
# Does not seem to be working on newer travis machines. Valgrind has long been a
# pain point; abandon it for now.
Add valgrind build bots to CI This commit adds two build-bots to CI that test the release builds of jemalloc on linux and macOS under valgrind. The macOS build is not enabled because valgrind reports errors about reads of uninitialized memory in some tests and segfaults in others.
2018-07-10 19:58:37 +08:00
# Valgrind build bots
Travis: Unbreak the builds. In the hopes of future-proofing as much as possible, jump to the latest distribution Travis supports.
2021-06-22 05:07:10 +08:00
#include_rows += '''
# # Valgrind
# - os: linux
# arch: amd64
# env: CC=gcc CXX=g++ COMPILER_FLAGS="" CONFIGURE_FLAGS="" EXTRA_CFLAGS="-Werror -Wno-array-bounds" JEMALLOC_TEST_PREFIX="valgrind"
# addons:
# apt:
# packages:
# - valgrind
#'''
Add valgrind build bots to CI This commit adds two build-bots to CI that test the release builds of jemalloc on linux and macOS under valgrind. The macOS build is not enabled because valgrind reports errors about reads of uninitialized memory in some tests and segfaults in others.
2018-07-10 19:58:37 +08:00
# To enable valgrind on macosx add:
#
# - os: osx
# env: CC=gcc CXX=g++ COMPILER_FLAGS="" CONFIGURE_FLAGS="" EXTRA_CFLAGS="-Werror -Wno-array-bounds" JEMALLOC_TEST_PREFIX="valgrind"
# install: brew install valgrind
#
# It currently fails due to: https://github.com/jemalloc/jemalloc/issues/1274
Travis: Unbreak the builds. In the hopes of future-proofing as much as possible, jump to the latest distribution Travis supports.
2021-06-22 05:07:10 +08:00
print(travis_template % include_rows)
Reference in New Issue Copy Permalink