Introduce scripts to run all possible tests

In 6e7d0890 we added better travis continuous integration tests. This is nice,
but has two problems:
- We run only a subset of interesting tests.
- The travis builds can take hours to give us back results (especially on OS X).

This adds scripts/gen_run_tests.py, and its output, run_tests.sh, which builds
and runs a larger portion of possible configurations on the local machine.

While a travis run takes several hours to complete , I can run these scripts on
my (OS X) latop and (Linux) devserve, and get a more exhaustive set of results
back in around 10 minutes.
This commit is contained in:
David Goldblatt 2017-01-27 13:16:56 -08:00 committed by David Goldblatt
parent 6e7d0890cb
commit 5260d9c12f
2 changed files with 45 additions and 0 deletions

1
run_tests.sh Executable file
View File

@ -0,0 +1 @@
$(dirname "$)")/scripts/gen_run_tests.py | bash

44
scripts/gen_run_tests.py Executable file
View File

@ -0,0 +1,44 @@
#!/usr/bin/env python
from itertools import combinations
def powerset(items):
result = []
for i in xrange(len(items) + 1):
result += combinations(items, i)
return result
MAKE_J_VAL = 32
possible_compilers = [('gcc', 'g++'), ('clang', 'clang++')]
possible_compiler_opts = [
'-m32',
]
possible_config_opts = [
'--enable-debug',
'--enable-prof',
'--disable-stats',
'--disable-tcache',
]
print 'set -e'
print 'autoconf'
print 'unamestr=`uname`'
for cc, cxx in possible_compilers:
for compiler_opts in powerset(possible_compiler_opts):
for config_opts in powerset(possible_config_opts):
config_line = (
'./configure '
+ 'CC="{} {}" '.format(cc, " ".join(compiler_opts))
+ 'CXX="{} {}" '.format(cxx, " ".join(compiler_opts))
+ " ".join(config_opts)
)
# Heap profiling is not supported on OS X.
if '--enable-prof' in config_opts:
print 'if [[ "$unamestr" != "Darwin" ]]; then'
print config_line
print "make clean"
print "make -j" + str(MAKE_J_VAL) + " check"
if '--enable-prof' in config_opts:
print 'fi'