support pyside CI testing with filtering

PySide is currently in an unfinished state. Therefore, running all the
tests will never return a success.

To make this more useful, this script is controlled by a blacklist,
in order to keep record of the known-to-fail tests.

On success: zero error status
On Failure: nonzero (raises ValueError)

The test parser uses a blacklist file that is compatible to standard
Qt blacklist files.

macOS, Ubuntu and windows (thanks to fkleint) are supported.

The blacklist has a feature where new configurations are learned.
It first removes old versions of the same tests, before it inserts the new ones.

Blacklisting is now optional.
We support now py2, py3, qt5, qt5.6 etc.

Some consideration about collapsing the data are added.
The parser has now a sub-command "test" that is mandatory.

Support for moving the build_dir to another location without loosing the tests so far.

Implement BPASS.
Support CTEST_OUTPUT_ON_FAILURE

Change-Id: If66d3cf1733a3b0c8a47fe9b8c6aec8ea430c699
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>

1 files changed, 14 insertions, 0 deletions

diff --git a/setup.py b/setup.py
index 79a82cfe2..ee3cfb6de 100644
--- a/setup.py
+++ b/setup.py
@@ -1,3 +1,5 @@
+from __future__ import print_function
+
 """This is a distutils setup-script for the PySide2 project
 
 To build the PySide2, simply execute:
@@ -630,6 +632,18 @@ class pyside_build(_build):
             for ext in ['shiboken2', 'pyside2', 'pyside2-tools']:
                 self.build_extension(ext)
 
+            if OPTION_BUILDTESTS:
+                # we record the latest successful build and note the build
+                # directory for supporting the tests.
+                timestamp = time.strftime('%Y-%m-%d_%H%M%S')
+                build_history = os.path.join(script_dir, 'build_history')
+                unique_dir = os.path.join(build_history, timestamp)
+                os.makedirs(unique_dir)
+                fpath = os.path.join(unique_dir, 'build_dir.txt')
+                with open(fpath, 'w') as f:
+                    print(build_dir, file=f)
+                log.info("Created %s" % build_history)
+
         if not OPTION_SKIP_PACKAGING:
             # Build patchelf if needed
             self.build_patchelf()