testrunner: Fix disrupted lines in the error log

Windows inserts extra newlines into the error log when certain
errors occur like
    "Exit code 0xc0000409\n***Exception:"
and that newline caused the parser match to fail.

Note that this is the normal stdout. CMake does not use stderr.
It makes no sense to fix the pipe structure of the script.
Instead, the only fix needed was recognition of line breaks
in the error log. You can see this also in the failure listing

    The following tests FAILED:
       1 - pysidetest_constructor_properties_test (Exit code 0xc0000409
    )

The following improvements were done:
- add an extra pass that checks for broken lines in the error log
- add ad extra plausibility check or consecutive test numbers
- improve the output, program structure and add some documentation

When there should still an unforeseen bug occurs, it will be recognized
by the plausi-check and the test repetitions are immediately cancelled.

We could also fix the output not to contain the line breaks, but
that breaks the principle of keeping the original output and needs
discussion by the developers.

Fixes: PYSIDE-1229
Change-Id: Ib71f3361e78eb59f3469da172c74c719e9f08706
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>

1 files changed, 7 insertions, 0 deletions

diff --git a/testrunner.py b/testrunner.py
index a16a07ab2..6ce0b1be1 100644
--- a/testrunner.py
+++ b/testrunner.py
@@ -39,6 +39,13 @@
 
 from __future__ import print_function, absolute_import
 
+"""
+testrunner.py
+
+Run ctest on the last build.
+See the notes in testing/command.py .
+"""
+
 import sys
 import testing
 import testing.blacklist # just to be sure it's us...