QTest: add opt-in changing QCOMPARE etc to exit with throw, not return

Add exception classes and use them to change the control flow for
QTEST_{FAIL,SKIP}_ACTION from return'ing from just the immediate
function to the full way to the QTestLib infrastructure, here we
filter them out.

There are three modes:
- If QT_NO_EXCEPTION, then we return
  - If QTEST_THROW_ON_... is also defined, #error out
- Otherwise, if QTEST_THROW_ON_... is defined, always throw
- Otherwise, the decision is made at runtime (with defaults read from
  QTEST_THROW_ON_... environment variables).

Three selftests depend on the old behavior, as they explicitly check
that multiple FAIL SKIP etc emerge, which the new framework, of
course, prevents. Locally disable throwing at the test function level.

Add initial docs and enable exceptions in all of the selftest
subprograms to facilitate switching between the two runtime-selectable
modes.

[ChangeLog][QtTest] Added QTEST_THROW_ON_FAIL and QTEST_THROW_ON_SKIP
C++ macros and environment variables that, when defined, change how
QCOMPARE/QVERIFY/QSKIP etc exit the test function on failure. Instead
of a return, exiting only the immediately-surrounding function, they
throw a special exception instead, thereby exiting from subfunctions
of the test function, all the way to QtTestLib.

Fixes: QTBUG-66320
Change-Id: I96c38d2a1dcdd9de84942cf448a8bbf3ab6d3679
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>

2 files changed, 3 insertions, 0 deletions

diff --git a/tests/auto/testlib/selftests/warnings/CMakeLists.txt b/tests/auto/testlib/selftests/warnings/CMakeLists.txt
index 12ccea5f5b..553a023a90 100644
--- a/tests/auto/testlib/selftests/warnings/CMakeLists.txt
+++ b/tests/auto/testlib/selftests/warnings/CMakeLists.txt
@@ -7,6 +7,7 @@
 
 qt_internal_add_executable(warnings
     NO_INSTALL
+    EXCEPTIONS
     OUTPUT_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
     SOURCES
         tst_warnings.cpp
diff --git a/tests/auto/testlib/selftests/warnings/tst_warnings.cpp b/tests/auto/testlib/selftests/warnings/tst_warnings.cpp
index 3dd9a7afe2..174e496cd0 100644
--- a/tests/auto/testlib/selftests/warnings/tst_warnings.cpp
+++ b/tests/auto/testlib/selftests/warnings/tst_warnings.cpp
@@ -1,6 +1,7 @@
 // Copyright (C) 2020 The Qt Company Ltd.
 // SPDX-License-Identifier: LicenseRef-Qt-Commercial OR GPL-3.0-only WITH Qt-GPL-exception-1.0
 
+#undef QTEST_THROW_ON_FAILURE // fails ### investigate
 
 #include <QtCore/QCoreApplication>
 #include <QtCore/QRegularExpression>
@@ -183,6 +184,7 @@ void tst_Warnings::testFailOnWarningsWithData()
 
 void tst_Warnings::testFailOnWarningsFailInHelper()
 {
+    const QTest::ThrowOnFailDisabler nothrow; // tests repeated QFAILs
     [](){ QFAIL("This failure message should be printed but not cause the test to abort"); }();
     // So we've already failed, but we get more messages - that don't increment counters.
     const auto warnRegex = QRegularExpression("Ran out of .*!");