Add QTest::failOnWarning

This solves the long-standing problem of not being able to easily
fail a test when a certain warning is output.

[ChangeLog][QtTest] Added QTest::failOnWarning. When called in a test
function, any warning that matches the given pattern will cause a test
failure. The test will continue execution when a failure is added.
All patterns are cleared at the end of each test function.

Fixes: QTBUG-70029
Change-Id: I5763f8d4acf1cee8178be43a503619fbfb0f4f36
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>

1 files changed, 47 insertions, 1 deletions

diff --git a/tests/auto/testlib/selftests/expected_warnings.txt b/tests/auto/testlib/selftests/expected_warnings.txt
index 0a65213b79..02361eb39d 100644
--- a/tests/auto/testlib/selftests/expected_warnings.txt
+++ b/tests/auto/testlib/selftests/expected_warnings.txt
@@ -23,6 +23,52 @@ FAIL!  : tst_Warnings::testMissingWarningsWithData(first row) Not all expected m
 INFO   : tst_Warnings::testMissingWarningsWithData(second row) Did not receive message: "Warning0"
 INFO   : tst_Warnings::testMissingWarningsWithData(second row) Did not receive message: "Warning1"
 FAIL!  : tst_Warnings::testMissingWarningsWithData(second row) Not all expected messages were received
+QWARN  : tst_Warnings::testFailOnWarnings() Ran out of space!
+FAIL!  : tst_Warnings::testFailOnWarnings() Received a warning that resulted in a failure:
+Ran out of cabbage!
+   Loc: [qtbase/tests/auto/testlib/selftests/warnings/tst_warnings.cpp(0)]
+QDEBUG : tst_Warnings::testFailOnWarnings() Ran out of tortillas!
+QINFO  : tst_Warnings::testFailOnWarnings() Ran out of oil!
+QWARN  : tst_Warnings::testFailOnWarnings() nope
+FAIL!  : tst_Warnings::testFailOnWarnings() Received a warning that resulted in a failure:
+Ran out of biscuits!
+   Loc: [qtbase/tests/auto/testlib/selftests/warnings/tst_warnings.cpp(0)]
+FAIL!  : tst_Warnings::testFailOnWarnings() Received a warning that resulted in a failure:
+Running low on toothpaste!
+   Loc: [qtbase/tests/auto/testlib/selftests/warnings/tst_warnings.cpp(0)]
+QWARN  : tst_Warnings::testFailOnWarnings() Running low on flour!
+QDEBUG : tst_Warnings::testFailOnWarnings() Running low on toothpaste!
+QINFO  : tst_Warnings::testFailOnWarnings() Running low on toothpaste!
+QWARN  : tst_Warnings::testFailOnWarningsCleared() Ran out of muffins!
+PASS   : tst_Warnings::testFailOnWarningsCleared()
+FAIL!  : tst_Warnings::testFailOnWarningsWithData(warning1) Received a warning that resulted in a failure:
+warning1
+   Loc: [qtbase/tests/auto/testlib/selftests/warnings/tst_warnings.cpp(0)]
+QWARN  : tst_Warnings::testFailOnWarningsWithData(warning1) warning2
+QWARN  : tst_Warnings::testFailOnWarningsWithData(warning1) warning3
+QWARN  : tst_Warnings::testFailOnWarningsWithData(warning2) warning1
+FAIL!  : tst_Warnings::testFailOnWarningsWithData(warning2) Received a warning that resulted in a failure:
+warning2
+   Loc: [qtbase/tests/auto/testlib/selftests/warnings/tst_warnings.cpp(0)]
+QWARN  : tst_Warnings::testFailOnWarningsWithData(warning2) warning3
+QWARN  : tst_Warnings::testFailOnWarningsWithData(warning3) warning1
+QWARN  : tst_Warnings::testFailOnWarningsWithData(warning3) warning2
+FAIL!  : tst_Warnings::testFailOnWarningsWithData(warning3) Received a warning that resulted in a failure:
+warning3
+   Loc: [qtbase/tests/auto/testlib/selftests/warnings/tst_warnings.cpp(0)]
+FAIL!  : tst_Warnings::testFailOnWarningsFailInHelper() This failure message should be printed but not cause the test to abort
+   Loc: [qtbase/tests/auto/testlib/selftests/warnings/tst_warnings.cpp(0)]
+FAIL!  : tst_Warnings::testFailOnWarningsFailInHelper() Received a warning that resulted in a failure:
+Ran out of cabbage!
+   Loc: [qtbase/tests/auto/testlib/selftests/warnings/tst_warnings.cpp(0)]
+FAIL!  : tst_Warnings::testFailOnWarningsFailInHelper() My cabbage! :(
+   Loc: [qtbase/tests/auto/testlib/selftests/warnings/tst_warnings.cpp(0)]
+FAIL!  : tst_Warnings::testFailOnWarningsThenSkip() Received a warning that resulted in a failure:
+Ran out of cabbage!
+   Loc: [qtbase/tests/auto/testlib/selftests/warnings/tst_warnings.cpp(0)]
+SKIP   : tst_Warnings::testFailOnWarningsThenSkip() My cabbage! :(
+   Loc: [qtbase/tests/auto/testlib/selftests/warnings/tst_warnings.cpp(0)]
+PASS   : tst_Warnings::testFailOnWarningsAndIgnoreWarnings()
 PASS   : tst_Warnings::cleanupTestCase()
-Totals: 3 passed, 4 failed, 0 skipped, 0 blacklisted, 0ms
+Totals: 5 passed, 14 failed, 1 skipped, 0 blacklisted, 0ms
 ********* Finished testing of tst_Warnings *********