testlib: Add Test Anything Protocol (TAP) reporter

The Test Anything Protocol (TAP), was originally Perl's simple text-based
interface between testing modules and test harnesses, but has since been
adopted by a large number of producers and consumers in many different
languages, which allows colorizing and summarizing test results.

The format is very simple:

TAP version 13
ok 1 - test description
not ok 2 - test description
  ---
  message: 'Failure message'
  severity: fail
  expected: 123
  actual: 456
  ...
ok 3 - test description # SKIP
1..3

The specification [1] is very brief, so the implementation has been
based on how typical consumers behave, especially when it comes to
the undefined diagnostics block.

[1] http://testanything.org/tap-version-13-specification.html

Change-Id: I616e802ea380165c678510e940ddc6607d39c92d
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>

1 files changed, 53 insertions, 0 deletions

diff --git a/tests/auto/testlib/selftests/expected_verifyexceptionthrown.tap b/tests/auto/testlib/selftests/expected_verifyexceptionthrown.tap
new file mode 100644
index 0000000000..2b04c08cdb
--- /dev/null
+++ b/tests/auto/testlib/selftests/expected_verifyexceptionthrown.tap
@@ -0,0 +1,53 @@
+TAP version 13
+# tst_VerifyExceptionThrown
+ok 1 - initTestCase()
+ok 2 - testCorrectStdTypes()
+ok 3 - testCorrectStdExceptions()
+ok 4 - testCorrectMyExceptions()
+not ok 5 - testFailInt()
+  ---
+  # Expected exception of type double to be thrown but unknown exception caught
+  at: tst_VerifyExceptionThrown::testFailInt() (qtbase/tests/auto/testlib/selftests/verifyexceptionthrown/tst_verifyexceptionthrown.cpp:115)
+  file: qtbase/tests/auto/testlib/selftests/verifyexceptionthrown/tst_verifyexceptionthrown.cpp
+  line: 115
+  ...
+not ok 6 - testFailStdString()
+  ---
+  # Expected exception of type char* to be thrown but unknown exception caught
+  at: tst_VerifyExceptionThrown::testFailStdString() (qtbase/tests/auto/testlib/selftests/verifyexceptionthrown/tst_verifyexceptionthrown.cpp:120)
+  file: qtbase/tests/auto/testlib/selftests/verifyexceptionthrown/tst_verifyexceptionthrown.cpp
+  line: 120
+  ...
+not ok 7 - testFailStdRuntimeError()
+  ---
+  # Expected exception of type std::runtime_error to be thrown but std::exception caught with message: logic error
+  at: tst_VerifyExceptionThrown::testFailStdRuntimeError() (qtbase/tests/auto/testlib/selftests/verifyexceptionthrown/tst_verifyexceptionthrown.cpp:125)
+  file: qtbase/tests/auto/testlib/selftests/verifyexceptionthrown/tst_verifyexceptionthrown.cpp
+  line: 125
+  ...
+not ok 8 - testFailMyException()
+  ---
+  # Expected exception of type MyBaseException to be thrown but std::exception caught with message: logic error
+  at: tst_VerifyExceptionThrown::testFailMyException() (qtbase/tests/auto/testlib/selftests/verifyexceptionthrown/tst_verifyexceptionthrown.cpp:130)
+  file: qtbase/tests/auto/testlib/selftests/verifyexceptionthrown/tst_verifyexceptionthrown.cpp
+  line: 130
+  ...
+not ok 9 - testFailMyDerivedException()
+  ---
+  # Expected exception of type std::runtime_error to be thrown but std::exception caught with message: MyDerivedException
+  at: tst_VerifyExceptionThrown::testFailMyDerivedException() (qtbase/tests/auto/testlib/selftests/verifyexceptionthrown/tst_verifyexceptionthrown.cpp:135)
+  file: qtbase/tests/auto/testlib/selftests/verifyexceptionthrown/tst_verifyexceptionthrown.cpp
+  line: 135
+  ...
+not ok 10 - testFailNoException()
+  ---
+  # Expected exception of type std::exception to be thrown but no exception caught
+  at: tst_VerifyExceptionThrown::testFailNoException() (qtbase/tests/auto/testlib/selftests/verifyexceptionthrown/tst_verifyexceptionthrown.cpp:140)
+  file: qtbase/tests/auto/testlib/selftests/verifyexceptionthrown/tst_verifyexceptionthrown.cpp
+  line: 140
+  ...
+ok 11 - cleanupTestCase()
+1..11
+# tests 11
+# pass 5
+# fail 6