testlib: Add Test Anything Protocol (TAP) reporter

The Test Anything Protocol (TAP), was originally Perl's simple text-based
interface between testing modules and test harnesses, but has since been
adopted by a large number of producers and consumers in many different
languages, which allows colorizing and summarizing test results.

The format is very simple:

TAP version 13
ok 1 - test description
not ok 2 - test description
  ---
  message: 'Failure message'
  severity: fail
  expected: 123
  actual: 456
  ...
ok 3 - test description # SKIP
1..3

The specification [1] is very brief, so the implementation has been
based on how typical consumers behave, especially when it comes to
the undefined diagnostics block.

[1] http://testanything.org/tap-version-13-specification.html

Change-Id: I616e802ea380165c678510e940ddc6607d39c92d
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>

1 files changed, 68 insertions, 0 deletions

diff --git a/tests/auto/testlib/selftests/expected_subtest.tap b/tests/auto/testlib/selftests/expected_subtest.tap
new file mode 100644
index 0000000000..67fe7be570
--- /dev/null
+++ b/tests/auto/testlib/selftests/expected_subtest.tap
@@ -0,0 +1,68 @@
+TAP version 13
+# tst_Subtest
+# initTestCase initTestCase (null)
+ok 1 - initTestCase()
+# init test1 (null)
+# test1 test1 (null)
+# cleanup test1 (null)
+ok 2 - test1()
+# test2_data test2 (null)
+# test2_data end
+# init test2 data0
+# test2 test2 data0
+# test2 end
+# cleanup test2 data0
+ok 3 - test2(data0)
+# init test2 data1
+# test2 test2 data1
+# test2 end
+# cleanup test2 data1
+ok 4 - test2(data1)
+# init test2 data2
+# test2 test2 data2
+# test2 end
+# cleanup test2 data2
+ok 5 - test2(data2)
+# test3_data test3 (null)
+# test3_data end
+# init test3 data0
+# test2 test3 data0
+# test2 end
+# cleanup test3 data0
+ok 6 - test3(data0)
+# init test3 data1
+# test2 test3 data1
+not ok 7 - test3(data1)
+  ---
+  type: QCOMPARE
+  message: Compared values are not the same
+  wanted: "hello0" (QString("hello0"))
+  found: "hello1" (str)
+  expected: "hello0" (QString("hello0"))
+  actual: "hello1" (str)
+  at: tst_Subtest::test3() (qtbase/tests/auto/testlib/selftests/subtest/tst_subtest.cpp:141)
+  file: qtbase/tests/auto/testlib/selftests/subtest/tst_subtest.cpp
+  line: 141
+  ...
+# cleanup test3 data1
+# init test3 data2
+# test2 test3 data2
+not ok 8 - test3(data2)
+  ---
+  type: QCOMPARE
+  message: Compared values are not the same
+  wanted: "hello0" (QString("hello0"))
+  found: "hello2" (str)
+  expected: "hello0" (QString("hello0"))
+  actual: "hello2" (str)
+  at: tst_Subtest::test3() (qtbase/tests/auto/testlib/selftests/subtest/tst_subtest.cpp:141)
+  file: qtbase/tests/auto/testlib/selftests/subtest/tst_subtest.cpp
+  line: 141
+  ...
+# cleanup test3 data2
+# cleanupTestCase cleanupTestCase (null)
+ok 9 - cleanupTestCase()
+1..9
+# tests 9
+# pass 7
+# fail 2