TAP test logger: move messages into the diagnostics block

Our TAP output was delivering messages as comments before the test
line, where TAP clearly expects the details of a test to follow its
test line. Version 13 provides a YAML block to deliver diagnostics and
encourages use of this, so accumulate our messages in a
QTestCharBuffer instead of emitting them one by one.

However, messages produced after a test has produced its test line
belong to that test, but are too late to be included in its
diagnostics block, so should be emitted immediately as before, albeit
now with a type prefix. This at least separates such messages, from
the end of one test, from messages produced early in the next.

In the process, add a type-prefix to each, to make clear what type of
message it was. Since the Yamlish supported by TAP consumers doesn't
support a way to have many messages, use the extensions: top-level
hash tag with a messages: sub-tag to gather our messages as a list.
(This expands at least one expected output file significantly and
substantially rewrites some others.)

Add methods to QTestCharBuffer, and a helper function, to support this.

Task-number: QTBUG-96844
Change-Id: If44a33da5879ed1670ef0980042599afd516f9d2
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>

1 files changed, 122 insertions, 36 deletions

diff --git a/tests/auto/testlib/selftests/expected_globaldata.tap b/tests/auto/testlib/selftests/expected_globaldata.tap
index 4d61b7437b..f0e71674a2 100644
--- a/tests/auto/testlib/selftests/expected_globaldata.tap
+++ b/tests/auto/testlib/selftests/expected_globaldata.tap
@@ -1,56 +1,142 @@
 TAP version 13
 # tst_globaldata
-# initTestCase initTestCase (null)
 ok 1 - initTestCase()
-# init testGlobal local=false
-# global: false
-# local: false
-# cleanup testGlobal local=false
+  ---
+  extensions:
+    messages:
+    - severity: debug
+      message: initTestCase initTestCase (null)
+  ...
 ok 2 - testGlobal(global=false:local=false)
-# init testGlobal local=true
-# global: false
-# local: true
-# cleanup testGlobal local=true
+  ---
+  extensions:
+    messages:
+    - severity: debug
+      message: init testGlobal local=false
+    - severity: debug
+      message: global: false
+    - severity: debug
+      message: local: false
+    - severity: debug
+      message: cleanup testGlobal local=false
+  ...
 ok 3 - testGlobal(global=false:local=true)
-# init testGlobal local=false
-# global: true
-# local: false
-# cleanup testGlobal local=false
+  ---
+  extensions:
+    messages:
+    - severity: debug
+      message: init testGlobal local=true
+    - severity: debug
+      message: global: false
+    - severity: debug
+      message: local: true
+    - severity: debug
+      message: cleanup testGlobal local=true
+  ...
 ok 4 - testGlobal(global=true:local=false)
-# init testGlobal local=true
-# global: true
-# local: true
-# cleanup testGlobal local=true
+  ---
+  extensions:
+    messages:
+    - severity: debug
+      message: init testGlobal local=false
+    - severity: debug
+      message: global: true
+    - severity: debug
+      message: local: false
+    - severity: debug
+      message: cleanup testGlobal local=false
+  ...
 ok 5 - testGlobal(global=true:local=true)
+  ---
+  extensions:
+    messages:
+    - severity: debug
+      message: init testGlobal local=true
+    - severity: debug
+      message: global: true
+    - severity: debug
+      message: local: true
+    - severity: debug
+      message: cleanup testGlobal local=true
+  ...
 ok 6 - skip(global=false) # SKIP skipping
-# init skipLocal local=false
 ok 7 - skipLocal(global=false:local=false) # SKIP skipping
-# cleanup skipLocal local=false
-# init skipLocal local=true
+  ---
+  extensions:
+    messages:
+    - severity: debug
+      message: init skipLocal local=false
+  ...
+# debug: cleanup skipLocal local=false
 ok 8 - skipLocal(global=false:local=true) # SKIP skipping
-# cleanup skipLocal local=true
-# init skipLocal local=false
+  ---
+  extensions:
+    messages:
+    - severity: debug
+      message: init skipLocal local=true
+  ...
+# debug: cleanup skipLocal local=true
 ok 9 - skipLocal(global=true:local=false) # SKIP skipping
-# cleanup skipLocal local=false
-# init skipLocal local=true
+  ---
+  extensions:
+    messages:
+    - severity: debug
+      message: init skipLocal local=false
+  ...
+# debug: cleanup skipLocal local=false
 ok 10 - skipLocal(global=true:local=true) # SKIP skipping
-# cleanup skipLocal local=true
-# init skipSingle local=false
-# global: false local: false
-# cleanup skipSingle local=false
+  ---
+  extensions:
+    messages:
+    - severity: debug
+      message: init skipLocal local=true
+  ...
+# debug: cleanup skipLocal local=true
 ok 11 - skipSingle(global=false:local=false)
-# init skipSingle local=true
+  ---
+  extensions:
+    messages:
+    - severity: debug
+      message: init skipSingle local=false
+    - severity: debug
+      message: global: false local: false
+    - severity: debug
+      message: cleanup skipSingle local=false
+  ...
 ok 12 - skipSingle(global=false:local=true) # SKIP Skipping
-# cleanup skipSingle local=true
-# init skipSingle local=false
+  ---
+  extensions:
+    messages:
+    - severity: debug
+      message: init skipSingle local=true
+  ...
+# debug: cleanup skipSingle local=true
 ok 13 - skipSingle(global=true:local=false) # SKIP Skipping
-# cleanup skipSingle local=false
-# init skipSingle local=true
-# global: true local: true
-# cleanup skipSingle local=true
+  ---
+  extensions:
+    messages:
+    - severity: debug
+      message: init skipSingle local=false
+  ...
+# debug: cleanup skipSingle local=false
 ok 14 - skipSingle(global=true:local=true)
-# cleanupTestCase cleanupTestCase (null)
+  ---
+  extensions:
+    messages:
+    - severity: debug
+      message: init skipSingle local=true
+    - severity: debug
+      message: global: true local: true
+    - severity: debug
+      message: cleanup skipSingle local=true
+  ...
 ok 15 - cleanupTestCase()
+  ---
+  extensions:
+    messages:
+    - severity: debug
+      message: cleanupTestCase cleanupTestCase (null)
+  ...
 1..15
 # tests 15
 # pass 8