diff options
| author | Jason McDonald <jason.mcdonald@nokia.com> | 2012-01-31 14:07:58 +1000 |
|---|---|---|
| committer | Qt by Nokia <qt-info@nokia.com> | 2012-02-02 05:42:07 +0100 |
| commit | 58c617c798c0d0db244735451acd316306068491 (patch) | |
| tree | 6cc2ee72fc0f0802d3034bb6573a1d8fb841f6e1 /tests/auto/testlib/selftests/expected_counting.xunitxml | |
| parent | 835c53490b4fa9a248e6e4ca56965f9f7276da4d (diff) | |
Add testlib selftest for counting of test results.
The test simply generates each possible pair of Pass, Fail and Skip
results. At present the test simply serves to demonstrate the current
shortcomings of testlib's plain text logging, namely:
* If a test function passes for all data rows, that is counted as one
pass, but each skipped or failed row counts as one skip or fail.
* Only skipped and failed rows are reported individually in the test
output. Passed rows are not reported, so it is impossible to see how
many rows were executed.
* A skip followed by a pass will be reported as an overall pass for the
test function, but the same rows in reverse order will not report any
overall result for the test function.
Future commits will attempt to correct these problems.
Task-number: QTBUG-22124
Change-Id: If8c7ea15fc43ba9a1bccd0e881c1efc18e705b25
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Diffstat (limited to 'tests/auto/testlib/selftests/expected_counting.xunitxml')
| -rw-r--r-- | tests/auto/testlib/selftests/expected_counting.xunitxml | 46 |
1 files changed, 46 insertions, 0 deletions
diff --git a/tests/auto/testlib/selftests/expected_counting.xunitxml b/tests/auto/testlib/selftests/expected_counting.xunitxml new file mode 100644 index 0000000000..c19a1612cb --- /dev/null +++ b/tests/auto/testlib/selftests/expected_counting.xunitxml @@ -0,0 +1,46 @@ +<?xml version="1.0" encoding="UTF-8" ?> +<testsuite errors="6" failures="6" tests="11" name="tst_Counting"> + <properties> + <property value="@INSERT_QT_VERSION_HERE@" name="QTestVersion"/> + <property value="@INSERT_QT_VERSION_HERE@" name="QtVersion"/> + </properties> + <testcase result="pass" name="initTestCase"/> + <testcase result="pass" name="testPassPass"/> + <testcase name="testPassSkip"> + <!-- tag="row 2" message="Skipping" type="skip" --> + </testcase> + <testcase result="fail" name="testPassFail"> + <failure tag="row 2" message="'false' returned FALSE. ()" result="fail"/> + </testcase> + <testcase result="pass" name="testSkipPass"> + <!-- tag="row 1" message="Skipping" type="skip" --> + </testcase> + <testcase name="testSkipSkip"> + <!-- tag="row 1" message="Skipping" type="skip" --> + <!-- tag="row 2" message="Skipping" type="skip" --> + </testcase> + <testcase result="fail" name="testSkipFail"> + <!-- tag="row 1" message="Skipping" type="skip" --> + <failure tag="row 2" message="'false' returned FALSE. ()" result="fail"/> + </testcase> + <testcase result="fail" name="testFailPass"> + <failure tag="row 1" message="'false' returned FALSE. ()" result="fail"/> + </testcase> + <testcase result="fail" name="testFailSkip"> + <failure tag="row 1" message="'false' returned FALSE. ()" result="fail"/> + <!-- tag="row 2" message="Skipping" type="skip" --> + </testcase> + <testcase result="fail" name="testFailFail"> + <failure tag="row 1" message="'false' returned FALSE. ()" result="fail"/> + <failure tag="row 2" message="'false' returned FALSE. ()" result="fail"/> + </testcase> + <testcase result="pass" name="cleanupTestCase"/> + <system-err> +<![CDATA[Skipping]]> +<![CDATA[Skipping]]> +<![CDATA[Skipping]]> +<![CDATA[Skipping]]> +<![CDATA[Skipping]]> +<![CDATA[Skipping]]> + </system-err> +</testsuite> |