| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
| |
There used to be an examples page; this is no longer the case. Thus,
remove the \note that indicates you can access the source code.
Fixes: QTBUG-122166
Pick-to: 6.7 6.6 6.5
Change-Id: I1abe88000ae406e8d2cc9cc1deed42664607626b
Reviewed-by: Topi Reiniö <topi.reinio@qt.io>
|
|
|
|
|
| |
Change-Id: I7865afbdaed266edfb54866cddacc6d6e5ab725c
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add exception classes and use them to change the control flow for
QTEST_{FAIL,SKIP}_ACTION from return'ing from just the immediate
function to the full way to the QTestLib infrastructure, here we
filter them out.
There are three modes:
- If QT_NO_EXCEPTION, then we return
- If QTEST_THROW_ON_... is also defined, #error out
- Otherwise, if QTEST_THROW_ON_... is defined, always throw
- Otherwise, the decision is made at runtime (with defaults read from
QTEST_THROW_ON_... environment variables).
Three selftests depend on the old behavior, as they explicitly check
that multiple FAIL SKIP etc emerge, which the new framework, of
course, prevents. Locally disable throwing at the test function level.
Add initial docs and enable exceptions in all of the selftest
subprograms to facilitate switching between the two runtime-selectable
modes.
[ChangeLog][QtTest] Added QTEST_THROW_ON_FAIL and QTEST_THROW_ON_SKIP
C++ macros and environment variables that, when defined, change how
QCOMPARE/QVERIFY/QSKIP etc exit the test function on failure. Instead
of a return, exiting only the immediately-surrounding function, they
throw a special exception instead, thereby exiting from subfunctions
of the test function, all the way to QtTestLib.
Fixes: QTBUG-66320
Change-Id: I96c38d2a1dcdd9de84942cf448a8bbf3ab6d3679
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add a command-line option "-skipblacklisted" to testlib to skip
blacklisted test cases. Currently, blacklisted test cases are run,
but results are ignored. For test code coverage measurements, it's
important to see the code coverage of the actually tested code in
comparison to the code that was run but not actually tested.
The default approach remains unchanged, meaning that blacklisted
tests are run with the results ignored.
Fixes: QTBUG-112793
Change-Id: I6fe0a6353cb1c021e0232c79bb4f404632fb0bce
Reviewed-by: Jason McDonald <macadder1@gmail.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Repeated test execution can be useful, under a debugger, to catch an
intermittent failure or, under memory instrumentation, to make memory
leaks easier to recognize.
The new -repeat flag allows running the entire test suite multiple times
within the same process. It works by executing all tests sequentially
before repeating the execution again.
This switch is a developer tool, and is not intended for CI. It can only
be used with the plain text logger.
Change-Id: I2439462c5c44d1c8aa3d3b5656de3eef44898c68
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
These pages are designed as tutorials, so
they can be \page instead of \example.
Also, reorganized the tutorials, moving them out
of the testlib manual, into several qdoc files.
Task-number: QTBUG-115248
Pick-to: 6.5 6.6
Change-Id: I2cbd66ecc1082ecc9d3d1742b621ee009daf1031
Reviewed-by: Kai Köhne <kai.koehne@qt.io>
Reviewed-by: Topi Reiniö <topi.reinio@qt.io>
|
|
|
|
|
|
| |
Pick-to: 6.6 6.5
Change-Id: I71c38e40db9ff222016ed24a43f646ceef749180
Reviewed-by: Leena Miettinen <riitta-leena.miettinen@qt.io>
|
|
|
|
|
|
|
|
|
|
|
| |
Mention addRow() as well as newRow(), even though the example only
uses the latter. Link the best-practice section to the fuller story.
Fixed a minor grammar glitch in the manual while I was about it.
Pick-to: 6.5
Change-Id: Ib1c52cd8d2b6a04ea944d24d9d26c901b6cdf4e7
Reviewed-by: Mitch Curtis <mitch.curtis@qt.io>
|
|
|
|
|
|
|
|
|
|
|
| |
It previously only covered ignoring them; we can now turn them into
errors. Expand the section to include some motivation for why it is
best practice to purge warnings from test output. That prepares the
scene for how to suppress what's expected and make errors of the rest.
Pick-to: 6.5
Change-Id: Ieb4b14b2b048d1db2fead004ee7471b82507722f
Reviewed-by: Mitch Curtis <mitch.curtis@qt.io>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Tero Heikkinen caught tst_QQuaternion() using a duplicated test data
tag and was surprised that testlib let it get away with that. That
seems like a reasonable thing to discourage. While I'm at it,
duplicate columns should be discouraged.
[ChangeLog][QtTest] Duplicate data tags are now warned about. Every
call to QTest::newRow() or QTest::addRow() should result in a distinct
data tag. If they do not, a warning is produced. Likewise, duplicate
column names are forbidden; each call to QTest::addColumn() should use
a distinct name.
Fixes: QTBUG-107185
Pick-to: 6.5
Change-Id: Idfdb7cdfdd71e1fe3db5cadb243eeecc83032922
Reviewed-by: Jason McDonald <macadder1@gmail.com>
|
|
|
|
|
|
|
|
|
| |
Documentation on QTEST_FUNCTION_TIMEOUT environment variable
Pick-to: 5.15 6.2 6.4 6.5
Fixes: QTBUG-88652
Change-Id: Ib851eb2312088cf6b9ab277faa571757f4076ad4
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
|
|
|
|
|
|
|
|
|
| |
qtestlib-tools is dead.
Fixes: QTBUG-107806
Pick-to: 6.4 6.2
Change-Id: Ic479b9e133c1889a046d1b7483af9d8ad8ac4968
Reviewed-by: Topi Reiniö <topi.reinio@qt.io>
|
|
|
|
|
|
|
|
|
| |
Also follow this best practice in testlib's own documentation and
examples.
Pick-to: 6.4
Change-Id: I8b57dfa8f88835adae8fceeb122a16635708e338
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A fragment of inline code wasn't wrapped in \c{}.
Doing so made the quotes it was wrapped in redundant.
Documentation of of QSKIP's parameter claimed it does something it
should, but it's up to the client code using it to write a description
that lives up to that.
Pick-to: 6.4
Change-Id: Ib355b9b85c9986f244ae01479134245f182c912c
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
|
|
|
|
|
|
|
|
|
| |
Clarify that the hyphen is the actual character that one can use.
The way the documentation was written might have been misleading.
Pick-to: 6.4
Change-Id: I55a9ede9903a8ac5bd33d8e138b3a0a21a820406
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
|
|
|
|
|
|
|
|
|
|
|
| |
- Make it explicit that each chapter can be run as a stand-alone test
application
- Add a CMake section on how to build the executable
Task-number: QTBUG-103730
Pick-to: 6.4 6.3
Change-Id: Id4c87c454521f698dcf16cfdc176318dd3e16786
Reviewed-by: Kai Koehne <kai.koehne@qt.io>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Replace the current license disclaimer in files by
a SPDX-License-Identifier.
Files that have to be modified by hand are modified.
License files are organized under LICENSES directory.
Task-number: QTBUG-67283
Change-Id: Id880c92784c40f3bbde861c0d93f58151c18b9f1
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In the same ways as for blacklisting. In the process, move reporting
of an unrecognised tag outside the global-data loop, as the tag might
be, or start with, a global data tag; and also report the global data
alongside the test-specific data. Indent the lines reporting tags, as
this makes the structure of the lists more evident. Converted the tag
argument to QLatin1String() to facilitate searching and matching.
Updated documentation, including expanding on the option of specifying
the data tags on the command-line. Drive-by: fixed a link in the
documentation, removed a snippet that was nowhere used.
[ChangeLog][QtTest] When specifying what tests to run on a QtTest
command-line, it is now possible to include a global data tag in the
same way as a function-specific data tag or in combination with it.
Thus if the test reports itself as tst_Class::function(global:local),
command-line option function:global:local will select that specific
test and function:global will run all data-rows for the function on
the given global data tag.
Fixes: QTBUG-102269
Change-Id: I7c86d6026933b05f7994bfc3663a2ea6a47f5f38
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
|
|
|
|
|
|
|
|
|
| |
Reorder the different formats to match the order in the help output
and include missing entries from the list of formats supported by the
-o filename,format variant.
Change-Id: I884f9facc7cfde7cec0bee75944efdd72d66a83e
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
|
|
|
|
|
|
|
|
|
| |
Provide basic (internal) documentation of each logging class and link
the command-line documentation to pages relevant to the formats not
defined by Qt.
Change-Id: I3251dd1304203c6ab87dfe1f2dec0e9787ab69f8
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
|
|
|
|
|
|
| |
Pick-to: 6.2 5.15
Change-Id: I9bcff18ca11fbbfdff968e29190cae488de56263
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
|
|
|
|
|
|
|
|
|
| |
Present behavior is to continue or abort, according to the mode,
regardless of whether the condition marked does fail.
Change-Id: I2eb5eefbbf173326101d08f3bbb378d214e93e71
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When debugging a spurious failure it's extremely useful to
run the test repeadtly into a debugger until a failure appears.
When that happens, one wants to immediately start debugging.
In so far, this has only been possible by placing breakpoints
inside Qt itself (when a failure is logged). Add another way:
an env variable, similar to QT_FATAL_WARNINGS, that makes
any failure fatal (terminate() gets called. Bonus: you can
control the termination handler!)
[ChangeLog][QtTestLib][QtTest] When the QTEST_FATAL_FAIL
environment variable is set to a non-zero value, a test
immediately aborts its execution. This is useful to debug
intermittent failures.
Change-Id: If2395f964ea482c30b8c8feab98db7fdee701cd3
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
Reviewed-by: Jason McDonald <macadder1@gmail.com>
|
|
|
|
|
|
|
|
|
|
| |
The usual behavior for env variables is that any nonzero
value means "enable", so do the same for a couple of QTestLib
env vars. While at it: document them.
Change-Id: I854285df5d7af5be771f9d6532ddec5d8a6f8987
Pick-to: 6.0
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
|
|
|
|
|
|
| |
Task-number: QTBUG-86295
Change-Id: If3c48bee8c898a228128ade18fbdeaa2b8de8b20
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
|
|
|
|
|
|
| |
Task-number: QTBUG-86295
Change-Id: I27f6bbdadffb08a8794520a14dfe0e2334979575
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
|
|
|
|
|
|
| |
Change-Id: Ibadce68775858c524b998aacad310905ba2c2e8e
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
|
|
|
|
|
|
|
| |
Change-Id: If8044a339cab754d427fd7626dd6813c7cc99e56
Pick-to: 5.15 5.12
Fixes: QTBUG-85827
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
|
|
|
|
|
|
| |
Task-number: QTBUG-81498
Change-Id: I22f07cd539e5e317b6cf15eb369d59915146bd13
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
|
|
|
|
|
|
|
|
| |
The command is deprecated and has no effect apart from generating
a documentation warning.
Change-Id: I30871bfd6975f8268930cce99993a8579242fdb8
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The reporter was probably named 'xunit' based on the historical use of
xUnit to refer to testing frameworks derived from Smalltalk's SUnit.
These frameworks typically added their own prefix, e.g. JUnit for Java,
RUnit for R, etc.
The most popular of these was the JUnit framework, and the corresponding
XML output produced by the Ant built tool became somewhat of a de facto
standard, which is probably why we chose to model our reporter after it.
Nowadays however, naming it 'xunit' is problematic as there is actually
a testing famework named xUnit.net, typically shortened to, you guessed
it: xunit.
Test report consumers will typically have a junit mode, and an xunit
mode, and the latter could easily be mistaken for what testlib outputs,
unless we clarify this.
The clarification also allows us to safely extend our support for the
JUnit XML format to incorporate some elements that are nowadays common,
but where we are lagging behind the standard.
[ChangeLog][QTestLib] The formerly named 'xunitxml' test reporter has
been renamed to what it actually is: a JUnit test reporter, and is now
triggered by passing -o junitxml to the test binary.
Change-Id: Ieb20d3d2b5905c74e55b98174948cc70870c0ef9
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
|
|
|
|
|
|
| |
Task-number: QTBUG-79824
Change-Id: I05caaccab1efa16bc0a01ce3045a9377f9ef640e
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
|
|
|
|
|
|
|
| |
Task-number: QTBUG-63987
Change-Id: I4b6e8f35afc9d3ca10b393a0305bbb51bf81ec26
Reviewed-by: Christian Stenger <christian.stenger@qt.io>
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
|
|\
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Conflicts:
.qmake.conf
mkspecs/features/mac/default_post.prf
src/corelib/tools/qsimd_p.h
src/platformsupport/fontdatabases/mac/qcoretextfontdatabase.mm
util/qfloat16-tables/gen_qfloat16_tables.cpp
Change-Id: If48fa8a3bc3c983706b609a6d3822cb67c1352a4
|
| |
| |
| |
| |
| |
| |
| | |
From https://wiki.qt.io/Writing_Unit_Tests
Change-Id: Idc0bafb32690f443e1f49cd4f8efb653d3aa46a4
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
|
| |
| |
| |
| |
| |
| |
| | |
From https://wiki.qt.io/Writing_Unit_Tests
Change-Id: I20027066640ca797a2330f6daa81468f03921a69
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
|
|\|
| |
| |
| |
| |
| |
| |
| |
| | |
Conflicts:
src/corelib/io/qstandardpaths_unix.cpp
src/corelib/tools/qsharedpointer_impl.h
tests/auto/corelib/tools/qsharedpointer/tst_qsharedpointer.cpp
Change-Id: Iae95c5778dc091058f16f6db76f04a0178a9e809
|
| |
| |
| |
| |
| | |
Change-Id: I1f6947acec4494c151317e1faf79720dad0da6bb
Reviewed-by: Martin Smith <martin.smith@qt.io>
|
| |
| |
| |
| |
| |
| | |
Fixes: QTBUG-24342
Change-Id: I8f8f3726c5d31e34af9bfe054572c08fc07e01e0
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
|
|/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When running Qt autotests on a developer machine with a high
resolution, failures occur due to either some widget becoming too
small, some rounding fuzz appearing when Qt High DPI scaling is active,
or some test taking screenshots failing to deal with device pixel
ratios != 1 in the obtained pixmaps.
It is not feasible to adapt all tests to pass on high resolution
monitors in both modes (Qt High DPI scaling enabled/disabled). It
should be possible to specify the High DPI setting per test.
Previously, it was not possible to set the Qt High DPI scaling
attributes since they must be applied before QApplication
instantiation.
Enable this by checking for the presence of a static void initMain()
function on the test object and invoking it before QApplication
instantiation.
Prototypically use it in tst_qtimer and to turn off High DPI scaling for
tst_QGL.
[ChangeLog][QtTestLib] It is now possible to perform static
initialization before QApplication instantiation by implementing a
initMain() function in the test class.
Change-Id: Idec0134b189710a14c41a451fa8445bc0c5b1cf3
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
|
|\
| |
| |
| | |
Change-Id: I66c7f18a2abd13601da0947919436f7da3549ae9
|
| |
| |
| |
| |
| |
| | |
Change-Id: Icc7552b46a2657c81958e40f33596ddeee045172
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
Reviewed-by: Topi Reiniö <topi.reinio@qt.io>
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
The Test Anything Protocol (TAP), was originally Perl's simple text-based
interface between testing modules and test harnesses, but has since been
adopted by a large number of producers and consumers in many different
languages, which allows colorizing and summarizing test results.
The format is very simple:
TAP version 13
ok 1 - test description
not ok 2 - test description
---
message: 'Failure message'
severity: fail
expected: 123
actual: 456
...
ok 3 - test description # SKIP
1..3
The specification [1] is very brief, so the implementation has been
based on how typical consumers behave, especially when it comes to
the undefined diagnostics block.
[1] http://testanything.org/tap-version-13-specification.html
Change-Id: I616e802ea380165c678510e940ddc6607d39c92d
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
|
|\|
| |
| |
| | |
Change-Id: Ib21e6b9030b4d5363f440d082ce3df28098d1b95
|
| |
| |
| |
| |
| | |
Change-Id: Ib8cd1b4926652984b41b5a05bf4dbf3214f2856f
Reviewed-by: Topi Reiniö <topi.reinio@qt.io>
|
|/
|
|
|
| |
Change-Id: Iea32858a7ae5d7450682a18247a76830b42a4ae4
Reviewed-by: Nico Vertriest <nico.vertriest@qt.io>
|
|
|
|
|
|
|
|
| |
This is now documented in the qt_attribution.json file added in commit
41a7d74385eece725435159ca021151e871bf116 .
Change-Id: I4f13397a14de7d9f98ba146cabf64dafac5dada4
Reviewed-by: Topi Reiniö <topi.reinio@qt.io>
|
|
|
|
|
|
|
|
|
| |
The platform is not supported since Qt 5.7
Task-number: QTBUG-55331
Change-Id: If6202b347efdfb38eba4c4c3a65dde515d066112
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
Reviewed-by: Topi Reiniö <topi.reinio@theqtcompany.com>
|
|\
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Conflicts:
src/corelib/global/qglobal.cpp
src/corelib/io/qsettings.cpp
src/corelib/itemmodels/qstringlistmodel.cpp
tests/auto/gui/image/qimagewriter/tst_qimagewriter.cpp
Change-Id: I1c6c306ef42c3c0234b19907914b19da706b4a03
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
As of version 10.12 (Sierra), the name of Apple's desktop operating
system will be macOS. Change the occurrences where the Mac platform
is discussed to use a macro \macos, which expands to 'macOS'. This
helps with adapting to future renaming.
Update the instructions on mac-specific Q_OS_* macro usage.
Add a \target for the old 'Qt for OS X' topic to keep links working
for other documentation modules that try to link with the old name.
Change-Id: Id33fb0cd985df702a4ae4efb4c5fd428e77d9b85
Reviewed-by: Leena Miettinen <riitta-leena.miettinen@qt.io>
|