| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We need to be able to handle warnings that may occur when temporary
objects, that were created in a test function, are destroyed.
For example, now we miss all warnings that might be triggered from
the object destructor, if the object's deletion was delayed (e.g. via
deleteLater()). Also we miss all the warnings that were triggered on
the test's cleanup() call.
To fix this we need simply move QTestLog::clearFailOnWarnings()
from QTestResult::finishedCurrentTestData() to the later stage,
i.e. into QTestLog::clearCurrentTestState() which is actually called
in appropriate time from QTestResult::finishedCurrentTestDataCleanup().
Same for QTestLog::clearIgnoreMessages(), since they are interrelated,
so we need to clear them at the same time.
We need this change for QML tests in particularly, to be able
fail on warnings that might be triggered from Component.onDestruction()
of some temporary test object.
Pick-to: 6.6 6.5
Change-Id: I58a57691f20761619f56bd1bea3a862f2c26f569
Reviewed-by: Mitch Curtis <mitch.curtis@qt.io>
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Added tests for repeated skips and failures (from within void lambdas,
to simulate skips and failures from within event handlers). These
exhibit yet more ways to count more than one outcome for a test. The
new QTest::failOnWarning() can also provoke more than one failure from
a single test, and several existing selftests exhibited various ways
for the Totals line's counts to add up to more than the number of
actual tests run.
Fixed counting so that only the first decisive incident is counted.
Tests can still report later failure or skipping, but only the first
is counted.
Added a currentTestState in qtestlog.cpp, by which it keeps track of
whether the test has resolved to a result, and clearCurrentTestState()
by which other code can reset that at the end of each test. This
brought to light various places where test-end clean-up was not being
handled - due to failure or skipping in a *_data() method or init, or
a skip in cleanup.
Fixes: QTBUG-95661
Change-Id: I5d24a37a53d3db225fa602649d8aad8f5ed6c1ad
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
This solves the long-standing problem of not being able to easily
fail a test when a certain warning is output.
[ChangeLog][QtTest] Added QTest::failOnWarning. When called in a test
function, any warning that matches the given pattern will cause a test
failure. The test will continue execution when a failure is added.
All patterns are cleared at the end of each test function.
Fixes: QTBUG-70029
Change-Id: I5763f8d4acf1cee8178be43a503619fbfb0f4f36
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We try our best to pass on the file location of a failure, including for
fatal errors, but the reporting or logging machinery should not assume
there is one.
By passing on nullptr for the file location we allow the logging backends
to decide how to handle the situation, e.g. by not emitting extra fields
for failure location.
This effectively reverts c25687fa0b6e4be043e1f8c92c093d8b06ca06c4,
in favor of relying on the backends to cope with null filename,
which they already did.
As qFatal uses QMessageLogger, which by default disables file/line
information in release builds, we need to explicitly enable this in
our self-tests, to get uniform test results. Similarly, we disable
file/line info from testlib itself, as reporting Qt internal file
and line information for user diagnostics is less useful. The odd
one out there is qtestdata.cpp, which still ends up in test output
due to using QTEST_ASSERT instead of qFatal for its diagnostics.
Cleaning up that, and unifying how we report testlib issues to the
user, is left for another day.
Pick-to: 6.2
Change-Id: Ib9451b8eed86fe3ade4a4dcaf0037e1a3450321c
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
|
| |
|
|
|
|
|
|
| |
addBFail() asserts on the file being non-null. The convention seems to
be "Unknown File" for cases where we cannot determine the file.
Change-Id: I3a4d0130352d77d75f264fad6f3bd47c6700ef4c
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In selftests.qrc, imposed alphabetic order (on stem of name, then on
suffix, effectively treating . as sorting before any letter) while
removing old tests and adding new tests and data. Updated all non-csv
files and added many missing files. (Not clear on csv support status;
the script seems to have dropped it after 5.6, but the test still uses
it.)
Left expected_crashes* alone (no new files added, no update to old) as
I don't get results resembling those anticipated.
Omitted printdatatagswithglobaltags, printdatatags due to dangling
hspace on output lines, which upset sanity-bot. A change to the test
cpp is needed to make it viable to skip that dangling hspace.
Change-Id: Iab3fb626c44a91c249b2fb626c12c75ea0317098
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add an 'info' message type that can be used for messages that are neither
warnings (QtWarningMsg), nor for debugging only (QtDebugMsg). This is
useful mainly for applications that do not have to adhere to the
'do not print anything by default' paradigm that we have for
the Qt libraries itself.
[ChangeLog][QtCore][Logging] QtInfoMsg got added as a new QtMsgType.
Use the new qInfo(), qCInfo() macros to log to it.
Change-Id: I810995d63de46c41a9a99a34d37c0d417fa87a05
Reviewed-by: Jason McDonald <macadder1@gmail.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We need to have a finer grained control over the tests
we skip in our CI system. This adds a blacklisting
mechanism that allows blacklisting individual test
functions (or even test data) using a set of predefined
matching keys for the operating system and some other
relevant variables.
QTestlib will search for a file called BLACKLIST in the test
directory and parse it if found. The file contains a simple
ini style list of functions to blacklist. For details see
qtestblacklist.cpp.
Change-Id: Id3fae4b264ca99970cbf9f45bfb85fa75c1fd823
Reviewed-by: Lars Knoll <lars.knoll@digia.com>
|
| |
|
|
|
|
|
| |
Make it possible to match messages by a pattern.
Change-Id: I713312e86db5471755459f1ecc43e8f1ac7a95fb
Reviewed-by: Jason McDonald <macadder1@gmail.com>
|
| |
|
|
|
|
|
|
|
|
|
| |
"QTest" is the C++ namespace; "QtTest" is the library name
- Edited the logger output in qplaintestlogger.cpp
- Updated documentation
- Updated expected outputs for self-tests
Change-Id: I43c525c43221a8d4e843a00d6d55b0f06ef55fd7
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Previously, ignored messages were only cleared at the end of each test
function, i.e. after all data rows were finished. This meant that if a
data row in a data-driven test function didn't cause all of the expected
messages to be generated, the remaining messages would be carried over
to the next data row. This would result in errors about missing
messages being associated with the last data row rather than with the
correct data row.
This commit makes testlib check for missing ignored messages after
running each data row rather than only doing so after the last data
row.
This commit also adds a regression test to demonstrate that ignored
messages can no longer be carried over from one data row to another.
Change-Id: Ibee51aa6e96866fbcbcb4acee1a8340a86a6a4ba
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
The selftests already used the @INSERT_QT_VERSION_HERE@ placeholder
in place of the real Qt and QTestlib versions in the expected test
output of the subtests. For unknown reasons, the same was not true for
the expected plain text output. In the past, this has caused Release
Managers to waste time incrementing the version numbers in these files.
Change-Id: I52f7870486fce128c04d53ff06978afa947474fd
Reviewed-on: http://codereview.qt-project.org/5375
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The warnings selftest verifies that qtestlib correctly handles output
sent via qDebug() and friends. The test had a number of printf calls
that were used to tell the user what output they should expect to see
next, but by bypassing testlib's logging, these made the testlog differ
depending on whether the test log was directed to the console or a file.
The printf calls are no longer needed, as any regressions in qDebug will
be detected by tst_selftests, which compares the output of the warnings
test with a reference copy of the expected output.
Change-Id: I8c3a3237463141fa494c50ca02062760cb583090
Reviewed-on: http://codereview.qt-project.org/5200
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
|
|
|
Task-number: QTBUG-21353
Change-Id: I2e2ad4b7570f80b3e6500821f769e967f0a75027
Reviewed-on: http://codereview.qt-project.org/4438
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
|
