summaryrefslogtreecommitdiffstats
path: root/tests/auto/testlib/selftests/blacklisted
Commit message (Collapse)AuthorAgeFilesLines
* tests: Remove remains of qmake conversion from CMakeLists.txt filesFriedemann Kleint2023-02-171-6/+2
| | | | | | | Pick-to: 6.5 Change-Id: I8d106554bb86ac1ec9bb7a4083de4c376bcbab1d Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org> Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
* Change the license of all CMakeLists.txt and *.cmake files to BSDLucie Gérard2022-08-231-1/+1
| | | | | | | Task-number: QTBUG-105718 Change-Id: I5d3ef70a31235868b9be6cb479b7621bf2a8ba39 Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org> Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
* Add license headers to cmake filesLucie Gérard2022-08-031-0/+3
| | | | | | | | | | | | CMakeLists.txt and .cmake files of significant size (more than 2 lines according to our check in tst_license.pl) now have the copyright and license header. Existing copyright statements remain intact Task-number: QTBUG-88621 Change-Id: I3b98cdc55ead806ec81ce09af9271f9b95af97fa Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
* CMake: Don't use PUBLIC_LIBRARIES for tests and test helpersAlexandru Croitor2022-07-281-1/+1
| | | | | Change-Id: I9b7404e1d3a78fe0726ec0f5ce1461f6c209e90d Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
* Use SPDX license identifiersLucie Gérard2022-05-161-27/+2
| | | | | | | | | | | | | Replace the current license disclaimer in files by a SPDX-License-Identifier. Files that have to be modified by hand are modified. License files are organized under LICENSES directory. Task-number: QTBUG-67283 Change-Id: Id880c92784c40f3bbde861c0d93f58151c18b9f1 Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org> Reviewed-by: Lars Knoll <lars.knoll@qt.io> Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
* Make counts of various types of test result add up correctlyEdward Welbourne2022-01-132-2/+25
| | | | | | | | | | | | | | | | | | | | | | | | | Added tests for repeated skips and failures (from within void lambdas, to simulate skips and failures from within event handlers). These exhibit yet more ways to count more than one outcome for a test. The new QTest::failOnWarning() can also provoke more than one failure from a single test, and several existing selftests exhibited various ways for the Totals line's counts to add up to more than the number of actual tests run. Fixed counting so that only the first decisive incident is counted. Tests can still report later failure or skipping, but only the first is counted. Added a currentTestState in qtestlog.cpp, by which it keeps track of whether the test has resolved to a result, and clearCurrentTestState() by which other code can reset that at the end of each test. This brought to light various places where test-end clean-up was not being handled - due to failure or skipping in a *_data() method or init, or a skip in cleanup. Fixes: QTBUG-95661 Change-Id: I5d24a37a53d3db225fa602649d8aad8f5ed6c1ad Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
* Add more tests of blacklisting, combined with XPASS and XFAILEdward Welbourne2021-08-272-1/+76
| | | | | | | | | | Include counting of test types, to catch two more cases where totals don't add up. Task-number: QTBUG-95661 Change-Id: I9fe5424bc6652c61a065bf2889333e2ed9437c81 Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io> Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
* testlib: Clean up tst_blacklistTor Arne Vestbø2021-08-052-40/+2
| | | | | | | | | | | | | | | | | | | | | | | 33d7f76f0e847d7e0fb00dd6056e7bba45b8b1e7 attempted to exercise more of the blacklisting code, but in doing so didn't account for the fact that the features it added didn't actually work. Adding a global blacklisting of * will result in blacklisting all tests, regardless of any inverse conditions "!*" later on, such as for the 'pass' test. This would have been evident by looking at the expectation files, which still showed the test as a BPASS. The broken feature should be fixed, but in the meantime let's clean up the test so it doesn't misrepresent reality. At the same time we also remove the messages() tests, which was copied straight out of tst_silent, but doesn't serve any purpose for the blacklist selftest. Pick-to: 6.2 Change-Id: I20e4693300aad5e9ff5f17ad22e3dfcb5c49fd0e Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
* testlib: Deprecate QWARN() in favor of qWarning()Tor Arne Vestbø2021-08-041-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | The QtTest best practices documentations recommends using output mechanisms such as qDebug() and qWarning() for diagnostic messages, and this is also what most of our own tests do. The QWARN() macro and corresponding internal QTest::qWarn() function was added when QtTest was first implemented, but was likely meant as an internal implementation detail, like its cousin QTestLog::info(), which does not have any corresponding macro. This theory is backed by our own QtTest self-test (tst_silent) describing the output from QWARN() as "an internal testlib warning". The only difference between QWARN() and qWarning(), besides the much richer feature set of the latter, is that qWarning() will not pass on file and line number information in release mode, but QWARN() will. This is an acceptable loss of functionality, considering that the user can override this behavior by defining QT_MESSAGELOGCONTEXT. [ChangeLog][QtTest] QWARN() has been deprecated in favor of qWarning() Pick-to: 6.2 Change-Id: I5a2431ce48c47392244560dd520953b9fc735c85 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* Remove .prev_CMakeLists.txt filesJoerg Bornemann2021-01-121-15/+0
| | | | | | | | | | Those serve no purpose anymore, now that the .pro files are gone. Task-number: QTBUG-88742 Change-Id: I39943327b8c9871785b58e9973e4e7602371793e Reviewed-by: Cristian Adam <cristian.adam@qt.io> Reviewed-by: Edward Welbourne <edward.welbourne@qt.io> Reviewed-by: Kai Koehne <kai.koehne@qt.io>
* Remove the qmake project filesJoerg Bornemann2021-01-071-9/+0
| | | | | | | | | | | | | | | | Remove the qmake project files for most of Qt. Leave the qmake project files for examples, because we still test those in the CI to ensure qmake does not regress. Also leave the qmake project files for utils and other minor parts that lack CMake project files. Task-number: QTBUG-88742 Change-Id: I6cdf059e6204816f617f9624f3ea9822703f73cc Reviewed-by: Edward Welbourne <edward.welbourne@qt.io> Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org> Reviewed-by: Kai Koehne <kai.koehne@qt.io>
* Replace QtTest headers with QTestDavid Skoland2020-12-221-1/+1
| | | | | | | | | | | Complete search and replace of QtTest and QtTest/QtTest with QTest, as QtTest includes the whole module. Replace all such instances with correct header includes. See Jira task for more discussion. Fixes: QTBUG-88831 Change-Id: I981cfae18a1cabcabcabee376016b086d9d01f44 Pick-to: 6.0 Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
* CMake: Regenerate projects to use new qt_internal_ APIAlexandru Croitor2020-09-232-3/+3
| | | | | | | | | | | Modify special case locations to use the new API as well. Clean up some stale .prev files that are not needed anymore. Clean up some project files that are not used anymore. Task-number: QTBUG-86815 Change-Id: I9947da921f98686023c6bb053dfcc101851276b5 Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io> Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
* CMake: Regenerate tests with new qt_ prefixed APIsAlexandru Croitor2020-07-092-2/+2
| | | | | | | | Use pro2cmake with '--api-version 2' to force regenerate projects to use the new prefixed qt_foo APIs. Change-Id: I055c4837860319e93aaa6b09d646dda4fc2a4069 Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
* CMake: Regenerate selftest with latest fixesLeander Beernaert2020-03-192-2/+0
| | | | | Change-Id: I7453c2439a62331cdc2d7ffaeafb05cd831191c6 Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
* CMake: Enable testlib selftestsLeander Beernaert2020-03-181-0/+4
| | | | | | | | | | | This patch adds the equivalent of testlib's selfcover.pri and is enabled for both testlib and the respective selftests test. This patch also fixes the selftests so that they can run without FEATURE_testlib_selfcover enabled. Change-Id: I15913de2d572ac79804ce3e652cee66de74318f8 Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
* Merge remote-tracking branch 'origin/wip/cmake' into devAlexandru Croitor2020-02-132-6/+2
|\ | | | | | | | | | | | | | | | | | | Conflicts: tests/manual/rhi/hellominimalcrossgfxtriangle/CMakeLists.txt Hopefully final merge from wip/cmake, and then all cmake changes should target dev directly. Change-Id: I29b04c9b0284e97334877c77a32ffdf887dbf95b
| * Regenerate projects one last time before mergewip/cmakeAlexandru Croitor2020-02-122-6/+2
| | | | | | | | | | | | Change-Id: Ia24cf56b79ca6dacd370a7e397024e9b663e0167 Reviewed-by: Liang Qi <liang.qi@qt.io> Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
* | Merge remote-tracking branch 'origin/5.15' into devLiang Qi2020-02-131-1/+1
|\ \ | |/ |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: examples/widgets/graphicsview/boxes/scene.h src/corelib/Qt5CoreMacros.cmake src/corelib/Qt6CoreMacros.cmake src/network/ssl/qsslsocket.cpp src/network/ssl/qsslsocket.h src/platformsupport/fontdatabases/windows/qwindowsfontenginedirectwrite.cpp src/testlib/CMakeLists.txt src/testlib/.prev_CMakeLists.txt tests/auto/corelib/tools/qscopeguard/tst_qscopeguard.cpp Disabled building manual tests with CMake for now, because qmake doesn't do it, and it confuses people. Done-With: Alexandru Croitor <alexandru.croitor@qt.io> Done-With: Volker Hilsheimer <volker.hilsheimer@qt.io> Change-Id: I865ae347bd01f4e59f16d007b66d175a52f1f152
| * Fix unused argument warning in testTor Arne Vestbø2020-02-011-1/+1
| | | | | | | | | | Change-Id: I9f55efee82f8d855d89977b3e4db32855e110c87 Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
* | Regenerate tests/auto/testlibAlexandru Croitor2019-11-132-12/+0
| | | | | | | | | | | | Change-Id: If1b2e105836b73b1a68ed5a8e11feb8e114d66e2 Reviewed-by: Leander Beernaert <leander.beernaert@qt.io> Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
* | Include the .prev_CMakeLists.txt files for selftestsAlexandru Croitor2019-06-041-0/+24
| | | | | | | | | | | | | | | | | | | | This will help with future regeneration. Amends 6d1c695b0e32b644d13a088ab3363b6aef75de68 Change-Id: I2454dc36cfc00f7aad72488fb7927d7807978fd8 Reviewed-by: Qt CMake Build Bot Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
* | cmake: Fix testlib/selftestsAlbert Astals Cid2019-06-041-0/+26
|/ | | | | | | | | | Actually build tst_selftest Fix the subdirs of it not to be actually built as tests (so that ctest doesn't pick them up and tries to run them) Change-Id: Id9d069f44241027dbd103884d36551163382b216 Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io> Reviewed-by: Qt CMake Build Bot
* Merge remote-tracking branch 'origin/5.13' into devQt Forward Merge Bot2019-02-201-3/+3
|\ | | | | | | | | | | | | | | | | | | | | Conflicts: tests/auto/widgets/itemviews/qheaderview/tst_qheaderview.cpp Added tests/auto/testlib/selftests/expected_crashes_5.txt to work round the output of the crashes test (which exercises UB, see QTBUG-73903) being truncated on one test platform. Change-Id: I9cd3f2639b4e50c3c4513e14629a40bdca8f8273
| * Merge remote-tracking branch 'origin/5.12' into 5.13Liang Qi2019-02-081-3/+3
| |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: src/android/templates/AndroidManifest.xml src/network/ssl/qsslsocket_mac.cpp src/widgets/styles/qstylesheetstyle.cpp tests/auto/corelib/kernel/qtimer/BLACKLIST tests/auto/testlib/selftests/blacklisted/tst_blacklisted.cpp tests/auto/testlib/selftests/expected_blacklisted.lightxml tests/auto/testlib/selftests/expected_blacklisted.tap tests/auto/testlib/selftests/expected_blacklisted.teamcity tests/auto/testlib/selftests/expected_blacklisted.txt tests/auto/testlib/selftests/expected_blacklisted.xml tests/auto/testlib/selftests/expected_blacklisted.xunitxml tests/auto/testlib/selftests/expected_float.tap tests/auto/testlib/selftests/expected_float.teamcity tests/auto/testlib/selftests/expected_float.txt tests/auto/testlib/selftests/expected_float.xunitxml Done-With: Christian Ehrlicher <ch.ehrlicher@gmx.de> Done-With: Edward Welbourne <edward.welbourne@qt.io> Done-With: Timur Pocheptsov <timur.pocheptsov@qt.io> Change-Id: If93cc432a56ae3ac1b6533d0028e4dc497415a52
| | * testlib: Add BXPASS and BXFAILOliver Wolff2019-01-291-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Prioritize blacklisting over QEXPECT_FAIL so that a test that is blacklisted no longer fails if QEXPECT_FAIL returns true unexpectedly. To reflect this state properly, the two values of BXPASS and BXFAIL were added to testlib's output. [ChangeLog][Important Behavior Changes][QtTestLib] Blacklisting of tests will be taken into account for XPASS and XFAIL. A blacklisted test that causes an XPASS will no longer be a fail. Task-number: QTBUG-72928 Change-Id: Ia2232fdc714d405fa3fd9aea6c89eb2836bc5950 Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
* | | Dodge qFatal() so as to get coverage results despite itEdward Welbourne2019-02-111-1/+20
|/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The silent and blacklisted selftests of testlib end in a qFatal(), to test its messaging is handled correctly. However, this prevents hooks in main() from saving coverage data when we're gathering that. So use a transient signal handler that longjmp()s back to a setjmp() just before the qFatal() to let the test complete normally (but, since qFatal() does something different on MS-Win, don't apply this to it). Note that testlib's internal FatalSignalHandler handles all fatal signals *except* SIGABRT, so this isn't over-riding it. (In any case, this restores the prior signal handler in setjmp()'s catch branch.) Added missing expected_silent.tap test output while checking that this change doesn't affect (the rest of) the test output. Change-Id: I7e460581ad93e26639c066b3229438a66fd299de Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
* | Add a feature to enable CodeCoverage analysis of testlibEdward Welbourne2019-01-241-0/+2
| | | | | | | | | | | | | | | | | | | | Based on Asmo Saarela's advice (QTPM-686), adapted on advice from FrogLogic support and converted to a feature so that the selftest and testlib qmake config can be co-ordinated. Task-number: QTPM-1385 Change-Id: Icd706f086009e1e08b3f8c5cd553f792402e28c0 Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
* | Tweak a selftest blacklist to exercise more of the blacklisting codeEdward Welbourne2018-11-012-5/+14
|/ | | | | | | | | | | In the process, corrected an inaccurate XFAIL message (an XPASS is normally an error, unless blacklisting ignores it so turns it into a BPASS). Added the missing .tap file to its expected output. Documented the similarity to the silent/ selftest. Task-number: QTPM-1385 Change-Id: Id74a1353d54af2f3bfe2c764e33c1f051958ab21 Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
* Updated license headersJani Heikkinen2016-01-211-17/+12
| | | | | | | | | | | | | From Qt 5.7 -> tools & applications are lisenced under GPL v3 with some exceptions, see http://blog.qt.io/blog/2016/01/13/new-agreement-with-the-kde-free-qt-foundation/ Updated license headers to use new GPL-EXCEPT header instead of LGPL21 one (in those files which will be under GPL 3 with exceptions) Change-Id: I42a473ddc97101492a60b9287d90979d9eb35ae1 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com> Reviewed-by: Lars Knoll <lars.knoll@theqtcompany.com>
* TestCase: Also check main source path when looking for test dataCaroline Chao2015-05-053-0/+113
This is for example useful when looking for a possible BLACKLIST file while doing a shadow build. Add autotests for blacklist. Change-Id: I41d3939d31d21d10187fefcb82604736d911b6ad Reviewed-by: Friedemann Kleint <Friedemann.Kleint@theqtcompany.com> Reviewed-by: Simon Hausmann <simon.hausmann@theqtcompany.com>
generated by cgit v1.2.3 (git 2.25.1) at 2024-06-01 16:49:38 +0000