summaryrefslogtreecommitdiffstats
path: root/tests/auto/testlib/selftests/tst_selftests.cpp
Commit message (Collapse)AuthorAgeFilesLines
* Changed selftests unittest to use specific headers instead of QtCore.Kurt Korbatits2012-01-101-1/+1
| | | | | | | | | | | Changed selftests unittest to use specific classes of QtCore instead of pulling in all of QtCore headers by using include <QtCore> - Decreasing build time. Change-Id: Ic53a70423ecafeb342cc05ae6d7dee795533d205 Reviewed-by: Kurt Korbatits <kurt.korbatits@nokia.com> Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com> Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
* Update copyright year in license headers.Jason McDonald2012-01-051-1/+1
| | | | | Change-Id: I02f2c620296fcd91d4967d58767ea33fc4e1e7dc Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* Changed tests: qmake,selftest and lancelot to use QFINDTESTDATA.Kurt Korbatits2012-01-051-1/+10
| | | | | | | | | | Changed these tests to use QFINDTESTDATA macro to detect location of testdata. Checking for a specific file contained in the testdata so as not to be confused by empty directories created during configure. Change-Id: Iac2ac6304b6b9ac79e00886025b93ec0af5a8507 Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com> Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* QTestLib-selftest: Unblock on Windows.Friedemann Kleint2011-12-221-1/+4
| | | | | | | Do not run tests that pop up assert boxes. Change-Id: Idac94398ff1109605eb33a042dab734eafb1a17a Reviewed-by: Sergio Ahumada <sergio.ahumada@nokia.com>
* QTestlib: Make QImage comparison more verbose.Friedemann Kleint2011-12-211-6/+22
| | | | | | | | | | | | | | | | | | Introduce a specialization for qCompare(QImage,QImage) that checks isNull, size and format and outputs verbose messages. Check isNull, size similarly for QPixmap. Add an autotest: - Add test to cmptest and make it a GUI application since QImage requires QGuiApplication. - Make testlib/selftests capable of running X11-GUI applications by passing DISPLAY. - Ignore stderr output for cmptest - Add test data Change-Id: I2b29c7822fbeedf2b22c90889739ed7ff859ce92 Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
* QTestlib-Selftest: Add initTestCase() setting the directory.Friedemann Kleint2011-12-151-2/+15
| | | | | | | | | Required for launching the sub-processes. The test worked only when launched locally, not from 'make check'. Change-Id: I42c9202a7726c3135f94445fb336b2b8241535a4 Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com> Reviewed-by: Friedemann Kleint <Friedemann.Kleint@nokia.com>
* Revert "Remove unused overload of QTest::qExec."Friedemann Kleint2011-12-121-0/+4
| | | | | | | | | | | | | | | | The overload is used in Qt Creator (see src/libs/extensionsystem/pluginmanager.cpp). The use case here is an application whose internal QObjects can be tested by passing a command line parameter. For this use case, it is inconvenient to have to allocate memory and create a char argv[]- array. This reverts commit ad80d42f8eefd72d9297c272139acc70e24bfa13. Change-Id: I2a2f91e2840100fd62743f6d03b33005d67b18f8 Reviewed-by: Daniel Teske <daniel.teske@nokia.com> Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
* QTestlib/Selftest: Fix Windows.Friedemann Kleint2011-12-081-2/+20
| | | | | | | | | - Do not run with empty environment. At least PATH is required at least (Qt + MSVC/MinGW runtime) - Account for MSVC's different formatting of double numbers. Change-Id: Ic7b1cf4a16a88c5384347b2651b011ac13c92d70 Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
* Remove unused overload of QTest::qExec.Jason McDonald2011-12-081-4/+0
| | | | | | | | | | | | | The overload of QTest::qExec() that takes a QStringList is not used anywhere in Qt's autotests, despite having been in the qtestlib API since Qt 4.4. This lack of use most likely derives from the fact that none of the QTEST_MAIN macros use the overload, and more than 99% of Qt's tests use those macros to avoid explicitly calling QTest::qExec(). Change-Id: I264b21d7fe1a9f2d565f748cf8bbe32414a73bb0 Reviewed-by: Lars Knoll <lars.knoll@nokia.com>
* testlib: add QFINDTESTDATA macro for finding testdata filesRohan McGovern2011-12-011-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Automated tests often need to load some data from external files. Currently, a wide variety of approaches for this have been used in Qt autotests, including: - embed the source directory into the test binary at compile time, and find the testdata relative to that; this fails when the source tree is no longer available (e.g. when the tests are deployed to a device). - use a path relative to the current working directory, and trust that the caller always sets the current working directory such that the testdata can be found; this fails when the caller uses a different working directory than expected. - use a path relative to QCoreApplication::applicationDirPath(); this fails when source tree != build tree (since testdata is not automatically copied into the build tree). - compile the files into the binary using the Qt resource system; this should work, but does not allow for testing of code which genuinely needs external files. It seems that there is not a simple method for determining the testdata path which can be reliably used in all circumstances, so various tests have reinvented the testdata location method in different ways. Therefore, this is a good candidate for an addition to the testlib API. The current implementation of QFINDTESTDATA is able to find testdata in all three of (build tree, install tree, source tree), in that order. Change-Id: Ib2fed860723ccf437240da3b00db22dfe1a6b56c Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
* Added -datatags option to QTestLibJo Asplin2011-11-041-0/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (Note: This feature is ported from Qt 4.8. See the following commits: 01575deafb7d26ca2431374e92c6d71de96547c7 4866d1ba8afbab61e102942d1ea93b81fea053d6 ) Passing the -datatags option to a QTestLib program prints the available data tags to standard output. For completeness, the test case name is also printed at the start of each output line. (Although the file name is supposed to match the lower-case version of the test case name, this is currently not true in all cases (particularly not under tests/benchmarks/). Even if there was a script to enforce this convention, the -datatags option provides this information in a reliable way.) Data tags for each test function (f() in this case) are printed in four different ways depending on the presence of local and global data tags: Case 1: No tags: tst_MyTestCasetst_MyTestCase f Case 2: Local tags only: tst_MyTestCase f local tag 1 tst_MyTestCase f local tag 2 ... Case 3: Global tags only: tst_MyTestCase f __global__ global tag 1 tst_MyTestCase f __global__ global tag 2 ... Case 4: Local and global tags: tst_MyTestCase f local tag 1 __global__ global tag 1 tst_MyTestCase f local tag 2 __global__ global tag 1 ... tst_MyTestCase f local tag 1 __global__ global tag 2 tst_MyTestCase f local tag 2 __global__ global tag 2 ... ... Note that the string __global__ is assumed to be highly unlikely to occur in a data tag (if it does, an ambiguity results). Change-Id: Ib51aa0c3c32ad52e52ce519729292cf8f0ec5d50 Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com> Reviewed-by: Sergio Ahumada <sergio.ahumada@nokia.com>
* Allow extra selftests without modifying source.Jason McDonald2011-11-031-7/+10
| | | | | Change-Id: I21708b725611eb64bdb0414cee08ca00dd62b213 Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* Remove skipglobal testlib selftest.Jason McDonald2011-10-191-1/+0
| | | | | | | | This test duplicates the skipinitdata selftest and has slightly less informative output. Change-Id: Ifd40e3ef8030059ec8fa0089ce5b2a994624abeb Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* Reduce the runtime of tst_selftestsRohan McGovern2011-10-181-0/+6
| | | | | | | | | | | The addition of testing for multiple loggers greatly increased the total runtime of this test, in fact making it the slowest testcase in Qt. Fortunately this is only due to a couple of slow subtests whose behavior is unlikely to be affected by the loggers. Change it to run these slow subtests just for a couple of loggers, instead of all the combinations. Change-Id: Ie90f6c0ca29470ed6a7c4e2e185f852602a2d162 Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
* Tidy and sort the list of selftestsJason McDonald2011-09-261-44/+38
| | | | | | | Change-Id: I123a775e2d56fc0c64d43e98a1d2e3ab68394076 Reviewed-on: http://codereview.qt-project.org/5492 Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com> Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* Use placeholder for Qt version in expected outputJason McDonald2011-09-261-2/+0
| | | | | | | | | | | | | The selftests already used the @INSERT_QT_VERSION_HERE@ placeholder in place of the real Qt and QTestlib versions in the expected test output of the subtests. For unknown reasons, the same was not true for the expected plain text output. In the past, this has caused Release Managers to waste time incrementing the version numbers in these files. Change-Id: I52f7870486fce128c04d53ff06978afa947474fd Reviewed-on: http://codereview.qt-project.org/5375 Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com> Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* Remove obsolete exclusion from selftest checkingJason McDonald2011-09-261-3/+0
| | | | | | | | | There is no test with a datatag "float", so this code is never executed. Change-Id: I9dd234e6575eb1c33075705edf77eb1d977061c9 Reviewed-on: http://codereview.qt-project.org/5362 Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com> Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* Don't XFAIL on the contents of assert messages.Jason McDonald2011-09-261-28/+6
| | | | | | | | | | | | | | | Q_ASSERT uses __FILE__, which has undefined contents, and thus the selftest cannot always match Q_ASSERT messages in actual output with those in expected output. The test was calling QEXPECT_FAIL() in this case, but doing so implies that (a) it's a failure and (b) we intend to fix it one day. Neither of these is true, so instead we should simply tolerate this difference in actual and expected outputs, in the same way that we do in a couple of other similar cases. Change-Id: I8b8609a400e9b798936875f6eb1ca18a83660efc Reviewed-on: http://codereview.qt-project.org/5358 Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com> Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* Allow tests to log to multiple destinationsJason McDonald2011-09-231-20/+116
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Each destination and the format of output to write there is specified by adding "-o filename,format" to the command-line. The special filename "-" indicates that the log output is written to the standard output stream, though standard output can be used as a destination at most once. The old-style testlib output options are still supported, but can only be used to specify one logging destination, as before. If no logging options are given on the command-line, a plain text log will go to the console, as before. To log to the console in plain text and to the file "test_output" in xunit format, one would invoke a test in the following way: tst_foo -o test_output,xunitxml -o -,txt This commit also enhances the selftests to test with multiple loggers, but negative tests (e.g. bad combinations of command-line options) are left for future task QTBUG-21567. Task-number: QTBUG-20615 Change-Id: If91e752bc7001657e15e427aba9d25ab0a29a0b0 Reviewed-on: http://codereview.qt-project.org/4125 Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com> Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* Enable testlib self-testing with multiple loggersJason McDonald2011-09-231-129/+164
| | | | | | | | | | | | Modify the selftest to be able to run each subtest with a list of one or more test loggers. The addition of tests that use this capability will be part of a subsequent commit. Task-number: QTBUG-20615 Change-Id: Iac3efe8220e8245aa7e5589348d2c86b8034dd28 Reviewed-on: http://codereview.qt-project.org/5292 Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com> Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* Don't run multiexec test with file loggingJason McDonald2011-09-211-2/+7
| | | | | | | | | | | | | | The multiexec test runs the same test object five times. If the -o option is given, the output file is overwritten by each run of the test object, meaning that tst_selftest only sees 1/5 of the test output in a file compared with what it sees on the console. This makes it impossible to use the expected output file to verify the test for both console and file output. This issue is noted in QTBUG-21561. Change-Id: I00031a2ea43a7ef78e8317473a089306ec062d8e Reviewed-on: http://codereview.qt-project.org/5270 Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com> Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* Split subtest selftestJason McDonald2011-09-201-2/+3
| | | | | | | | | | | | This test was attempting to verify two completely unrelated things, so this commit splits it into two tests. Also, printf calls are replaced by qDebug so that the test does not bypass the testlib loggers. Change-Id: I1a202af38ce2c69690a32d93405ba604ec6cabee Reviewed-on: http://codereview.qt-project.org/5178 Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* Remove waitwithoutgui testlib selftestJason McDonald2011-09-201-4/+0
| | | | | | | | | | | | | This test is not useful for finding bugs in qtestlib's logging code, because it bypasses the qtestlib loggers and doesn't play nice with tst_selftest. Neither is this test very useful for finding bugs in QTest::qWait(), as the test only proves the qWait() terminates, not that it waits accurately, or even that it waits at all. Change-Id: Ia5dd7cbaf3a6fbb4e94e54ed155263580e495694 Reviewed-on: http://codereview.qt-project.org/5173 Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com> Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* Simplify selftestJason McDonald2011-09-171-17/+11
| | | | | | | | | | Don't store separate strings for the logger name and the associated file suffix -- just use the same string everywhere. Change-Id: Ie7d1af6bf906b5ac09fbd5fcc5754b68036fb370 Reviewed-on: http://codereview.qt-project.org/5060 Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com> Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* Improve cleanup of selftestsJason McDonald2011-09-171-2/+3
| | | | | | | | | | | | The selftest was only attempting to delete the output file at the end of the entire test run. This file should actually be deleted after each subtest to ensure that it does not cause confusing errors if the next subtest fails to overwrite the output file. Change-Id: Ia3827926f71a697c6108de3f2b08c969f761d978 Reviewed-on: http://codereview.qt-project.org/5051 Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com> Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* Remove obsolete QEXPECT_FAIL callsJason McDonald2011-09-171-2/+0
| | | | | | | | | | The removed calls related to tests that were removed when redundant logging functionality was removed. Change-Id: Iaebbc109eaaddb440ae78e1f4a1b4e874a8b2960 Reviewed-on: http://codereview.qt-project.org/5050 Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com> Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* Improve compliance with Qt coding guidelinesJason McDonald2011-09-171-60/+48
| | | | | | | | | | | | * Use C++ comment delimiters consistently * Use a space after control flow keywords * Don't nest "else if" blocks unnecessarily * Follow indenting rules * Added comments describing some internal functions Change-Id: I72fd1eaf5c5f3130945fbd2b64fa59e19ad2913d Reviewed-on: http://codereview.qt-project.org/5049 Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* Moved autotests in category 'testlib' into new directory structureJo Asplin2011-09-121-0/+633
Task-number: QTBUG-21353 Change-Id: I2e2ad4b7570f80b3e6500821f769e967f0a75027 Reviewed-on: http://codereview.qt-project.org/4438 Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com> Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
generated by cgit v1.2.3 (git 2.25.1) at 2024-05-21 11:24:00 +0000