summaryrefslogtreecommitdiffstats
path: root/tests/auto/testlib
Commit message (Collapse)AuthorAgeFilesLines
* support building with -no-guiOswald Buddenhagen2013-10-112-1/+8
| | | | | | Change-Id: I1c418e2aa6c01dbc40156a3494a7f7ddc95beca3 Reviewed-by: Sergio Ahumada <sergio.ahumada@digia.com> Reviewed-by: Jason McDonald <macadder1@gmail.com>
* Add QT_NO_PROCESS guards in tests where they are missingAndrew Knight2013-09-031-0/+8
| | | | | | | | | | A handful of tests lack QT_NO_PROCESS guards, making them non-compilable on Qt builds with no QProcess support. This commit does not change QProcess-specific tests, which should be left out of the build using the .pro file mechanism. Change-Id: Iac8562428abc1f59ccbb23bf5c3a919785e41f12 Reviewed-by: Friedemann Kleint <Friedemann.Kleint@digia.com>
* TestLib: align the colons so the "actual" and "expected" align tooThiago Macieira2013-07-2117-39/+39
| | | | | | | | | | | | | | | Before: FAIL! tst_testcase::testcase: Compared values are not the same Actual (actual): F0O Expected (expected): FOO Now: FAIL! tst_testcase::testcase: Compared values are not the same Actual (actual) : F0O Expected (expected): FOO Change-Id: I6f0768e4ef53e065b85a56879cecbad06fa34aef Reviewed-by: Jason McDonald <macadder1@gmail.com>
* QtTest: Output correct library nameSze Howe Koh2013-05-2743-56/+56
| | | | | | | | | | | "QTest" is the C++ namespace; "QtTest" is the library name - Edited the logger output in qplaintestlogger.cpp - Updated documentation - Updated expected outputs for self-tests Change-Id: I43c525c43221a8d4e843a00d6d55b0f06ef55fd7 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* QtTest tests: Remove DEFINES += QT_DISABLE_DEPRECATED_BEFORE=0Debao Zhang2013-03-2746-46/+0
| | | | | Change-Id: Id67ab0ea3be47d3b365aa01a354829b8745b7250 Reviewed-by: Friedemann Kleint <Friedemann.Kleint@digia.com>
* Whitespace cleanup: remove trailing whitespaceAxel Waggershauser2013-03-161-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | Remove all trailing whitespace from the following list of files: *.cpp *.h *.conf *.qdoc *.pro *.pri *.mm *.rc *.pl *.qps *.xpm *.txt *README excluding 3rdparty, test-data and auto generated code. Note A): the only non 3rdparty c++-files that still have trailing whitespace after this change are: * src/corelib/codecs/cp949codetbl_p.h * src/corelib/codecs/qjpunicode.cpp * src/corelib/codecs/qbig5codec.cpp * src/corelib/xml/qxmlstream_p.h * src/tools/qdoc/qmlparser/qqmljsgrammar.cpp * src/tools/uic/ui4.cpp * tests/auto/other/qtokenautomaton/tokenizers/* * tests/benchmarks/corelib/tools/qstring/data.cpp * util/lexgen/tokenizer.cpp Note B): in about 30 files some overlapping 'leading tab' and 'TAB character in non-leading whitespace' issues have been fixed to make the sanity bot happy. Plus some general ws-fixes here and there as asked for during review. Change-Id: Ia713113c34d82442d6ce4d93d8b1cf545075d11d Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@digia.com>
* Merge remote-tracking branch 'origin/stable' into devFrederik Gladhorn2013-01-281-1/+1
|\ | | | | | | Change-Id: I12b4d8b99bdccae53b1a978cd6eb8f4ac6fb3c76
| * Doc: Fix module name formatSze Howe Koh2013-01-251-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Follow the conventions at http://qt-project.org/wiki/Spelling_Module_Names_in_Qt_Documentation QtCore -> Qt Core QtDBus -> Qt D-Bus QtDesigner -> Qt Designer QtGui -> Qt GUI QtImageFormats -> Qt Image Formats QtNetwork -> Qt Network QtPrintSupport -> Qt Print Support QtScript -> Qt Script QtSql -> Qt SQL QtSvg -> Qt SVG QtTest -> Qt Test QtWebKit -> Qt WebKit QtWidgets -> Qt Widgets QtXml -> Qt XML QtConcurrent -> Qt Concurrent (partial) QtQuick -> Qt Quick (partial) Also, distinguish between "module" and "library" Change-Id: Icb8aa695ae60b0e45920b0c8fce4dc763a12b0cd Reviewed-by: Jerome Pasion <jerome.pasion@digia.com>
* | Merge "Merge remote-tracking branch 'origin/stable' into dev" into ↵Frederik Gladhorn2013-01-2345-45/+45
|\ \ | | | | | | | | | refs/staging/dev
| * | Merge remote-tracking branch 'origin/stable' into devFrederik Gladhorn2013-01-2245-45/+45
| |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: src/corelib/io/qsavefile_p.h src/corelib/tools/qregularexpression.cpp src/gui/util/qvalidator.cpp src/gui/util/qvalidator.h Change-Id: I58fdf0358bd86e2fad5d9ad0556f3d3f1f535825
| | * Update copyright year in Digia's license headersSergio Ahumada2013-01-1845-45/+45
| | | | | | | | | | | | | | | Change-Id: Ic804938fc352291d011800d21e549c10acac66fb Reviewed-by: Lars Knoll <lars.knoll@digia.com>
* | | Fix testlib-selftest for MinGW.Friedemann Kleint2013-01-221-2/+5
|/ / | | | | | | | | | | | | | | Fix float format and exclude crashing sub-binary. Task-number: QTBUG-29014 Change-Id: I404f971edeb128263122a194f23e2806d6fd3bd0 Reviewed-by: Kai Koehne <kai.koehne@digia.com>
* / QTestLib: improve output when comparing lists fails.David Faure2013-01-225-22/+84
|/ | | | | | | | | | | | | | | | | | | | Only QStringList was handled before, now any QList is handled. A specialization for QStringList is still needed though, due to the way template matching works. Example with QList<int>. Before: FAIL! : tst_QTextCodec::threadSafety() Compared values are not the same Loc: [../tst_qtextcodec.cpp(2057)] After: FAIL! : tst_QTextCodec::threadSafety() Compared lists differ at index 0. Actual (res2.toList()): '0' Expected (mibList): '3' Loc: [../tst_qtextcodec.cpp(2057)] Change-Id: If0fdec3236ddb78a679ee549aba569ef5571c395 Reviewed-by: Jason McDonald <macadder1@gmail.com> Reviewed-by: Friedemann Kleint <Friedemann.Kleint@digia.com>
* Tests: Remove declaration of built-in and automatic metatypes.Stephen Kelly2013-01-061-1/+0
| | | | | | | | These types are either built-in or 'automatically declared' and so don't need to be explicitly declared as metatypes. Change-Id: Ifd116dee32a450ff89a9a1011e26b434765d6e95 Reviewed-by: David Faure <david.faure@kdab.com>
* Remove metatype registration of built-in types.Stephen Kelly2013-01-031-1/+0
| | | | | | | | As they are built-in, they are effectively registered at compile-time already. Change-Id: I7ae6ba16088eab5d19213fa7b07c2a7760988a86 Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@digia.com>
* add a .qmake.conf file which load()s qt_build_configOswald Buddenhagen2012-10-181-0/+1
| | | | | | | | | | | | | | that way we don't have to auto-generate code for that in the configures. note that we now load qt_build_config.prf instead of just qmodule.pri, which means that exceptions_off is set everywhere. we forcibly re-enable them for testcases to minimize the deviation from default 3rd party usage. testlib selftests are not qt testcases, so the one that needs exceptions needs to enable them explicitly. Change-Id: I1b9360bb11f2e80c92a2b63a7c45991ad17fda1b Reviewed-by: Joerg Bornemann <joerg.bornemann@digia.com> Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@digia.com>
* Make QUrl::errorString() usable for QtTest output in invalid URLsThiago Macieira2012-10-024-8/+8
| | | | | | | | | If an URL is invalid, let's indicate that in the test output. To be helpful, let's make QUrl::errorString() include the component form of the URL. Change-Id: Iaafe16973ded79c7ea688fbb23808d91253e8c14 Reviewed-by: David Faure <faure@kde.org>
* Change copyrights from Nokia to DigiaIikka Eklund2012-09-2245-1085/+1085
| | | | | | | | Change copyrights and license headers from Nokia to Digia Change-Id: If1cc974286d29fd01ec6c19dd4719a67f4c3f00e Reviewed-by: Lars Knoll <lars.knoll@digia.com> Reviewed-by: Sergio Ahumada <sergio.ahumada@digia.com>
* Make QSignalSpy copy QVariant parameters directlyKent Hansen2012-08-071-0/+9
| | | | | | | | | | | | | | | | | | | | | Previously, a QVariant parameter would be wrapped inside a new QVariant, and you would have to cast the QSignalSpy's QVariant to a QVariant to get the actual value. This behavior was unintuitive and undocumented. Check if the parameter type is QVariant, and copy it directly if it is. This makes the QSignalSpy's QVariant directly usable (no need to "unwrap" the value in user code). Existing tests that use QSignalSpy together with QVariant parameters (such as tst_QPropertyAnimation::valueChanged()) and do cast the QVariant parameter to a QVariant, continue to work after this change; this is because qvariant_cast<QVariant>() returns its input value (unchanged) when the type is not QMetaType::QVariant. Task-number: QTBUG-21645 Change-Id: Ibfb171edd60c0d3f7ca1d5419e5c5f3d0380d5b3 Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
* Set the Qt API level to compatibility mode in all tests.Thiago Macieira2012-08-0146-0/+46
| | | | | | | | | | | Qt 5.0 beta requires changing the default to the 5.0 API, disabling the deprecated code. However, tests should test (and often do) the compatibility API too, so turn it back on. Task-number: QTBUG-25053 Change-Id: I8129c3ef3cb58541c95a32d083850d9e7f768927 Reviewed-by: Lars Knoll <lars.knoll@nokia.com> Reviewed-by: Olivier Goffart <ogoffart@woboq.com>
* Add QSignalSpy::wait() method.David Faure2012-07-251-1/+54
| | | | | | Change-Id: I1f3b49e3dee19bf0b1d2933c6e6ad7972186e0d0 Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com> Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
* Provide public API for native event filtering, moved up from QPA.David Faure2012-07-202-4/+4
| | | | | | | | | | | | | | | | | | The previous API was hard to use (global function, no type safety, manual chaining), and confusing (app vs dispatcher split only made sense on Windows). Installing and removing out of order would have the risk of setting back a dangling pointer (crash). Meanwhile QPA added type safety, and this new API models the QObject::installEventFilter API for ease of use. The virtual method is in a new interface, QAbstractNativeEventFilter. QPA was even calling the dispatcher event filter with QPA-private event classes, which made no sense (refactoring leftover from when the code was in the dispatcher). Now the QPA plugins trigger the qcoreapp event filters with the actual native events directly. Change-Id: Ie35e47c59c862383bcaf857b28d54f7c72547882 Reviewed-by: Marius Storm-Olsen <marius.storm-olsen@nokia.com>
* Avoid load(testcase) for installing test helper appsRohan McGovern2012-07-041-2/+1
| | | | | | | | | | | | Make test projects declare TEST_HELPER_INSTALLS rather than calling a function exported by testcase.prf. load(testcase) may be unsafe, as testcase.prf should be processed after default_post.prf. Fixes silent disabling of various autotests. Change-Id: I56b35ffd653a637ad5ab18d64dd1a1edadfac59f Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@nokia.com> Reviewed-by: Kurt Korbatits <kurt.korbatits@nokia.com>
* Use UTF-8 in the QtTest data and benchmark tagsThiago Macieira2012-06-0932-32/+32
| | | | | | | | | | | Future-proofing. Since Qt source code is now mandated to be in UTF-8, it is entirely possible that someone will use non-ASCII in data tags. Though it would be interesting to see how to access them from the Windows command-line. Change-Id: I880fc312432b62143888ff1e1d9abbd54f704601 Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* Make tst_selftests parallel-safe.Rohan McGovern2012-05-292-3/+18
| | | | | | | | | Use a QTemporaryDir for temporary files, instead of the current working directory. Change-Id: Ifeb2944238f785a1f7beb0dc2a7c1e092d121db5 Reviewed-by: Kalle Lehtonen <kalle.ju.lehtonen@nokia.com> Reviewed-by: Toby Tomkins <toby.tomkins@nokia.com>
* Make the event notification on Windows be mandatory in all dispatchersThiago Macieira2012-05-042-0/+10
| | | | | | | | | | | | | This way, QWinEventNotifier will work on all Windows systems, not just with the default event dispatcher. Other dispatchers (other than QWin32EventDispatcher) are permitted, so the class should not abort just because of that. If a dispatcher really doesn't want to implement this, they need to implement the virtuals to do nothing, possibly print a warning. Change-Id: I2c132bcde95b9d5941c8906a0fcd2ad964087772 Reviewed-by: Bradley T. Hughes <bradley.hughes@nokia.com>
* Merge remote-tracking branch 'origin/master' into api_changesOswald Buddenhagen2012-04-103-7/+8
|\ | | | | | | | | | | | | | | | | | | Conflicts: configure src/widgets/styles/qwindowsxpstyle.cpp tests/auto/gui/kernel/qwindow/qwindow.pro tests/auto/gui/kernel/qwindow/tst_qwindow.cpp Change-Id: I624b6d26abce9874c610c04954c1c45bc074bef3
| * Fix MSVC warnings in tests.Friedemann Kleint2012-04-021-3/+4
| | | | | | | | | | | | | | | | | | - Unused variables - conversion truncations - Overflow in expressions like '-1 + sizeof()' Change-Id: Ibbd18497951e9e7e9dccaf596cb4e864b69ec02c Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
| * Fix some compiler warnings in tests.Friedemann Kleint2012-03-272-4/+4
| | | | | | | | | | | | | | | | - Unused variables - Deprecated conversion from const char * to char *. Change-Id: Iea0b9c4613ea74cead6d95ba12ad1028f531cbff Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
* | Add a remainingTime() method to the public interface of the QTimer classLaszlo Papp2012-04-032-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It is an extension coming from the use case when you, for instance, need to implement a countdown timer in client codes, and manually maintain a dedicated variable for counting down with the help of yet another Timer. There might be other use cases as well. The returned value is meant to be in milliseconds, as the method documentation says, since it is reasonable, and consistent with the rest (ie. the interval accessor). The elapsed time is already being tracked inside the event dispatcher, thus the effort is only exposing that for all platforms supported according to the desired timer identifier, and propagating up to the QTimer public API. It is done by using the QTimerInfoList class in the glib and unix dispatchers, and the WinTimeInfo struct for the windows dispatcher. It might be a good idea to to establish a QWinTimerInfo (qtimerinfo_win{_p.h,cpp}) in the future for resembling the interface for windows with the glib/unix management so that it would be consistent. That would mean abstracting out a base class (~interface) for the timer info classes. Something like that QAbstractTimerInfo. Test: Build test only on (Arch)Linux, Windows and Mac. I have also run the unit tests and they passed as well. Change-Id: Ie37b3aff909313ebc92e511e27d029abb070f110 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com> Reviewed-by: Bradley T. Hughes <bradley.hughes@nokia.com>
* | Remove all calls to, and deprecate qMalloc, qRealloc and qFree.Robin Burchell2012-03-311-2/+2
|/ | | | | | | | | | | | | | | | | | Callers should just call the standard allocation functions directly. Adding an extra function call onto all basic memory management for the sake of making it instrumentable in rare cases isn't really fair to everyone else. What's more, this wasn't completely reliable, as not everything was using them in a number of places. Memory management can still be overridden using tricks like LD_PRELOAD if needed. Their aligned equivilents cannot be deprecated, as no standard equivilents exist, although investigation into posix_memalign(3) is a possibility for the future. Change-Id: Ic5f74b14be33f8bc188fe7236c55e15c36a23fc7 Reviewed-by: Lars Knoll <lars.knoll@nokia.com>
* testlib: Improve formatting of QCOMPARE failure messages.Jason McDonald2012-03-1221-122/+122
| | | | | | | | | | | Make the various versions of the failure message align consistently so that it's a little easier to compare the actual and expected values. Of course, the value won't align nicely unless the "actual" and "expected" strings are the same length, but at least this commit makes that consistent across all versions of the message. Change-Id: If9ce231df3b5d279a06f6458fdb5da0aa4586068 Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* selftests: Improve reporting of problems with loading expected outputJason McDonald2012-03-081-4/+17
| | | | | | | | | | | | | | | | | | If the expected output file was missing (e.g. not included in selftests.qrc), tst_selftests would trigger an assert inside QList by calling QList::at() on an empty list. Make tst_selftests detect this error instead and give a meaningful error message. When loading expected output for the crashes selftest, where there are several alternative versions of the expected output, the code reused the "exp" variable when loading the alternative test output files. This caused the last file loaded to be used unintentionally if none of the alternative files had the correct number of lines. Use a different variable so that exp remains empty if none of the alternatives are valid and a failure can be reported. Change-Id: I35b2a3d905d069d3ee8dcb1447836eb68d5c8612 Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* testlib: Improve verbose and XPASS outputJason McDonald2012-03-089-35/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | | Previously, verbose (-v2) and XPASS test output showed all QCOMPAREs as "COMPARE()", making it impossible to see what was compared and difficult to match the output to the source of a test containing many calls to QCOMPARE. This commit changes testlib's internal compare_helper API so that string representations of the compared expressions are always passed to QTestResult::compare() when available, and can thus be shown in the verbose and XPASS output. The XPASS output has also been changed to state explicitly that the comparison succeeded unexpectedly, bringing it in line with the XPASS output resulting from a call to QVERIFY. This commit also changes all calls to compare_helper() to call the eight-argument version of the function, which simplifies much of the calling code. The now obsolete four-argument version of compare_helper() has been changed to output a warning that it is obsolete. It will be removed once other modules have had some time to catch up. The improved XPASS and verbose output is demonstrated by the expectfail and verbose2 selftests. Change-Id: I8baa46d5dd30e6c43b26f366c34dc5b64aab5f7c Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* testlib: Make verbose2 selftest use QCOMPARE as well as QVERIFY.Jason McDonald2012-03-0511-163/+236
| | | | | | | | | The verbose output for QCOMPARE is not very helpful. Make the verbose2 selftest (which reuses the counting selftest) demonstrate this, so that a future commit can demonstrate improvement. Change-Id: I6b3bc8f5199e984aa11d0a67b76a8c916be86380 Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* Merge remote-tracking branch 'origin/api_changes'Lars Knoll2012-03-041-1/+1
|\ | | | | | | | | | | | | | | | | | | Conflicts: dist/changes-5.0.0 mkspecs/features/qt_module_config.prf qmake/project.cpp qmake/property.cpp Change-Id: I6e4af40743a9aeff8ed18533a48036e332acc296
| * QEvent (and subclasses): make ctors explicitMarc Mutz2012-03-011-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | Do this regardless of whether the event subclass is public API or only used in examples. Examples are examples, used by others as templates or even copied verbatim, so they should also follow sound engineering rules. Anyway, there's only one in examples/... Change-Id: I586ff16407a956c9e89288fdd4377eed73f45c0f Reviewed-by: Samuel Rødal <samuel.rodal@nokia.com>
* | testlib: Add selftests for -v1 and -v2 command-line switches.Jason McDonald2012-03-0213-5/+1060
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | These tests don't have their own source code but rather reuse the counting selftest with additional command-line options. Note that currently the -v1 switch only changes the plain text output, and the expected xml output is identical to that of the counting test. This may change in the future however. This commit also restores a couple of lists to alphabetical order, where the findtestdata selftest was not sorted into the list correctly. Change-Id: Ie38e255f8029157b34162b3864b5fa66e137d74a Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* | Remove unused selftest data file.Jason McDonald2012-02-292-7/+0
|/ | | | | | | | | The private git history shows that the test program associated with this data file was missing from the original commit and was never subsequently added. Change-Id: I3401724ac04168158a48eb06436db83d3557711f Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* testlib: Improve the silent logging modeJason McDonald2012-02-286-0/+124
| | | | | | | | | | | | | | Previously the silent logging mode suppressed passes, skips and internal testlib info messages, but did not suppress debugging output, making it hard to see the fails in a noisy test. This commit changes silent mode so that it suppresses all output except test failures and fatal errors, making silent mode truly useful for seeing just the important test output. This commit also adds a selftest to verify the behaviour of silent mode. Change-Id: I75420aead03682306210746a87e2a3b608b58fc6 Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* Fix selftests while using QStringBuilderOlivier Goffart2012-02-271-1/+1
| | | | | Change-Id: I6dcd830b96023765447c9683fd95209d4312fb1b Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
* Port badxml autotest to QMetaObjectBuilderKent Hansen2012-02-212-16/+16
| | | | | | | | | | | | | | The meta-object format is going to change for Qt5. Use QMOB to insulate the badxml test from such changes. (It just so happens that the QFAIL("a failure") statement is still on line 109 after the refactoring, so the expected_badxml.* files' location tags did not have to be changed.) Change-Id: I04421d13c4df71c8004fa71cafc4823a59079a41 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com> Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com> Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
* testlib: Report one test result per benchmark test.Jason McDonald2012-02-2016-70/+314
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Prior to this commit, a benchmark test could report 0..n passes and 0..m fails or skips, where n is the number of accumulation iterations used to collect benchmark data and m is the number of times the test function was invoked. Depending on the type of benchmark measurer being used, this could result in a very large volume of test output and inconsistent pass, fail and skip counts between test runs. This commit changes the behaviour so that each benchmark test reports one pass, fail or skip, regardless of the number of iterations used to collect benchmark data. This commit also prevents benchmark data being reported in the test output if the benchmark test failed or skipped, as any benchmark data is of dubious value in such cases. The latter change in behaviour requires a minor modification to the badxml selftest, which now tests quoting of literal strings in xml test output for both passing and failing benchmarks. Finally, this commit also adds a new selftest specifically for verifying correct behaviour for benchmarks that fail or skip. Task-number: QTBUG-24313 Change-Id: I3426dc659a7511b62fd183a031c7235bc753f497 Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* testlib: Add selftests for XPASS and XFAIL involving QCOMPARE.Jason McDonald2012-02-205-59/+200
| | | | | Change-Id: I4f897b411aaa51ff453208eda50e12665489074b Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* testlib: make XPASS output slightly less confusing.Jason McDonald2012-02-208-12/+12
| | | | | | | | | | | | | | Prior to this commit, when an unexpected pass occurred the test output showed the message "XPASS : tst_foo::function() 'expr' returned FALSE", where the problem was actually that the expression evaluated to true when it was expected to be false. This commit changes the output to make it clear that the expression evaluated to true unexpectedly. Task-number: QTBUG-22118 Change-Id: Id22c178073d3b75789675ca37a8ef019029b1f91 Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* testlib: Skip test function if init() fails.Jason McDonald2012-02-175-38/+273
| | | | | | | | | | | | | | | | | Prior to this commit, the following statement in the qtestlib documentation was untrue: "If init() fails, the following testfunction will not be executed, the test will proceed to the next testfunction." If init() called QSKIP, the test function would be skipped, but if init() reported a failure, the test function would still be executed (even though doing so could be unsafe). This commit makes testlib skip a test function if init() reports a failure and enhances the selftests to cover skips and fails in both init() and cleanup(). Task-number: QTBUG-20371 Change-Id: Id1cc8464ae0b8c257ae1b74dbe9189a501f5366b Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* testlib: Count passes, fails and skips consistently.Jason McDonald2012-02-1338-143/+599
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | For data-driven tests, testlib previously counted one fail or skip for each data row that failed or skipped, while it counted only one pass for a test function where all rows passed and counted no passes for a test function where some rows passed and some rows failed. A similar problem also existed for benchmark tests, which could run multiple iterations of the same test, with each fail and skip being counted but only a single pass being counted for the entire series of iterations. This commit makes testlib count one pass, fail or skip for each data row. Test functions that are not data-driven count one result for the test function, as before. Benchmark tests count one pass, fail or skip per iteration. A side-effect of this change is that the test output in plain text, xml and light xml formats now shows a result for every data row and benchmark iteration executed, allowing post-processors to correctly calculate the total number of tests executed. Previously, individual rows were not shown in the test output if they passed, making such calculations impossible. The only change to the xunitxml output format is to correct a bug where no test result was recorded for a test function if the last data row was skipped and all other rows passed -- in which case the overall result should be a pass. Note that there is also a pre-existing bug in the xunit logger, where no result is reported if all rows are skipped; that bug is unaffected by this commit. Task-number: QTBUG-21848 Task-number: QTBUG-22124 Change-Id: I7e17177e10d6e89e55b9684c159bd506f21d002b Reviewed-by: Ed Baak <ed.baak@nokia.com>
* CodeCoverage: Handle QTest based subtests.Caroline Chao2012-02-121-11/+14
| | | | | | | | | | | | | | | | Set QT_TESTCOCOON_ACTIVE environment variable when the coverage is installed for a test and unset it when the coverage data is saved. Tests that run when QT_TESTCOCOON_ACTIVE is set are subtests and will not be considered as stand-alone tests for the coverage. When a test is run as a subtest its coverage data will not be saved for itself but for the main test it is merged with. Also its status will not be reported since only the status of the main test is expected in the test report, e.g. the test tests/auto/testlib/selftests. Change-Id: Icfdf99300aae18040e1a3441a8af21f68df4c0db Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com> Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
* Make selftest.pri more readable/maintainable.Jason McDonald2012-02-091-7/+41
| | | | | Change-Id: Ic5a74beebb718df4cce5a9b02cdfd6f6d4cec97d Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* Improve float selftestJason McDonald2012-02-092-10/+23
| | | | | | | | | | | | Correct a bug in the test and use unique names for data rows. The bug was that the test assumed that QCOMPARE for float values 100001 and 100002 would fail, but it actually succeeds. QCOMPARE for floats uses qFuzzyCompare(), which succeeds if the numbers differ by no more than 1/100,000th of the smaller value. Thus QCOMPARE(100001, 100002) passes, while QCOMPARE(99998, 99999) fails. Change-Id: Ia35d3126c2e3ebe91d64daa309048514a365d9fb Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
generated by cgit v1.2.3 (git 2.25.1) at 2024-04-30 18:49:28 +0000