summaryrefslogtreecommitdiffstats
path: root/src/testlib/qtestlog.cpp
Commit message (Collapse)AuthorAgeFilesLines
* Extend QTest::failOnWarning() to a no-parameter fail-on-any-warningEdward Welbourne3 days1-0/+5
| | | | | | | | | | | | | | | | | | | | Many users (albeit mostly in qtdeclarative) of failOnWarning() are passing a catch-all regexp, which is only supported when regular expression support is enabled. Make their lives easier and those checks independent of the feature by adding a third overload, taking no parameter, that fails on any (unanticipated) warning. Implementation is trivial - just put a null QVariant in failOnWarningList; it won't match either check on the entry that might exempt a test from failing on a warning, so any warning at all will trigger failure. [ChangeLog][QtTest] QTest::failOnWarning() now has a no-parameter overload to support the common case of fail-on-any-warning, without needing to construct a match-everything regular expression. Change-Id: Ic693f1c8619fd6e495543b85737d566134cf9d20 Reviewed-by: Mitch Curtis <mitch.curtis@qt.io>
* QTest: make failOnWarning() functional on temp objects destructionVladimir Belyavsky2023-11-221-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | We need to be able to handle warnings that may occur when temporary objects, that were created in a test function, are destroyed. For example, now we miss all warnings that might be triggered from the object destructor, if the object's deletion was delayed (e.g. via deleteLater()). Also we miss all the warnings that were triggered on the test's cleanup() call. To fix this we need simply move QTestLog::clearFailOnWarnings() from QTestResult::finishedCurrentTestData() to the later stage, i.e. into QTestLog::clearCurrentTestState() which is actually called in appropriate time from QTestResult::finishedCurrentTestDataCleanup(). Same for QTestLog::clearIgnoreMessages(), since they are interrelated, so we need to clear them at the same time. We need this change for QML tests in particularly, to be able fail on warnings that might be triggered from Component.onDestruction() of some temporary test object. Pick-to: 6.6 6.5 Change-Id: I58a57691f20761619f56bd1bea3a862f2c26f569 Reviewed-by: Mitch Curtis <mitch.curtis@qt.io> Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
* Add QTest option for repeating the entire test executionJøger Hansegård2023-11-151-0/+15
| | | | | | | | | | | | | | | | | Repeated test execution can be useful, under a debugger, to catch an intermittent failure or, under memory instrumentation, to make memory leaks easier to recognize. The new -repeat flag allows running the entire test suite multiple times within the same process. It works by executing all tests sequentially before repeating the execution again. This switch is a developer tool, and is not intended for CI. It can only be used with the plain text logger. Change-Id: I2439462c5c44d1c8aa3d3b5656de3eef44898c68 Reviewed-by: Edward Welbourne <edward.welbourne@qt.io> Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
* Doc: fix link to QTest::failOnWarningVolker Hilsheimer2023-03-051-1/+1
| | | | | | | | | Also fix incorrect failOnWarning*s* elsewhere in comments. Pick-to: 6.5 6.5.0 Change-Id: I0bbc5e71060348153876f3d9a7c77c67f3c2e00d Reviewed-by: Mitch Curtis <mitch.curtis@qt.io> Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
* QPlainTestLogger: don't print "RESULT" before each additional resultThiago Macieira2022-11-091-2/+2
| | | | | | | | | | | | | | | | | We do that by passing the full list of results to the logger, to a virtual that is present in the base class to call the existing function. For all but the plain logger, we'll just print multiple results. The plain logger now prints: RESULT : tst_MyClass::QString_toInt() 383 nsecs per iteration (total: 3,837,324, iterations: 10000) 1,069 CPU cycles per iteration (total: 10,692,457, iterations: 10000) 3,123 instructions per iteration (total: 31,230,101, iterations: 10000) 536 branch instructions per iteration (total: 5,360,022, iterations: 10000) Change-Id: I3c79b7e08fa346988dfefffd17203cb5802693dd Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org> Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
* Simplify a list-removal operation in QTestLog's handleIgnoredMessage()Edward Welbourne2022-10-181-3/+1
| | | | | | | | When list->next is nullptr, setting ignoreResultList to list->next is the same as setting it to nullptr. Change-Id: If328ce31db4344cf0136b7a827be871eb89e35b5 Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
* Port from qAsConst() to std::as_const()Marc Mutz2022-10-111-1/+1
| | | | | | | | | | | | | | | | We've been requiring C++17 since Qt 6.0, and our qAsConst use finally starts to bother us (QTBUG-99313), so time to port away from it now. Since qAsConst has exactly the same semantics as std::as_const (down to rvalue treatment, constexpr'ness and noexcept'ness), there's really nothing more to it than a global search-and-replace, with manual unstaging of the actual definition and documentation in dist/, src/corelib/doc/ and src/corelib/global/. Task-number: QTBUG-99313 Change-Id: I4c7114444a325ad4e62d0fcbfd347d2bbfb21541 Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
* Port from container.count()/length() to size()Marc Mutz2022-10-041-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is semantic patch using ClangTidyTransformator: auto QtContainerClass = expr(hasType(namedDecl(hasAnyName(<classes>)))).bind(o) makeRule(cxxMemberCallExpr(on(QtContainerClass), callee(cxxMethodDecl(hasAnyName({"count", "length"), parameterCountIs(0))))), changeTo(cat(access(o, cat("size"), "()"))), cat("use 'size()' instead of 'count()/length()'")) a.k.a qt-port-to-std-compatible-api with config Scope: 'Container'. <classes> are: // sequential: "QByteArray", "QList", "QQueue", "QStack", "QString", "QVarLengthArray", "QVector", // associative: "QHash", "QMultiHash", "QMap", "QMultiMap", "QSet", // Qt has no QMultiSet Change-Id: Ibe8837be96e8d30d1846881ecd65180c1bc459af Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org> Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
* Apply Q_CONSTINIT where beneficialSona Kurazyan2022-09-011-2/+2
| | | | | | | | | | Applied Q_CONSTINIT to variables with static storage duration, but skipped the POD types with core constant initializers. Task-number: QTBUG-100486 Change-Id: Iaabf824e9cb0f29a405a149912200d4e4b3573c1 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com> Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
* Fix race condition with QTest::ignoreMessageAndré Klitzing2022-07-281-0/+9
| | | | | | | | | | | If another spawned thread will log at the same time a crash was possible because IgnoreResultList was not thread-safe. Task-number: QTBUG-104840 Pick-to: 6.4 6.3 6.2 5.15 Change-Id: I251c83bb28cb75e55d477706bf2524c061a6eab7 Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
* Use SPDX license identifiersLucie Gérard2022-05-161-38/+2
| | | | | | | | | | | | | Replace the current license disclaimer in files by a SPDX-License-Identifier. Files that have to be modified by hand are modified. License files are organized under LICENSES directory. Task-number: QTBUG-67283 Change-Id: Id880c92784c40f3bbde861c0d93f58151c18b9f1 Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org> Reviewed-by: Lars Knoll <lars.knoll@qt.io> Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
* QTestLog: simplify construction of a log messageSona Kurazyan2022-05-021-2/+5
| | | | | Change-Id: I055efa45ec468cae43a38a9746a47af0382ca8c1 Reviewed-by: Marc Mutz <marc.mutz@qt.io>
* QtTestLib: stop using QLatin1Char constructor for creating char literalsSona Kurazyan2022-04-281-3/+3
| | | | | | | | Required for porting away from QLatin1Char/QLatin1String in scope of QTBUG-98434. Change-Id: I3debae5f481037958bfb65caabca97a0d4681eb9 Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
* Make counts of various types of test result add up correctlyEdward Welbourne2022-01-131-6/+62
| | | | | | | | | | | | | | | | | | | | | | | | | Added tests for repeated skips and failures (from within void lambdas, to simulate skips and failures from within event handlers). These exhibit yet more ways to count more than one outcome for a test. The new QTest::failOnWarning() can also provoke more than one failure from a single test, and several existing selftests exhibited various ways for the Totals line's counts to add up to more than the number of actual tests run. Fixed counting so that only the first decisive incident is counted. Tests can still report later failure or skipping, but only the first is counted. Added a currentTestState in qtestlog.cpp, by which it keeps track of whether the test has resolved to a result, and clearCurrentTestState() by which other code can reset that at the end of each test. This brought to light various places where test-end clean-up was not being handled - due to failure or skipping in a *_data() method or init, or a skip in cleanup. Fixes: QTBUG-95661 Change-Id: I5d24a37a53d3db225fa602649d8aad8f5ed6c1ad Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
* Make Skip an incident in test loggingEdward Welbourne2021-12-091-1/+1
| | | | | | | | | | | | | | | Skip ends the test (albeit inconclusively). Rearrange the enums in the abstract logger, move code to handle skip between relevant function and tidy up various things that became simpler as a result. Also reorder the message enum, and its switches, to separate testlib's internals from the usual Qt messages, and put each group in ascending order of severity. Task-number: QTBUG-96844 Change-Id: I2c7a634b9f849830d64eafa750155e66e244b729 Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io> Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* Add QTest::failOnWarningMitch Curtis2021-11-261-0/+51
| | | | | | | | | | | | | | This solves the long-standing problem of not being able to easily fail a test when a certain warning is output. [ChangeLog][QtTest] Added QTest::failOnWarning. When called in a test function, any warning that matches the given pattern will cause a test failure. The test will continue execution when a failure is added. All patterns are cleared at the end of each test function. Fixes: QTBUG-70029 Change-Id: I5763f8d4acf1cee8178be43a503619fbfb0f4f36 Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
* QTestLog: Properly own the loggersUlf Hermann2021-11-091-8/+10
| | | | | | | | | | | | | | | Previously, the loggers would leak if the application failed to call stopLogging(). Now they are owned by the global static which will delete them in that case. Also, since we have to adapt loggerCount() to the fact that std::vector uses size_t, recognize that we only ever want to know whether the number of loggers is 0. Change the method to only provide that information rather than the actual number. Change-Id: Ieb2e185048d573ec7f36373ad49bb2a0ca391ce3 Reviewed-by: Mitch Curtis <mitch.curtis@qt.io> Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
* Make clear why QTestLog::addB?XFail() don't add to countersEdward Welbourne2021-08-271-2/+4
| | | | | | | | | | | | | This reverts commit 904617dfb83f39a6a379635b64fea6fcd00f241a and makes clear to future readers why that wouldn't be a sensible change. Update the test's data, eliminating a case of duplicate counting that was caused by the reverted commit. Task-number: QTBUG-95661 Pick-to: 5.12 5.15 6.1 6.2 Change-Id: Ice6d3ab06ca171e0d6eb0fac757f1ab774e229f0 Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io> Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
* testlib: Rename QAbstractTestLogger::QSystem to QCriticalTor Arne Vestbø2021-08-101-1/+1
| | | | | | | | | | | | | It's used to handle qCritical() messages, and corresponds to QtMsgType's QtCriticalMsg, so change its name to reflect what it really is. In the process, reclassify the -maxwarnings overflow message as a warning, rather than a critical message. Pick-to: 6.2 Change-Id: I87626117a547ae4498d5dc352b93bd6db8bfb332 Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io> Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
* testlib: Pass on file location on failure, but don't assume we have oneTor Arne Vestbø2021-08-041-7/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We try our best to pass on the file location of a failure, including for fatal errors, but the reporting or logging machinery should not assume there is one. By passing on nullptr for the file location we allow the logging backends to decide how to handle the situation, e.g. by not emitting extra fields for failure location. This effectively reverts c25687fa0b6e4be043e1f8c92c093d8b06ca06c4, in favor of relying on the backends to cope with null filename, which they already did. As qFatal uses QMessageLogger, which by default disables file/line information in release builds, we need to explicitly enable this in our self-tests, to get uniform test results. Similarly, we disable file/line info from testlib itself, as reporting Qt internal file and line information for user diagnostics is less useful. The odd one out there is qtestdata.cpp, which still ends up in test output due to using QTEST_ASSERT instead of qFatal for its diagnostics. Cleaning up that, and unifying how we report testlib issues to the user, is left for another day. Pick-to: 6.2 Change-Id: Ib9451b8eed86fe3ade4a4dcaf0037e1a3450321c Reviewed-by: Edward Welbourne <edward.welbourne@qt.io> Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io> Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* testlib: Add private API to add test loggerTor Arne Vestbø2020-10-311-5/+19
| | | | | | | Allows adding test loggers outside of testlib. Change-Id: Iabcc780e441de96032a05fc0a386dd52e2f2f404 Reviewed-by: Simon Hausmann <hausmann@gmail.com>
* testlib: Let logger report whether it is logging to stdoutTor Arne Vestbø2020-10-311-5/+6
| | | | | Change-Id: I7df61b4fdc8c431a73c4b399e2f620923a7c7217 Reviewed-by: Simon Hausmann <hausmann@gmail.com>
* Use QList instead of QVector in qtbaseJarek Kobus2020-07-071-3/+3
| | | | | | | | Fixes all other QVector occurrences Task-number: QTBUG-84469 Change-Id: I5f9311298d341a9a3061a6a640539583d1618939 Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
* Use QList instead of QVector in testlibJarek Kobus2020-06-251-1/+1
| | | | | | Task-number: QTBUG-84469 Change-Id: Icbc3c3130399296f6b5a7e9a313ad4737669de00 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
* Port testlib from QStringRef to QStringViewLars Knoll2020-06-111-1/+1
| | | | | | Task-number: QTBUG-84319 Change-Id: I780945c8923c0a03e9c0a32d039da0c793f650fd Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* QTest::ignoreMessage(): interpret the message in UTF-8Edward Welbourne2020-05-081-2/+2
| | | | | | | | | | | | The message to ignore is given in source code, hence UTF-8; it was being ingested as local 8-bit, which lead to problems when a debug message wasn't 7-bit clean and the system's native encoding wasn't UTF-8. Modified QtTest's selftest to check encoding failure. Pick-to: 5.15 Change-Id: I898744a450115b6d2ee992f1d3b36d8efaeeff7e Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org> Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* Fix namespaced build of QTestLibKai Koehne2020-02-141-2/+2
| | | | | | | | This amends 93b8cdc974e584d80dbaabf459610a555af25b81 Fixes: QTBUG-82142 Change-Id: Id77a35a1e1c244b6778159e85b50728f2e4103db Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
* testlib: Make testlogs enumerable and printableTor Arne Vestbø2020-02-101-0/+2
| | | | | Change-Id: Id0ac30fe0a63757ef722634e6dd90806015467aa Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
* testlib: Clarify that our XUnit reporter is actually a JUnit reporterTor Arne Vestbø2020-02-031-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The reporter was probably named 'xunit' based on the historical use of xUnit to refer to testing frameworks derived from Smalltalk's SUnit. These frameworks typically added their own prefix, e.g. JUnit for Java, RUnit for R, etc. The most popular of these was the JUnit framework, and the corresponding XML output produced by the Ant built tool became somewhat of a de facto standard, which is probably why we chose to model our reporter after it. Nowadays however, naming it 'xunit' is problematic as there is actually a testing famework named xUnit.net, typically shortened to, you guessed it: xunit. Test report consumers will typically have a junit mode, and an xunit mode, and the latter could easily be mistaken for what testlib outputs, unless we clarify this. The clarification also allows us to safely extend our support for the JUnit XML format to incorporate some elements that are nowadays common, but where we are lagging behind the standard. [ChangeLog][QTestLib] The formerly named 'xunitxml' test reporter has been renamed to what it actually is: a JUnit test reporter, and is now triggered by passing -o junitxml to the test binary. Change-Id: Ieb20d3d2b5905c74e55b98174948cc70870c0ef9 Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io> Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
* Replace most use of QVariant::type and occurrences of QVariant::TypeOlivier Goffart2020-01-231-2/+2
| | | | | | | | | | | | | | | | | | | | | I made a clazy automated check that replaced the use of QVariant::Type by the equivalent in QMetaType. This has been deprecated since Qt 5.0, but many uses were not yet removed. In addition, there was some manual changes to fix the compilation errors. Adapted the Private API of QDateTimeParser and QMimeDataPrivate and adjust QDateTimeEdit and QSpinBox. QVariant(QVariant::Invalid) in qstylesheet made no sense. But note that in QVariant::save, we actually wanted to use the non-user type. In the SQL module, many changes were actually reverted because the API still expects QVarient::Type. Change-Id: I98c368490e4ee465ed3a3b63bda8b8eaa50ea67e Reviewed-by: Lars Knoll <lars.knoll@qt.io>
* QTestLib: Introduce nullptrFriedemann Kleint2019-07-111-5/+5
| | | | | | | | Apply Fixits by Qt Creator with some amendments. Task-number: QTBUG-69413 Change-Id: I620e40a277dc2b20b0ec26fc32577e66b0456bb3 Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
* QTestLib: Use member initializationFriedemann Kleint2019-07-111-2/+2
| | | | | | | | Apply Fixits by Qt Creator with some amendments. Task-number: QTBUG-69413 Change-Id: Iba0834dc89cbfc215acc5873f31fa6eeed74177d Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
* Port from QAtomic::load() to loadRelaxed()Giuseppe D'Angelo2019-06-201-1/+1
| | | | | | | | | | | | | | | Semi-automated, just needed ~20 manual fixes: $ find \( -iname \*.cpp -or -iname \*.h \) -exec perl -pe 's/(\.|->)load\(\)/$1loadRelaxed\(\)/g' -i \{\} + $ find \( -iname \*.cpp -or -iname \*.h \) -exec perl -pe 's/(\.|->)store\(/$1storeRelaxed\(/g' -i \{\} + It can be easily improved (e.g. for store check that there are no commas after the opening parens). The most common offender is QLibrary::load, and some code using std::atomic directly. Change-Id: I07c38a3c8ed32c924ef4999e85c7e45cf48f0f6c Reviewed-by: Marc Mutz <marc.mutz@kdab.com>
* Merge remote-tracking branch 'origin/5.12' into 5.13Liang Qi2019-03-061-137/+58
|\ | | | | | | | | | | | | Conflicts: src/network/access/http2/hpacktable_p.h Change-Id: Ie0c296667dfdebba84f4858056a1ac80c24ee7df
| * testlib: Prevent Apple test logger from stomping on other loggersTor Arne Vestbø2019-03-051-8/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | We were potentially adding the Apple test logger multiple times, and we didn't consider whether the existing loggers were logging to file or not when circumventing them. We now don't use the Apple logger if it would touch stderr and some other logger is using stdout. In the case of no explicit logger being specified on the command line, we allow the Apple logger to take priority over the default plain test logger. Change-Id: I31bbec4f4b3ab84ba9a2be35e8e5db08fee071a7 Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
| * testlib: Replace manual logger linked list with QVectorTor Arne Vestbø2019-03-051-129/+53
| | | | | | | | | | | | | | Removes useless indirection and builds on existing primitives in Qt. Change-Id: I9fe50b21f5f77fc02566d5f5ff04c3e94c830e81 Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
* | Merge remote-tracking branch 'origin/5.12' into 5.13Qt Forward Merge Bot2019-02-211-0/+2
|\| | | | | | | Change-Id: I830beea26863323ab78a5d4b093f7763d77ad3da
| * Add missing increment of blacklisted tests for BXFAIL caseEdward Welbourne2019-02-201-0/+2
| | | | | | | | | | | | | | | | | | The counter is just how many tests were blacklisted, regardless of their success or otherwise. Skipping its increment for BXFAIL is apt to introduce noise in our tracking of how many tests are blacklisted. Change-Id: I1dd74e5f6619121c21d8741be7bc4e2d1cb43fa9 Reviewed-by: Christian Stenger <christian.stenger@qt.io>
* | Merge remote-tracking branch 'origin/5.12' into 5.13Liang Qi2019-02-081-0/+18
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: src/android/templates/AndroidManifest.xml src/network/ssl/qsslsocket_mac.cpp src/widgets/styles/qstylesheetstyle.cpp tests/auto/corelib/kernel/qtimer/BLACKLIST tests/auto/testlib/selftests/blacklisted/tst_blacklisted.cpp tests/auto/testlib/selftests/expected_blacklisted.lightxml tests/auto/testlib/selftests/expected_blacklisted.tap tests/auto/testlib/selftests/expected_blacklisted.teamcity tests/auto/testlib/selftests/expected_blacklisted.txt tests/auto/testlib/selftests/expected_blacklisted.xml tests/auto/testlib/selftests/expected_blacklisted.xunitxml tests/auto/testlib/selftests/expected_float.tap tests/auto/testlib/selftests/expected_float.teamcity tests/auto/testlib/selftests/expected_float.txt tests/auto/testlib/selftests/expected_float.xunitxml Done-With: Christian Ehrlicher <ch.ehrlicher@gmx.de> Done-With: Edward Welbourne <edward.welbourne@qt.io> Done-With: Timur Pocheptsov <timur.pocheptsov@qt.io> Change-Id: If93cc432a56ae3ac1b6533d0028e4dc497415a52
| * testlib: Add BXPASS and BXFAILOliver Wolff2019-01-291-0/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Prioritize blacklisting over QEXPECT_FAIL so that a test that is blacklisted no longer fails if QEXPECT_FAIL returns true unexpectedly. To reflect this state properly, the two values of BXPASS and BXFAIL were added to testlib's output. [ChangeLog][Important Behavior Changes][QtTestLib] Blacklisting of tests will be taken into account for XPASS and XFAIL. A blacklisted test that causes an XPASS will no longer be a fail. Task-number: QTBUG-72928 Change-Id: Ia2232fdc714d405fa3fd9aea6c89eb2836bc5950 Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
* | Add a feature to enable CodeCoverage analysis of testlibEdward Welbourne2019-01-241-0/+5
|/ | | | | | | | | | Based on Asmo Saarela's advice (QTPM-686), adapted on advice from FrogLogic support and converted to a feature so that the selftest and testlib qmake config can be co-ordinated. Task-number: QTPM-1385 Change-Id: Icd706f086009e1e08b3f8c5cd553f792402e28c0 Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
* Merge remote-tracking branch 'origin/5.11' into devQt Forward Merge Bot2018-03-211-4/+6
|\ | | | | | | Change-Id: I35a6555e3885e489f88aa9b4b0142e1017f7a959
| * Modernize the "regularexpression" featureUlf Hermann2018-03-201-4/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | Use QT_CONFIG(regularexpression), disentangle it from QT_BOOTSTRAPPED, switch it off in the bootstrap build, remove the #ifdefs from qregularexpression.{h|cpp}, and add QT_REQUIRE_CONFIG(regularexpression) to the header. qregularexpression.{h|cpp} are already correctly excluded in tools.pri if !qtConfig(regularexpression). Change-Id: I21de154a6a118b76f99003d3acb72ac1e220d302 Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@qt.io>
* | testlib: Add Test Anything Protocol (TAP) reporterTor Arne Vestbø2018-03-141-0/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The Test Anything Protocol (TAP), was originally Perl's simple text-based interface between testing modules and test harnesses, but has since been adopted by a large number of producers and consumers in many different languages, which allows colorizing and summarizing test results. The format is very simple: TAP version 13 ok 1 - test description not ok 2 - test description --- message: 'Failure message' severity: fail expected: 123 actual: 456 ... ok 3 - test description # SKIP 1..3 The specification [1] is very brief, so the implementation has been based on how typical consumers behave, especially when it comes to the undefined diagnostics block. [1] http://testanything.org/tap-version-13-specification.html Change-Id: I616e802ea380165c678510e940ddc6607d39c92d Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
* | testlib: Feed test log messages to Apple Unified LoggingTor Arne Vestbø2018-03-091-0/+13
|/ | | | | | | | | | | | | | | | | | | QtTestLib uses its own message handler for plain text logging, so it doesn't go though the logic in qDefaultMessageHandler that dispatches logging to alternate logging sinks. Building on the approach of Android and Windows, we log to AUL from the plain text logger. A future improvement in this area would be to make QtTestLib make more use of the infrastructure in qlogging.cpp, but this is a bigger task and requires a better overview of the requirements. The AUL code has been isolated into a wrapper, to allow logging both to AUl and to e.g. the QPlainTextLogger. This is needed so that we short circuit the plain text logger in case AUL already has handled printing to stderr. Change-Id: Iee0c3902190bd081b7ffbaf77c41b3118ce7f6da Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
* testlib: Pass on log message context to loggersTor Arne Vestbø2018-02-211-20/+12
| | | | | Change-Id: I1e0bb09d4f96b45f1e9757ea25ae3aba1ae42447 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* Add TeamCity logging feature to testlibBorgar Ovsthus2016-01-231-0/+4
| | | | | | | | | | | | | This allows TeamCity to parse realtime test-results instead of using post-build XML Report Processing. This does not support logging of benchmarks. [ChangeLog][QTest] Added a new logging mode that allow test-results to be parsed on-the-fly when using Jetbrains TeamCity as CI-server. This mode is enabled by using the -teamcity option on the command-line. Change-Id: Ie730beb643043eb0f448f99abe6c0b5ac48aaf03 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* Updated license headersJani Heikkinen2016-01-151-14/+20
| | | | | | | | | | | From Qt 5.7 -> LGPL v2.1 isn't an option anymore, see http://blog.qt.io/blog/2016/01/13/new-agreement-with-the-kde-free-qt-foundation/ Updated license headers to use new LGPL header instead of LGPL21 one (in those files which will be under LGPL v3) Change-Id: I046ec3e47b1876cd7b4b0353a576b352e3a946d9 Reviewed-by: Lars Knoll <lars.knoll@theqtcompany.com>
* Testlib: Output function / total time along with crash dump.Friedemann Kleint2015-10-131-0/+17
| | | | | | | | | | | | | | | | | | | Previously, only QXmlTestLogger had timers to take elapsed times and log them. Move those into class QTestLog for access by the loggers and output the times in the crash dump to make it easier to spot hangs/recursion crashes. Produces: QFATAL : foo() Received signal 11 Function time: 22ms Total time: 23ms A crash occurred in ... Function time: 24ms Total time: 26ms Task-number: QTBUG-47370 Change-Id: Ia530a63104087daffc9a15f68c15d93378b9407e Reviewed-by: Simon Hausmann <simon.hausmann@theqtcompany.com>
* Add XCTest logger backend to QtTestLibTor Arne Vestbø2015-03-271-0/+9
| | | | | | | | | | | | | | | | Will be active when running test apps through Xcode's 'test' action, and reports QtTestLib test objects and functions to Xcode as XCTest cases. This allows running tests on both iOS Simulator and iOS devices from the command line, through xcodebuild, without relying on any 3rd party tools. It also integrates Qt test failures and passes into the Xcode IDE, which may be useful for closer investigation of test failures. The feature is limited to Xcode 6.x. Change-Id: I33d39edbabdbaebef48d2d0eb7e08a1ffb72c397 Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@theqtcompany.com>
generated by cgit v1.2.3 (git 2.25.1) at 2024-05-09 07:30:52 +0000