summaryrefslogtreecommitdiffstats
path: root/src/testlib/qabstracttestlogger_p.h
Commit message (Collapse)AuthorAgeFilesLines
* Add QTest option for repeating the entire test executionJøger Hansegård2023-11-151-0/+2
| | | | | | | | | | | | | | | | | Repeated test execution can be useful, under a debugger, to catch an intermittent failure or, under memory instrumentation, to make memory leaks easier to recognize. The new -repeat flag allows running the entire test suite multiple times within the same process. It works by executing all tests sequentially before repeating the execution again. This switch is a developer tool, and is not intended for CI. It can only be used with the plain text logger. Change-Id: I2439462c5c44d1c8aa3d3b5656de3eef44898c68 Reviewed-by: Edward Welbourne <edward.welbourne@qt.io> Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
* QPlainTestLogger: don't print "RESULT" before each additional resultThiago Macieira2022-11-091-0/+1
| | | | | | | | | | | | | | | | | We do that by passing the full list of results to the logger, to a virtual that is present in the base class to call the existing function. For all but the plain logger, we'll just print multiple results. The plain logger now prints: RESULT : tst_MyClass::QString_toInt() 383 nsecs per iteration (total: 3,837,324, iterations: 10000) 1,069 CPU cycles per iteration (total: 10,692,457, iterations: 10000) 3,123 instructions per iteration (total: 31,230,101, iterations: 10000) 536 branch instructions per iteration (total: 5,360,022, iterations: 10000) Change-Id: I3c79b7e08fa346988dfefffd17203cb5802693dd Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org> Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
* Use SPDX license identifiersLucie Gérard2022-05-161-38/+2
| | | | | | | | | | | | | Replace the current license disclaimer in files by a SPDX-License-Identifier. Files that have to be modified by hand are modified. License files are organized under LICENSES directory. Task-number: QTBUG-67283 Change-Id: Id880c92784c40f3bbde861c0d93f58151c18b9f1 Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org> Reviewed-by: Lars Knoll <lars.knoll@qt.io> Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
* QAbstractTestLogger: disable copyingMarc Mutz2022-05-031-0/+1
| | | | | | | | | Fixes clazy-copyable-polymorphic. Pick-to: 6.3 6.2 5.15 Change-Id: I31a68da6e402ada91099b09540b29a1d56f9cc56 Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io> Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
* TAP test logger: move messages into the diagnostics blockEdward Welbourne2022-03-291-2/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | Our TAP output was delivering messages as comments before the test line, where TAP clearly expects the details of a test to follow its test line. Version 13 provides a YAML block to deliver diagnostics and encourages use of this, so accumulate our messages in a QTestCharBuffer instead of emitting them one by one. However, messages produced after a test has produced its test line belong to that test, but are too late to be included in its diagnostics block, so should be emitted immediately as before, albeit now with a type prefix. This at least separates such messages, from the end of one test, from messages produced early in the next. In the process, add a type-prefix to each, to make clear what type of message it was. Since the Yamlish supported by TAP consumers doesn't support a way to have many messages, use the extensions: top-level hash tag with a messages: sub-tag to gather our messages as a list. (This expands at least one expected output file significantly and substantially rewrites some others.) Add methods to QTestCharBuffer, and a helper function, to support this. Task-number: QTBUG-96844 Change-Id: If44a33da5879ed1670ef0980042599afd516f9d2 Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
* Make sure all qtbase private headers include at least one otherThiago Macieira2022-02-241-0/+1
| | | | | | | | | | See script in qtbase/util/includeprivate for the rules. Since these files are being touched anyway, I also ran the updatecopyright.pl script too. Change-Id: Ib056b47dde3341ef9a52ffff13ef677e471674b6 Reviewed-by: Marc Mutz <marc.mutz@qt.io>
* Make Skip an incident in test loggingEdward Welbourne2021-12-091-5/+6
| | | | | | | | | | | | | | | Skip ends the test (albeit inconclusively). Rearrange the enums in the abstract logger, move code to handle skip between relevant function and tidy up various things that became simpler as a result. Also reorder the message enum, and its switches, to separate testlib's internals from the usual Qt messages, and put each group in ascending order of severity. Task-number: QTBUG-96844 Change-Id: I2c7a634b9f849830d64eafa750155e66e244b729 Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io> Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* testlib: Rename QAbstractTestLogger::QSystem to QCriticalTor Arne Vestbø2021-08-101-1/+1
| | | | | | | | | | | | | It's used to handle qCritical() messages, and corresponds to QtMsgType's QtCriticalMsg, so change its name to reflect what it really is. In the process, reclassify the -maxwarnings overflow message as a warning, rather than a critical message. Pick-to: 6.2 Change-Id: I87626117a547ae4498d5dc352b93bd6db8bfb332 Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io> Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
* QTestCharBuffer: kill copies and movesGiuseppe D'Angelo2021-06-201-0/+2
| | | | | | | The compiler generated ones are broken. The RO5 exists for a reason... Change-Id: Ibaa5322a7a1ad4ad06bd00e9d1c21bbbea6eb08c Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
* testlib: Add private API to add test loggerTor Arne Vestbø2020-10-311-3/+4
| | | | | | | Allows adding test loggers outside of testlib. Change-Id: Iabcc780e441de96032a05fc0a386dd52e2f2f404 Reviewed-by: Simon Hausmann <hausmann@gmail.com>
* testlib: Let logger report whether it is logging to stdoutTor Arne Vestbø2020-10-311-0/+2
| | | | | Change-Id: I7df61b4fdc8c431a73c4b399e2f620923a7c7217 Reviewed-by: Simon Hausmann <hausmann@gmail.com>
* QTestLib: Use member initializationFriedemann Kleint2019-07-111-3/+2
| | | | | | | | Apply Fixits by Qt Creator with some amendments. Task-number: QTBUG-69413 Change-Id: Iba0834dc89cbfc215acc5873f31fa6eeed74177d Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
* More nullptr usage in headersKevin Funk2019-03-141-3/+3
| | | | | | | | | | | Diff generated by running clang-tidy's modernize-use-nullptr checker on the CMake-based Qt version. Skipping src/3rdparty, examples/, tests/ Change-Id: Ib182074e2e2fd52f63093f73b3e2e4c0cb7af188 Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io> Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
* testlib: Add BXPASS and BXFAILOliver Wolff2019-01-291-1/+3
| | | | | | | | | | | | | | | Prioritize blacklisting over QEXPECT_FAIL so that a test that is blacklisted no longer fails if QEXPECT_FAIL returns true unexpectedly. To reflect this state properly, the two values of BXPASS and BXFAIL were added to testlib's output. [ChangeLog][Important Behavior Changes][QtTestLib] Blacklisting of tests will be taken into account for XPASS and XFAIL. A blacklisted test that causes an XPASS will no longer be a fail. Task-number: QTBUG-72928 Change-Id: Ia2232fdc714d405fa3fd9aea6c89eb2836bc5950 Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
* testlib: Add Test Anything Protocol (TAP) reporterTor Arne Vestbø2018-03-141-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The Test Anything Protocol (TAP), was originally Perl's simple text-based interface between testing modules and test harnesses, but has since been adopted by a large number of producers and consumers in many different languages, which allows colorizing and summarizing test results. The format is very simple: TAP version 13 ok 1 - test description not ok 2 - test description --- message: 'Failure message' severity: fail expected: 123 actual: 456 ... ok 3 - test description # SKIP 1..3 The specification [1] is very brief, so the implementation has been based on how typical consumers behave, especially when it comes to the undefined diagnostics block. [1] http://testanything.org/tap-version-13-specification.html Change-Id: I616e802ea380165c678510e940ddc6607d39c92d Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
* testlib: Feed test log messages to Apple Unified LoggingTor Arne Vestbø2018-03-091-0/+5
| | | | | | | | | | | | | | | | | | | QtTestLib uses its own message handler for plain text logging, so it doesn't go though the logic in qDefaultMessageHandler that dispatches logging to alternate logging sinks. Building on the approach of Android and Windows, we log to AUL from the plain text logger. A future improvement in this area would be to make QtTestLib make more use of the infrastructure in qlogging.cpp, but this is a bigger task and requires a better overview of the requirements. The AUL code has been isolated into a wrapper, to allow logging both to AUl and to e.g. the QPlainTextLogger. This is needed so that we short circuit the plain text logger in case AUL already has handled printing to stderr. Change-Id: Iee0c3902190bd081b7ffbaf77c41b3118ce7f6da Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
* testlib: Pass on log message context to loggersTor Arne Vestbø2018-02-211-0/+3
| | | | | Change-Id: I1e0bb09d4f96b45f1e9757ea25ae3aba1ae42447 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* Updated license headersJani Heikkinen2016-01-151-14/+20
| | | | | | | | | | | From Qt 5.7 -> LGPL v2.1 isn't an option anymore, see http://blog.qt.io/blog/2016/01/13/new-agreement-with-the-kde-free-qt-foundation/ Updated license headers to use new LGPL header instead of LGPL21 one (in those files which will be under LGPL v3) Change-Id: I046ec3e47b1876cd7b4b0353a576b352e3a946d9 Reviewed-by: Lars Knoll <lars.knoll@theqtcompany.com>
* Update copyright headersJani Heikkinen2015-02-111-7/+7
| | | | | | | | | | | | | | | | | | Qt copyrights are now in The Qt Company, so we could update the source code headers accordingly. In the same go we should also fix the links to point to qt.io. Outdated header.LGPL removed (use header.LGPL21 instead) Old header.LGPL3 renamed to header.LGPL3-COMM to match actual licensing combination. New header.LGPL-COMM taken in the use file which were using old header.LGPL3 (src/plugins/platforms/android/extract.cpp) Added new header.LGPL3 containing Commercial + LGPLv3 + GPLv2 license combination Change-Id: I6f49b819a8a20cc4f88b794a8f6726d975e8ffbe Reviewed-by: Matti Paaso <matti.paaso@theqtcompany.com>
* Add QtInfoMsgKai Koehne2015-01-091-2/+3
| | | | | | | | | | | | | | Add an 'info' message type that can be used for messages that are neither warnings (QtWarningMsg), nor for debugging only (QtDebugMsg). This is useful mainly for applications that do not have to adhere to the 'do not print anything by default' paradigm that we have for the Qt libraries itself. [ChangeLog][QtCore][Logging] QtInfoMsg got added as a new QtMsgType. Use the new qInfo(), qCInfo() macros to log to it. Change-Id: I810995d63de46c41a9a99a34d37c0d417fa87a05 Reviewed-by: Jason McDonald <macadder1@gmail.com>
* Update license headers and add new license filesMatti Paaso2014-09-241-19/+11
| | | | | | | | | - Renamed LICENSE.LGPL to LICENSE.LGPLv21 - Added LICENSE.LGPLv3 - Removed LICENSE.GPL Change-Id: Iec3406e3eb3f133be549092015cefe33d259a3f2 Reviewed-by: Iikka Eklund <iikka.eklund@digia.com>
* Add support for blacklisting test functionsLars Knoll2014-09-211-1/+3
| | | | | | | | | | | | | | | | | We need to have a finer grained control over the tests we skip in our CI system. This adds a blacklisting mechanism that allows blacklisting individual test functions (or even test data) using a set of predefined matching keys for the operating system and some other relevant variables. QTestlib will search for a file called BLACKLIST in the test directory and parse it if found. The file contains a simple ini style list of functions to blacklist. For details see qtestblacklist.cpp. Change-Id: Id3fae4b264ca99970cbf9f45bfb85fa75c1fd823 Reviewed-by: Lars Knoll <lars.knoll@digia.com>
* Testlib: Use QString for messages in QAbstractTestLogger::addMessage()Friedemann Kleint2014-01-211-1/+1
| | | | | | | | Task-number: QTBUG-35743 [ChangeLog][QtTest][Windows] Use correct UTF-8 encoding for XML test results on platforms with different console encoding. Change-Id: Ice9d03192098f931e5dac358928e0c4421ab715e Reviewed-by: Frederik Gladhorn <frederik.gladhorn@digia.com>
* Update copyright year in Digia's license headersSergio Ahumada2013-01-181-1/+1
| | | | | Change-Id: Ic804938fc352291d011800d21e549c10acac66fb Reviewed-by: Lars Knoll <lars.knoll@digia.com>
* Change copyrights from Nokia to DigiaIikka Eklund2012-09-221-24/+24
| | | | | | | | Change copyrights and license headers from Nokia to Digia Change-Id: If1cc974286d29fd01ec6c19dd4719a67f4c3f00e Reviewed-by: Lars Knoll <lars.knoll@digia.com> Reviewed-by: Sergio Ahumada <sergio.ahumada@digia.com>
* Remove all calls to, and deprecate qMalloc, qRealloc and qFree.Robin Burchell2012-03-311-3/+4
| | | | | | | | | | | | | | | | | | Callers should just call the standard allocation functions directly. Adding an extra function call onto all basic memory management for the sake of making it instrumentable in rare cases isn't really fair to everyone else. What's more, this wasn't completely reliable, as not everything was using them in a number of places. Memory management can still be overridden using tricks like LD_PRELOAD if needed. Their aligned equivilents cannot be deprecated, as no standard equivilents exist, although investigation into posix_memalign(3) is a possibility for the future. Change-Id: Ic5f74b14be33f8bc188fe7236c55e15c36a23fc7 Reviewed-by: Lars Knoll <lars.knoll@nokia.com>
* Remove "All rights reserved" line from license headers.Jason McDonald2012-01-301-1/+1
| | | | | | | | | | As in the past, to avoid rewriting various autotests that contain line-number information, an extra blank line has been inserted at the end of the license text to ensure that this commit does not change the total number of lines in the license header. Change-Id: I311e001373776812699d6efc045b5f742890c689 Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* Update contact information in license headers.Jason McDonald2012-01-231-1/+1
| | | | | | | Replace Nokia contact email address with Qt Project website. Change-Id: I431bbbf76d7c27d8b502f87947675c116994c415 Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* Update copyright year in license headers.Jason McDonald2012-01-051-1/+1
| | | | | Change-Id: I02f2c620296fcd91d4967d58767ea33fc4e1e7dc Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* Filter unprintable chars out of all test output.Jason McDonald2011-12-121-0/+1
| | | | | | | | | | | | | | | | | | Previously, unprintable characters were filtered out of test output while the output strings were being formatted by either qt_snprintf() or qt_asprintf(). Any strings not formatted by one of those functions weren't filtered at all, and any strings passed more than once would be filtered more than once. This commit separates the filtering of output strings from their formatting, leaving the filtering until just before the strings are written to the output stream. For now, the filtering is done by a protected method of QAbstractTestLogger, but this could easily be changed to a virtual method in future to allow different filtering for loggers with different output character sets. Change-Id: Ia4bb49cd10d37c84af75d2cf58325d27f0e16d99 Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* Pass log file name when constructing loggers.Jason McDonald2011-09-051-3/+3
| | | | | | | | | | | Pass the filename when creating the logger rather than when commencing output. This will simplify forthcoming changes for running multiple loggers. Change-Id: I563aa97661caf279a5dcccb1321ff3ce4725b332 Reviewed-on: http://codereview.qt.nokia.com/4116 Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com> Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* Remove circular dependency in testlib loggingJason McDonald2011-09-021-1/+1
| | | | | | | | | | | | Remove one of the circular dependencies between QPlainTestLogger and QTestLog by directly checking whether we're writing to stdout rather than inferring the same by examining whether the output file name is null. Change-Id: I798288482c9e2e071e17a8622e8a8f8d5016dc7e Reviewed-on: http://codereview.qt.nokia.com/4052 Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com> Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* Remove ability to run tests in random order.Jason McDonald2011-08-311-2/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | Remove the undocumented feature that allows test functions to be executed in random order. The feature was designed to expose unintended dependencies between test functions -- test functions are only supposed to depend on the initTestCase() and init() functions. Aside from the lack of documentation, there are a number of problems with this feature. Most importantly, running the tests in random order has only a 50% chance of exposing dependencies between test functions. A better strategy would be to run the test functions in reverse order and complain if that produces different results to running the tests in the normal order. Additionally, the random order is not deterministic, so even if a dependency is exposed during a test run, there's no guarantee that it will be exposed again. The feature allows the user to optionally supply a random seed to make the "random" order deterministic, but as rand() implementations are not identical across platforms, even that does not guarantee that dependencies between test functions will be exposed deterministically. Change-Id: I39eac34c532ccb988116778bbc5ab05d835874c5 Reviewed-on: http://codereview.qt.nokia.com/3720 Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com> Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* Enable multiple instances of QAbstractTestLogger.Jason McDonald2011-08-291-2/+6
| | | | | | | | | | | | | | | | | | | | Previously QAbstractTestLogger used a global variable for the file pointer to which it was writing test output. This effectively meant that only one instance of this or its derived classes could exist at any time. This commit moves the file pointer inside the class, so that multiple loggers can exist at the same time. This means that the outputString() method can no longer be static, which in turn means that several functions used by QPlainTestLogger need to move from the QTest namespace into the class, and also that QTestBasicStreamer must hold a non-const pointer to its associated logger instead of a const pointer. Task-number: QTBUG-20615 Change-Id: If941f1f9399cf20fb93e3e87f3390bceeca1cbfc Reviewed-on: http://codereview.qt.nokia.com/3576 Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com> Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* Remove color logging feature from qtestlib.Jason McDonald2011-08-291-1/+0
| | | | | | | | | | | | | | | | This feature was undocumented and only worked if a secret environment variable was set, the test output was going to a real console, and plain text test results were being generated. Including code in testlib for fancy presentation of test results conflicts with testlib's stated goal of being a lightweight framework where every feature contributes to finding bugs, so this feature is being removed. If fancy presentation of test output is required, it should be achieved by post-processing the test output outside of the test framework. Change-Id: I872165c4d2c3d2498c3aa039070ecf319e237ca1 Reviewed-on: http://codereview.qt.nokia.com/3432 Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com> Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* Remove circular dependency in testlib logging codeJason McDonald2011-08-291-1/+1
| | | | | | | | | | | Pass the output file name from the QTestLog to the test logger when commencing logging rather than having the logger call back into the QTestLog. Change-Id: Id484635f9fcfca08a66c92f3442887e9473b6f9b Reviewed-on: http://codereview.qt.nokia.com/3454 Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com> Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* Update licenseheader text in source files for qtbase Qt moduleJyri Tahtela2011-05-241-17/+17
| | | | | | | Updated version of LGPL and FDL licenseheaders. Apply release phase licenseheaders for all source files. Reviewed-by: Trust Me
* Initial import from the monolithic Qt.Qt by Nokia2011-04-271-0/+174
This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
generated by cgit v1.2.3 (git 2.25.1) at 2024-04-27 12:21:49 +0000