/**************************************************************************** ** ** Copyright (C) 2012 Digia Plc and/or its subsidiary(-ies). ** Contact: http://www.qt-project.org/legal ** ** This file is part of the test suite of the Qt Toolkit. ** ** $QT_BEGIN_LICENSE:LGPL$ ** Commercial License Usage ** Licensees holding valid commercial Qt licenses may use this file in ** accordance with the commercial license agreement provided with the ** Software or, alternatively, in accordance with the terms contained in ** a written agreement between you and Digia. For licensing terms and ** conditions see http://qt.digia.com/licensing. For further information ** use the contact form at http://qt.digia.com/contact-us. ** ** GNU Lesser General Public License Usage ** Alternatively, this file may be used under the terms of the GNU Lesser ** General Public License version 2.1 as published by the Free Software ** Foundation and appearing in the file LICENSE.LGPL included in the ** packaging of this file. Please review the following information to ** ensure the GNU Lesser General Public License version 2.1 requirements ** will be met: http://www.gnu.org/licenses/old-licenses/lgpl-2.1.html. ** ** In addition, as a special exception, Digia gives you certain additional ** rights. These rights are described in the Digia Qt LGPL Exception ** version 1.1, included in the file LGPL_EXCEPTION.txt in this package. ** ** GNU General Public License Usage ** Alternatively, this file may be used under the terms of the GNU ** General Public License version 3.0 as published by the Free Software ** Foundation and appearing in the file LICENSE.GPL included in the ** packaging of this file. Please review the following information to ** ensure the GNU General Public License version 3.0 requirements will be ** met: http://www.gnu.org/copyleft/gpl.html. ** ** ** $QT_END_LICENSE$ ** ****************************************************************************/ /*! \qmltype TestCase \instantiates TestCase \brief Represents a unit test case \ingroup qtquick-utility \since 4.8 \ingroup qtest::qml \section1 Introduction to QML test cases Test cases are written as JavaScript functions within a TestCase element: \code import QtQuick 2.0 import QtTest 1.0 TestCase { name: "MathTests" function test_math() { compare(2 + 2, 4, "2 + 2 = 4") } function test_fail() { compare(2 + 2, 5, "2 + 2 = 5") } } \endcode Functions whose names start with "test_" are treated as test cases to be executed. The \l name property is used to prefix the functions in the output: \code ********* Start testing of MathTests ********* Config: Using QTest library 4.7.2, Qt 4.7.2 PASS : MathTests::initTestCase() FAIL! : MathTests::test_fail() 2 + 2 = 5 Actual (): 4 Expected (): 5 Loc: [/home/.../tst_math.qml(12)] PASS : MathTests::test_math() PASS : MathTests::cleanupTestCase() Totals: 3 passed, 1 failed, 0 skipped ********* Finished testing of MathTests ********* \endcode Because of the way JavaScript properties work, the order in which the test functions are found is unpredictable. To assist with predictability, the test framework will sort the functions on ascending order of name. This can help when there are two tests that must be run in order. Multiple TestCase elements can be supplied. The test program will exit once they have all completed. If a test case doesn't need to run (because a precondition has failed), then \l optional can be set to true. \section1 Data-driven tests Table data can be provided to a test using a function name that ends with "_data". Alternatively, the \c init_data() function can be used to provide default test data for all test functions in a TestCase element: \code import QtQuick 2.0 import QtTest 1.0 TestCase { name: "DataTests" function init_data() { return [ {tag:"init_data_1", a:1, b:2, answer: 3}, {tag:"init_data_2", a:2, b:4, answer: 6} ]; } function test_table_data() { return [ {tag: "2 + 2 = 4", a: 2, b: 2, answer: 4 }, {tag: "2 + 6 = 8", a: 2, b: 6, answer: 8 }, ] } function test_table(data) { //data comes from test_table_data compare(data.a + data.b, data.answer) } function test__default_table(data) { //data comes from init_data compare(data.a + data.b, data.answer) } } \endcode The test framework will iterate over all of the rows in the table and pass each row to the test function. As shown, the columns can be extracted for use in the test. The \c tag column is special - it is printed by the test framework when a row fails, to help the reader identify which case failed amongst a set of otherwise passing tests. \section1 Benchmarks Functions whose names start with "benchmark_" will be run multiple times with the Qt benchmark framework, with an average timing value reported for the runs. This is equivalent to using the \c{QBENCHMARK} macro in the C++ version of QTestLib. \code TestCase { id: top name: "CreateBenchmark" function benchmark_create_component() { var component = Qt.createComponent("item.qml") var obj = component.createObject(top) obj.destroy() component.destroy() } } RESULT : CreateBenchmark::benchmark_create_component: 0.23 msecs per iteration (total: 60, iterations: 256) PASS : CreateBenchmark::benchmark_create_component() \endcode To get the effect of the \c{QBENCHMARK_ONCE} macro, prefix the test function name with "benchmark_once_". \section1 Simulating keyboard and mouse events The keyPress(), keyRelease(), and keyClick() methods can be used to simulate keyboard events within unit tests. The events are delivered to the currently focused QML item. \code Rectangle { width: 50; height: 50 focus: true TestCase { name: "KeyClick" when: windowShown function test_key_click() { keyClick(Qt.Key_Left) ... } } } \endcode The mousePress(), mouseRelease(), mouseClick(), mouseDoubleClick(), and mouseMove() methods can be used to simulate mouse events in a similar fashion. \b{Note:} keyboard and mouse events can only be delivered once the main window has been shown. Attempts to deliver events before then will fail. Use the \l when and windowShown properties to track when the main window has been shown. \sa SignalSpy */ /*! \qmlproperty string TestCase::name This property defines the name of the test case for result reporting. The default is the empty string. \code TestCase { name: "ButtonTests" ... } \endcode */ /*! \qmlproperty bool TestCase::when This property should be set to true when the application wants the test cases to run. The default value is true. In the following example, a test is run when the user presses the mouse button: \code Rectangle { id: foo width: 640; height: 480 color: "cyan" MouseArea { id: area anchors.fill: parent } property bool bar: true TestCase { name: "ItemTests" when: area.pressed id: test1 function test_bar() { verify(bar) } } } \endcode The test application will exit once all \l TestCase elements have been triggered and have run. The \l optional property can be used to exclude a \l TestCase element. \sa optional, completed */ /*! \qmlproperty bool TestCase::optional Multiple \l TestCase elements can be supplied in a test application. The application will exit once they have all completed. If a test case does not need to run (because a precondition has failed), then this property can be set to true. The default value is false. \code TestCase { when: false optional: true function test_not_run() { verify(false) } } \endcode \sa when, completed */ /*! \qmlproperty bool TestCase::completed This property will be set to true once the test case has completed execution. Test cases are only executed once. The initial value is false. \sa running, when */ /*! \qmlproperty bool TestCase::running This property will be set to true while the test case is running. The initial value is false, and the value will become false again once the test case completes. \sa completed, when */ /*! \qmlproperty bool TestCase::windowShown This property will be set to true after the QML viewing window has been displayed. Normally test cases run as soon as the test application is loaded and before a window is displayed. If the test case involves visual elements and behaviors, then it may need to be delayed until after the window is shown. \code Button { id: button onClicked: text = "Clicked" TestCase { name: "ClickTest" when: windowShown function test_click() { button.clicked(); compare(button.text, "Clicked"); } } } \endcode */ /*! \qmlmethod TestCase::fail(message = "") Fails the current test case, with the optional \a message. Similar to \c{QFAIL(message)} in C++. */ /*! \qmlmethod TestCase::verify(condition, message = "") Fails the current test case if \a condition is false, and displays the optional \a message. Similar to \c{QVERIFY(condition)} or \c{QVERIFY2(condition, message)} in C++. */ /*! \qmlmethod TestCase::compare(actual, expected, message = "") Fails the current test case if \a actual is not the same as \a expected, and displays the optional \a message. Similar to \c{QCOMPARE(actual, expected)} in C++. \sa tryCompare(), fuzzyCompare */ /*! \qmlmethod TestCase::fuzzyCompare(actual, expected, delta, message = "") Fails the current test case if the difference betwen \a actual and \a expected is greater than \a delta, and displays the optional \a message. Similar to \c{qFuzzyCompare(actual, expected)} in C++ but with a required \a delta value. This funtion can also be used for color comparisons if both the \a actual and \a expected values can be converted into color values. If any of the differences for RGBA channel values are greater than \a delta, the test fails. \sa tryCompare(), compare() */ /*! \qmlmethod TestCase::tryCompare(obj, property, expected, timeout = 5000) Fails the current test case if the specified \a property on \a obj is not the same as \a expected. The test will be retried multiple times until the \a timeout (in milliseconds) is reached. This function is intended for testing applications where a property changes value based on asynchronous events. Use compare() for testing synchronous property changes. \code tryCompare(img, "status", BorderImage.Ready) compare(img.width, 120) compare(img.height, 120) compare(img.horizontalTileMode, BorderImage.Stretch) compare(img.verticalTileMode, BorderImage.Stretch) \endcode SignalSpy::wait() provides an alternative method to wait for a signal to be emitted. \sa compare(), SignalSpy::wait() */ /*! \qmlmethod object TestCase::grabImage(item) Returns a snapshot image object of the given \a item. The returned image object has the following methods: \list \li red(x, y) Returns the red channel value of the pixel at \a x, \a y position \li green(x, y) Returns the green channel value of the pixel at \a x, \a y position \li blue(x, y) Returns the blue channel value of the pixel at \a x, \a y position \li alpha(x, y) Returns the alpha channel value of the pixel at \a x, \a y position \li pixel(x, y) Returns the color value of the pixel at \a x, \a y position For example: \code var image = grabImage(rect); compare(image.red(10, 10), 255); compare(image.pixel(20, 20), Qt.rgba(255, 0, 0, 255); \endcode \endlist \sa */ /*! \qmlmethod TestCase::skip(message = "") Skips the current test case and prints the optional \a message. If this is a data-driven test, then only the current row is skipped. Similar to \c{QSKIP(message)} in C++. */ /*! \qmlmethod TestCase::expectFail(tag, message) In a data-driven test, marks the row associated with \a tag as expected to fail. When the fail occurs, display the \a message, abort the test, and mark the test as passing. Similar to \c{QEXPECT_FAIL(tag, message, Abort)} in C++. If the test is not data-driven, then \a tag must be set to the empty string. \sa expectFailContinue() */ /*! \qmlmethod TestCase::expectFailContinue(tag, message) In a data-driven test, marks the row associated with \a tag as expected to fail. When the fail occurs, display the \a message, and then continue the test. Similar to \c{QEXPECT_FAIL(tag, message, Continue)} in C++. If the test is not data-driven, then \a tag must be set to the empty string. \sa expectFail() */ /*! \qmlmethod TestCase::warn(message) Prints \a message as a warning message. Similar to \c{QWARN(message)} in C++. \sa ignoreWarning() */ /*! \qmlmethod TestCase::ignoreWarning(message) Marks \a message as an ignored warning message. When it occurs, the warning will not be printed and the test passes. If the message does not occur, then the test will fail. Similar to \c{QTest::ignoreMessage(QtWarningMsg, message)} in C++. \sa warn() */ /*! \qmlmethod TestCase::wait(ms) Waits for \a ms milliseconds while processing Qt events. \sa sleep(), waitForRendering() */ /*! \qmlmethod TestCase::waitForRendering(item, timeout = 5000) Waits for \a timeout milliseconds or until the \a item is rendered by the renderer. Returns true if \c item is rendered in \a timeout milliseconds, otherwise returns false. The default \a timeout value is 5000. \sa sleep(), wait() */ /*! \qmlmethod TestCase::sleep(ms) Sleeps for \a ms milliseconds without processing Qt events. \sa wait(), waitForRendering() */ /*! \qmlmethod TestCase::keyClick(key, modifiers = Qt.NoModifier, delay = -1) Simulates clicking of \a key with an optional \a modifier on the currently focused item. If \a delay is larger than 0, the test will wait for \a delay milliseconds. \sa keyPress(), keyRelease() */ /*! \qmlmethod TestCase::keyPress(key, modifiers = Qt.NoModifier, delay = -1) Simulates pressing a \a key with an optional \a modifier on the currently focused item. If \a delay is larger than 0, the test will wait for \a delay milliseconds. \b{Note:} At some point you should release the key using keyRelease(). \sa keyRelease(), keyClick() */ /*! \qmlmethod TestCase::keyRelease(key, modifiers = Qt.NoModifier, delay = -1) Simulates releasing a \a key with an optional \a modifier on the currently focused item. If \a delay is larger than 0, the test will wait for \a delay milliseconds. \sa keyPress(), keyClick() */ /*! \qmlmethod TestCase::mousePress(item, x, y, button = Qt.LeftButton, modifiers = Qt.NoModifier, delay = -1) Simulates pressing a mouse \a button with an optional \a modifier on an \a item. The position is defined by \a x and \a y. If \a delay is specified, the test will wait for the specified amount of milliseconds before the press. The position given by \a x and \a y is transformed from the co-ordinate system of \a item into window co-ordinates and then delivered. If \a item is obscured by another item, or a child of \a item occupies that position, then the event will be delivered to the other item instead. \sa mouseRelease(), mouseClick(), mouseDoubleClick(), mouseMove(), mouseDrag(), mouseWheel() */ /*! \qmlmethod TestCase::mouseRelease(item, x, y, button = Qt.LeftButton, modifiers = Qt.NoModifier, delay = -1) Simulates releasing a mouse \a button with an optional \a modifier on an \a item. The position of the release is defined by \a x and \a y. If \a delay is specified, the test will wait for the specified amount of milliseconds before releasing the button. The position given by \a x and \a y is transformed from the co-ordinate system of \a item into window co-ordinates and then delivered. If \a item is obscured by another item, or a child of \a item occupies that position, then the event will be delivered to the other item instead. \sa mousePress(), mouseClick(), mouseDoubleClick(), mouseMove(), mouseDrag(), mouseWheel() */ /*! \qmlmethod TestCase::mouseClick(item, x, y, button = Qt.LeftButton, modifiers = Qt.NoModifier, delay = -1) Simulates clicking a mouse \a button with an optional \a modifier on an \a item. The position of the click is defined by \a x and \a y. If \a delay is specified, the test will wait for the specified amount of milliseconds before pressing and before releasing the button. The position given by \a x and \a y is transformed from the co-ordinate system of \a item into window co-ordinates and then delivered. If \a item is obscured by another item, or a child of \a item occupies that position, then the event will be delivered to the other item instead. \sa mousePress(), mouseRelease(), mouseDoubleClick(), mouseMove(), mouseDrag(), mouseWheel() */ /*! \qmlmethod TestCase::mouseDoubleClick(item, x, y, button = Qt.LeftButton, modifiers = Qt.NoModifier, delay = -1) Simulates double-clicking a mouse \a button with an optional \a modifier on an \a item. The position of the click is defined by \a x and \a y. If \a delay is specified, the test will wait for the specified amount of milliseconds before pressing and before releasing the button. The position given by \a x and \a y is transformed from the co-ordinate system of \a item into window co-ordinates and then delivered. If \a item is obscured by another item, or a child of \a item occupies that position, then the event will be delivered to the other item instead. \sa mousePress(), mouseRelease(), mouseClick(), mouseMove(), mouseDrag(), mouseWheel() */ /*! \qmlmethod TestCase::mouseMove(item, x, y, delay = -1) Moves the mouse pointer to the position given by \a x and \a y within \a item. If a \a delay (in milliseconds) is given, the test will wait before moving the mouse pointer. The position given by \a x and \a y is transformed from the co-ordinate system of \a item into window co-ordinates and then delivered. If \a item is obscured by another item, or a child of \a item occupies that position, then the event will be delivered to the other item instead. \sa mousePress(), mouseRelease(), mouseClick(), mouseDoubleClick(), mouseDrag(), mouseWheel() */ /*! \qmlmethod TestCase::mouseDrag(item, x, y, dx, dy, button = Qt.LeftButton, modifiers = Qt.NoModifier, delay = -1) Simulates dragging the mouse on an \a item with \a button pressed and an optional \a modifier. The initial drag position is defined by \a x and \a y, and drag distance is defined by \a dx and \a dy. If \a delay is specified, the test will wait for the specified amount of milliseconds before releasing the button. The position given by \a x and \a y is transformed from the co-ordinate system of \a item into window co-ordinates and then delivered. If \a item is obscured by another item, or a child of \a item occupies that position, then the event will be delivered to the other item instead. Note: this method does not imply a drop action, to make a drop, an additional mouseRelease(item, x + dx, y + dy) is needed. \sa mousePress(), mouseClick(), mouseDoubleClick(), mouseMove(), mouseRelease(), mouseWheel() */ /*! \qmlmethod TestCase::mouseWheel(item, x, y, xDelta, yDelta, button = Qt.LeftButton, modifiers = Qt.NoModifier, delay = -1) Simulates rotating the mouse wheel on an \a item with \a button pressed and an optional \a modifier. The position of the wheel event is defined by \a x and \a y. If \a delay is specified, the test will wait for the specified amount of milliseconds before releasing the button. The position given by \a x and \a y is transformed from the co-ordinate system of \a item into window co-ordinates and then delivered. If \a item is obscured by another item, or a child of \a item occupies that position, then the event will be delivered to the other item instead. The \a xDelta and \a yDelta contain the wheel rotation distance in eighths of a degree. see \l QWheelEvent::angleDelta() for more details. \sa mousePress(), mouseClick(), mouseDoubleClick(), mouseMove(), mouseRelease(), mouseDrag(), QWheelEvent::angleDelta() */ /*! \qmlmethod TestCase::initTestCase() This function is called before any other test functions in the \l TestCase element. The default implementation does nothing. The application can provide its own implementation to perform test case initialization. \sa cleanupTestCase(), init() */ /*! \qmlmethod TestCase::cleanupTestCase() This function is called after all other test functions in the \l TestCase element have completed. The default implementation does nothing. The application can provide its own implementation to perform test case cleanup. \sa initTestCase(), cleanup() */ /*! \qmlmethod TestCase::init() This function is called before each test function that is executed in the \l TestCase element. The default implementation does nothing. The application can provide its own implementation to perform initialization before each test function. \sa cleanup(), initTestCase() */ /*! \qmlmethod TestCase::cleanup() This function is called after each test function that is executed in the \l TestCase element. The default implementation does nothing. The application can provide its own implementation to perform cleanup after each test function. \sa init(), cleanupTestCase() */