path: root/src/imports/testlib/testcase.qdoc

/****************************************************************************
**
** Copyright (C) 2010 Nokia Corporation and/or its subsidiary(-ies).
** All rights reserved.
** Contact: Nokia Corporation (qt-info@nokia.com)
**
** This file is part of the test suite of the Qt Toolkit.
**
** $QT_BEGIN_LICENSE:LGPL$
** No Commercial Usage
** This file contains pre-release code and may not be distributed.
** You may use this file in accordance with the terms and conditions
** contained in the Technology Preview License Agreement accompanying
** this package.
**
** GNU Lesser General Public License Usage
** Alternatively, this file may be used under the terms of the GNU Lesser
** General Public License version 2.1 as published by the Free Software
** Foundation and appearing in the file LICENSE.LGPL included in the
** packaging of this file.  Please review the following information to
** ensure the GNU Lesser General Public License version 2.1 requirements
** will be met: http://www.gnu.org/licenses/old-licenses/lgpl-2.1.html.
**
** In addition, as a special exception, Nokia gives you certain additional
** rights.  These rights are described in the Nokia Qt LGPL Exception
** version 1.1, included in the file LGPL_EXCEPTION.txt in this package.
**
** If you have questions regarding the use of this file, please contact
** Nokia at qt-info@nokia.com.
**
**
**
**
**
**
**
**
** $QT_END_LICENSE$
**
****************************************************************************/

/*!
    \qmlclass TestCase TestCase
    \brief The TestCase item represents a unit test case.
    \since 4.8
    \ingroup qtest::qml

    \section1 Introduction to QML test cases

    Test cases are written as JavaScript functions within a TestCase
    element:

    \code
    import QtQuick 1.0
    import QtQuickTest 1.0

    TestCase {
        name: "MathTests"

        function test_math() {
            compare(2 + 2, 4, "2 + 2 = 4")
        }

        function test_fail() {
            compare(2 + 2, 5, "2 + 2 = 5")
        }
    }
    \endcode

    Functions whose names start with "test_" are treated as test cases
    to be executed.  The \l name property is used to prefix the functions
    in the output:

    \code
    ********* Start testing of MathTests *********
    Config: Using QTest library 4.7.2, Qt 4.7.2
    PASS   : MathTests::initTestCase()
    FAIL!  : MathTests::test_fail() 2 + 2 = 5
       Actual (): 4
       Expected (): 5
       Loc: [file:///.../tst_math.qml(12)]
    PASS   : MathTests::test_math()
    PASS   : MathTests::cleanupTestCase()
    Totals: 3 passed, 1 failed, 0 skipped
    ********* Finished testing of MathTests *********
    \endcode

    Because of the way JavaScript properties work, the order in which the
    test functions are found is unpredictable.  To assist with predictability,
    the test framework will sort the functions on ascending order of name.
    This can help when there are two tests that must be run in order.

    \section1 Data-driven tests

    Table data can be provided to a test using a function name that ends
    with "_data":

    \code
    import QtQuick 1.0
    import QtQuickTest 1.0

    TestCase {
        name: "DataTests"

        function test_table_data() {
            return [
                {tag: "2 + 2 = 4", a: 2, b: 2, answer: 4 },
                {tag: "2 + 6 = 8", a: 2, b: 6, answer: 8 },
            ]
        }

        function test_table(data) {
            compare(data.a + data.b, data.answer)
        }
    }
    \endcode

    The test framework will iterate over all of the rows in the table
    and pass each row to the test function.  As shown, the columns can be
    extracted for use in the test.  The \c tag column is special - it is
    printed by the test framework when a row fails, to help the reader
    identify which case failed amongst a set of otherwise passing tests.

    \section1 Benchmarks

    Functions whose names start with "benchmark_" will be run multiple
    times with the Qt benchmark framework, with a average timing value
    reported for the runs.  This is equivalent to using the \c{QBENCHMARK}
    macro in the C++ version of QTestLib.

    \code
    TestCase {
        id: top
        name: "CreateBenchmark"

        function benchmark_create_component() {
            var component = Qt.createComponent("item.qml")
            var obj = component.createObject(top)
            obj.destroy()
            component.destroy()
        }
    }

    RESULT : CreateBenchmark::benchmark_create_component:
         0.23 msecs per iteration (total: 60, iterations: 256)
    PASS   : CreateBenchmark::benchmark_create_component()
    \endcode

    To get the effect of the \c{QBENCHMARK_ONCE} macro, prefix the test
    function name with "benchmark_once_".
*/

/*!
    \qmlproperty string TestCase::name

    This property defines the name of the test case for result reporting.
    The default is the empty string.

    \code
    TestCase {
        name: "ButtonTests"
        ...
    }
    \endcode
*/

/*!
    \qmlproperty bool TestCase::when

    This property should be set to true when the application wants
    the test cases to run.  The default value is true.

    \code
    TestCase {
        name: "ButtonTests"
        when: button.pressed
        ...
    }
    \endcode

    The test application will exit once all \l TestCase elements
    have been triggered and have run.  The \l optional property can
    be used to exclude a \l TestCase element.

    \sa optional, completed
*/

/*!
    \qmlproperty bool TestCase::optional

    Multiple \l TestCase elements can be supplied in a test application.
    The application will exit once they have all completed.  If a test case
    does not need to run (because a precondition has failed), then this
    property can be set to true.  The default value is false.

    \code
    TestCase {
        when: false
        optional: true
        function test_not_run() {
            verify(false)
        }
    }
    \endcode

    \sa when, completed
*/

/*!
    \qmlproperty bool TestCase::completed

    This property will be set to true once the test case has completed
    execution.  Test cases are only executed once.  The initial value
    is false.

    \sa running, when
*/

/*!
    \qmlproperty bool TestCase::running

    This property will be set to true while the test case is running.
    The initial value is false, and the value will become false again
    once the test case completes.

    \sa completed, when
*/

/*!
    \qmlproperty bool TestCase::windowShown

    This property will be set to true after the QML viewing window has
    been displayed.  Normally test cases run as soon as the test application
    is loaded and before a window is displayed.  If the test case involves
    visual elements and behaviors, then it may need to be delayed until
    after the window is shown.

    \code
    Button {
        id: button
        onClicked: text = "Clicked"
        TestCase {
            name: "ClickTest"
            when: windowShown
            function test_click() {
                button.clicked();
                compare(button.text, "Clicked");
            }
        }
    }
    \endcode
*/

/*!
    \qmlmethod TestCase::fail()
    \qmlmethod TestCase::fail(message)

    Fails the current test case, with the optional \a message.
    Similar to \c{QFAIL(message)} in C++.
*/

/*!
    \qmlmethod TestCase::verify(condition)
    \qmlmethod TestCase::verify(condition, message)

    Fails the current test case if \a condition is false, and
    displays the optional \a message.  Similar to \c{QVERIFY(condition)}
    or \c{QVERIFY2(condition, message)} in C++.
*/

/*!
    \qmlmethod TestCase::compare(actual, expected)
    \qmlmethod TestCase::compare(actual, expected, message)

    Fails the current test case if \a actual is not the same as
    \a expected, and displays the optional \a message.  Similar
    to \c{QCOMPARE(actual, expected)} in C++.

    \sa tryCompare()
*/

/*!
    \qmlmethod TestCase::tryCompare(obj, property, expected)
    \qmlmethod TestCase::tryCompare(obj, property, expected, timeout)

    Fails the current test case if the specified \a property on \a obj
    is not the same as \a expected.  The test will be retried multiple
    times until the \a timeout is reached (default is 5000 milliseconds).

    This function is intended for testing applications where a property
    changes value based on asynchronous events.  Use compare() for testing
    synchronous property changes.

    \sa compare()
*/

/*!
    \qmlmethod TestCase::skip()
    \qmlmethod TestCase::skip(message)

    Skips the current test case and prints the optional \a message.
    If this is a data-driven test, then only the current row is skipped.
    Similar to \c{QSKIP(message, SkipSingle)} in C++.

    \sa skipAll()
*/

/*!
    \qmlmethod TestCase::skipAll()
    \qmlmethod TestCase::skipAll(message)

    Skips the current test case and prints the optional \a message.
    If this is a data-driven test, then all remaining rows are skipped.
    Similar to \c{QSKIP(message, SkipAll)} in C++.

    \sa skip()
*/

/*!
    \qmlmethod TestCase::expectFail(tag)
    \qmlmethod TestCase::expectFail(tag, message)

    In a data-driven test, marks the row associated with \a tag as
    expected to fail.  When the fail occurs, display \a message,
    abort the test, and mark the test as passing.  Similar to
    \c{QEXPECT_FAIL(tag, message, Abort)} in C++.

    \sa expectFailContinue()
*/

/*!
    \qmlmethod TestCase::expectFailContinue(tag)
    \qmlmethod TestCase::expectFailContinue(tag, message)

    In a data-driven test, marks the row associated with \a tag as
    expected to fail.  When the fail occurs, display \a message,
    and then continue the test.  Similar to
    \c{QEXPECT_FAIL(tag, message, Continue)} in C++.

    \sa expectFail()
*/

/*!
    \qmlmethod TestCase::warn(message)

    Prints \a message as a warning message.  Similar to
    \c{QWARN(message)} in C++.

    \sa ignoreWarning()
*/

/*!
    \qmlmethod TestCase::ignoreWarning(message)

    Marks \a message as an ignored message.  When it occurs, the message
    will not be printed and the test passes.  If the message does not
    occur, then the test will fail.  Similar to
    \c{QTest::ignoreMessage(QtWarningMsg, message)} in C++.

    \sa warn()
*/

/*!
    \qmlmethod TestCase::wait(ms)

    Waits for \a ms milliseconds while processing Qt events.

    \sa sleep()
*/

/*!
    \qmlmethod TestCase::sleep(ms)

    Sleeps for \a ms milliseconds without processing Qt events.

    \sa wait()
*/

/*!
    \qmlmethod TestCase::initTestCase()

    This function is called before any other test functions in the
    \l TestCase element.  The default implementation does nothing.
    The application can provide its own implementation to perform
    test case initialization.

    \sa cleanupTestCase(), init()
*/

/*!
    \qmlmethod TestCase::cleanupTestCase()

    This function is called after all other test functions in the
    \l TestCase element have completed.  The default implementation
    does nothing.  The application can provide its own implementation
    to perform test case cleanup.

    \sa initTestCase(), cleanup()
*/

/*!
    \qmlmethod TestCase::init()

    This function is called before each test function that is
    executed in the \l TestCase element.  The default implementation
    does nothing.  The application can provide its own implementation
    to perform initialization before each test function.

    \sa cleanup(), initTestCase()
*/

/*!
    \qmlmethod TestCase::cleanup()

    This function is called after each test function that is
    executed in the \l TestCase element.  The default implementation
    does nothing.  The application can provide its own implementation
    to perform cleanup after each test function.

    \sa init(), cleanupTestCase()
*/