diff options
Diffstat (limited to 'old/doc/html/qtuitest-manual.html')
| -rw-r--r-- | old/doc/html/qtuitest-manual.html | 290 |
1 files changed, 290 insertions, 0 deletions
diff --git a/old/doc/html/qtuitest-manual.html b/old/doc/html/qtuitest-manual.html new file mode 100644 index 0000000..946f903 --- /dev/null +++ b/old/doc/html/qtuitest-manual.html @@ -0,0 +1,290 @@ +<?xml version="1.0" encoding="iso-8859-1"?> +<!DOCTYPE html + PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "DTD/xhtml1-strict.dtd"> +<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> +<!-- ../../doc/src/qtuitest_manual.qdoc --> +<head> + <title>QtUiTest Manual</title> + <link href="classic.css" rel="stylesheet" type="text/css" /> +</head> +<body> +<table border="0" cellpadding="0" cellspacing="0" width="100%"> +<tr> +<td align="left" valign="top" width="32"><a href="http://qt.nokia.com/"><img src="images/qt-logo.png" align="left" border="0" /></a></td> +<td width="1"> </td><td align="right" valign="top" width="230"></td></tr></table><h1 class="title">QtUiTest Manual<br /><span class="subtitle"></span> +</h1> +<p>The <a href="qtuitest.html">QtUiTest</a> framework is a tool for automatic and semi-automatic User Interface/System level testing of Qt based applications and libraries. <a href="qtuitest.html">QtUiTest</a> provides functionality commonly found in system test frameworks but with a focus on maintainable cross-platform tests.</p> +<p>Contents: <ul><li><a href="#introduction">Introduction</a></li> +<li><a href="#the-scripting-language">The Scripting Language</a></li> +<ul><li><a href="#language-design-philosophy">Language design philosophy</a></li> +</ul> +<li><a href="#using-qtuitest">Using QtUiTest</a></li> +<ul><li><a href="#creating-a-test">Creating a Test</a></li> +<ul><li><a href="#writing-a-manual-system-test">Writing a Manual System Test</a></li> +</ul> +<li><a href="#running-a-test">Running a Test</a></li> +<ul><li><a href="#running-tests-in-a-continuous-integration-system">Running tests in a Continuous Integration System</a></li> +</ul> +<li><a href="#qtuitest-tutorial">QtUiTest Tutorial</a></li> +</ul> +<li><a href="#qtuitest-api-overview">QtUiTest API Overview</a></li> +<ul><li><a href="#system-architecture">System Architecture</a></li> +<li><a href="#query-paths">Query Paths</a></li> +<ul><li><a href="#labels">Labels</a></li> +<li><a href="#signatures">Signatures</a></li> +<li><a href="#querying-widgets-by-property">Querying Widgets By Property</a></li> +</ul> +<li><a href="#learn-mode">Learn Mode</a></li> +<li><a href="#application-management">Application Management</a></li> +<li><a href="#file-management">File Management</a></li> +<li><a href="#querying-objects">Querying Objects</a></li> +<li><a href="#text-entry-and-navigation">Text Entry and Navigation</a></li> +<li><a href="#key-simulation">Key Simulation</a></li> +<li><a href="#mouse-touchscreen-simulation">Mouse / Touchscreen Simulation</a></li> +<li><a href="#other">Other</a></li> +</ul> +<li><a href="#extending-qtuitest-for-custom-widgets">Extending QtUiTest for Custom Widgets</a></li> +<li><a href="#qtuitest-api-reference">QtUiTest API Reference</a></li> +</ul> +</p> +<a name="introduction"></a> +<h2>Introduction</h2> +<p><a href="qtuitest.html">QtUiTest</a> is, by design, quite similar to the QTestLib API that is used for Unit level testing. In contrast to QTestLib however, <a href="qtuitest.html">QtUiTest</a> based tests are written in QtScript. Please refer to the QTestLib Manual if you are looking for a Unit test framework.</p> +<p><a href="qtuitest.html">QtUiTest</a> is a scripting test language plus some basic test execution tools and as such does not provide a complete Integrated Development Environment (IDE) to write and execute tests. Tests can be written with any standard editor or development IDE. Tests can be executed from the command line or in continuous integration test systems.</p> +<a name="the-scripting-language"></a> +<h2>The Scripting Language</h2> +<a name="language-design-philosophy"></a> +<h3>Language design philosophy</h3> +<p><a href="qtuitest.html">QtUiTest</a> aims to provide a test scripting language that is:</p> +<ul> +<li>simple, reliable and easy to learn,</li> +<li>resilient to UI changes,</li> +<li>transparent to device changes (i.e. a touchscreen and keyboard based device should run the same test),</li> +<li>suitable for performance testing.</li> +</ul> +<p><a href="qtuitest.html">QtUiTest</a> scripts are written from a User perspective. When a User uses a GUI application all he/she does is apply a few concepts that form the basis of using a GUI application. For instance, a button can be clicked on and then 'the text on the button' will happen.</p> +<p>It is irrelevant to the user that the actual UI implementation uses a button. The UI could just as well have been an html page with a hyperlink in it with a similar text as on the button. In both cases, the user will 'select' the text to initiate a certain action from the application. So, today the implementation can be a button, tomorrow it can be a hyperlink, the day after tomorrow it can be something that hasn't been invented yet: the principle remains the same.</p> +<p>So conceptually a user does not 'click' keys, but rather uses the concept of a button to 'select' an action or hits keys on a real or virtual keyboard to 'enter' text. The user wants something to happen and has an understanding of what needs to be done to "get there".</p> +<p>On a new system a user first needs to learn the User Manual. The more the system behaves in a way the user already knows, and the more the system provides functionality in a way the user can predict, the more "Intuitive" and "Usable" the system is experienced by the user.</p> +<p>System testing is, in our view, very much related to these fundamental issues and just like a real user, <a href="qtuitest.html">QtUiTest</a> understands the UI concepts that form the foundation of a UI application and uses a simple syntax to enable the tester to describe it's Use Cases.</p> +<p>Next to being able to write tests in a language that is as close as possible to the user experience it is also important to write maintainable code. Automating tests is well known to be major investment, and tests being broken for every small UI change, or tests that need to be modified for each slightly different device (or Operating System) are a major problem. <a href="qtuitest.html">QtUiTest</a> is designed in such a way that the script will result in good readable and understandable code which reduces the need for additional documentation, as well as tests that are platform agnostic.</p> +<ul> +<li>The four most important commands are <a href="qsystemtest.html#select">select()</a>, <a href="qsystemtest.html#enter">enter()</a>, <a href="qsystemtest.html#compare">compare()</a> and <a href="qsystemtest.html#verify">verify()</a>.</li> +<li>Every command is making verification steps along the way. For instance, <a href="qsystemtest.html#enter">enter()</a> will perform a verification at the end to ensure that the text has been entered correctly.</li> +<li>There is no need to explicitly code key clicks or mouse clicks. The usage of key/mouse clicks typically results in low level code that is difficult to understand and should only be used in exceptional cases.</li> +<li>The test can be executed equally well with a keyboard based and/or a mouse/touchscreen based system.</li> +<li>There is usually no need for explicit synchronisation steps such as a <a href="qsystemtest.html#wait">wait()</a>: <a href="qtuitest.html">QtUiTest</a> understands that certain actions may result in a background task and will wait automatically.</li> +<li><a href="qtuitest.html">QtUiTest</a> uses the Labels that are visible on the screen to find the editable 'buddy fields'.</li> +<li><a href="qtuitest.html">QtUiTest</a> is forgiving to UI changes. For instance, the user "Name" field may start as a simple line edit, then may become a full text edit, and finally become a combobox but all this time the test will pass without as much as a single complaint: as long as a field labeled "Name" exists and a text can be successfully entered into a buddy field the test is a success from user perspective.</li> +<li>Test Data and Test Logic are separated.</li> +</ul> +<p><a href="#introduction">Back to QtUiTest Manual</a></p> +<a name="using-qtuitest"></a> +<h2>Using QtUiTest</h2> +<a name="creating-a-test"></a> +<h3>Creating a Test</h3> +<p>Each System Test is stored in its own file ending with the <tt>.js</tt> extension.</p> +<p>Within the test script, a <tt>testcase</tt> object is created containing several test functions. Within each test function, the <a href="#qtuitest-api-reference">QtUiTest API</a> scripting commands may be used, as well as standard commands provided by QtScript.</p> +<p>Helper functions and global variables can be written outside of the testcase object.</p> +<p>For example:</p> +<pre> testcase = { + testFunction1: function() { + startApplication("Dog Walker"); + select("Walk the dog...", optionsMenu()); + compare( getText(), "Dog was walked." ); + }, + + testFunction2: function() { + var i = 1; + helper(i); + } + } + + function helper(num) { + verify( num > 0 ); + ... + }</pre> +<p>The project files for System tests are usually very simple, for example:</p> +<pre> CONFIG+=systemtest + SOURCES+=sys_mytest.js</pre> +<p>Adding "systemtest" to CONFIG enables the use of the "make test" command.</p> +<p><a href="#introduction">Back to QtUiTest Manual</a></p> +<a name="writing-a-manual-system-test"></a> +<h4>Writing a Manual System Test</h4> +<p>Writing a Manual System test is typically the first step in many test automation efforts. It is far better to document and run through the steps manually to check that an idea is executable (and repeatable) before time is invested in automating all steps.</p> +<p>The downside about Manual tests is that the test is usually 'somewhere else', i.e. in a text document or spreadsheet that isn't necessarily saved in the same version control system as the source code it tests, or in a format that can be easily merged by the version control system.</p> +<p><a href="qtuitest.html">QtUiTest</a> integrates the Manual test process into the Automation process and makes it possible to seamlessly transition from a full Manual to a Semi-Automatic to a Full-Automatic test. All work is done in the same text file, so the intermediate steps are all recorded in the version control system for easy retrieval, merging and tracking.</p> +<p>A Manual test is written by using the prompt() command:</p> +<pre> my_manual_test: function() + { + prompt( "* Do something\n"+ + "* Do something else\n"+ + "* Verify that the result is as expected"); + }</pre> +<p>When a System test is executed it can be a combination of fully automated and semi-automated test functions. The automated tests simply execute without user intervention and result in a PASS/FAIL/etc. A Manual test will be executed until a prompt is detected at which time a Dialog is shown on the Desktop (host) machine containing the specified prompt text. In the dialog, the stars (*) will be replaced with "1:", "2:", etc to indicate the steps that need to be taken in sequence. The advantage of using stars (*) in the code is that lines can be inserted/removed without having to renumber the remaining steps until the end of the prompt.</p> +<p>The Dialog will also contain "Pass" and "Fail" buttons as well as a "Record" button that puts the test system into recording mode. The Pass and Fail buttons obviously must be used to indicate the result of the manual step.</p> +<p>It is perfectly fine, and to be expected, that prompts() will be combined with automated steps. For instance, one could launch an application and then select a few menu options before arriving at the section of the test that requires manual intervention.</p> +<p>For instance:</p> +<pre> startApplication("Dog Walker"); + select("Walk the dog..."); + prompt( "* Verify that the dog is having a good time");</pre> +<p><a href="#introduction">Back to QtUiTest Manual</a></p> +<a name="running-a-test"></a> +<h3>Running a Test</h3> +<p>System tests can usually be run using the command "make test". Alternatively, the <a href="qtuitest.html">QtUiTest</a> Script Execution engine (qtuitestrunner) can be called directly:</p> +<pre> qtuitestrunner <scriptname> <options></pre> +<p>These commands should be run on the desktop machine (host environment).</p> +<a name="running-tests-in-a-continuous-integration-system"></a> +<h4>Running tests in a Continuous Integration System</h4> +<p><a href="qtuitest.html">QtUiTest</a> tests can be executed from the command line of a desktop (Host) machine and as such are perfectly suited to be executed in Continuous Integration Test Systems. The output format of the test, the testresults, can be printed to the screen or saved into a file in a number of formats (such as XML) to allow easy post processing by test-reporting and defect-tracking systems.</p> +<p><a href="#introduction">Back to QtUiTest Manual</a></p> +<a name="qtuitest-tutorial"></a> +<h3>QtUiTest Tutorial</h3> +<p>Please refer to the <a href="qtuitest-tutorial.html">QtUiTest Tutorial</a> for a detailed example about writing <a href="qtuitest.html">QtUiTest</a> test cases.</p> +<p><a href="#introduction">Back to QtUiTest Manual</a></p> +<a name="qtuitest-api-overview"></a> +<h2>QtUiTest API Overview</h2> +<p>To write a System Test, create a new file containing a test script, which defines a "testcase" object with one or more test functions.</p> +<p>For example:</p> +<pre> testcase = { + testFunction1: function() { + ... + }, + testFunction2: function() { + ... + } + }</pre> +<p>The testfunctions can be written with a combination of standard QtScript and special <a href="qtuitest.html">QtUiTest</a> commands, which are grouped below.</p> +<ul> +<li><a href="#system-architecture">System Architecture</a> - briefly describes the architecture of <a href="qtuitest.html">QtUiTest</a>.</li> +<li><a href="#query-paths">Query Paths</a> - specifying widgets and objects to access.</li> +<li><a href="#querying-objects">Querying Objects</a> - getting information from widgets or objects.</li> +<li><a href="#learn-mode">Learn Mode</a> - dynamically generating test data.</li> +<li><a href="#application-management">Application Management</a> - starting and stopping applications.</li> +<li><a href="#file-management">File Management</a> - accessing files on the test system.</li> +<li>Time Management - functionality to manage the system time on the device.</li> +<li><a href="#key-simulation">Key Simulation</a> - simulating entering text, selection of items from the menus or lists, and generation of arbitrary key events.</li> +<li><a href="#mouse-touchscreen-simulation">Mouse / Touchscreen Simulation</a> - simulating user interaction via a mouse or touchscreen.</li> +<li><a href="#other">Other</a> - some miscellaneous functionality.</li> +</ul> +<a name="system-architecture"></a> +<h3>System Architecture</h3> +<p>The <a href="qtuitest.html">QtUiTest</a> framework is designed around a script execution engine (qtuitestrunner) that is executed on the desktop (host machine) and a set of <a href="qtuitest.html">QtUiTest</a> plugins that are loaded by the System Under Test (SUT). The plugins are running in the SUT process space, i.e. may be running on a device, and communicate with the System Test script execution engine via a socket based communication protocol.</p> +<p>Each test command is sent as a query to the SUT, and the script runner will be blocked until a reply is received from the SUT.</p> +<p><a href="#introduction">Back to QtUiTest Manual</a></p> +<a name="query-paths"></a> +<h3>Query Paths</h3> +<p>Most <a href="qtuitest.html">QtUiTest</a> functions use 'query paths' to identify the object on which the query needs to be performed. A query path is a string value, and is one of the following:</p> +<ul> +<li>a Label</li> +<li>a Signature</li> +</ul> +<p><a href="#introduction">Back to QtUiTest Manual</a></p> +<a name="labels"></a> +<h4>Labels</h4> +<p>A <b>Label</b> is a (usually static) text description that is associated with an editable field. In some cases, a widget is self-labelling (for example, the text property of a QPushButton), but often the label is seperate widget (usually a QLabel). The Label and Edit widgets form a "buddy pair". This relationship can be hardcoded in the application, but this is not mandatory.</p> +<p>For labels that have no buddy widget set, <a href="qtuitest.html">QtUiTest</a> resolves the buddy relationships based on relative positions of the label and edit widgets on screen. By default, for Left-to-Right languages, labels are expected to be to the left of their corresponding edit widgets (and to the right for Right-to-Left languages). However, this can be changed by using the setLabelOrientation() function.</p> +<p>For example, the code below will enter the text "dog walker" in a text field next to a Label with text "Occupation".</p> +<pre> enter( "dog walker", "Occupation" );</pre> +<p>In words the example says: "enter the text 'dog walker' into a field labeled 'Occupation'.</p> +<p>A special case of a Label is an empty string. This will resolve to the widget that currently has focus, or (for touchscreen devices) the first field in the dialog. Note that for tests that should run both on keyboard and touchscreen devices it may be risky to rely on this feature. Always explicitly specifying a Label is the most safe and portable thing to do.</p> +<p>Using a Label to identify a field is the preferred mechanism for identifying fields. It is reliable, easy to write, easy to understand and easy to debug.</p> +<p><a href="#introduction">Back to QtUiTest Manual</a></p> +<a name="signatures"></a> +<h4>Signatures</h4> +<p>In some cases it is not possible to use a label to uniquely identify a widget. In these cases it is necessary to use a <b>Signature</b> to identify the widget.</p> +<p>A Signature is a string that uniquely identifies an object. Signatures are generated by the system at run-time; the signature of a widget will be different each time a test is run, so it is not possible to hard-code a signature value in a test case.</p> +<p>The signature() function can be used to return the signature of an object based on its position relative to another widget.</p> +<pre> enter( "dog walker", signature("Label",1));</pre> +<p>In this case, the signature() function returns the first editable field to the right of or below the field associated with label "Label".</p> +<a name="querying-widgets-by-property"></a> +<h4>Querying Widgets By Property</h4> +<p>Widgets can also be specified using the findByProperty() function. This finds all widgets that match the specified property values, sorted by position (top left to bottom right). The values returned are returned as a QStringList of signatures. In addition to the properties supported by the widget, "className" and "inherits" are also supported. The "className" matches if the specified value matches the class name of the object; "inherits" matches if the class of the object inherits the specified class name.</p> +<p>For example:</p> +<pre> var toolButtons = findByProperty( { className: "QToolButton" } ); + ... + select (toolButtons[3]); + var addNewPage = findByProperty( { tooltip: "Add new page" } );</pre> +<p>Multiple property value may be specified, for example:</p> +<pre> var password = findByProperty( { className: "QLineEdit", echoMode: 2 } );</pre> +<p>Using findByProperty requires some knowledge of the inner workings of the application under test, and as such breaks some of the principles of black-box testing. It is recommended that, where possible, logic using findByProperty should be wrapped in a utility function, which can be called by the test case.</p> +<p><a href="#introduction">Back to QtUiTest Manual</a></p> +<a name="learn-mode"></a> +<h3>Learn Mode</h3> +<p>Some testcases require the existence of non-volatile test data, such as screen snapshots. To simplify the process of data gathering, the test system supports the concept of a learn mode. When a test is run in learn mode, certain actions will result in the generation of test data, which will be presented to the user to manually verify once, then stored for use in subsequent test runs. A system test is run in learn mode by supplying the <tt>-learn</tt> or <tt>-learn-all</tt> command line options.</p> +<p>The system test framework will store generated test data in the <tt>testdata</tt> subdirectory of the directory containing the test script.</p> +<p>NOTE: It is up to the tester to ensure that new learned elements are added to a version control system.</p> +<p><a href="#introduction">Back to QtUiTest Manual</a></p> +<a name="application-management"></a> +<h3>Application Management</h3> +<ul> +<li><a href="qsystemtest.html#startApplication">startApplication()</a> - start an application.</li> +</ul> +<p><a href="#introduction">Back to QtUiTest Manual</a></p> +<a name="file-management"></a> +<h3>File Management</h3> +<p>It is possible to access files on the System Under Test (even if this is another device) from within a system test.</p> +<ul> +<li><a href="qsystemtest.html#putFile">putFile()</a>, <a href="qsystemtest.html#putData">putData()</a> - copy data to the test system.</li> +<li><a href="qsystemtest.html#deletePath">deletePath()</a> - delete files from the test system.</li> +<li><a href="qsystemtest.html#getDirectoryEntries">getDirectoryEntries()</a> - read the contents of a directory on the test system.</li> +</ul> +<p><a href="#introduction">Back to QtUiTest Manual</a></p> +<a name="querying-objects"></a> +<h3>Querying Objects</h3> +<p>The system test framework provides powerful methods of obtaining information on particular objects and widgets in the system.</p> +<ul> +<li><a href="qsystemtest.html#isVisible">isVisible()</a>, <a href="qsystemtest.html#isChecked">isChecked()</a>, <a href="qsystemtest.html#isEnabled">isEnabled()</a> - retrieve simple information from widgets.</li> +<li><a href="qsystemtest.html#getText">getText()</a>, <a href="qsystemtest.html#getSelectedText">getSelectedText()</a> - retrieve a textual representation of widgets.</li> +<li><a href="qsystemtest.html#getProperty">getProperty()</a>, <a href="qsystemtest.html#setProperty">setProperty()</a>, <a href="qsystemtest.html#invokeMethod">invokeMethod()</a> - access Qt properties, signals and slots on any QObject.</li> +</ul> +<p><a href="#introduction">Back to QtUiTest Manual</a></p> +<a name="text-entry-and-navigation"></a> +<h3>Text Entry and Navigation</h3> +<p>The system test framework provides several ways of entering text into widgets and for navigating through applications.</p> +<ul> +<li><a href="qsystemtest.html#enter">enter()</a> - navigate to a widget and enter any text.</li> +<li><a href="qsystemtest.html#select">select()</a> - choose a particular item from a list, combo box, menu or navigate to a particular widget or tab.</li> +</ul> +<p>These functions are input method independant and will have the same syntax for both Keyboard and Mouse/Touchscreen based devices and are therefore the preferred commands to use in tests.</p> +<p><a href="#introduction">Back to QtUiTest Manual</a></p> +<a name="key-simulation"></a> +<h3>Key Simulation</h3> +<p>The system test framework provides several ways of generating arbitrary key events.</p> +<ul> +<li><a href="qsystemtest.html#keyPress">keyPress()</a>, <a href="qsystemtest.html#keyRelease">keyRelease()</a>, <a href="qsystemtest.html#keyRelease">keyRelease()</a>, <a href="qsystemtest.html#keyClickHold">keyClickHold()</a> - press/release/click/hold any key.</li> +</ul> +<p>Note that Key simulation should be used in exceptional cases only since it makes the test specific for keyboard based devices. Other functions such as select() are the preferred mechanism to write tests because these will work both for Keyboard and Mouse/Touchscreen based devices.</p> +<p><a href="#introduction">Back to QtUiTest Manual</a></p> +<a name="mouse-touchscreen-simulation"></a> +<h3>Mouse / Touchscreen Simulation</h3> +<p>The system test framework provides methods to simulate input from a mouse or touchscreen.</p> +<ul> +<li><a href="qsystemtest.html#mousePress">mousePress()</a>, <a href="qsystemtest.html#mouseRelease">mouseRelease()</a>, <a href="qsystemtest.html#mouseClick">mouseClick()</a>, <a href="qsystemtest.html#mouseClickHold">mouseClickHold()</a> - press/release/click with mouse or touchscreen.</li> +</ul> +<p>Note that Mouse simulation should be used in exceptional cases only since it makes the test specific for mouse/touchscreen based devices. Other functions such as select() are the preferred mechanism to write tests because these will work both for Keyboard and Mouse/Touchscreen based devices.</p> +<p><a href="#introduction">Back to QtUiTest Manual</a></p> +<a name="other"></a> +<h3>Other</h3> +<p>Some functionality does not fall into the aforementioned categories:</p> +<ul> +<li><a href="qsystemtest.html#setSetting">setSetting()</a>, <a href="qsystemtest.html#getSetting">getSetting()</a> - access any settings stored within QSettings objects.</li> +</ul> +<p><a href="#introduction">Back to QtUiTest Manual</a></p> +<a name="extending-qtuitest-for-custom-widgets"></a> +<h2>Extending QtUiTest for Custom Widgets</h2> +<p><a href="qtuitest.html">QtUiTest</a>'s high-level commands require knowledge of the widgets they interact with. For example, <a href="qsystemtest.html#select">select()</a> must know how to select items from both a combobox and a list view.</p> +<p>When testing an application which uses custom widgets, some additional steps may be required in order to use high-level <a href="qtuitest.html">QtUiTest</a> commands. The logic for interacting with custom widgets must be implemented either in the application under test or in a plugin.</p> +<p>See <a href="qtuitest-plugins.html">Creating a QtUiTest Test Widget</a> for more details.</p> +<p><a href="#introduction">Back to QtUiTest Manual</a></p> +<a name="qtuitest-api-reference"></a> +<h2>QtUiTest API Reference</h2> +<p>For a complete description of the <a href="qtuitest.html">QtUiTest</a> API please refer to <a href="qsystemtest.html">QSystemTest API</a>.</p> +<p><a href="#introduction">Back to QtUiTest Manual</a></p> +<p /><address><hr /><div align="center"> +<table width="100%" cellspacing="0" border="0"><tr class="address"> +<td width="30%" align="left">Copyright © 2009 Nokia Corporation and/or its subsidiary(-ies)</td> +<td width="40%" align="center"><a href="trademarks.html">Trademarks</a></td> +<td width="30%" align="right"><div align="right">QtUiTest</div></td> +</tr></table></div></address></body> +</html> |