1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
|
/****************************************************************************
**
** Copyright (C) 2016 The Qt Company Ltd.
** Contact: https://www.qt.io/licensing/
**
** This file is part of the Qt Creator documentation.
**
** Commercial License Usage
** Licensees holding valid commercial Qt licenses may use this file in
** accordance with the commercial license agreement provided with the
** Software or, alternatively, in accordance with the terms contained in
** a written agreement between you and The Qt Company. For licensing terms
** and conditions see https://www.qt.io/terms-conditions. For further
** information use the contact form at https://www.qt.io/contact-us.
**
** GNU Free Documentation License Usage
** Alternatively, this file may be used under the terms of the GNU Free
** Documentation License version 1.3 as published by the Free Software
** Foundation and appearing in the file included in the packaging of
** this file. Please review the following information to ensure
** the GNU Free Documentation License version 1.3 requirements
** will be met: https://www.gnu.org/licenses/fdl-1.3.html.
**
****************************************************************************/
/*!
\contentspage {Qt Creator Manual}
\previouspage creator-cpu-usage-analyzer.html
\page creator-autotest.html
\nextpage creator-advanced.html
\title Running Autotests
\QC integrates the \l{Qt Test} framework and
\l{https://github.com/google/googletest}{Google C++ Testing Framework} for
unit testing applications and libraries. You can use \QC to build and run
Qt tests, Qt Quick tests (QML-based Qt tests), and Google tests for your
projects.
\image qtcreator-autotests.png
\section1 Creating Tests
You can use a wizard to create projects that contain Qt or Google tests.
\section2 Creating Qt Tests
To create a Qt test:
\list 1
\li Select \uicontrol File > \uicontrol {New File or Project} >
\uicontrol {Other Project} > \uicontrol {Auto Test} >
\uicontrol Choose to create a project with boilerplate code for a
Qt test.
\li In the \uicontrol {Project and Test Information} dialog, specify
settings for the project and test:
\list 1
\li In the \uicontrol {Test framework} field, select
\uicontrol {Qt Test}.
\li Select the \uicontrol {GUI Application} check box to create
a Qt application.
\li In the \uicontrol {Test case name} field, enter a name for
the test case.
\li Select the \uicontrol {Requires QApplication} check box to
add the include statement for QApplication to the main.cpp
file of the project.
\li Select the \uicontrol {Generate initialization and cleanup
code} checkbox to add functions to your test that are
executed by the testing framework to initialize and clean
up the test.
\li In the \uicontrol {Build auto tests} field, select
\uicontrol Always to always build the test when building
the project or \uicontrol {Debug Only} to only build it
during debug builds.
\endlist
\endlist
\QC creates the test in the specified project directory. Edit the .cpp file
to add private slots for each test function in your test. For more information
about creating Qt tests, see \l{Creating a Test}.
\section2 Creating Google Tests
To create a Google test:
\list 1
\li Select \uicontrol File > \uicontrol {New File or Project} >
\uicontrol {Other Project} > \uicontrol {Auto Test} >
\uicontrol Choose to create a project with boilerplate code for a
Google test.
\li In the \uicontrol {Project and Test Information} dialog, specify
settings for the project and test:
\list 1
\li In the \uicontrol {Test framework} field, select
\uicontrol {Google Test}.
\li In the \uicontrol {Test case name} field, enter a name for
the test case.
\li In the \uicontrol {Test set name} field, enter a name for
the test set.
\li Select the \uicontrol {Enable C++ 11} check box to
support C++ 11 features in the test.
\li In the \uicontrol {Build auto tests} field, select
\uicontrol Always to always build the test when building
the project or \uicontrol {Debug Only} to only build it
during debug builds.
\li In the \uicontrol {Google test repository} field, select
a directory that contains a clone of the googletest
repository.
To use an installed Google C++ Testing framework instead,
see \l{Setting Up the Google C++ Testing Framework}.
\endlist
\endlist
\QC creates the test in the specified project directory.
\section1 Setting Up the Google C++ Testing Framework
To build and run Google tests, you must have the Google C++ Testing
framework installed and configured on the development host. You can either
clone it from Git Hub or install it from an installation package.
To configure a project to use a cloned Google testing framework, edit the
\c INCLUDEPATH variable in the project file (.pro) to include the source
and \c include folders of Google Test's \c googletest and \c googlemock.
Usually, you need to add the following subfolders:
\list
\li \c googletest
\li \c googlemock
\li \c googletest/include
\li \c googlemock/include
\endlist
You also need to add the necessary files to the \c SOURCES variable. For
example:
\list
\li \c googletest/src/gtest-all.cc
\li \c googlemock/src/gmock-all.cc
\endlist
To configure a project to use an installed Google testing framework package,
add the following include paths to the .pro file:
\list
\li \c <googletest_install_path>/include/gtest
\li \c <googletest_install_path>/include/gmock
\endlist
Then add linker options to be able to find the libraries and to link against
them. For example, for qmake based projects, you typically need to add the
following values to the .pro file:
\list
\li \c {LIBS += -lgtest -L<path_to_gtest_lib>}
\li \c {LIBS += -lgmock -L<path_to_gmock_lib>}
\endlist
To specify settings for running Google tests, select \uicontrol Tools >
\uicontrol Options > \uicontrol {Test Settings} > \uicontrol {Google Test}.
\image qtcreator-autotests-options-google.png
To run disabled tests, select the \uicontrol {Run disabled tests} check box.
To run several iterations of the tests, select the \uicontrol {Repeat tests}
check box and enter the number of times the tests should be run in the
\uicontrol Iterations field. To make sure that the tests are independent and
repeatable, you can run them in a different order each time by selecting the
\uicontrol {Shuffle tests} check box.
To turn failures into debugger breakpoints, select the
\uicontrol {Break on failure while debugging} check box. To turn assertion
failures into C++ exceptions, select the \uicontrol {Throw on failure} check
box.
For more information about creating Google tests, see the
\l{https://github.com/google/googletest/blob/master/googletest/docs/Primer.md}
{Google Test Primer}.
\section1 Building and Running Tests
To build and run tests:
\list 1
\li Open a project that contains tests.
\li In the \uicontrol {Test Results} output pane, select
\inlineimage run_small.png
(\uicontrol {Run All Tests}) to run all test or
\inlineimage qtcreator-run-selected-tests.png
(\uicontrol {Run Selected Tests}) to run the selected tests.
You can select the tests to run in the \uicontrol Tests view in the
\uicontrol Projects pane. For more information, see
\l{Viewing Tests}.
\note By default, \QC builds a project before deploying and running
it.
\endlist
If a test takes more than a minute to execute, the default timeout might
stop the test execution. To increase the timeout, select \uicontrol Tools >
\uicontrol Options > \uicontrol {Test Settings} > \uicontrol General.
\image qtcreator-autotests-options.png
\QC scans the project for tests when you open the project and updates the
test list for the currently active test frameworks when you edit tests.
The code inside a benchmark test is measured, and possibly also repeated
several times in order to get an accurate measurement. This depends on the
measurement back-end that you can select in the
\uicontrol {Benchmark Metrics} group in \uicontrol Tools >
\uicontrol Options > \uicontrol {Test Settings} > \uicontrol {Qt Test}:
walltime, CPU tick counter, event counter, Valgrind Callgrind, and Linux
Perf. For more information, see \l{Creating a Benchmark}.
\image qtcreator-autotests-options-qt.png
To allow the debugger to interrupt Qt tests on assertions, select the
\uicontrol {Disable crash handler while debugging} check box.
\section1 Viewing Test Output
The test results are displayed in the \uicontrol {Test Results} output pane.
\table
\header
\li Result
\li Description
\row
\li BENCH
\li Benchmark test.
\row
\li BFAIL
\li Blacklisted test case failed. Since Qt 5.4, you can
provide a BLACKLIST file for tests. It is mainly used internally
by the Qt CI system.
\row
\li BPASS
\li Blacklisted test case passed.
\row
\li DEBUG
\li Debug message.
\row
\li XFAIL
\li Test case is expected to fail, so it is marked by using the
QEXPECT_FAIL macro. If the test case passes instead, an
unexpected pass (XPASS) is written to the test log.
\row
\li FAIL
\li Test case failed. Double-click the line for more information.
\row
\li INTERNAL
\li Internal message.
\row
\li PASS
\li Test case passed.
\row
\li SKIP
\li Test case was skipped.
\row
\li XPASS
\li Test case passed even though it was expected to fail.
\row
\li WARN
\li Warning message.
\endtable
To view only messages of a particular type, select
\inlineimage filtericon.png
(\uicontrol {Filter Test Results}), and then select the types of messages to
show.
To hide internal messages and run configuration warnings, select
\uicontrol Tools > \uicontrol Options > \uicontrol {Test Settings} >
\uicontrol General.
*/
|
