1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
|
/****************************************************************************
**
** Copyright (C) 2019 Luxoft Sweden AB
** Contact: https://www.qt.io/licensing/
**
** This file is part of the Neptune 3 UI.
**
** $QT_BEGIN_LICENSE:FDL-QTAS$
** Commercial License Usage
** Licensees holding valid commercial Qt Automotive Suite licenses may use
** this file in accordance with the commercial license agreement provided
** with the Software or, alternatively, in accordance with the terms
** contained in a written agreement between you and The Qt Company. For
** licensing terms and conditions see https://www.qt.io/terms-conditions.
** For further information use the contact form at https://www.qt.io/contact-us.
**
** GNU Free Documentation License Usage
** Alternatively, this file may be used under the terms of the GNU Free
** Documentation License version 1.3 as published by the Free Software
** Foundation and appearing in the file included in the packaging of
** this file. Please review the following information to ensure
** the GNU Free Documentation License version 1.3 requirements
** will be met: https://www.gnu.org/licenses/fdl-1.3.html.
** $QT_END_LICENSE$
**
****************************************************************************/
/*!
\page neptune3ui-testing-squish.html
\title Neptune 3 UI - Squish Automated UI Testing
\brief How to use Squish within Neptune 3 UI.
\l{Squish} is a commercial product from \l{Froglogic} which is used for automated GUI testing.
The goals of these tests and their setup are so that we can:
\list
\li Provide a basic set of scripts for \l{Neptune 3 UI}, \l{QmlLive}, and
\l{Qt Application Manager} for smoke testing on a CI.
\li Document any specifics in Qt Automotive Suite, especially the multi-processing aspect,
which users should pay attention to for automated UI testing with Squish.
\li Provide reference scripts as a part of Neptune 3 UI.
\li Enable developers working with UI, in the Qt Automotive Suite, to write their own
tests, if needed.
\endlist
To use these tests, you need:
\list
\li \l{Qt Automotive Suite}
\li Squish installed on the test machine
\endlist
\note Squish and the accompanying license must be obtained from \l{Froglogic}.
\section1 Set Up Your Test Environment
Squish can be run:
\list 1
\li within an IDE - the Squish IDE is available for most developer platforms.
\li as a service - the service program, \c squishserver, is either distributed
with the IDE or can be adapted from an embedded source package. This source package
runs on different platforms and is downloadable from \l{Froglogic}.
\endlist
The \e{target} referenced embedded system is a \l{Yocto} Linux x64 build which, on the
Qt Automotive Suite, can be installed as \b{Intel NUC}.
Tests are provided in Neptune 3 UI's \c squishtest sub folder, and can be imported as a suite
directly into the Squish IDE.
\section2 Application(s) Under Test (AUTs)
Whether you use the Squish IDE or \e squishserver, you must define all of the AUTs, that also
includes all sub-processes that are called from within Neptune 3 UI apps. For multi-process,
each of the \l{Neptune 3 UI - App Development}{Neptune 3 UI apps} run as
\c{appman-launcher-qml} processes.
To distinguish and attach to these processes you need to use the \b{Squish hook plugin} that
extends Squish's attachable AUT feature. The \c server.ini file must be adapted to specify
where Neptune 3 UI and the other AUTs are located.
Because in multi-process, all Neptune 3 UI apps must be connected via Squish's attachable AUT
feature, these apps must be defined as attachable AUTs with a corresponding port number.
Additionally, the port in the \c server.ini has to match the port defined in its \c{info.yaml}
manifest file as \c{squishPort} property.
\note The Neptune 3 UI apps and their manifest files are located in \c {neptune3/apps}).
To run these tests directly from the IDE, it is possible and recommended to add or modify AUTs
and attachable AUTs with port numbers, in the IDE itself.
For more details, see \l{https://doc.froglogic.com/squish/latest/rg-ini-files.html}{Squish
Initialization Files}.
\badcode
[General]
AUT/appman = "/home/user/qt/bin"
AUT/appman-launcher-qml = "/home/user/qt/bin"
AUT/neptune3-ui = "/home/user/install/neptune3"
attachableAUT/neptune3-ui = "localhost:3334"
attachableAUT/appman = "localhost:7777"
attachableAUT/app_appstore = "localhost:7720"
attachableAUT/app_calendar = "localhost:7721"
attachableAUT/app_climate = "localhost:7722"
attachableAUT/app_cluster = "localhost:7723"
attachableAUT/app_downloads = "localhost:7724"
attachableAUT/app_hud = "localhost:7725"
attachableAUT/app_map = "localhost:7726"
attachableAUT/app_music = "localhost:7727"
attachableAUT/app_phone = "localhost:7728"
attachableAUT/app_settings = "localhost:7729"
attachableAUT/app_sheets = "localhost:7730"
attachableAUT/app_tuner = "localhost:7731"
attachableAUT/app_vehicle = "localhost:7732"
\endcode
\section2 Exclude Disruptive Sub-Processes
Typically, Squish tries to connect to all sub-processes that are created from an initial AUT.
However, this behavior can lead to issues where Squish stops, does not create these processes,
or causes these processes to not behave as expected.
Remember, that these processes can also be external programs or services, which in our case
are:
\list
\li D-Bus process
\li remotesettings-server
\endlist
Consequently, for Neptune 3 UI you need to force Squish to ignore these processes to run the
tests. Otherwise these tests might not succeed from the beginning - for example because of a
stopped D-Bus session.
To \l{https://doc.froglogic.com/squish/latest/rg-ini-files.html#ignoredauts.txt}{ignore these processes}
you need to add or edit the \c ignoredauts.txt file in your Squish installation directory.
For macOS and other Unix-like Operating Systems, the file is in \c{SQUISHDIR/etc/}.
For Linux, make the following change:
\badcode
# ignore the following processes
remotesettings-server
dbus-daemon
\endcode
For other Operating Systems, you might need to modify the D-Bus process name.
\note It might be a good idea to ignore all sub-processes created that are not relevant
to test in the test suite.
\section2 Additional Libraries
Squish needs additional Qt libraries to run Neptune 3 UI tests. One solution is to copy the
following libraries from your installation folder to Squish's installation library
folder \c{SQUISHDIR/etc/lib}:
\list
\li \c libQt5Widgets.so
\li \c libQt5QuickWidgets.so
\endlist
\note Different Qt versions: If the Qt version for your Squish release differs from the
Qt version used in Neptune 3 UI, when the tests start, you may notice a warning in the
logs from \c squishserver indicating: \br"\c{Loaded Qt version X.xx differs from Squish/Qt
expected version X.yy, automating this AUT may not work.}"\br Currently, with Neptune 3 UI's
5.12 and Squish's 5.11 there haven't been issues and tests can run using the additional
libraries; but this warning mustn't be ignored.
\section1 Run Squish from the Command Line or on a Target
Once you have set up your test environment, you have the option to run tests via the
Squish IDE, the command line, or as a service: such as a \c systemd service (recommended),
for your target \l{Yocto} environment.
Remember to use a D-Bus session beforehand, as this is how Neptune 3 UI communicates
with its sub-processes.
Normally, for simple cases you can use a server, \c squishserver, to run the tests and a
corresponding client, \c squishrunner, that can request the server to run these tests
and return their results.
The AUT is on the server side, the test cases are transmitted from \c squishrunner to
\c squishserver; they need not be stored on the test machine. If you use the Squish IDE, when
performing tests, both \c squishserver and \c squishrunner are run seamlessly on the same
machine.
When you use the the command line, start the server first before the clients.
Ensure that all IP addresses for \c squishrunner are able to connect to \c squishserver. On
\c squishserver, this is stored in \c $SQUISH_DIR/etc/squishserverrc. Make sure that the
IP ranges match, \e{localhost} is always valid.
The server starts with the following:
\badcode
$SQUISH_DIR/bin/squishserver --port 3333
\endcode
\note \c squishserver has a default port: 4322
Change the directory to the \c squishtest path in your Neptune 3 UI installation, and run a
single test case with the following command:
\badcode
$SQUISH_DIR/bin/squishrunner --host SQUISH_SERVER_IP --port 3333 --testsuite suite_neptune3 --testcase tst_popup_climate
\endcode
The test cases are run on a client, and the results are returned.
\section1 Squish on Target Environment (Yocto)
You can add a test environment on target hardware with Neptune 3 UI installed. As mentioned
before, for a Yocto environment, you need to create a custom \c squishserver using the Yocto
SDK. A squish-embedded source package can be downloaded from the \l{Froglogic} website.
Additionally, you need to have a binary desktop Squish installation to use the \c squishidl
command.
\section2 Create a Custom (Yocto) Squishserver
The Yocto environment is an x64 Linux but unlike a standard Ubuntu x64 version,
libraries in Yocto are stored in \c /lib64 versus \c /lib .
Download and install the Yocto SDK separately, or use it from your Qt Automotive Suite
installation (Qt Automotive Suite, Intel NUC).
Next, download the squish-embedded source package \c squish-x.x.x-qt-embedded-src.tar.gz from
the \l{Froglogic} website.
To enable cross compiling, run the following command:
\badcode
source <SDK-install-location>/environment-setup-corei7-64-poky-linux
\endcode
Then, change the directory to where the squish-embedded source package is located. Typically,
the binary desktop edition is installed in \c SQUISH_DIR:
\badcode
mkdir -p yocto/build && cd yocto/build
\endcode
Run the next two commands consecutively:
\badcode
../../configure --with-qmake=<SDK-install-location>/sysroots/x86_64-pokysdk-linux/usr/bin/qmake --enable-qmake-config --with-squishidl=$SQUISH_DIR/bin/squishidl --disable-all --enable-qt --enable-server
\endcode
\badcode
./build
\endcode
Next, create a destination directory:
\badcode
mkdir ../install
\endcode
Build and install Yocto into it. Then, create an archive for transferability and storability.
Make sure to use the same version as Squish.
\badcode
./build install DESTDIR=../install && tar cfz ../squish-x.x.x-qt-yocto.tgz ../install
\endcode
\section2 Install and Set Up Custom Yocto Squishserver on Target Hardware
The affected directory structure on the Neptune 3 target is:
\badcode
./home/root
./home/.squish
./opt
./opt/squish
\endcode
Perform the following steps on the target:
\list
\li Unpack the \c squish.x.x.x-qt.yocto.tgz into \c /opt/squish-for-qt-x.x.x and rename the
directory according to the version number, if needed.
\li For convenience, create a symlink from the directory you just created to \c{/opt/squish}.
Consider this symlink as your \c{$SQUISH_DIR}.
\li Edit \c $SQUISH_DIR/etc/squishserverrc according to your network arrangement.
\li Edit \c $SQUISH_DIR/etc/ignoredauts.txt to exclude disruptive sub-processes.
\li Place your license key in the \c{/home/root} directory.
\li Transfer the \c $USER\.squish configuration files from your working Squish installation
directory on your local machine. This directory is created when you first start the
Squish IDE.
\li Then, adapt the paths to the Neptune 3 UI installation: \c /opt/.
The files involved are: \c server.ini and \c paths.ini in the \c .squish/ver1/
sub-folder. For more information, see
\l{https://doc.froglogic.com/squish/latest/rg-ini-files.html}{Squish initialization files}.
\endlist
\note Tests cannot be performed while Neptune 3 UI is running. You need to make
sure that the \c neptune3-ui process is terminated, prior to running the tests.
*/
|
