1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
|
This repository contains much of the configuration which defines what Pulse
projects do.
HOW IT WORKS:
=============
Each Pulse project is made up of several stages. A stage is used to test
a particular configuration of the tested product. Like this:
Qt 4.7 Autotests
|
+-- linux-g++-32 Ubuntu 9.10
|
+-- win32-msvc2008 Windows XP
|
+-- macx-g++ cocoa OSX 10.6
|
...etc
The combination of project & stage names are used to look up all the settings
for the current test. Effectively, all Pulse projects and stages should
call exactly the same top-level script, and that script looks up the settings
for the test in a big hash, where the key is the combination of project and
stage name, and the value is the settings and script needed to do the test.
LAYOUT:
=======
Note that project and stage names have whitespace and special characters
replaced with an `_' when mapping to a filename.
projects/<projectname>/properties
This directory contains one file named after each Pulse property to set.
During the test, the Pulse property named after the file will be set to
the first line of content in the file (properties with values containing
a newline character aren't supported).
The values here may refer to other Pulse properties using the syntax
$(my.property), and may refer to environment variables using the syntax
${MY_ENVIRONMENT_VARIABLE}. If you
want to include a literal $ or \ in the value, you must escape it with
\ (i.e., use \$ or \\). Having a cyclic dependency between properties
is an error.
These properties are global to all build stages for the given project.
projects/<projectname>/environment
This directory contains one file named after each environment variable to
set. During the test, the environment variable named after the file will
be set to the first line of content in the file.
The values here may refer to other Pulse properties in the same way
as values in the `properties' directory. Please note that environment
variables can refer to Pulse properties, but not to other environment
variables.
These environment variables are global to all build stages for the given
project.
projects/<projectname>/repository
This file contains a single line specifying an additional git repository
needed to do the test. The format is:
git/uri#gitbranch directory
...which will cause the specified git repository and branch to be checked
out into the given directory.
For Qt5, this will usually be:
git://gitorious.org/qt/qtqa _qtqa_latest
... as most test scripts are located in this repository at time of writing.
By convention, the destination directory for the clone starts with the _
character, to differentiate it in build logs from directories contained in
the source under test.
This file is optional if no extra repo needs to be checked out for the test.
projects/<projectname>/script
This file contains a single line specifying the script to run in order to
do the test.
Please do not try to do anything tricky here. In particular, don't put any
test logic here! Just run a single command stored _outside_ of this
repository.
This will usually be something like:
perl _qtqa_latest/scripts/setup.pl --install && perl _qtqa_latest/scripts/<some_script>.pl
projects/<projectname>/forcesuccess
If this file exists and contains a true value, then test.pl will exit with
a 0 exit code even when the test fails.
This is intended to be used for "trial" stages which should be put into Pulse
but may not yet be 100% reliable. Using this, the stage can be put into the
true production environment with minimal risk of causing build failures. The
forcesuccess file can later be deleted once the stage is thought to be stable.
Note that `forcesuccess' is unable to catch certain classes of critical
failure, such as a hanging build.
projects/<projectname>/inherits
projects/<projectname>/stages/<stagename>/inherits
If present, these are symbolic links to another project or stage which the
current project or stage inherits from. All settings from the parent
project/stage will apply, unless otherwise overriden.
projects/<projectname>/stages/<stagename>/properties
projects/<projectname>/stages/<stagename>/environment
projects/<projectname>/stages/<stagename>/repository
projects/<projectname>/stages/<stagename>/script
projects/<projectname>/stages/<stagename>/forcesuccess
These work identically to their counterparts in the project directory.
When a file exists in both the project and stage directory, the version
in the stage directory is preferred. Therefore, one can set default
values in the project directory and override them on a per-stage
basis.
manual_overrides/properties
manual_overrides/environment
These work identically to their counterparts in the project directories
but they're applied only when running test.pl outside of Pulse.
Use this sparingly! It's mainly a hack to prevent people from
accidentally uploading binary packages, test results, etc. Without
this, one could easily clobber real test artifacts when testing locally.
test.pl guesses that it's running outside of Pulse based on the
absence of certain environment variables set by Pulse.
CAVEATS:
========
Adding projects/stages in this source tree does _not_ automatically
create the corresponding projects/stages in Pulse. That step needs
to be done manually.
There is no way to represent "required resources" here. This means
that the part of the Pulse configuration specifying which machines
a stage can run on remains not under source control. This can
be mitigated by reducing our usage of "required resources" as much
as possible.
In particular, requiring a resource should ideally _not_ set any
additional properties when doing a test.
There is (currently) no way to represent per-agent properties here.
This could be implemented if it seems worth the effort.
Since this repository uses symbolic links in some places, please be
very careful making any changes on Windows. It is easy to accidentally
destroy the symbolic links when working on that operating system.
|
