| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
| |
PATH should be a ';'-separated string on Windows, not a
JS array.
Change-Id: If4711fc76c530376812328bced971babb26abac9
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
| |
Change-Id: Ia9a874f09d87020c6f6999e6267596a03b64e721
Reviewed-by: Ivan Komissarov <ABBAPOH@gmail.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
===================
Problem description
===================
The ModuleLoader class as it exists before this patch has a number of
serious problems:
- It instantiates modules over and over again everywhere a
Depends item appears. The different instances are later
merged in a hopelessly obscure and error-prone way.
- It seriously pollutes the module scopes so that sloppily written
project files appear to work even though they shouldn't.
- Dependencies between products are handled twice: Once using the
normal module instantiation code (with the Export item
acting like a Module item), and also with a parallel mechanism
that does strange, seemingly redundant things especially regarding
transitive dependencies, which appear to introduce enormous
run-time overhead. It is also split up between ModuleLoader and
ProjectResolver, adding even more confusion.
- The code is messy, seriously under-documented and hardly
understood even by its original authors. It presents a huge obstacle
to potential contributors.
=================
Patch description
=================
- Each module is instantiated once per product. Property values are
merged on the fly. Special handling for dependencies between
products are kept to the absolutely required minimum.
- There are no more extra passes for establishing inter-product
dependencies. Instead, whenever an unhandled dependency is
encountered, processing the current product is paused and resumed
once the dependency is ready, with the product state getting saved
and restored in between so no work is done twice.
- The ModuleLoader class now really only locates and loads modules.
The new main class is called ProjectTreeBuilder, and we have split
off small, self-contained pieces wherever possible. This process
will be continued in follow-up patches (see next section).
=======
Outlook
=======
The ProjectTreeBuilder ist still too large and should be split up
further into small, easily digestible parts with clear responsibilities,
until the top-level code becomes tidy and self-documenting. In the end,
it can probably be merged with ProjectResolver and Loader.
However, this first requires the tight coupling between
ProjectTreeBuilder/ModuleProviderLoader/ProbesResolver/ProjectResolver
to be broken; otherwise we can't produce clean interfaces. As this would
involve touching a lot of otherwise unrelated code, it is out of scope
for this patch.
=================
Benchmarking data
=================
We first present wall-time clock results gathered by running
"qbs resolve --log-time" for qbs itself and Qt Creator on macOS
and Windows. The numbers are the average of several runs, with
outliers removed.
Then the same for a simple QML project using a static Qt on Linux
(this case is special because our current way of handling plugins
causes a huge amount of modules to be loaded).
Finally, we show the output of the qbs_benchmarker tool for
resolving qbs and Qt Creator on Linux.
The data shows a speed-up that is hardly noticeable for simple
projects, but increases sharply with project complexity.
This suggests that our new way of resolving does not suffer
anymore from the non-linear slowdown when the number
of dependencies gets large.
Resolving qbs on Windows:
Before this patch: ModuleLoader 3.6s, ProjectResolver 870ms
With this patch: ProjectTreeBuilder 3.6s, ProjectResolver 840ms
Resolving Qt Creator on Windows:
Before this patch: ModuleLoader 17s, ProjectResolver 6.8s
With this patch: ProjectTreeBuilder 10.0s, ProjectResolver 6.5s
Resolving qbs on macOS:
Before this patch: ModuleLoader 4.0s, ProjectResolver 2.3s
With this patch: ProjectTreeBuilder 4.0s, ProjectResolver 2.3s
Resolving Qt Creator on macOS:
Before this patch: ModuleLoader 32.0s, ProjectResolver 15.6s
With this patch: ProjectTreeBuilder 23.0s, ProjectResolver 15.3s
Note that the above numbers are for an initial resolve, so they include
the time for running Probes. The speed-up for re-resolving (with cached
Probes) is even higher, in particular on macOS, where Probes take
excessively long.
Resolving with static Qt on Linux (QBS-1521):
Before this patch: ModuleLoader 36s, ProjectResolver 18s
With this patch: ProjectTreeBuilder 1.5s, ProjectResolver 14s
Output of qbs_benchmarker for resolving qbs on Linux:
Old instruction count: 10029744668
New instruction count: 9079802661
Relative change: -10 %
Old peak memory usage: 69881840 Bytes
New peak memory usage: 82434624 Bytes
Relative change: +17 %
Output of qbs_benchmarker for resolving Qt Creator on Linux:
Old instruction count: 87364681688
New instruction count: 53634332869
Relative change: -39 %
Old peak memory usage: 578458840 Bytes
New peak memory usage: 567271960 Bytes
Relative change: -2 %
I don't know where the increased memory footprint for a small project
comes from, but since it goes away for larger projects, it doesn't seem
worth investigating.
Fixes: QBS-1037
Task-number: QBS-1521
Change-Id: Ieeebce8a7ff68cdffc15d645e2342ece2426fa94
Reviewed-by: Ivan Komissarov <ABBAPOH@gmail.com>
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
|
|
|
|
|
|
|
| |
contains -> includes
Change-Id: Id893c2ea0659d9bded4f5cec1a3160ccd802118b
Reviewed-by: Ivan Komissarov <ABBAPOH@gmail.com>
|
|
|
|
|
|
|
|
| |
Replace all the possible use of the qbs module that can be replaced by
the Host and FileInfo services.
Change-Id: I0d6f820bb2577eacd9081ec2df1eb53adf94d50e
Reviewed-by: Ivan Komissarov <ABBAPOH@gmail.com>
|
|
|
|
|
|
|
|
|
| |
It makes sense to use the {assembler|compiler}OutputTags() and
the {assembler|compiler}OutputArtifacts() functions also in GCC
and MSVC toolchains, that simplifies and unifies a code.
Change-Id: Ie8cfd185c30322139fb8871b75ea69a888c1f471
Reviewed-by: Ivan Komissarov <ABBAPOH@gmail.com>
|
|
|
|
|
| |
Change-Id: I4e87e8aa98330c9d12a5d23cd2183edbb55d1a4d
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
|
|
|
| |
This patch unifies the functions to generate the output file tags, and the
output artifacts for the precompiled headers.
Change-Id: I84be3f56037220ae5ade6e3d1d1a4e7d0a46847f
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
... to unify a codebase.
Resource output artifacts are now tagged with the 'res' instead of
the 'obj' as before (also the output file extensions are also changed
from '*.obj' to '*.res'). This only affected to the additional
processing for the new 'res' tags in the linker rules for the MSVC
and the MinGW.
The reason for adding the new 'res' tag is that for some compilers
(e.g. the future Open Watcom), the resource files must be linked using
the special flags (and not passed as like regular object files to the
linker).
Change-Id: Ibfc52a384816d3769a7c1f6564db079c118011a0
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
|
|
|
|
| |
It makes sense to use the FileInfo.joinPaths() instead of the '/'
separators in the paths, because the FileInfo.joinPaths() automatically
use the best separators for a specific platforms.
Change-Id: If518cca7f9edf0f199c146b1f22f07ddb51c1d51
Reviewed-by: Ivan Komissarov <ABBAPOH@gmail.com>
|
|
|
|
|
|
|
|
| |
It makes sense to share the cpp.compilerExtension property,
because this property is used in almost all CPP modules.
Change-Id: I0bf44a83c722772b93d7f2febbef7b4e38d11103
Reviewed-by: Ivan Komissarov <ABBAPOH@gmail.com>
|
|
|
|
|
|
|
|
|
| |
It makes sense to minimize the copy-paste errors by moving the
repetitive code into the separate functions to the cpp.js file.
This will simplify the code and improve the maintenance.
Change-Id: Id5a2914ff28619553b855cc6f09810ad370c408c
Reviewed-by: Ivan Komissarov <ABBAPOH@gmail.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
We need to generate a list of file tags and artifacts taking
into account the dependent properties from the product.
For example, we should only build tags for generating linker
files if the generateLinkerMapFile property has been set.
Change-Id: I286c566ffe119eebf24b60113dda65403f7af3dd
Reviewed-by: Ivan Komissarov <ABBAPOH@gmail.com>
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
It makes sense to add the cpp.linkerMapSuffix property to the base
CppModule due the following reasons:
1. It is possible that the user wants to change the extension for
the generated map files, which makes working with Qbs more flexible.
2. It will be easier to write an autotests that check the generation
of the map files for a bare metal platforms, where object files can
have various extensions such as ".map", ".m51", ".m66", and so forth.
Change-Id: I1e5e3b39bf84fddfe8c06f2db9d3936c6a3ff027
Reviewed-by: Ivan Komissarov <ABBAPOH@gmail.com>
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
|
|
| |
We should only sign the resulting binary during the lipo step.
Change-Id: If4d508bcdf347bf2fc68d345ed8d5913a7457f8d
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
It makes sense to add the cpp.objectSuffix property to the base
CppModule due the following reasons:
1. It is possible that the user wants to change the extension for
the generated object files, which makes working with Qbs more flexible.
2. It will be easier to write an autotests that check the generation
of the object files for a bare metal platforms, where object files can
have various extensions such as ".rel", ".r90", ".r43", and so forth.
Change-Id: Ib73836d5e614a6924d312d061f02437c3b318785
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
|
|
|
| |
Android cannot sign individual artifacts so simply checking if
codesign.enableCodeSigning is true is not enough
Change-Id: Ie27ec2b9b7b3b7d7356324590c78b4b967c1b54e
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This moves code signing functionality into a dedicated module, and also
implements automatic provisioning for Apple platforms, which
automatically selects appropriate signing identities and provisioning
profiles based on the product being built.
This also results in a significant performance improvement since all
code signing setup information is retrieved in process instead of
forking off the openssl and security command line tools.
Task-number: QBS-899
Change-Id: I60d0aeaeb2d1004929505bcb1e0bc77512fe77bc
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
|
|
|
| |
This property re-declared in each of toolchain modules. So, it
makes sense to declare this property once in a base CppModule.
Change-Id: I1df76b288660d4b999c918a0019a01825ee0890a
Reviewed-by: Ivan Komissarov <ABBAPOH@gmail.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A problem was in that the GenericGCC.qbs file missing the
initialization for the staticLibrary{Prefix|Suffix} properties.
This properties are initialized only in the derived
UnixGCC.qbs or MingwBaseModule.qbs modules, which are does
work only for the 'unix' or 'windows' platforms.
So, we need to initialize required properties inside
of a base GenericGCC.qbs module.
Change-Id: Ie25f1a19acbc5d86e65eb5ac8f8205ae8079ebd9
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
To be able to use cpp.driverFlags and cpp.driverLinkerFlags with clang-
cl. This patchset makes possible to use clang-cl with "-
fsanitize=address" flag without passing the sanitizer libraries manually
to the linker
There's also a behavior change in which linker is used - clang-cl uses
native linker by default. Old behavior can be restored by setting
cpp.linkerVariant to "lld"
Fixes: QBS-1522
Change-Id: I9528ce40aa5fdfab987672b15fffd830fa2d6376
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
|
|
|
|
| |
Now it is possible to generate a linker map file
from the GCC, MinGW and CLang toolchains.
Change-Id: I69b07e907129daac0ff29bd62034d8807cd5fdf1
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
Reviewed-by: Richard Weickelt <richard@weickelt.de>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The handling of version numbers in the cpp module was overly complicated and
too restrictive. Version numbers that are valid according to SemVer (for
instance 1.2.3-beta+47) were not accepted although being valid.
This patch allows the usage of any version number as long as it starts with
x[.y[.z]]. It also simplifies version number handling in the cpp module.
Task-number: QBS-1486
Change-Id: I2790f8a80bd78bf549f5d46c0748f88c788addec
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is about clang in "mingw mode", not clang-cl.
When targeting Windows, clang is a lot like mingw, so factor out the
common parts into a new base module.
Testing uncovered a number of invalid assumptions in our autotests,
which are also fixed in this patch. In addition, minor adjustments had
to be made to the Qt.core module and to the qbscore lib.
Change-Id: I73085dc62a65e2a9d0397cf234c8641989246f22
Reviewed-by: Ivan Komissarov <ABBAPOH@gmail.com>
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
|
|
|
|
|
|
|
|
| |
Qt Creator uses non-latin letters in path to the build directory in case
of ru_RU.UTF-8 $LANG on Linux. Now QBS does handle that correctly.
Change-Id: Ifc727b120a5f0ee60972c5e3ed24067426db8df2
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Linkerscripts may contain INCLUDE and SEARCH_DIR commands that allows
it to include other linkerscripts in a similary way as headers are
included in C++. This commit adds a scanner that adds these additional
dependencies.
[ChangeLog] Added recursive dependency scanning of GNU ld
linkerscripts that contain INCLUDE and SEARCH_DIR commands.
Change-Id: I7549e27aad4fe7ade2a6a26eba14f66880261077
Reviewed-by: Ola Røer Thorsen <ola@silentwings.no>
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
|
|
|
|
|
|
|
| |
...via cpp.linkerVariant.
[ChangeLog] Added new cpp property linkerVariant to force use of gold,
bfd or lld.
Task-number: QBS-1421
Change-Id: I98f9d2581515ac8e9c984a2f8d70f4a7a3ab230b
Reviewed-by: Orgad Shaneh <orgads@gmail.com>
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
|
|
|
|
|
|
|
|
| |
[ChangeLog] Introduced the property Qt.core.enableBigResources for the
creation of "big" Qt resources.
Change-Id: Ic87aa70132240e67243c1cbe137583a371558261
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Commands can now be assigned to an arbitrary job pool and a limit for
the number of concurrently running jobs in such pools can be provided in
a number of ways:
- via the build command line: qbs --job-limits linker:1
- via the settings: qbs config preferences.jobLimit.linker 1
- in a project file: JobLimit {
jobPool: "linker";
jobCount: 1
}
We provide two job pools ourselves with the cpp module: "compiler" and
"linker".
[ChangeLog] Added the concept of job pools for limiting concurrent
execution of commands by type
Task-number: QBS-743
Change-Id: Ib3f361dbc73093e342bf0eba0daf2079a2b3a8ce
Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@qt.io>
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
|
|\
| |
| |
| | |
Change-Id: I0ce6c28f9117f945c9fec0654bd06daf3d13ace0
|
| |
| |
| |
| |
| |
| |
| |
| |
| | |
It is unrealistic to cover all its possible uses via dedicated
properties.
Task-number: QBS-1018
Change-Id: Iacfb9617ef130255c35766f057c60be9f816bca2
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
The artifacts a dynamic rule produces need to be consistent with what it
declares in its outputFileTags property.
Somewhat suprisingly, such a check was missing so far, so we managed to
make such mistakes in our very own modules. These are also fixed here.
Change-Id: I9052a8c4b0423e8fd146c9fc67bf5c3ce752c638
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
|
| |
| |
| |
| |
| |
| |
| | |
This is the platform standard, and it gives us smart relinking for free.
Change-Id: Iee9e14fe66c9418adaa14893fe2b4f573408f77d
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
|
| |
| |
| |
| |
| | |
Change-Id: I5270e033dc41b5bd76a417099514c69cc073008b
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|/
|
|
|
|
|
|
|
|
| |
The ids are currently unused and so are the names.
But while the ids are of no value at all, the names can be useful for
debugging. Also, we might use them later as a way to identify rules /
groups of "similar" rules (e.g. linker rules).
Change-Id: I6dfac1db7c36a6cae52298ceb0691a106379ae3c
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
|
|
|
|
|
| |
We forgot to adapt the static library rule back when we implemented
smart relinking.
Change-Id: Iea952c868ca7b5c0bda89690865d554be4893bb5
Reviewed-by: Kai Koehne <kai.koehne@qt.io>
Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@qt.io>
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
|
|
|
|
|
|
|
|
|
|
| |
Of our existing backends, only QCC has that requirement.
The old code caused enormous slowness on Darwin, because the build
environment there contains product-specific values.
Task-number: QBS-1277
Change-Id: Ie26c148ff49030eb2626324d6a9f658a794dda23
Reviewed-by: Jake Petroules <jake.petroules@qt.io>
|
|
|
|
|
|
|
|
|
|
|
|
| |
This allows the toolchain prefix to be extracted from the probe itself,
which is preparation for refactoring which determines the prefix
automatically if it is left unspecified.
Right now it's obviously always going to be empty string,
or equal to _toolchainPrefix. A subsequent commit will make use of it.
Change-Id: I0f44c4a9d401e75ba48f3e499a23940954a8b619
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
|
|
| |
This helps to block inappropriate host compilers when cross compiling.
Change-Id: Ied22fce094fdb726babea4e94a7ef1d78afc9a98
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|\
| |
| |
| | |
Change-Id: Iab942ed25d06038cf8c172eb70dcd8e9a720e1c8
|
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This was fixed before in 8176a4d419 and broken again in 72eedbb0a4. Fix
it for good this time using module priorities.
Task-number: QBS-1263
Change-Id: Ib1702299ef6c51b19a9082da96485ef5387dc1bb
Reviewed-by: Jake Petroules <jake.petroules@qt.io>
|
|/
|
|
|
|
|
|
|
|
|
|
| |
...because that reflects the purpose of this artifact much better.
For quite some time we do not create copies of shared objects anymore.
The file tag is internal. Users are not negatively affected.
Change-Id: I0df85587d9f8eb5f5f1c7f272ec57a66588dac29
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
Reviewed-by: Jake Petroules <jake.petroules@qt.io>
Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@qt.io>
|
|
|
|
|
|
|
|
| |
The binutils probe's condition was reversed in a refactoring accident
(commit 2856d903).
Change-Id: I52f01bd98a75c31fb8cd3d51226aa90b2b1c866d
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
|
|
|
|
|
|
| |
We claim that build directories are opaque, yet there's a ".obj" subdir
where object files are placed. This needlessly adds a level in the build
dir structure. Remove the creation and usage of this directory, which is
clearly qmake legacy.
Change-Id: I3afa6bd61b76941ecebbc2cd8f4109621694e371
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
Reviewed-by: Jake Petroules <jake.petroules@qt.io>
|
|\
| |
| |
| | |
Change-Id: I6da76b0a53bf87105c4e7b48724d9760cb68e831
|
| |
| |
| |
| |
| |
| |
| |
| |
| | |
The toolchain prefix was missing in the compiler file name, which lead
to interesting effects, as the probe would potentially detect a wrong
toolchain install path.
Change-Id: I09fe63c5b9860506fe86ade60a74851d428438c8
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
|
|/
|
|
|
|
|
|
| |
This provides users with access to the full list of macros defined by
the MSVC, GCC, Clang, or MinGW compiler in use.
Change-Id: I6e7f6841bbc18aed13958724b268283d97b17320
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
|
| |
Task-number: QBS-1163
Change-Id: I56790b32796f33f7474ffbc95c7f3c86ee8d3c1d
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Now it can actually be useful, in order to select a specific endianness
when building for architectures that are bi-endian. For example, MIPS
and PowerPC.
This patch makes it possible to build for ppc64le, which is the only
variant of the PowerPC architecture that seems to be relevant today.
ppc32 and all big-endian variants seem to have fallen out of use.
Change-Id: Id1b9f9b397990136ab9aa899829345d426465824
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The compiler version is needed in the GccProbe itself (to determine
which compiler flags are available to use) so it cannot be done in a
single probe. This change will be taken advantage of in a subsequent
patch.
This patch also fixes a silent failure that caused the wrong compiler
flags to be used in the GccProbe due to the compiler version being
undefined at the point where feature checks based on the compiler
version were being made... specifically, that the -target flag was
assumed to be unavailable due to the missing compiler version.
Change-Id: I13626c37f3fd21b5b8b6632714ba9dd9373b7838
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|