diff options
author | Christian Kandeler <christian.kandeler@qt.io> | 2017-02-23 16:42:37 +0100 |
---|---|---|
committer | Christian Kandeler <christian.kandeler@qt.io> | 2019-01-23 14:35:02 +0000 |
commit | c2833b1a009bc7c382b30d94109b9b7a25a404a6 (patch) | |
tree | 5b960940919310f8867e503234ca6d4f4295bfbe | |
parent | 1bfc30065371a3d28421e2e7af5653e1e78259f3 (diff) |
Introduce module providers
If a dependency is not found, we now search for a matching module
provider that can generate one for us.
We also provide a generic fall-back provider which uses pkg-config to
locate the dependency (but could be extended to incorporate other
methods in the future). This is the most important part of this change
for practical purposes, as it makes hundreds of popular libraries
available for use in qbs projects without users having to write any
boilerplate code.
In a future patch, a module provider could also be used to implement the
functionality of the qtprofilesetup library, relieving users of the need
to create a profile for building Qt applications.
[ChangeLog] The Depends item now falls back to pkg-config to locate
dependencies whose names do not correspond to a qbs module.
Fixes: QBS-1107
Change-Id: Ifd4f05c237cf58cd9fe707c3da648d3dbb33e82b
Reviewed-by: Leena Miettinen <riitta-leena.miettinen@qt.io>
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
52 files changed, 1129 insertions, 44 deletions
diff --git a/doc/howtos.qdoc b/doc/howtos.qdoc index 7ef1fc086..a2595cffd 100644 --- a/doc/howtos.qdoc +++ b/doc/howtos.qdoc @@ -41,6 +41,7 @@ \li \l{How do I make sure my generated sources are getting compiled?} \li \l{How do I run my autotests?} \li \l{How do I create a module for a third-party library?} + \li \l{How do I build against libraries that provide pkg-config files?} \li \l{How do I create application bundles and frameworks on iOS, macOS, tvOS, and watchOS?} \li \l{How do I apply C/C++ preprocessor macros to only a subset of the files in my product?} \li \l{How do I make the state of my Git repository available to my source files?} @@ -315,6 +316,20 @@ static library; see the \l{How do I make my app build against my library?} section for an example. + \section1 How do I build against libraries that provide pkg-config files? + + Just add a \l Depends item that matches the name of the pkg-config module, and \QBS + will automatically employ \l{https://www.freedesktop.org/wiki/Software/pkg-config}{pkg-config} + to find the headers and libraries if no matching \QBS module can be found. For instance, + to build against the OpenSSL library, you would write this: + \code + Depends { name: "openssl" } + \endcode + That's it. The pkg-config behavior can be fine-tuned via the \l pkgconfig module, + but normally you will not need to pull it in explicitly. + + Internally, this functionality is implemented via \l {Module Providers} + \section1 How do I apply C/C++ preprocessor macros to only a subset of the files in my product? Use a \l{Group} item to define a subset of project files. To add diff --git a/doc/qbs.qdoc b/doc/qbs.qdoc index 14787ffbf..44678bc40 100644 --- a/doc/qbs.qdoc +++ b/doc/qbs.qdoc @@ -68,6 +68,7 @@ \li \l{Generators} \li \l{Multiplexing} \li \l{Custom Modules and Items} + \li \l{Module Providers} \endlist \li \l{How-tos} \li \l{Reference} @@ -791,6 +792,7 @@ \li \l{Generators} \li \l{Multiplexing} \li \l{Custom Modules and Items} + \li \l{Module Providers} \endlist */ @@ -1351,7 +1353,7 @@ \contentspage index.html \previouspage multiplexing.html \page custom-modules.html - \nextpage howtos.html + \nextpage module-providers.html \title Custom Modules and Items @@ -1421,6 +1423,62 @@ /*! \contentspage index.html + \previouspage custom-modules.html + \page module-providers.html + \nextpage howtos.html + + \title Module Providers + + There are use cases for which a pre-defined module is not flexible enough. + For instance, the overall set of modules related to a certain task might depend + on some information present on the local platform. + + \note Module providers are an advanced concept that you will rarely need to use directly. + Reading this section is not required for most people's everyday work. + + \section1 How \QBS Uses Module Providers + + If \QBS encounters a \l Depends item whose name does not match a known module, + it checks whether such a module can be generated. This procedure works as follows: + \list 1 + \li All \l{Project::qbsSearchPaths}{search paths} are scanned for a file called + \c {module-providers/<name>/provider.qbs}, where \c <name> is the name of the dependency + as specified in the \c Depends item. Multi-component names such as "a.b" are turned + into nested directories, and each of them is scanned, starting with the deepest path. + For instance, if the dependency's name is \c {a.b}, then \QBS will look for + \c {a/b/provider.qbs} and then \c {a/provider.qbs}. + \li If such a file is found, it needs to contain a \l ModuleProvider item. This item + is instantiated, which potentially leads to the creation of one or more modules, + and \QBS retrieves the search paths to find these modules from the item. + The details are described in the \l ModuleProvider documentation. + \li If a matching module provider was found and provided new search paths, + a second attempt will be made to locate the dependency using the new paths. + The search for a matching module provider ends as soon as one was found, regardless + of whether it created any modules or not. + \li If no matching module provider was found in any of the search paths, \QBS will fall back + to a generic module provider, which creates a module that attempts to locate the + dependency via \c pkg-config. + This fallback mechanism can be disabled in the respective + \l{Depends::enableFallback}{Depends} item or globally via the + \l{no-fallback-module-provider}{--no-fallback-module-provider} option. + \endlist + + \section1 Parameterizing Module Providers + + You can pass information to module providers from the command line, via profiles or + from within a product, in a similar way as you would do for modules. For instance, the + following invocation of \QBS passes information to two module providers \c a and \c b: + \code + $ qbs moduleProviders.a.p1:true moduleProviders.a.p2:true moduleProviders.b.p:false + \endcode + \QBS will set the properties of the respective module providers accordingly. + In the above example, module provider \c a needs to declare two boolean properties \c p1 + and \c p2, and they will be set to \c true and \c false, respectively. + +*/ + +/*! + \contentspage index.html \previouspage shell.html \page generators.html \nextpage multiplexing.html diff --git a/doc/reference/cli/builtin/cli-build.qdoc b/doc/reference/cli/builtin/cli-build.qdoc index cffb19d49..7844dceea 100644 --- a/doc/reference/cli/builtin/cli-build.qdoc +++ b/doc/reference/cli/builtin/cli-build.qdoc @@ -79,6 +79,8 @@ \include cli-options.qdocinc products-specified \include cli-options.qdocinc settings-dir \include cli-options.qdocinc show-progress + \target no-fallback-module-provider + \include cli-options.qdocinc no-fallback-module-provider \include cli-options.qdocinc wait-lock \section1 Parameters diff --git a/doc/reference/cli/builtin/cli-resolve.qdoc b/doc/reference/cli/builtin/cli-resolve.qdoc index 7170856f2..b13f3de3d 100644 --- a/doc/reference/cli/builtin/cli-resolve.qdoc +++ b/doc/reference/cli/builtin/cli-resolve.qdoc @@ -56,6 +56,7 @@ \include cli-options.qdocinc more-verbose \include cli-options.qdocinc settings-dir \include cli-options.qdocinc show-progress + \include cli-options.qdocinc no-fallback-module-provider \section1 Parameters diff --git a/doc/reference/cli/cli-options.qdocinc b/doc/reference/cli/cli-options.qdocinc index 254444dcb..2111d8a2d 100644 --- a/doc/reference/cli/cli-options.qdocinc +++ b/doc/reference/cli/cli-options.qdocinc @@ -459,6 +459,16 @@ //! [show-progress] +//! [no-fallback-module-provider] + + \section2 \c --no-fallback-module-provider + + If this option is set, then \QBS will not fall back to a pkg-config based + \l{Module Providers}{module provider} if a dependency is not found. + +//! [no-fallback-module-provider] + + //! [setup-tools-system] \section2 \c {--system} diff --git a/doc/reference/items/language/depends.qdoc b/doc/reference/items/language/depends.qdoc index b57c4a79f..8a3e23ba9 100644 --- a/doc/reference/items/language/depends.qdoc +++ b/doc/reference/items/language/depends.qdoc @@ -38,25 +38,21 @@ A Depends item can appear inside a \l{Product} or \l{Module} item. For example, the following product will load the \l{cpp} module. In - addition, it will try to load modules that may or may not exist, and in the - latter case use a fallback. + addition, it will try to load modules that may or may not exist, and + pass this information on to the compiler. \code Product { Depends { name: "cpp" } Depends { - name: "awesome_module" + name: "optional_module" versionAtLeast: "2.0" required: false } - Depends { - name: "adequate_module" - condition: !awesome_module.present - required: false - } - Depends { - name: "inferior_module" - condition: !awesome_module.present && !adequate_module.present + + Properties { + condition: optional_module.present + cpp.defines: "HAS_OPTIONAL_MODULE" } // ... @@ -190,3 +186,12 @@ \nodefaultvalue */ + +/*! + \qmlproperty bool Depends::enableFallback + + Whether to fall back to a pkg-config based \l{Module Providers}{module provider} + if the dependency is not found. + + \defaultvalue \c true +*/ diff --git a/doc/reference/items/language/module.qdoc b/doc/reference/items/language/module.qdoc index e5472983f..7dd5249b2 100644 --- a/doc/reference/items/language/module.qdoc +++ b/doc/reference/items/language/module.qdoc @@ -27,7 +27,7 @@ /*! \contentspage list-of-language-items.html \previouspage JobLimit - \nextpage Parameter + \nextpage ModuleProvider \qmltype Module \inqmlmodule QbsLanguageItems \ingroup list-of-items diff --git a/doc/reference/items/language/moduleprovider.qdoc b/doc/reference/items/language/moduleprovider.qdoc new file mode 100644 index 000000000..ddbc25959 --- /dev/null +++ b/doc/reference/items/language/moduleprovider.qdoc @@ -0,0 +1,108 @@ +/**************************************************************************** +** +** Copyright (C) 2017 The Qt Company Ltd. +** Contact: https://www.qt.io/licensing/ +** +** This file is part of Qbs. +** +** $QT_BEGIN_LICENSE:FDL$ +** Commercial License Usage +** Licensees holding valid commercial Qt licenses may use this file in +** accordance with the commercial license agreement provided with the +** Software or, alternatively, in accordance with the terms contained in +** a written agreement between you and The Qt Company. For licensing terms +** and conditions see https://www.qt.io/terms-conditions. For further +** information use the contact form at https://www.qt.io/contact-us. +** +** GNU Free Documentation License Usage +** Alternatively, this file may be used under the terms of the GNU Free +** Documentation License version 1.3 as published by the Free Software +** Foundation and appearing in the file included in the packaging of +** this file. Please review the following information to ensure +** the GNU Free Documentation License version 1.3 requirements +** will be met: https://www.gnu.org/licenses/fdl-1.3.html. +** $QT_END_LICENSE$ +** +****************************************************************************/ +/*! + \contentspage list-of-language-items.html + \previouspage Module + \nextpage Parameter + \qmltype ModuleProvider + \inqmlmodule QbsLanguageItems + \ingroup list-of-items + \keyword QML.ModuleProvider + + \brief Creates modules on demand. + + The \c ModuleProvider item implements the module creation part of the procedure described + in the \l {Module Providers} overview. It is always located in a file called \c provider.qbs. + + The actual module creation is done on the right-hand side of the + \l{ModuleProvider::relativeSearchPaths}{relativeSearchPaths} property. + + Here is a complete minimal example of a module provider. It just creates an empty module. + If you put this item into the file \c {module-providers/mymodule/provider.qbs} + in your project source directory, you will be able to successfully build a product which + contains a dependency on the module \c mymodule. + \code + import qbs.File + import qbs.FileInfo + import qbs.TextFile + + ModuleProvider { + relativeSearchPaths: { + var moduleDir = FileInfo.joinPaths(outputBaseDir, "modules", name); + File.makePath(moduleDir); + var moduleFilePath = FileInfo.joinPaths(moduleDir, name + ".qbs"); + var moduleFile = new TextFile(moduleFilePath, TextFile.WriteOnly); + moduleFile.writeLine("Module {"); + moduleFile.writeLine("}"); + moduleFile.close(); + return ""; + } + } + \endcode +*/ + +/*! + \qmlproperty string ModuleProvider::name + + The name of the module provider. + + This property is set by \QBS. For simple dependency names, it is the name of the dependency + as specified in the \l Depends item. If the dependency name consists of multiple components, + the value is the name up until (and including) the component that corresponds to the directory + the provider was found in. For instance, if the dependency is \c {x.m1} and the provider was + found in \c {module-providers/x/m1/provider.qbs}, then \c name is \c {x.m1}. + If the provider was found in \c {module-providers/x/provider.qbs}, then \c name is \c x. +*/ + +/*! + \qmlproperty string ModuleProvider::outputBaseDir + + The path under which the new modules should be created when \l relativeSearchPaths + is evaluated. The path is unique for the current provider in the given configuration. + + This property is set by \QBS. +*/ + +/*! + \qmlproperty stringList ModuleProvider::relativeSearchPaths + + This property gets evaluated by \QBS to retrieve new search paths with which + to re-attempt the module look-up. + + It is here where you need to put the code that creates the new module files. + Use the directory structure explained in \l {Custom Modules and Items}. + That is, the file for a module called \c m will be located in a directory \c {modules/m/}, + anchored at \l outputBaseDir. + + The return value is the list of search paths required to find the new module, + relative to \l outputBaseDir. In most cases, only a single search path will be required, + in which case a single-element list containing an empty string should be returned + (or just the empty string, because of \QBS' auto-conversion feature). + + The returned list can also be empty, which means that the module provider was not able + to generate any modules in this environment. +*/ diff --git a/doc/reference/modules/pkgconfig-module.qdoc b/doc/reference/modules/pkgconfig-module.qdoc new file mode 100644 index 000000000..898349628 --- /dev/null +++ b/doc/reference/modules/pkgconfig-module.qdoc @@ -0,0 +1,69 @@ +/**************************************************************************** +** +** Copyright (C) 2019 The Qt Company Ltd. +** Contact: https://www.qt.io/licensing/ +** +** This file is part of Qbs. +** +** $QT_BEGIN_LICENSE:FDL$ +** Commercial License Usage +** Licensees holding valid commercial Qt licenses may use this file in +** accordance with the commercial license agreement provided with the +** Software or, alternatively, in accordance with the terms contained in +** a written agreement between you and The Qt Company. For licensing terms +** and conditions see https://www.qt.io/terms-conditions. For further +** information use the contact form at https://www.qt.io/contact-us. +** +** GNU Free Documentation License Usage +** Alternatively, this file may be used under the terms of the GNU Free +** Documentation License version 1.3 as published by the Free Software +** Foundation and appearing in the file included in the packaging of +** this file. Please review the following information to ensure +** the GNU Free Documentation License version 1.3 requirements +** will be met: https://www.gnu.org/licenses/fdl-1.3.html. +** $QT_END_LICENSE$ +** +****************************************************************************/ + +/*! + \contentspage index.html + \qmltype pkgconfig + \inqmlmodule QbsModules + \since 1.13 + + \brief Allows to configure the pkg-config tool. + + The \c pkgconfig module is used to fine-tune the behavior of the \c {pkg-config} tool, + which is + \l{How do I build against libraries that provide pkg-config files?}{potentially employed} + when looking up dependencies. +*/ + +/*! + \qmlproperty string pkgconfig::executableFilePath + + The path to the \c {pkg-config} executable. + + \defaultvalue auto-detected +*/ + +/*! + \qmlproperty stringList pkgconfig::libDirs + + Set this if you need to overwrite the default search directories. The values + given here will be forwarded to the tool via the \c PKG_CONFIG_LIBDIR environment + variable. + \note You do not need to set this for cross-compilation in order to point + \c {pkg-config} to the sysroot. \QBS does that for you. + + \nodefaultvalue +*/ + +/*! + \qmlproperty bool pkgconfig::staticMode + + If this property is \c true, then calls to \c{pkg-config} will include the + \c{--static} option. Set this if your product is to be linked statically. + + \defaultvalue \c false +*/ diff --git a/share/qbs/module-providers/__fallback/fallback.qbs b/share/qbs/module-providers/__fallback/fallback.qbs new file mode 100644 index 000000000..e23851951 --- /dev/null +++ b/share/qbs/module-providers/__fallback/fallback.qbs @@ -0,0 +1,76 @@ +/**************************************************************************** +** +** Copyright (C) 2019 The Qt Company Ltd. +** Contact: https://www.qt.io/licensing/ +** +** This file is part of Qbs. +** +** $QT_BEGIN_LICENSE:LGPL$ +** Commercial License Usage +** Licensees holding valid commercial Qt licenses may use this file in +** accordance with the commercial license agreement provided with the +** Software or, alternatively, in accordance with the terms contained in +** a written agreement between you and The Qt Company. For licensing terms +** and conditions see https://www.qt.io/terms-conditions. For further +** information use the contact form at https://www.qt.io/contact-us. +** +** GNU Lesser General Public License Usage +** Alternatively, this file may be used under the terms of the GNU Lesser +** General Public License version 3 as published by the Free Software +** Foundation and appearing in the file LICENSE.LGPL3 included in the +** packaging of this file. Please review the following information to +** ensure the GNU Lesser General Public License version 3 requirements +** will be met: https://www.gnu.org/licenses/lgpl-3.0.html. +** +** GNU General Public License Usage +** Alternatively, this file may be used under the terms of the GNU +** General Public License version 2.0 or (at your option) the GNU General +** Public license version 3 or any later version approved by the KDE Free +** Qt Foundation. The licenses are as published by the Free Software +** Foundation and appearing in the file LICENSE.GPL2 and LICENSE.GPL3 +** included in the packaging of this file. Please review the following +** information to ensure the GNU General Public License requirements will +** be met: https://www.gnu.org/licenses/gpl-2.0.html and +** https://www.gnu.org/licenses/gpl-3.0.html. +** +** $QT_END_LICENSE$ +** +****************************************************************************/ + +import qbs +import qbs.FileInfo +import qbs.Probes + +Module { + Depends { name: "cpp" } + Depends { name: "pkgconfig"; required: false } + + property string theName: FileInfo.completeBaseName(filePath) + + Probes.PkgConfigProbe { + id: pkgConfigProbe + condition: pkgconfig.present + name: theName + executable: pkgconfig.executableFilePath + libDirs: pkgconfig.libDirs + forStaticBuild: pkgconfig.staticMode + } + + Properties { + condition: pkgConfigProbe.found + version: pkgConfigProbe.modversion + cpp.dynamicLibraries: pkgConfigProbe.libraries + cpp.libraryPaths: pkgConfigProbe.libraryPaths + cpp.includePaths: pkgConfigProbe.includePaths + cpp.defines: pkgConfigProbe.defines + cpp.driverLinkerFlags: pkgConfigProbe.linkerFlags + cpp.commonCompilerFlags: pkgConfigProbe.compilerFlags + } + + validate: { + if (!pkgConfigProbe.found) { + throw "Dependency '" + theName + "' not found for product '" + product.name + "'. " + + "Locating a package of this name via pkg-config also failed."; + } + } +} diff --git a/share/qbs/module-providers/__fallback/provider.qbs b/share/qbs/module-providers/__fallback/provider.qbs new file mode 100644 index 000000000..72740f3c3 --- /dev/null +++ b/share/qbs/module-providers/__fallback/provider.qbs @@ -0,0 +1,53 @@ +/**************************************************************************** +** +** Copyright (C) 2019 The Qt Company Ltd. +** Contact: https://www.qt.io/licensing/ +** +** This file is part of Qbs. +** +** $QT_BEGIN_LICENSE:LGPL$ +** Commercial License Usage +** Licensees holding valid commercial Qt licenses may use this file in +** accordance with the commercial license agreement provided with the +** Software or, alternatively, in accordance with the terms contained in +** a written agreement between you and The Qt Company. For licensing terms +** and conditions see https://www.qt.io/terms-conditions. For further +** information use the contact form at https://www.qt.io/contact-us. +** +** GNU Lesser General Public License Usage +** Alternatively, this file may be used under the terms of the GNU Lesser +** General Public License version 3 as published by the Free Software +** Foundation and appearing in the file LICENSE.LGPL3 included in the +** packaging of this file. Please review the following information to +** ensure the GNU Lesser General Public License version 3 requirements +** will be met: https://www.gnu.org/licenses/lgpl-3.0.html. +** +** GNU General Public License Usage +** Alternatively, this file may be used under the terms of the GNU +** General Public License version 2.0 or (at your option) the GNU General +** Public license version 3 or any later version approved by the KDE Free +** Qt Foundation. The licenses are as published by the Free Software +** Foundation and appearing in the file LICENSE.GPL2 and LICENSE.GPL3 +** included in the packaging of this file. Please review the following +** information to ensure the GNU General Public License requirements will +** be met: https://www.gnu.org/licenses/gpl-2.0.html and +** https://www.gnu.org/licenses/gpl-3.0.html. +** +** $QT_END_LICENSE$ +** +****************************************************************************/ + +import qbs.File +import qbs.FileInfo + +ModuleProvider { + relativeSearchPaths: { + console.debug("Running fallback provider for module '" + name + "'."); + var inputFilePath = FileInfo.joinPaths(path, "fallback.qbs"); + var outputDir = FileInfo.joinPaths(outputBaseDir, "modules", name.replace(".", "/")); + File.makePath(outputDir); + var outputFilePath = FileInfo.joinPaths(outputDir, name + ".qbs"); + File.copy(inputFilePath, outputFilePath); + return ""; + } +} diff --git a/share/qbs/modules/pkgconfig/pkgconfig.qbs b/share/qbs/modules/pkgconfig/pkgconfig.qbs new file mode 100644 index 000000000..92e0bfb25 --- /dev/null +++ b/share/qbs/modules/pkgconfig/pkgconfig.qbs @@ -0,0 +1,62 @@ +/**************************************************************************** +** +** Copyright (C) 2019 The Qt Company Ltd. +** Contact: https://www.qt.io/licensing/ +** +** This file is part of Qbs. +** +** $QT_BEGIN_LICENSE:LGPL$ +** Commercial License Usage +** Licensees holding valid commercial Qt licenses may use this file in +** accordance with the commercial license agreement provided with the +** Software or, alternatively, in accordance with the terms contained in +** a written agreement between you and The Qt Company. For licensing terms +** and conditions see https://www.qt.io/terms-conditions. For further +** information use the contact form at https://www.qt.io/contact-us. +** +** GNU Lesser General Public License Usage +** Alternatively, this file may be used under the terms of the GNU Lesser +** General Public License version 3 as published by the Free Software +** Foundation and appearing in the file LICENSE.LGPL3 included in the +** packaging of this file. Please review the following information to +** ensure the GNU Lesser General Public License version 3 requirements +** will be met: https://www.gnu.org/licenses/lgpl-3.0.html. +** +** GNU General Public License Usage +** Alternatively, this file may be used under the terms of the GNU +** General Public License version 2.0 or (at your option) the GNU General +** Public license version 3 or any later version approved by the KDE Free +** Qt Foundation. The licenses are as published by the Free Software +** Foundation and appearing in the file LICENSE.GPL2 and LICENSE.GPL3 +** included in the packaging of this file. Please review the following +** information to ensure the GNU General Public License requirements will +** be met: https://www.gnu.org/licenses/gpl-2.0.html and +** https://www.gnu.org/licenses/gpl-3.0.html. +** +** $QT_END_LICENSE$ +** +****************************************************************************/ + +import qbs +import qbs.File +import qbs.Probes + +Module { + Probes.BinaryProbe { + id: pkgconfigProbe + names: "pkg-config" + } + + property string executableFilePath: pkgconfigProbe.filePath + property stringList libDirs + property bool staticMode: false + + validate: { + if (!executableFilePath) { + throw "No pkg-config executable found. " + + "Please set modules.pkgconfig.executableFilePath."; + } + if (!File.exists(executableFilePath)) + throw "The pkg-config executable '" + executableFilePath + "' does not exist."; + } +} diff --git a/share/share.qbs b/share/share.qbs index 6c99f64d5..9349d5c8b 100644 --- a/share/share.qbs +++ b/share/share.qbs @@ -56,6 +56,15 @@ Product { } Group { + name: "Module providers" + files: ["qbs/module-providers/**/*"] + fileTags: ["qbs resources"] + qbs.install: true + qbs.installDir: qbsbuildconfig.resourcesInstallDir + "/share" + qbs.installSourceBase: "." + } + + Group { name: "Examples as resources" files: ["../examples/**/*"] fileTags: [] diff --git a/src/app/qbs/commandlinefrontend.cpp b/src/app/qbs/commandlinefrontend.cpp index 978fdbb7a..c848b24d0 100644 --- a/src/app/qbs/commandlinefrontend.cpp +++ b/src/app/qbs/commandlinefrontend.cpp @@ -146,6 +146,7 @@ void CommandLineFrontend::start() params.setDryRun(m_parser.dryRun()); params.setForceProbeExecution(m_parser.forceProbesExecution()); params.setWaitLockBuildGraph(m_parser.waitLockBuildGraph()); + params.setFallbackProviderEnabled(!m_parser.disableFallbackProvider()); params.setLogElapsedTime(m_parser.logTime()); params.setSettingsDirectory(m_settings->baseDirectory()); params.setOverrideBuildGraphData(m_parser.command() == ResolveCommandType); diff --git a/src/app/qbs/parser/commandlineoption.cpp b/src/app/qbs/parser/commandlineoption.cpp index e18658751..dc5b4e440 100644 --- a/src/app/qbs/parser/commandlineoption.cpp +++ b/src/app/qbs/parser/commandlineoption.cpp @@ -681,6 +681,17 @@ QString WaitLockOption::longRepresentation() const return QLatin1String("--wait-lock"); } +QString DisableFallbackProviderOption::description(CommandType) const +{ + return Tr::tr("%1\n\tDo not fall back to pkg-config if a dependency is not found.\n") + .arg(longRepresentation()); +} + +QString DisableFallbackProviderOption::longRepresentation() const +{ + return QLatin1String("--no-fallback-module-provider"); +} + QString RunEnvConfigOption::description(CommandType command) const { Q_UNUSED(command); diff --git a/src/app/qbs/parser/commandlineoption.h b/src/app/qbs/parser/commandlineoption.h index d57ec76b7..414f90489 100644 --- a/src/app/qbs/parser/commandlineoption.h +++ b/src/app/qbs/parser/commandlineoption.h @@ -75,6 +75,7 @@ public: GeneratorOptionType, WaitLockOptionType, RunEnvConfigOptionType, + DisableFallbackProviderType, }; virtual ~CommandLineOption(); @@ -414,6 +415,14 @@ public: QString longRepresentation() const override; }; +class DisableFallbackProviderOption : public OnOffOption +{ +public: + QString description(CommandType command) const override; + QString shortRepresentation() const override { return QString(); } + QString longRepresentation() const override; +}; + } // namespace qbs #endif // QBS_COMMANDLINEOPTION_H diff --git a/src/app/qbs/parser/commandlineoptionpool.cpp b/src/app/qbs/parser/commandlineoptionpool.cpp index 9964f051a..63711f623 100644 --- a/src/app/qbs/parser/commandlineoptionpool.cpp +++ b/src/app/qbs/parser/commandlineoptionpool.cpp @@ -128,6 +128,9 @@ CommandLineOption *CommandLineOptionPool::getOption(CommandLineOption::Type type case CommandLineOption::WaitLockOptionType: option = new WaitLockOption; break; + case CommandLineOption::DisableFallbackProviderType: + option = new DisableFallbackProviderOption; + break; case CommandLineOption::RunEnvConfigOptionType: option = new RunEnvConfigOption; break; @@ -273,6 +276,12 @@ WaitLockOption *CommandLineOptionPool::waitLockOption() const return static_cast<WaitLockOption *>(getOption(CommandLineOption::WaitLockOptionType)); } +DisableFallbackProviderOption *CommandLineOptionPool::disableFallbackProviderOption() const +{ + return static_cast<DisableFallbackProviderOption *>( + getOption(CommandLineOption::DisableFallbackProviderType)); +} + RunEnvConfigOption *CommandLineOptionPool::runEnvConfigOption() const { return static_cast<RunEnvConfigOption *>(getOption(CommandLineOption::RunEnvConfigOptionType)); diff --git a/src/app/qbs/parser/commandlineoptionpool.h b/src/app/qbs/parser/commandlineoptionpool.h index 6a4669165..c7ac263e1 100644 --- a/src/app/qbs/parser/commandlineoptionpool.h +++ b/src/app/qbs/parser/commandlineoptionpool.h @@ -77,6 +77,7 @@ public: RespectProjectJobLimitsOption *respectProjectJobLimitsOption() const; GeneratorOption *generatorOption() const; WaitLockOption *waitLockOption() const; + DisableFallbackProviderOption *disableFallbackProviderOption() const; RunEnvConfigOption *runEnvConfigOption() const; private: diff --git a/src/app/qbs/parser/commandlineparser.cpp b/src/app/qbs/parser/commandlineparser.cpp index c2e265336..2ec0df1df 100644 --- a/src/app/qbs/parser/commandlineparser.cpp +++ b/src/app/qbs/parser/commandlineparser.cpp @@ -231,6 +231,11 @@ bool CommandLineParser::waitLockBuildGraph() const return d->optionPool.waitLockOption()->enabled(); } +bool CommandLineParser::disableFallbackProvider() const +{ + return d->optionPool.disableFallbackProviderOption()->enabled(); +} + bool CommandLineParser::logTime() const { return d->logTime; diff --git a/src/app/qbs/parser/commandlineparser.h b/src/app/qbs/parser/commandlineparser.h index e2ef8ad77..d47657b16 100644 --- a/src/app/qbs/parser/commandlineparser.h +++ b/src/app/qbs/parser/commandlineparser.h @@ -76,6 +76,7 @@ public: bool dryRun() const; bool forceProbesExecution() const; bool waitLockBuildGraph() const; + bool disableFallbackProvider() const; bool logTime() const; bool withNonDefaultProducts() const; bool buildBeforeInstalling() const; diff --git a/src/app/qbs/parser/parsercommand.cpp b/src/app/qbs/parser/parsercommand.cpp index 33f93ce53..79636ff0a 100644 --- a/src/app/qbs/parser/parsercommand.cpp +++ b/src/app/qbs/parser/parsercommand.cpp @@ -210,7 +210,8 @@ static QList<CommandLineOption::Type> resolveOptions() << CommandLineOption::ShowProgressOptionType << CommandLineOption::DryRunOptionType << CommandLineOption::ForceProbesOptionType - << CommandLineOption::LogTimeOptionType; + << CommandLineOption::LogTimeOptionType + << CommandLineOption::DisableFallbackProviderType; } QList<CommandLineOption::Type> ResolveCommand::supportedOptions() const diff --git a/src/lib/corelib/buildgraph/buildgraphloader.cpp b/src/lib/corelib/buildgraph/buildgraphloader.cpp index a1ca7afdb..e6d1cb75a 100644 --- a/src/lib/corelib/buildgraph/buildgraphloader.cpp +++ b/src/lib/corelib/buildgraph/buildgraphloader.cpp @@ -340,6 +340,8 @@ void BuildGraphLoader::trackProjectChanges() ldr.setSearchPaths(m_parameters.searchPaths()); ldr.setProgressObserver(m_evalContext->observer()); ldr.setOldProjectProbes(restoredProject->probes); + if (!m_parameters.forceProbeExecution()) + ldr.setStoredModuleProviderInfo(restoredProject->moduleProviderInfo); ldr.setLastResolveTime(restoredProject->lastResolveTime); QHash<QString, std::vector<ProbeConstPtr>> restoredProbes; for (const auto &restoredProduct : qAsConst(allRestoredProducts)) diff --git a/src/lib/corelib/corelib.qbs b/src/lib/corelib/corelib.qbs index c947cb484..db00a7005 100644 --- a/src/lib/corelib/corelib.qbs +++ b/src/lib/corelib/corelib.qbs @@ -299,6 +299,7 @@ QbsLibrary { "moduleloader.h", "modulemerger.cpp", "modulemerger.h", + "moduleproviderinfo.h", "preparescriptobserver.cpp", "preparescriptobserver.h", "projectresolver.cpp", diff --git a/src/lib/corelib/language/builtindeclarations.cpp b/src/lib/corelib/language/builtindeclarations.cpp index bfdfab51e..3ca0608d4 100644 --- a/src/lib/corelib/language/builtindeclarations.cpp +++ b/src/lib/corelib/language/builtindeclarations.cpp @@ -69,6 +69,7 @@ BuiltinDeclarations::BuiltinDeclarations() { QLatin1String("Group"), ItemType::Group }, { QLatin1String("JobLimit"), ItemType::JobLimit }, { QLatin1String("Module"), ItemType::Module }, + { QLatin1String("ModuleProvider"), ItemType::ModuleProvider }, { QLatin1String("Parameter"), ItemType::Parameter }, { QLatin1String("Parameters"), ItemType::Parameters }, { QLatin1String("Probe"), ItemType::Probe }, @@ -90,6 +91,7 @@ BuiltinDeclarations::BuiltinDeclarations() addGroupItem(); addJobLimitItem(); addModuleItem(); + addModuleProviderItem(); addProbeItem(); addProductItem(); addProfileItem(); @@ -244,6 +246,8 @@ void BuiltinDeclarations::addDependsItem() item << PropertyDeclaration(StringConstants::multiplexConfigurationIdsProperty(), PropertyDeclaration::StringList, QString(), PropertyDeclaration::ReadOnlyFlag); + item << PropertyDeclaration(StringConstants::enableFallbackProperty(), + PropertyDeclaration::Boolean, StringConstants::trueValue()); insert(item); } @@ -316,6 +320,16 @@ void BuiltinDeclarations::addModuleItem() insert(item); } +void BuiltinDeclarations::addModuleProviderItem() +{ + ItemDeclaration item(ItemType::ModuleProvider); + item << nameProperty() + << PropertyDeclaration(QStringLiteral("outputBaseDir"), PropertyDeclaration::String) + << PropertyDeclaration(QStringLiteral("relativeSearchPaths"), + PropertyDeclaration::StringList); + insert(item); +} + ItemDeclaration BuiltinDeclarations::moduleLikeItem(ItemType type) { ItemDeclaration item(type); diff --git a/src/lib/corelib/language/builtindeclarations.h b/src/lib/corelib/language/builtindeclarations.h index ff16b395a..988f9ab81 100644 --- a/src/lib/corelib/language/builtindeclarations.h +++ b/src/lib/corelib/language/builtindeclarations.h @@ -78,6 +78,7 @@ private: void addGroupItem(); void addJobLimitItem(); void addModuleItem(); + void addModuleProviderItem(); static ItemDeclaration moduleLikeItem(ItemType type); void addProbeItem(); void addProductItem(); diff --git a/src/lib/corelib/language/evaluatorscriptclass.cpp b/src/lib/corelib/language/evaluatorscriptclass.cpp index a5e23a131..829cb7494 100644 --- a/src/lib/corelib/language/evaluatorscriptclass.cpp +++ b/src/lib/corelib/language/evaluatorscriptclass.cpp @@ -621,7 +621,8 @@ public: || itemOfProperty->type() == ItemType::Export)) { const VariantValueConstPtr varValue = itemOfProperty->variantProperty(StringConstants::nameProperty()); - QBS_ASSERT(varValue, return); + if (!varValue) + return; m_stackUpdate = true; const QualifiedId fullPropName = QualifiedId::fromString(varValue->value().toString()) << name.toString(); diff --git a/src/lib/corelib/language/itemtype.h b/src/lib/corelib/language/itemtype.h index 324a1fb87..724666cb4 100644 --- a/src/lib/corelib/language/itemtype.h +++ b/src/lib/corelib/language/itemtype.h @@ -55,6 +55,7 @@ enum class ItemType { Group, JobLimit, Module, + ModuleProvider, Parameter, Parameters, Probe, diff --git a/src/lib/corelib/language/language.cpp b/src/lib/corelib/language/language.cpp index d8a5d9162..2a9eb5b0e 100644 --- a/src/lib/corelib/language/language.cpp +++ b/src/lib/corelib/language/language.cpp @@ -594,6 +594,7 @@ TopLevelProject::TopLevelProject() TopLevelProject::~TopLevelProject() { + cleanupModuleProviderOutput(); delete bgLocker; } @@ -636,6 +637,8 @@ void TopLevelProject::store(Logger logger) qCDebug(lcBuildGraph) << "build graph is unchanged in project" << id(); return; } + for (ModuleProviderInfo &m : moduleProviderInfo) + m.transientOutput = false; const QString fileName = buildGraphFilePath(); qCDebug(lcBuildGraph) << "storing:" << fileName; PersistentPool pool(logger); @@ -661,6 +664,23 @@ void TopLevelProject::store(PersistentPool &pool) serializationOp<PersistentPool::Store>(pool); } +void TopLevelProject::cleanupModuleProviderOutput() +{ + QString error; + for (const ModuleProviderInfo &m : moduleProviderInfo) { + if (m.transientOutput) { + if (!removeDirectoryWithContents(m.outputDirPath(buildDirectory), &error)) + qCWarning(lcBuildGraph) << "Error removing module provider output:" << error; + } + } + QDir moduleProviderBaseDir(buildDirectory + QLatin1Char('/') + + ModuleProviderInfo::outputBaseDirName()); + if (moduleProviderBaseDir.exists() && moduleProviderBaseDir.isEmpty() + && !removeDirectoryWithContents(moduleProviderBaseDir.path(), &error)) { + qCWarning(lcBuildGraph) << "Error removing module provider output:" << error; + } +} + /*! * \class SourceWildCards * \brief Objects of the \c SourceWildCards class result from giving wildcards in a diff --git a/src/lib/corelib/language/language.h b/src/lib/corelib/language/language.h index f9d69efff..994ad6c55 100644 --- a/src/lib/corelib/language/language.h +++ b/src/lib/corelib/language/language.h @@ -43,6 +43,7 @@ #include "filetags.h" #include "forward_decls.h" #include "jsimports.h" +#include "moduleproviderinfo.h" #include "propertydeclaration.h" #include "resolvedfilecontext.h" @@ -691,6 +692,7 @@ public: QString buildDirectory; // Not saved QProcessEnvironment environment; std::vector<ProbeConstPtr> probes; + ModuleProviderInfoList moduleProviderInfo; QHash<QString, QString> canonicalFilePathResults; // Results of calls to "File.canonicalFilePath()." QHash<QString, bool> fileExistsResults; // Results of calls to "File.exists()". @@ -722,11 +724,14 @@ private: pool.serializationOp<opType>(m_id, canonicalFilePathResults, fileExistsResults, directoryEntriesResults, fileLastModifiedResults, environment, probes, profileConfigs, overriddenValues, buildSystemFiles, - lastResolveTime, warningsEncountered, buildData); + lastResolveTime, warningsEncountered, buildData, + moduleProviderInfo); } void load(PersistentPool &pool) override; void store(PersistentPool &pool) override; + void cleanupModuleProviderOutput(); + QString m_id; QVariantMap m_buildConfiguration; }; diff --git a/src/lib/corelib/language/language.pri b/src/lib/corelib/language/language.pri index 6d68f9643..e07a671b9 100644 --- a/src/lib/corelib/language/language.pri +++ b/src/lib/corelib/language/language.pri @@ -28,6 +28,7 @@ HEADERS += \ $$PWD/loader.h \ $$PWD/moduleloader.h \ $$PWD/modulemerger.h \ + $$PWD/moduleproviderinfo.h \ $$PWD/preparescriptobserver.h \ $$PWD/projectresolver.h \ $$PWD/property.h \ diff --git a/src/lib/corelib/language/loader.cpp b/src/lib/corelib/language/loader.cpp index 98a90e221..4d2eef983 100644 --- a/src/lib/corelib/language/loader.cpp +++ b/src/lib/corelib/language/loader.cpp @@ -104,6 +104,11 @@ void Loader::setStoredProfiles(const QVariantMap &profiles) m_storedProfiles = profiles; } +void Loader::setStoredModuleProviderInfo(const ModuleProviderInfoList &providerInfo) +{ + m_storedModuleProviderInfo = providerInfo; +} + TopLevelProjectPtr Loader::loadProject(const SetupProjectParameters &_parameters) { SetupProjectParameters parameters = _parameters; @@ -160,6 +165,7 @@ TopLevelProjectPtr Loader::loadProject(const SetupProjectParameters &_parameters moduleLoader.setOldProductProbes(m_oldProductProbes); moduleLoader.setLastResolveTime(m_lastResolveTime); moduleLoader.setStoredProfiles(m_storedProfiles); + moduleLoader.setStoredModuleProviderInfo(m_storedModuleProviderInfo); const ModuleLoaderResult loadResult = moduleLoader.load(parameters); ProjectResolver resolver(&evaluator, loadResult, parameters, m_logger); resolver.setProgressObserver(m_progressObserver); diff --git a/src/lib/corelib/language/loader.h b/src/lib/corelib/language/loader.h index 9883d5b66..48a0b6065 100644 --- a/src/lib/corelib/language/loader.h +++ b/src/lib/corelib/language/loader.h @@ -40,6 +40,7 @@ #define QBS_LOADER_H #include "forward_decls.h" +#include "moduleproviderinfo.h" #include <logging/logger.h> #include <tools/filetime.h> @@ -64,6 +65,7 @@ public: void setOldProductProbes(const QHash<QString, std::vector<ProbeConstPtr>> &oldProbes); void setLastResolveTime(const FileTime &time) { m_lastResolveTime = time; } void setStoredProfiles(const QVariantMap &profiles); + void setStoredModuleProviderInfo(const ModuleProviderInfoList &providerInfo); TopLevelProjectPtr loadProject(const SetupProjectParameters ¶meters); static void setupProjectFilePath(SetupProjectParameters ¶meters); @@ -75,6 +77,7 @@ private: QStringList m_searchPaths; std::vector<ProbeConstPtr> m_oldProjectProbes; QHash<QString, std::vector<ProbeConstPtr>> m_oldProductProbes; + ModuleProviderInfoList m_storedModuleProviderInfo; QVariantMap m_storedProfiles; FileTime m_lastResolveTime; }; diff --git a/src/lib/corelib/language/moduleloader.cpp b/src/lib/corelib/language/moduleloader.cpp index b34cfd6a9..c4d77ba40 100644 --- a/src/lib/corelib/language/moduleloader.cpp +++ b/src/lib/corelib/language/moduleloader.cpp @@ -57,6 +57,7 @@ #include <logging/translator.h> #include <tools/error.h> #include <tools/fileinfo.h> +#include <tools/jsliterals.h> #include <tools/preferences.h> #include <tools/profile.h> #include <tools/profiling.h> @@ -73,6 +74,8 @@ #include <QtCore/qdiriterator.h> #include <QtCore/qjsondocument.h> #include <QtCore/qjsonobject.h> +#include <QtCore/qtemporaryfile.h> +#include <QtCore/qtextstream.h> #include <QtScript/qscriptvalueiterator.h> #include <algorithm> @@ -267,6 +270,11 @@ void ModuleLoader::setStoredProfiles(const QVariantMap &profiles) m_storedProfiles = profiles; } +void ModuleLoader::setStoredModuleProviderInfo(const ModuleProviderInfoList &moduleProviderInfo) +{ + m_moduleProviderInfo = moduleProviderInfo; +} + ModuleLoaderResult ModuleLoader::load(const SetupProjectParameters ¶meters) { TimedActivityLogger moduleLoaderTimer(m_logger, Tr::tr("ModuleLoader"), @@ -290,6 +298,7 @@ ModuleLoaderResult ModuleLoader::load(const SetupProjectParameters ¶meters) static const QStringList prefixes({ StringConstants::projectPrefix(), QLatin1String("projects"), QLatin1String("products"), QLatin1String("modules"), + StringConstants::moduleProviders(), StringConstants::qbsModule()}); bool ok = false; for (const auto &prefix : prefixes) { @@ -309,6 +318,8 @@ ModuleLoaderResult ModuleLoader::load(const SetupProjectParameters ¶meters) e.append(QLatin1Char('\t') + Tr::tr("modules.<module-name>.<property-name>:value")); e.append(QLatin1Char('\t') + Tr::tr("products.<product-name>.<module-name>." "<property-name>:value")); + e.append(QLatin1Char('\t') + Tr::tr("moduleProviders.<provider-name>." + "<property-name>:value")); handlePropertyError(e, m_parameters, m_logger); } @@ -456,6 +467,9 @@ private: m_parentItems.push_back(item); for (Item::PropertyMap::const_iterator it = item->properties().constBegin(); it != item->properties().constEnd(); ++it) { + if (item->type() == ItemType::Product && it.key() == StringConstants::moduleProviders() + && it.value()->type() == Value::ItemValueType) + continue; const PropertyDeclaration decl = item->propertyDeclaration(it.key()); if (decl.isValid()) { if (!decl.isDeprecated()) @@ -547,6 +561,9 @@ void ModuleLoader::handleTopLevelProject(ModuleLoaderResult *loadResult, Item *p throw err; handleProductError(err, &productContext); } + for (std::size_t i = 0; i < productContext.newlyAddedModuleProviderSearchPaths.size(); ++i) + m_reader->popExtraSearchPaths(); + productContext.newlyAddedModuleProviderSearchPaths.clear(); } } if (!m_productsWithDeferredDependsItems.empty() || !m_exportsWithDeferredDependsItems.empty()) { @@ -585,6 +602,7 @@ void ModuleLoader::handleTopLevelProject(ModuleLoaderResult *loadResult, Item *p } loadResult->projectProbes = tlp.probes; + loadResult->moduleProviderInfo = m_moduleProviderInfo; m_reader->clearExtraSearchPathsStack(); AccumulatingTimer timer(m_parameters.logElapsedTime() @@ -2111,6 +2129,18 @@ void ModuleLoader::setSearchPathsForProduct(ModuleLoader::ProductContext *produc if (!currentSearchPaths.contains(p) && FileInfo(p).exists()) product->searchPaths << p; } + + // Existing module provider search paths are re-used if and only if the provider configuration + // at setup time was the same as the current one for the respective module provider. + if (!m_moduleProviderInfo.empty()) { + const QVariantMap configForProduct = moduleProviderConfig(*product); + for (const ModuleProviderInfo &c : m_moduleProviderInfo) { + if (configForProduct.value(c.name.toString()) == c.config) { + product->knownModuleProviders.insert(c.name); + product->searchPaths << c.searchPaths; + } + } + } } ModuleLoader::ShadowProductInfo ModuleLoader::getShadowProductInfo( @@ -2498,6 +2528,9 @@ void ModuleLoader::resolveDependsItem(DependsContext *dependsContext, Item *pare QString msg = Tr::tr("A Depends item with more than one module cannot have an id."); throw ErrorInfo(msg, dependsItem->location()); } + const FallbackMode fallbackMode = m_parameters.fallbackProviderEnabled() + && m_evaluator->boolValue(dependsItem, StringConstants::enableFallbackProperty()) + ? FallbackMode::Enabled : FallbackMode::Disabled; QList<QualifiedId> moduleNames; const QualifiedId nameParts = QualifiedId::fromString(name); @@ -2554,8 +2587,8 @@ void ModuleLoader::resolveDependsItem(DependsContext *dependsContext, Item *pare QVariantMap defaultParameters; Item *moduleItem = loadModule(dependsContext->product, dependsContext->exportingProductItem, parentItem, dependsItem->location(), dependsItem->id(), - moduleName, multiplexConfigurationIds.first(), isRequired, - &result.isProduct, &defaultParameters); + moduleName, multiplexConfigurationIds.first(), fallbackMode, + isRequired, &result.isProduct, &defaultParameters); if (!moduleItem) { const QString productName = ResolvedProduct::fullDisplayName( dependsContext->product->name, @@ -2746,11 +2779,12 @@ Item *ModuleLoader::moduleInstanceItem(Item *containerItem, const QualifiedId &m return instance; } -ModuleLoader::ProductModuleInfo *ModuleLoader::productModule( - ProductContext *productContext, const QString &name, const QString &multiplexId) +ModuleLoader::ProductModuleInfo *ModuleLoader::productModule(ProductContext *productContext, + const QString &name, const QString &multiplexId, bool &productNameMatch) { auto &exportsData = productContext->project->topLevelProject->productModules; const auto firstIt = exportsData.find(name); + productNameMatch = firstIt != exportsData.end(); for (auto it = firstIt; it != exportsData.end() && it.key() == name; ++it) { if (it.value().multiplexId == multiplexId) return &it.value(); @@ -2861,8 +2895,9 @@ private: Item *ModuleLoader::loadModule(ProductContext *productContext, Item *exportingProductItem, Item *item, const CodeLocation &dependsItemLocation, const QString &moduleId, const QualifiedId &moduleName, - const QString &multiplexId, bool isRequired, - bool *isProductDependency, QVariantMap *defaultParameters) + const QString &multiplexId, FallbackMode fallbackMode, + bool isRequired, bool *isProductDependency, + QVariantMap *defaultParameters) { qCDebug(lcModuleLoader) << "loadModule name:" << moduleName.toString() << "id:" << moduleId; @@ -2900,17 +2935,15 @@ Item *ModuleLoader::loadModule(ProductContext *productContext, Item *exportingPr Item *modulePrototype = nullptr; ProductModuleInfo * const pmi = productModule(productContext, moduleName.toString(), - multiplexId); + multiplexId, *isProductDependency); if (pmi) { - *isProductDependency = true; m_dependsChain.back().isProduct = true; modulePrototype = pmi->exportItem; if (defaultParameters) *defaultParameters = pmi->defaultParameters; - } else { - *isProductDependency = false; + } else if (!*isProductDependency) { modulePrototype = searchAndLoadModuleFile(productContext, dependsItemLocation, - moduleName, isRequired, moduleInstance); + moduleName, fallbackMode, isRequired, moduleInstance); } delayedPropertyChanger.applyNow(); if (!modulePrototype) @@ -2968,17 +3001,19 @@ static Item *chooseModuleCandidate(const std::vector<PrioritizedItem> &candidate Item *ModuleLoader::searchAndLoadModuleFile(ProductContext *productContext, const CodeLocation &dependsItemLocation, const QualifiedId &moduleName, - bool isRequired, Item *moduleInstance) + FallbackMode fallbackMode, bool isRequired, Item *moduleInstance) { bool triedToLoadModule = false; const QString fullName = moduleName.toString(); std::vector<PrioritizedItem> candidates; const QStringList &searchPaths = m_reader->allSearchPaths(); + bool matchingDirectoryFound = false; for (int i = 0; i < searchPaths.size(); ++i) { const QString &path = searchPaths.at(i); const QString dirPath = findExistingModulePath(path, moduleName); if (dirPath.isEmpty()) continue; + matchingDirectoryFound = true; QStringList moduleFileNames = m_moduleDirListCache.value(dirPath); if (moduleFileNames.empty()) { QDirIterator dirIter(dirPath, StringConstants::qbsFileWildcards()); @@ -2999,6 +3034,36 @@ Item *ModuleLoader::searchAndLoadModuleFile(ProductContext *productContext, } if (candidates.empty()) { + if (!matchingDirectoryFound) { + bool moduleAlreadyKnown = false; + ModuleProviderResult result; + for (QualifiedId providerName = moduleName; !providerName.empty(); + providerName.pop_back()) { + if (!productContext->knownModuleProviders.insert(providerName).second) { + moduleAlreadyKnown = true; + break; + } + qCDebug(lcModuleLoader) << "Module" << moduleName.toString() + << "not found, checking for module providers"; + result = findModuleProvider(providerName, *productContext, + ModuleProviderLookup::Regular, dependsItemLocation); + if (result.providerFound) + break; + } + if (fallbackMode == FallbackMode::Enabled && !result.providerFound + && !moduleAlreadyKnown) { + qCDebug(lcModuleLoader) << "Specific module provider not found for" + << moduleName.toString() << ", setting up fallback."; + result = findModuleProvider(moduleName, *productContext, + ModuleProviderLookup::Fallback, dependsItemLocation); + } + if (result.providerAddedSearchPaths) { + qCDebug(lcModuleLoader) << "Re-checking for module" << moduleName.toString() + << "with newly added search paths from module provider"; + return searchAndLoadModuleFile(productContext, dependsItemLocation, moduleName, + fallbackMode, isRequired, moduleInstance); + } + } if (!isRequired) return createNonPresentModule(fullName, QLatin1String("not found"), nullptr); if (Q_UNLIKELY(triedToLoadModule)) @@ -3177,8 +3242,8 @@ Item::Module ModuleLoader::loadBaseModule(ProductContext *productContext, Item * Item::Module baseModuleDesc; baseModuleDesc.name = baseModuleName; baseModuleDesc.item = loadModule(productContext, nullptr, item, CodeLocation(), QString(), - baseModuleName, QString(), true, &baseModuleDesc.isProduct, - nullptr); + baseModuleName, QString(), FallbackMode::Disabled, true, + &baseModuleDesc.isProduct, nullptr); if (productContext->item) { const Item * const qbsInstanceItem = moduleInstanceItem(productContext->item, baseModuleName); @@ -3599,6 +3664,145 @@ QString ModuleLoader::findExistingModulePath(const QString &searchPath, return dirPath; } +QVariantMap ModuleLoader::moduleProviderConfig(ModuleLoader::ProductContext &product) +{ + if (product.moduleProviderConfigRetrieved) + return product.theModuleProviderConfig; + const ItemValueConstPtr configItemValue + = product.item->itemProperty(StringConstants::moduleProviders()); + if (configItemValue) { + const std::function<void(const Item *, QualifiedId)> collectMap + = [this, &product, &collectMap](const Item *item, QualifiedId name) { + const Item::PropertyMap &props = item->properties(); + for (auto it = props.begin(); it != props.end(); ++it) { + QVariant value; + switch (it.value()->type()) { + case Value::ItemValueType: + collectMap(static_cast<ItemValue *>(it.value().get())->item(), + QualifiedId(name += it.key())); + return; + case Value::JSSourceValueType: + value = m_evaluator->value(item, it.key()).toVariant(); + break; + case Value::VariantValueType: + value = static_cast<VariantValue *>(it.value().get())->value(); + break; + } + QVariantMap m = product.theModuleProviderConfig.value(name.toString()).toMap(); + m.insert(it.key(), value); + product.theModuleProviderConfig.insert(name.toString(), m); + } + }; + collectMap(configItemValue->item(), QualifiedId()); + } + for (auto it = product.moduleProperties.begin(); it != product.moduleProperties.end(); ++it) { + if (!it.key().startsWith(QStringLiteral("moduleProviders."))) + continue; + const QString provider = it.key().mid(QStringLiteral("moduleProviders.").size()); + const QVariantMap providerConfigFromBuildConfig = it.value().toMap(); + if (providerConfigFromBuildConfig.empty()) + continue; + QVariantMap currentMapForProvider = product.theModuleProviderConfig.value(provider).toMap(); + for (auto propIt = providerConfigFromBuildConfig.begin(); + propIt != providerConfigFromBuildConfig.end(); ++propIt) { + currentMapForProvider.insert(propIt.key(), propIt.value()); + } + product.theModuleProviderConfig.insert(provider, currentMapForProvider); + } + product.moduleProviderConfigRetrieved = true; + return product.theModuleProviderConfig; +} + +ModuleLoader::ModuleProviderResult ModuleLoader::findModuleProvider(const QualifiedId &name, + ModuleLoader::ProductContext &product, ModuleProviderLookup lookupType, + const CodeLocation &dependsItemLocation) +{ + for (const QString &path : m_reader->allSearchPaths()) { + QString fullPath = FileInfo::resolvePath(path, QStringLiteral("module-providers")); + switch (lookupType) { + case ModuleProviderLookup::Regular: + for (const QString &component : name) + fullPath = FileInfo::resolvePath(fullPath, component); + break; + case ModuleProviderLookup::Fallback: + fullPath = FileInfo::resolvePath(fullPath, QStringLiteral("__fallback")); + break; + } + const QString providerFile = FileInfo::resolvePath(fullPath, + QStringLiteral("provider.qbs")); + if (!FileInfo::exists(providerFile)) { + qCDebug(lcModuleLoader) << "No module provider found at" << providerFile; + continue; + } + QTemporaryFile dummyItemFile; + if (!dummyItemFile.open()) { + throw ErrorInfo(Tr::tr("Failed to create temporary file for running module provider " + "for dependency '%1': %2").arg(name.toString(), + dummyItemFile.errorString())); + } + qCDebug(lcModuleLoader) << "Instantiating module provider at" << providerFile; + const QString projectBuildDir = product.project->item->variantProperty( + StringConstants::buildDirectoryProperty())->value().toString(); + const QString searchPathBaseDir = ModuleProviderInfo::outputDirPath(projectBuildDir, name); + const QVariant moduleConfig = moduleProviderConfig(product).value(name.toString()); + QTextStream stream(&dummyItemFile); + stream.setCodec("UTF-8"); + stream << "import qbs.FileInfo" << endl; + stream << "import qbs.Utilities" << endl; + stream << "import '" << providerFile << "' as Provider" << endl; + stream << "Provider {" << endl; + stream << " name: " << toJSLiteral(name.toString()) << endl; + stream << " property var config: (" << toJSLiteral(moduleConfig) << ')' << endl; + stream << " outputBaseDir: FileInfo.joinPaths(baseDirPrefix, " + " Utilities.getHash(JSON.stringify(config)))" << endl; + stream << " property string baseDirPrefix: " << toJSLiteral(searchPathBaseDir) << endl; + stream << " property stringList searchPaths: (relativeSearchPaths || [])" + " .map(function(p) { return FileInfo.joinPaths(outputBaseDir, p); })" + << endl; + stream << "}" << endl; + stream.flush(); + Item * const providerItem = loadItemFromFile(dummyItemFile.fileName(), dependsItemLocation); + if (providerItem->type() != ItemType::ModuleProvider) { + throw ErrorInfo(Tr::tr("File '%1' declares an item of type '%2', " + "but '%3' was expected.") + .arg(providerFile, providerItem->typeName(), + BuiltinDeclarations::instance().nameForType(ItemType::ModuleProvider))); + } + const QVariantMap configMap = moduleConfig.toMap(); + for (auto it = configMap.begin(); it != configMap.end(); ++it) { + const PropertyDeclaration decl = providerItem->propertyDeclaration(it.key()); + if (!decl.isValid()) { + throw ErrorInfo(Tr::tr("No such property '%1' in module provider '%2'.") + .arg(it.key(), name.toString())); + } + providerItem->setProperty(it.key(), VariantValue::create(it.value())); + } + EvalContextSwitcher contextSwitcher(m_evaluator->engine(), EvalContext::ProbeExecution); + const QStringList searchPaths + = m_evaluator->stringListValue(providerItem, QStringLiteral("searchPaths")); + if (searchPaths.empty()) { + qCDebug(lcModuleLoader) << "Module provider did run, but did not set up " + "any modules."; + return ModuleProviderResult(true, false); + } + qCDebug(lcModuleLoader) << "Module provider added" << searchPaths.size() + << "new search path(s)"; + + // (1) is needed so the immediate new look-up works. + // (2) is needed so the next use of SearchPathManager considers the new paths. + // (3) is needed for the code that removes the product-specific search paths when + // product handling is done. + // (4) is needed for possible re-use in subsequent products and builds. + m_reader->pushExtraSearchPaths(searchPaths); // (1) + product.searchPaths << searchPaths; // (2) + product.newlyAddedModuleProviderSearchPaths.push_back(searchPaths); // (3) + m_moduleProviderInfo.emplace_back(ModuleProviderInfo(name, moduleConfig.toMap(), // (4) + searchPaths, m_parameters.dryRun())); + return ModuleProviderResult(true, true); + } + return ModuleProviderResult(); +} + void ModuleLoader::setScopeForDescendants(Item *item, Item *scope) { for (Item * const child : item->children()) { @@ -3755,8 +3959,8 @@ void ModuleLoader::addTransitiveDependencies(ProductContext *ctx) } else { Item::Module dep; dep.item = loadModule(ctx, nullptr, ctx->item, ctx->item->location(), QString(), - module.name, QString(), module.required, &dep.isProduct, - &dep.parameters); + module.name, QString(), FallbackMode::Disabled, + module.required, &dep.isProduct, &dep.parameters); if (!dep.item) { throw ErrorInfo(Tr::tr("Module '%1' not found when setting up transitive " "dependencies for product '%2'.").arg(module.name.toString(), diff --git a/src/lib/corelib/language/moduleloader.h b/src/lib/corelib/language/moduleloader.h index 7b9e0bede..9e45908c3 100644 --- a/src/lib/corelib/language/moduleloader.h +++ b/src/lib/corelib/language/moduleloader.h @@ -44,6 +44,7 @@ #include "forward_decls.h" #include "item.h" #include "itempool.h" +#include "moduleproviderinfo.h" #include <logging/logger.h> #include <tools/filetime.h> #include <tools/qttools.h> @@ -106,6 +107,7 @@ struct ModuleLoaderResult Item *root; QHash<Item *, ProductInfo> productInfos; std::vector<ProbeConstPtr> projectProbes; + ModuleProviderInfoList moduleProviderInfo; Set<QString> qbsFiles; QVariantMap profileConfigs; }; @@ -128,6 +130,7 @@ public: void setOldProductProbes(const QHash<QString, std::vector<ProbeConstPtr>> &oldProbes); void setLastResolveTime(const FileTime &time) { m_lastResolveTime = time; } void setStoredProfiles(const QVariantMap &profiles); + void setStoredModuleProviderInfo(const ModuleProviderInfoList &moduleProviderInfo); Evaluator *evaluator() const { return m_evaluator; } ModuleLoaderResult load(const SetupProjectParameters ¶meters); @@ -181,6 +184,11 @@ private: std::unordered_map<const Item *, std::vector<ErrorInfo>> unknownProfilePropertyErrors; QStringList searchPaths; + std::vector<QStringList> newlyAddedModuleProviderSearchPaths; + Set<QualifiedId> knownModuleProviders; + QVariantMap theModuleProviderConfig; + bool moduleProviderConfigRetrieved = false; + // The key corresponds to DeferredDependsContext.exportingProductItem, which is the // only value from that data structure that we still need here. std::unordered_map<Item *, std::vector<Item *>> deferredDependsItems; @@ -297,16 +305,18 @@ private: QVariantMap extractParameters(Item *dependsItem) const; Item *moduleInstanceItem(Item *containerItem, const QualifiedId &moduleName); static ProductModuleInfo *productModule(ProductContext *productContext, const QString &name, - const QString &multiplexId); + const QString &multiplexId, bool &productNameMatch); static ProductContext *product(ProjectContext *projectContext, const QString &name); static ProductContext *product(TopLevelProjectContext *tlpContext, const QString &name); + + enum class FallbackMode { Enabled, Disabled }; Item *loadModule(ProductContext *productContext, Item *exportingProductItem, Item *item, const CodeLocation &dependsItemLocation, const QString &moduleId, - const QualifiedId &moduleName, const QString &multiplexId, bool isRequired, - bool *isProductDependency, QVariantMap *defaultParameters); + const QualifiedId &moduleName, const QString &multiplexId, FallbackMode fallbackMode, + bool isRequired, bool *isProductDependency, QVariantMap *defaultParameters); Item *searchAndLoadModuleFile(ProductContext *productContext, const CodeLocation &dependsItemLocation, const QualifiedId &moduleName, - bool isRequired, Item *moduleInstance); + FallbackMode fallbackMode, bool isRequired, Item *moduleInstance); Item *loadModuleFile(ProductContext *productContext, const QString &fullModuleName, bool isBaseModule, const QString &filePath, bool *triedToLoad, Item *moduleInstance); Item *getModulePrototype(ProductContext *productContext, const QString &fullModuleName, @@ -329,6 +339,20 @@ private: Item *wrapInProjectIfNecessary(Item *item); static QString findExistingModulePath(const QString &searchPath, const QualifiedId &moduleName); + + enum class ModuleProviderLookup { Regular, Fallback }; + struct ModuleProviderResult + { + ModuleProviderResult() = default; + ModuleProviderResult(bool ran, bool added) + : providerFound(ran), providerAddedSearchPaths(added) {} + bool providerFound = false; + bool providerAddedSearchPaths = false; + }; + ModuleProviderResult findModuleProvider(const QualifiedId &name, ProductContext &product, + ModuleProviderLookup lookupType, const CodeLocation &dependsItemLocation); + QVariantMap moduleProviderConfig(ProductContext &product); + static void setScopeForDescendants(Item *item, Item *scope); void overrideItemProperties(Item *item, const QString &buildConfigKey, const QVariantMap &buildConfig); @@ -425,6 +449,8 @@ private: std::unordered_map<ProductContext *, Set<DeferredDependsContext>> m_productsWithDeferredDependsItems; Set<Item *> m_exportsWithDeferredDependsItems; + ModuleProviderInfoList m_moduleProviderInfo; + SetupProjectParameters m_parameters; std::unique_ptr<Settings> m_settings; Version m_qbsVersion; diff --git a/src/lib/corelib/language/moduleproviderinfo.h b/src/lib/corelib/language/moduleproviderinfo.h new file mode 100644 index 000000000..fef9d9765 --- /dev/null +++ b/src/lib/corelib/language/moduleproviderinfo.h @@ -0,0 +1,90 @@ +/**************************************************************************** +** +** Copyright (C) 2019 The Qt Company Ltd. +** Contact: https://www.qt.io/licensing/ +** +** This file is part of Qbs. +** +** $QT_BEGIN_LICENSE:LGPL$ +** Commercial License Usage +** Licensees holding valid commercial Qt licenses may use this file in +** accordance with the commercial license agreement provided with the +** Software or, alternatively, in accordance with the terms contained in +** a written agreement between you and The Qt Company. For licensing terms +** and conditions see https://www.qt.io/terms-conditions. For further +** information use the contact form at https://www.qt.io/contact-us. +** +** GNU Lesser General Public License Usage +** Alternatively, this file may be used under the terms of the GNU Lesser +** General Public License version 3 as published by the Free Software +** Foundation and appearing in the file LICENSE.LGPL3 included in the +** packaging of this file. Please review the following information to +** ensure the GNU Lesser General Public License version 3 requirements +** will be met: https://www.gnu.org/licenses/lgpl-3.0.html. +** +** GNU General Public License Usage +** Alternatively, this file may be used under the terms of the GNU +** General Public License version 2.0 or (at your option) the GNU General +** Public license version 3 or any later version approved by the KDE Free +** Qt Foundation. The licenses are as published by the Free Software +** Foundation and appearing in the file LICENSE.GPL2 and LICENSE.GPL3 +** included in the packaging of this file. Please review the following +** information to ensure the GNU General Public License requirements will +** be met: https://www.gnu.org/licenses/gpl-2.0.html and +** https://www.gnu.org/licenses/gpl-3.0.html. +** +** $QT_END_LICENSE$ +** +****************************************************************************/ + +#ifndef QBS_MODULEPROVIDERINFO_H +#define QBS_MODULEPROVIDERINFO_H + +#include "qualifiedid.h" +#include <tools/persistence.h> + +#include <QtCore/qstringlist.h> +#include <QtCore/qvariant.h> + +#include <vector> + +namespace qbs { +namespace Internal { + +class ModuleProviderInfo +{ +public: + ModuleProviderInfo() = default; + ModuleProviderInfo(const QualifiedId &name, const QVariantMap &config, + const QStringList &searchPaths, bool transientOutput) + : name(name), config(config), searchPaths(searchPaths), transientOutput(transientOutput) + {} + + static QString outputBaseDirName() { return QStringLiteral("genmodules"); } + static QString outputDirPath(const QString &baseDir, const QualifiedId &name) + { + return baseDir + QLatin1Char('/') + outputBaseDirName() + QLatin1Char('/') + + name.toString(); + } + QString outputDirPath(const QString &baseDir) const + { + return outputDirPath(baseDir, name); + } + + template<PersistentPool::OpType opType> void completeSerializationOp(PersistentPool &pool) + { + pool.serializationOp<opType>(reinterpret_cast<QStringList &>(name), config, searchPaths); + } + + QualifiedId name; + QVariantMap config; + QStringList searchPaths; + bool transientOutput = false; // Not to be serialized. +}; + +using ModuleProviderInfoList = std::vector<ModuleProviderInfo>; + +} // namespace Internal +} // namespace qbs + +#endif // Include guard diff --git a/src/lib/corelib/language/projectresolver.cpp b/src/lib/corelib/language/projectresolver.cpp index 7deb964b3..d0af0b7ed 100644 --- a/src/lib/corelib/language/projectresolver.cpp +++ b/src/lib/corelib/language/projectresolver.cpp @@ -240,6 +240,7 @@ TopLevelProjectPtr ProjectResolver::resolveTopLevelProject() project->buildSystemFiles = m_loadResult.qbsFiles; project->profileConfigs = m_loadResult.profileConfigs; project->probes = m_loadResult.projectProbes; + project->moduleProviderInfo = m_loadResult.moduleProviderInfo; ProjectContext projectContext; projectContext.project = project; diff --git a/src/lib/corelib/tools/persistence.cpp b/src/lib/corelib/tools/persistence.cpp index ec412cf3b..b50084702 100644 --- a/src/lib/corelib/tools/persistence.cpp +++ b/src/lib/corelib/tools/persistence.cpp @@ -48,7 +48,7 @@ namespace qbs { namespace Internal { -static const char QBS_PERSISTENCE_MAGIC[] = "QBSPERSISTENCE-124"; +static const char QBS_PERSISTENCE_MAGIC[] = "QBSPERSISTENCE-125"; NoBuildGraphError::NoBuildGraphError(const QString &filePath) : ErrorInfo(Tr::tr("Build graph not found for configuration '%1'. Expected location was '%2'.") diff --git a/src/lib/corelib/tools/setupprojectparameters.cpp b/src/lib/corelib/tools/setupprojectparameters.cpp index 39304d4c9..5600d9b0b 100644 --- a/src/lib/corelib/tools/setupprojectparameters.cpp +++ b/src/lib/corelib/tools/setupprojectparameters.cpp @@ -92,6 +92,7 @@ public: bool logElapsedTime; bool forceProbeExecution; bool waitLockBuildGraph; + bool fallbackProviderEnabled = true; SetupProjectParameters::RestoreBehavior restoreBehavior; ErrorHandlingMode propertyCheckingMode; ErrorHandlingMode productErrorMode; @@ -505,6 +506,22 @@ void SetupProjectParameters::setWaitLockBuildGraph(bool wait) } /*! + * \brief Returns true if qbs should fall back to pkg-config if a dependency is not found. + */ +bool SetupProjectParameters::fallbackProviderEnabled() const +{ + return d->fallbackProviderEnabled; +} + +/*! + * Controls whether to fall back to pkg-config if a dependency is not found. + */ +void SetupProjectParameters::setFallbackProviderEnabled(bool enable) +{ + d->fallbackProviderEnabled = enable; +} + +/*! * \brief Gets the environment used while resolving the project. */ QProcessEnvironment SetupProjectParameters::environment() const diff --git a/src/lib/corelib/tools/setupprojectparameters.h b/src/lib/corelib/tools/setupprojectparameters.h index fe7e3d487..10e4310cd 100644 --- a/src/lib/corelib/tools/setupprojectparameters.h +++ b/src/lib/corelib/tools/setupprojectparameters.h @@ -123,6 +123,9 @@ public: bool waitLockBuildGraph() const; void setWaitLockBuildGraph(bool wait); + bool fallbackProviderEnabled() const; + void setFallbackProviderEnabled(bool enable); + QProcessEnvironment environment() const; void setEnvironment(const QProcessEnvironment &env); QProcessEnvironment adjustedEnvironment() const; diff --git a/src/lib/corelib/tools/stringconstants.h b/src/lib/corelib/tools/stringconstants.h index 1a9356e49..cd41f3768 100644 --- a/src/lib/corelib/tools/stringconstants.h +++ b/src/lib/corelib/tools/stringconstants.h @@ -86,6 +86,7 @@ public: static const QString &explicitlyDependsOnFromDependenciesProperty() { return explicitlyDependsOnFromDependencies(); } + QBS_STRING_CONSTANT(enableFallbackProperty, "enableFallback") static const QString &fileNameProperty() { return fileName(); } static const QString &filePathProperty() { return filePath(); } static const QString &filePathVar() { return filePath(); } @@ -111,6 +112,7 @@ public: QBS_STRING_CONSTANT(limitToSubProjectProperty, "limitToSubProject") QBS_STRING_CONSTANT(minimumQbsVersionProperty, "minimumQbsVersion") QBS_STRING_CONSTANT(moduleNameProperty, "moduleName") + QBS_STRING_CONSTANT(moduleProviders, "moduleProviders") QBS_STRING_CONSTANT(multiplexByQbsPropertiesProperty, "multiplexByQbsProperties") QBS_STRING_CONSTANT(multiplexConfigurationIdProperty, "multiplexConfigurationId") QBS_STRING_CONSTANT(multiplexConfigurationIdsProperty, "multiplexConfigurationIds") diff --git a/static.pro b/static.pro index e172648b0..86a9db24f 100644 --- a/static.pro +++ b/static.pro @@ -1,6 +1,6 @@ TEMPLATE = aux -DATA_DIRS = share/qbs/imports share/qbs/modules +DATA_DIRS = share/qbs/imports share/qbs/modules share/qbs/module-providers PYTHON_DATA_DIRS = src/3rdparty/python/lib win32:DATA_FILES = $$PWD/bin/ibmsvc.xml $$PWD/bin/ibqbs.bat LIBEXEC_FILES = $$PWD/src/3rdparty/python/bin/dmgbuild diff --git a/tests/auto/api/testdata/soft-dependency/soft-dependency.qbs b/tests/auto/api/testdata/soft-dependency/soft-dependency.qbs index 62713a481..42d2f2de4 100644 --- a/tests/auto/api/testdata/soft-dependency/soft-dependency.qbs +++ b/tests/auto/api/testdata/soft-dependency/soft-dependency.qbs @@ -1,12 +1,10 @@ -Application { +CppApplication { Depends { name: "nosuchmodule" required: false } - Depends { - name: "cpp" + Properties { condition: nosuchmodule.present + files: "main.cpp" } - - files: "main.cpp" } diff --git a/tests/auto/blackbox/testdata/fallback-module-provider/fallback-module-provider.qbs b/tests/auto/blackbox/testdata/fallback-module-provider/fallback-module-provider.qbs new file mode 100644 index 000000000..a798e15b3 --- /dev/null +++ b/tests/auto/blackbox/testdata/fallback-module-provider/fallback-module-provider.qbs @@ -0,0 +1,8 @@ +CppApplication { + name: "p" + property bool fallbacksEnabled + Depends { name: "pkgconfig"; required: false } + Depends { name: "qbsmetatestmodule"; required: false; enableFallback: fallbacksEnabled } + property bool dummy: { console.info("pkg-config present: " + pkgconfig.present); } + files: "main.cpp" +} diff --git a/tests/auto/blackbox/testdata/fallback-module-provider/libdir/qbsmetatestmodule.pc b/tests/auto/blackbox/testdata/fallback-module-provider/libdir/qbsmetatestmodule.pc new file mode 100644 index 000000000..ae4daba89 --- /dev/null +++ b/tests/auto/blackbox/testdata/fallback-module-provider/libdir/qbsmetatestmodule.pc @@ -0,0 +1,5 @@ +Name: qbsmetatestmodule +Description: just a test +Version: 0.0.1 + +Cflags: -DTHE_MAGIC_DEFINE diff --git a/tests/auto/blackbox/testdata/fallback-module-provider/main.cpp b/tests/auto/blackbox/testdata/fallback-module-provider/main.cpp new file mode 100644 index 000000000..442b755bf --- /dev/null +++ b/tests/auto/blackbox/testdata/fallback-module-provider/main.cpp @@ -0,0 +1,5 @@ +#ifndef THE_MAGIC_DEFINE +#error "missing the magic define" +#endif + +int main() {} diff --git a/tests/auto/blackbox/testdata/module-providers/main.cpp b/tests/auto/blackbox/testdata/module-providers/main.cpp new file mode 100644 index 000000000..9cd29b1fe --- /dev/null +++ b/tests/auto/blackbox/testdata/module-providers/main.cpp @@ -0,0 +1,6 @@ +#include <iostream> + +int main() +{ + std::cout << "The letters are " << LETTER1 << " and " << LETTER2 << std::endl; +} diff --git a/tests/auto/blackbox/testdata/module-providers/module-providers.qbs b/tests/auto/blackbox/testdata/module-providers/module-providers.qbs new file mode 100644 index 000000000..d1ff79269 --- /dev/null +++ b/tests/auto/blackbox/testdata/module-providers/module-providers.qbs @@ -0,0 +1,20 @@ +Project { + CppApplication { + name: "app1" + Depends { name: "mygenerator.module1" } + Depends { name: "mygenerator.module2" } + moduleProviders.mygenerator.chooseLettersFrom: "beginning" + files: "main.cpp" + } + CppApplication { + name: "app2" + Depends { name: "mygenerator.module1" } + Depends { name: "mygenerator.module2" } + Profile { + name: "myProfile" + moduleProviders.mygenerator.chooseLettersFrom: "end" + } + qbs.profile: "myProfile" + files: "main.cpp" + } +} diff --git a/tests/auto/blackbox/testdata/module-providers/module-providers/mygenerator/provider.qbs b/tests/auto/blackbox/testdata/module-providers/module-providers/mygenerator/provider.qbs new file mode 100644 index 000000000..dae02c03a --- /dev/null +++ b/tests/auto/blackbox/testdata/module-providers/module-providers/mygenerator/provider.qbs @@ -0,0 +1,31 @@ +import qbs.File; +import qbs.FileInfo; +import qbs.TextFile; + +ModuleProvider { + property string chooseLettersFrom + relativeSearchPaths: { + console.info("Running setup script for " + name); + var startAtBeginning = chooseLettersFrom === "beginning"; + var moduleBaseDir = FileInfo.joinPaths(outputBaseDir, "modules", "mygenerator"); + var module1Dir = FileInfo.joinPaths(moduleBaseDir, "module1"); + File.makePath(module1Dir); + var module1 = new TextFile(FileInfo.joinPaths(module1Dir, "module1.qbs"), TextFile.WriteOnly); + module1.writeLine("Module {"); + module1.writeLine(" Depends { name: 'cpp' }"); + module1.writeLine(" cpp.defines: 'LETTER1=" + (startAtBeginning ? "\\\'A\\\'" : "\\\'Z\\\'") + + "'"); + module1.writeLine("}"); + module1.close(); + var module2Dir = FileInfo.joinPaths(moduleBaseDir, "module2"); + File.makePath(module2Dir); + var module2 = new TextFile(FileInfo.joinPaths(module2Dir, "module2.qbs"), TextFile.WriteOnly); + module2.writeLine("Module {"); + module2.writeLine(" Depends { name: 'cpp' }"); + module2.writeLine(" cpp.defines: 'LETTER2=" + (startAtBeginning ? "\\\'B\\\'" : "\\\'Y\\\'") + + "'"); + module2.writeLine("}"); + module2.close(); + return ""; + } +} diff --git a/tests/auto/blackbox/tst_blackbox.cpp b/tests/auto/blackbox/tst_blackbox.cpp index fbcd16eb8..82f29f320 100644 --- a/tests/auto/blackbox/tst_blackbox.cpp +++ b/tests/auto/blackbox/tst_blackbox.cpp @@ -6310,6 +6310,106 @@ void TestBlackbox::maximumCxxLanguageVersion() m_qbsStdout.constData()); } +void TestBlackbox::moduleProviders() +{ + QDir::setCurrent(testDataDir + "/module-providers"); + + // Resolving in dry-run mode must not leave any data behind. + QCOMPARE(runQbs(QbsRunParameters("resolve", QStringList("-n"))), 0); + QCOMPARE(m_qbsStdout.count("Running setup script for mygenerator"), 2); + QVERIFY(!QFile::exists(relativeBuildDir())); + + // Initial build. + QCOMPARE(runQbs(QbsRunParameters("run", QStringList{"-p", "app1"})), 0); + QVERIFY(QFile::exists(relativeBuildDir())); + QCOMPARE(m_qbsStdout.count("Running setup script for mygenerator"), 2); + QVERIFY2(m_qbsStdout.contains("The letters are A and B"), m_qbsStdout.constData()); + QCOMPARE(runQbs(QbsRunParameters("run", QStringList{"-p", "app2"})), 0); + QVERIFY2(m_qbsStdout.contains("The letters are Z and Y"), m_qbsStdout.constData()); + + // Rebuild with overridden module provider config. The output for product 2 must change, + // but no setup script must be re-run, because both config values have already been + // handled in the first run. + const QStringList resolveArgs("moduleProviders.mygenerator.chooseLettersFrom:beginning"); + QCOMPARE(runQbs(QbsRunParameters("resolve", resolveArgs)), 0); + QVERIFY2(!m_qbsStdout.contains("Running setup script"), m_qbsStdout.constData()); + QCOMPARE(runQbs(QbsRunParameters("run", QStringList{"-p", "app1"})), 0); + QVERIFY2(m_qbsStdout.contains("The letters are A and B"), m_qbsStdout.constData()); + QCOMPARE(runQbs(QbsRunParameters("run", QStringList{"-p", "app2"})), 0); + QVERIFY2(m_qbsStdout.contains("The letters are A and B"), m_qbsStdout.constData()); + + // Forcing Probe execution triggers a re-run of the setup script. But only once, + // because the module provider config is the same now. + QCOMPARE(runQbs(QbsRunParameters("resolve", QStringList(resolveArgs) + << "--force-probe-execution")), 0); + QCOMPARE(m_qbsStdout.count("Running setup script for mygenerator"), 1); + QCOMPARE(runQbs(QbsRunParameters("run", QStringList{"-p", "app1"})), 0); + QVERIFY2(m_qbsStdout.contains("The letters are A and B"), m_qbsStdout.constData()); + QCOMPARE(runQbs(QbsRunParameters("run", QStringList{"-p", "app2"})), 0); + QVERIFY2(m_qbsStdout.contains("The letters are A and B"), m_qbsStdout.constData()); + + // Now re-run without the module provider config override. Again, the setup script must + // run once, for the config value that was not present in the last run. + QCOMPARE(runQbs(QbsRunParameters("resolve")), 0); + QCOMPARE(m_qbsStdout.count("Running setup script for mygenerator"), 1); + QCOMPARE(runQbs(QbsRunParameters("run", QStringList{"-p", "app1"})), 0); + QVERIFY2(m_qbsStdout.contains("The letters are A and B"), m_qbsStdout.constData()); + QCOMPARE(runQbs(QbsRunParameters("run", QStringList{"-p", "app2"})), 0); + QVERIFY2(m_qbsStdout.contains("The letters are Z and Y"), m_qbsStdout.constData()); +} + +void TestBlackbox::fallbackModuleProvider_data() +{ + QTest::addColumn<bool>("fallbacksEnabledGlobally"); + QTest::addColumn<bool>("fallbacksEnabledInProduct"); + QTest::addColumn<QStringList>("pkgConfigLibDirs"); + QTest::addColumn<bool>("successExpected"); + QTest::newRow("without custom lib dir, fallbacks disabled globally and in product") + << false << false << QStringList() << false; + QTest::newRow("without custom lib dir, fallbacks disabled globally, enabled in product") + << false << true << QStringList() << false; + QTest::newRow("without custom lib dir, fallbacks enabled globally, disabled in product") + << true << false << QStringList() << false; + QTest::newRow("without custom lib dir, fallbacks enabled globally and in product") + << true << true << QStringList() << false; + QTest::newRow("with custom lib dir, fallbacks disabled globally and in product") + << false << false << QStringList(testDataDir + "/fallback-module-provider/libdir") + << false; + QTest::newRow("with custom lib dir, fallbacks disabled globally, enabled in product") + << false << true << QStringList(testDataDir + "/fallback-module-provider/libdir") + << false; + QTest::newRow("with custom lib dir, fallbacks enabled globally, disabled in product") + << true << false << QStringList(testDataDir + "/fallback-module-provider/libdir") + << false; + QTest::newRow("with custom lib dir, fallbacks enabled globally and in product") + << true << true << QStringList(testDataDir + "/fallback-module-provider/libdir") + << true; +} + +void TestBlackbox::fallbackModuleProvider() +{ + QFETCH(bool, fallbacksEnabledInProduct); + QFETCH(bool, fallbacksEnabledGlobally); + QFETCH(QStringList, pkgConfigLibDirs); + QFETCH(bool, successExpected); + QDir::setCurrent(testDataDir + "/fallback-module-provider"); + static const auto b2s = [](bool b) { return QString(b ? "true" : "false"); }; + QbsRunParameters resolveParams("resolve", + QStringList{"modules.pkgconfig.libDirs:" + pkgConfigLibDirs.join(','), + "products.p.fallbacksEnabled:" + b2s(fallbacksEnabledInProduct)}); + if (!fallbacksEnabledGlobally) + resolveParams.arguments << "--no-fallback-module-provider"; + QCOMPARE(runQbs(resolveParams), 0); + const bool pkgConfigPresent = m_qbsStdout.contains("pkg-config present: true"); + const bool pkgConfigNotPresent = m_qbsStdout.contains("pkg-config present: false"); + QVERIFY(pkgConfigPresent != pkgConfigNotPresent); + if (pkgConfigNotPresent) + successExpected = false; + QbsRunParameters buildParams; + buildParams.expectFailure = !successExpected; + QCOMPARE(runQbs(buildParams) == 0, successExpected); +} + void TestBlackbox::minimumSystemVersion() { rmDirR(relativeBuildDir()); diff --git a/tests/auto/blackbox/tst_blackbox.h b/tests/auto/blackbox/tst_blackbox.h index 313ad5d40..624cd5fbb 100644 --- a/tests/auto/blackbox/tst_blackbox.h +++ b/tests/auto/blackbox/tst_blackbox.h @@ -174,6 +174,9 @@ private slots: void makefileGenerator(); void maximumCLanguageVersion(); void maximumCxxLanguageVersion(); + void moduleProviders(); + void fallbackModuleProvider_data(); + void fallbackModuleProvider(); void minimumSystemVersion(); void minimumSystemVersion_data(); void missingBuildGraph(); diff --git a/tests/auto/language/tst_language.cpp b/tests/auto/language/tst_language.cpp index be9dfb932..6b5cc2447 100644 --- a/tests/auto/language/tst_language.cpp +++ b/tests/auto/language/tst_language.cpp @@ -93,7 +93,7 @@ TestLanguage::TestLanguage(ILogSink *logSink, Settings *settings) { qsrand(QTime::currentTime().msec()); qRegisterMetaType<QList<bool> >("QList<bool>"); - defaultParameters.setBuildRoot("/some/build/directory"); + defaultParameters.setBuildRoot(m_tempDir.path() + "/buildroot"); defaultParameters.setPropertyCheckingMode(ErrorHandlingMode::Strict); defaultParameters.setSettingsDirectory(m_settings->baseDirectory()); } @@ -163,6 +163,7 @@ void TestLanguage::handleInitCleanupDataTags(const char *projectFileName, bool * void TestLanguage::init() { m_logSink->setLogLevel(LoggerInfo); + QVERIFY(m_tempDir.isValid()); } #define HANDLE_INIT_CLEANUP_DATATAGS(fn) {\ diff --git a/tests/auto/language/tst_language.h b/tests/auto/language/tst_language.h index 0fa44afe4..3fe6d8f2a 100644 --- a/tests/auto/language/tst_language.h +++ b/tests/auto/language/tst_language.h @@ -45,6 +45,7 @@ #include <logging/ilogsink.h> #include <tools/setupprojectparameters.h> +#include <QtCore/qtemporarydir.h> #include <QtTest/qtest.h> class TestLanguage : public QObject @@ -178,6 +179,9 @@ private slots: void versionCompare(); void wildcards_data(); void wildcards(); + +private: + QTemporaryDir m_tempDir; }; #endif // TST_LANGUAGE_H |