aboutsummaryrefslogtreecommitdiffstats
path: root/src/lib/corelib/buildgraph/processcommandexecutor.cpp
Commit message (Collapse)AuthorAgeFilesLines
* Merge 1.14 into masterChristian Kandeler2019-08-261-1/+3
|\ | | | | | | Change-Id: Ic9ee11d628ba278271517aae0af020aec28ecbf7
| * Process Command: Do not needlessly interpret output as textChristian Kandeler2019-08-261-1/+3
| | | | | | | | | | | | | | | | | | ... when redirecting process output. Only do that if there is an output filter function. Otherwise, we break redirection of binary data. Fixes: QBS-1488 Change-Id: Ic5a1d2653fa8fc9c4f0eb5d31b7315d7aba3e030 Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
* | Add command and AutotestRunner timeoutJochen Ulrich2019-08-011-9/+17
|/ | | | | | | Task-number: QBS-1454 Change-Id: I6e2514d10cca0cba0a14456ecd2abfb495539ee4 Reviewed-by: Christian Kandeler <christian.kandeler@qt.io> Reviewed-by: Denis Shienkov <denis.shienkov@gmail.com>
* Fix calls to non-const begin()/end() in range-for loopsIvan Komissarov2019-03-061-2/+4
| | | | | | | | This fixes -Wclazy-range-loop Change-Id: I08a435b700017cdbe27fbfdbda6e26840d9a3568 Reviewed-by: Christian Kandeler <christian.kandeler@qt.io> Reviewed-by: Denis Shienkov <denis.shienkov@gmail.com>
* corelib: Apply modernize-use-nullptrAlessandro Portale2018-08-201-1/+1
| | | | | | | Use nullptr instead of 0. Change-Id: Id5b7933844730e244a2dcf3fcebf24f9f048047f Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
* Add product name to command descriptionsChristian Kandeler2018-03-121-2/+2
| | | | | | | | | | | | | | | The pure file name that the description usually contains does not always provide enough context: It's not necessarily clear from a file name which product it belongs to, and some files can be built as part of more than one product. Because we print the full product name, including the multiplexing information if necessary, we also remove the JS code that provided this kind of disambiguation for GCC compiler and linker rules only. [ChangeLog] Command descriptions now also contain the product name. Change-Id: I9702a6cff3faffb086e5c93e02fffa6e65fb1043 Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
* Move transformer change tracking out of the build graph loaderChristian Kandeler2018-02-161-3/+6
| | | | | | | | | | | | | | | | | | | | The checks that determine whether to re-run prepare scripts and/or commands due to property or environment changes do not belong into the build graph loader. Instead, we now do them on demand during the build process. Advantages: - The code is at its "natural" place, making it easier to understand and less fragile. - There are a lot fewer unnecessary build data invalidations, speeding up both re-resolving and rebuilding in case only some rules/commands have changed. - Re-running commands due to property or environment changes no longer implies re-running prepare scripts. - We now catch property changes on generated artifacts, which was not possible before, because the build graph loader only had access to the source artifacts of the re-resolved project. Change-Id: I36b022cf631fa9e8293feec4d6f416c2686539c1 Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
* Process commands: Fix environment merging for some path list variablesChristian Kandeler2017-12-221-6/+25
| | | | | | | | | | | | | The value of Command.environment is supposed to be added to the general build environment. Variables containing lists should be merged rather than overwritten, as the code has done so far. This is difficult to get right in the general case, as we'd need a way to find out which variables contain lists. For now, use a fixed set of well-known and important path list variables (PATH is particularly relevant). Change-Id: I860302550b8005cbe10ee7d06dcca2b5edfaf475 Reviewed-by: Jake Petroules <jake.petroules@qt.io>
* STL compatibility: use size() instead of count()Jake Petroules2017-11-151-2/+2
| | | | | | | This is a simple find and replace with manual sanity check. Change-Id: Ia733befe7885dc3c643d5c84e151312bfd86a3c6 Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
* Fix race condition in rule execution / TextFile auto-closingJoerg Bornemann2017-09-011-0/+12
| | | | | | | | | | | | | | | Suppose there's a rule script that opens a TextFile and does a lengthy operation. If a ProcessCommand finishes that has a filterOutput script set, then it might happen that we call QScriptEngine::execute from a QScriptEngine::execute callstack. After the inner execute finishes we would auto-close the TextFile, unexpectedly for the code of the outer execute call. Delay ProcessCommand::onProcess{Finished|Error} if the ScriptEngine is already active to avoid this situation. Change-Id: Ibad32732f7efe9d5dbfbe1fdc143c0ea109dbb34 Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
* Remove superfluous includeJoerg Bornemann2017-08-311-1/+0
| | | | | Change-Id: I6bcd8a9da77b68800bbced425ebf99ea6f83c413 Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
* Use categorized logging in ExecutorJoerg Bornemann2017-08-021-12/+5
| | | | | Change-Id: Id06bc8b14aa31cb0dafa621477ca072952532099 Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
* Make command-line-with-environment mode print in an executable formJake Petroules2017-06-291-3/+6
| | | | | | | | | | command-line mode prints in a form that is directly copy-paste'able into a shell for re-execution, so command-line-with-environment should as well. On Windows the format remains unchanged, because it's not possible to set local environment variables for a command. Change-Id: I5e7a9db297fcbeda821f7b6b5e176d340b2c43cf Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
* Try harder to find out the correct location for evaluation errorsChristian Kandeler2017-05-041-1/+2
| | | | | | Task-number: QBS-946 Change-Id: I2fb5d573f7caf44f96fd4a96fd5069a44cdfb16a Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
* Support command files in cpp.{static,dynamic}LibrariesChristian Kandeler2017-04-281-1/+12
| | | | | | | We will make use of that in a follow-up patch. Change-Id: I1fb6ca0938bbcc6f53bd0046761023a1b8563949 Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
* Remove all usages of Q_FOREACH from the libraryChristian Kandeler2017-02-211-1/+1
| | | | | | | | This will make it easier to experiment with exchanging Qt containers for STL ones, for instance. Change-Id: Ie591fa54b5241ad4841b1ebcfb78b72932cd2b38 Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
* ProcessCommandExecutor: Make two debug statements conditionalChristian Kandeler2017-01-261-3/+8
| | | | | Change-Id: If7e3a60f1b927e89f4a515d2f95beda7c3d1c665 Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
* Use a dedicated launcher process to run all ProcessCommandsChristian Kandeler2017-01-251-5/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | On Linux (and very likely other Unices as well), QProcess::start() incurs a certain overhead, because forking off the child process duplicates some of the parent process' resources. This overhead appears to be proportional to the amount of memory allocated by the parent process, presumably due to page table entries getting copied. This has consequences for qbs, particularly when being used from an IDE such as Qt Creator, which has a higher memory footprint than the command line tool. When using a high job count, as is typical on machines with lots of CPU cores or in a distributed compilation environment, the following problems were observed: - High CPU load in the starting process (Qt Creator). Profiling showed that most of the time was spent in fork() and related functions. - As a result, the number of parallel jobs stalled at a value well below the requested one, slowing down the build. - In some cases, QProcess::start() failed altogether, emitting a message such as "fork() failed: Could not allocate memory". We solve these issues by outsourcing the starting of ProcessCommands to a dedicated launcher tool with modest memory requirements. For each qbs process, we have one instance of this tool running while a build job is going on. Communication with qbs happens via QLocalSocket. The protocol is encapsulated in a QProcess replacement, so almost no changes to existing code were necessary. No performance regressions were observed when using lower job counts. This patch will also enable us to properly support the incredibuild tool in IDEs such as Qt Creator, which you do not want to start via ibconsole. [ChangeLog] Improved scalability of parallel builds on Linux by starting Process commands via a dedicated launcher process. Change-Id: I8966c00a2d67a94c3a380f0e089d61eda122209e Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@qt.io> Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
* Give the two copies command.{cpp,h} non-ambiguous namesJake Petroules2017-01-051-1/+1
| | | | | | | | | | | The pair in tools/buildgraph becomes rulecommands.{cpp,h} The pair in parser becomes parsercommand.{cpp,h} This resolves a conflict which prevented compilation of the entire set of qbs sources with MSVC and the /Fo option. Change-Id: Ic909d02a230d8e59f3bf4d00fc895bf54739961a Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
* Change style of #includes of Qt headersJake Petroules2017-01-041-5/+6
| | | | | | | | | | | | | | | | By using q<lowercase>.h headers, we become more bootstrap-friendly, as no full syncqt is required any more. In the same go, prefix all includes with the module name. This helps grep-based analysis of which parts of Qt are used and where. It's also consistent with Qt's public headers (where syncqt enforces the style). Testdata and examples are excluded from the change, as they are not relevant for bootstrapping. Change-Id: I9539b1d795e87fca6e5fc6c91acbb775b79208d9 Reviewed-by: Christian Kandeler <christian.kandeler@qt.io> Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
* Do not use fromLocal8Bit() for string literalsKai Koehne2016-11-081-1/+1
| | | | | | | | | | The current locale encoding does have nothing to do with string literals. Use fromLatin1(), which tends to be the fastest for ASCII. Change-Id: I9f1ff603f26812a63e5b23a413ba240c47d5ac23 Reviewed-by: Orgad Shaneh <orgads@gmail.com> Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
* Update license headersIikka Eklund2016-08-291-14/+23
| | | | | | | | | | Patch-set 2 includes *.cpp *.c Every source file needs to have up-to-date license headers in order to pass ci. Change-Id: Ie6e493097af6f7dd6a8adff170eb856f496e689e Reviewed-by: Jake Petroules <jake.petroules@qt.io>
* Merge remote-tracking branch 'origin/1.5'Joerg Bornemann2016-06-081-1/+1
|\ | | | | | | | | | | | | | | Conflicts: doc/reference/items/transformer.qdoc share/qbs/modules/cpp/GenericGCC.qbs Change-Id: I45560834560019b18274c373c45651eb8aadd206
| * Replace 'the Qt build suite' with 'Qbs' in copyright notices.Jake Petroules2016-06-061-1/+1
| | | | | | | | | | | | Change-Id: I885d94bb14b325dc36767a840ebdb0be1fb59dd2 Reviewed-by: Leena Miettinen <riitta-leena.miettinen@qt.io> Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
* | Use Qt5-style connectsOrgad Shaneh2016-06-071-4/+6
| | | | | | | | | | | | | | The heavy lifting was done by clazy. Change-Id: Ibb13c517567b1b32bbda6d26225454d1b003934d Reviewed-by: Christian Kandeler <christian.kandeler@theqtcompany.com>
* | Provide the ability to let commands bypass dry-run mode.Jake Petroules2016-06-011-1/+1
| | | | | | | | | | | | | | | | | | This allows these commands to run even in dry-run mode. This is for internal use ONLY in building the helper tools for Java, TypeScript, etc., dependency tracking and is not intended to be exposed to users. Change-Id: I36130131363aa6247f10b9824726f73df9d7031f Reviewed-by: Christian Kandeler <christian.kandeler@theqtcompany.com>
* | Merge 1.5 into master.Christian Kandeler2016-05-271-1/+1
|\| | | | | | | Change-Id: I85f81532d88798d71cb88552dd150189d72829bd
| * Quote arguments in response filesPhilip Schuchardt2016-05-201-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | This bug fix adds quotes around arguments with spaces or tabs in the response files. It also escapes quotes if they're found in the argument list. Task-number: QBS-812 Autotest-by: Joerg Bornemann <joerg.bornemann@qt.io> Change-Id: I4f4479e413b467e64f55c3068475d0dbb1990798 Reviewed-by: Jake Petroules <jake.petroules@qt.io> Reviewed-by: Philip Schuchardt <vpicaver@gmail.com>
* | Add a new command line echo mode that also prints environment variables.Jake Petroules2016-05-231-7/+29
| | | | | | | | | | Change-Id: I88213103519ae4f69fedb4fe535c9c44e3b61d03 Reviewed-by: Christian Kandeler <christian.kandeler@theqtcompany.com>
* | Fix filterFunction evaluationJoerg Bornemann2016-05-101-2/+1
| | | | | | | | | | | | | | | | The global object was reset too early. Introduce RAII class for setting/resetting the global object safely. Change-Id: Ib04d5021a8dcb64974c54cf44721ef642d227d21 Reviewed-by: Christian Kandeler <christian.kandeler@theqtcompany.com>
* | Do not use QScriptEngine::push/popScope in rule evaluationJoerg Bornemann2016-05-101-3/+3
|/ | | | | | | | | | | | | | | This functionality is undocumented, has its flaws and is unavailable in other JS engines. Use the following pattern instead to inject global variables without polluting the global namespace: QScriptValue g = engine.newObject(); g.setPrototype(engine.globalObject()); // set properties on g engine.setGlobalObject(g); engine.evaluate(...); engine.setGlobalObject(g.prototype()); Change-Id: I5f92da8cccfa23c3722740c1852a71fd50e8eb8a Reviewed-by: Christian Kandeler <christian.kandeler@theqtcompany.com>
* Fix command lines being printed regardless of echo mode.Christian Kandeler2015-12-141-1/+1
| | | | | | | Bug was introduced by a3b5c6e944. Change-Id: Ic7a0a9dfa7f009905880dd6613e37bae4b74a3d2 Reviewed-by: Joerg Bornemann <joerg.bornemann@theqtcompany.com>
* ProcessCommandExecutor: Clean up process output handling.Christian Kandeler2015-11-231-32/+45
| | | | | | | | | - Get rid of code duplication. - Simplify code flow. - Fix potentially erroneous overwriting of error status. Change-Id: Ic7e3babe707e6d9e5c8c478de0891a5adecc0875 Reviewed-by: Joerg Bornemann <joerg.bornemann@theqtcompany.com>
* Make it easy to redirect the commands outputs to files.BogDan Vatra2015-11-201-12/+31
| | | | | | Change-Id: I0516527408d44b8f78d8da1fc65cdf609babfbc6 Reviewed-by: Christian Kandeler <christian.kandeler@theqtcompany.com> Reviewed-by: BogDan Vatra <bogdan@kdab.com>
* Allow more fine-grained error reporting in ProcessResult.Christian Kandeler2015-11-171-17/+19
| | | | | | | | We achieve this by forwarding QProcess::ProcessError instead of QProcess::ExitStatus. Change-Id: Id4b9ac7facbb92281e0a11da635131145114be9d Reviewed-by: Joerg Bornemann <joerg.bornemann@theqtcompany.com>
* Introduce Command.responseFileArgumentIndex.Jake Petroules2015-09-141-2/+3
| | | | | | | | This fixes an issue causing extremely long command lines to fail on Windows when used in conjunction with cpp.compilerWrapper. Change-Id: I7c844eeb00f9068d84e8c4fdcdf20db658438bbb Reviewed-by: Joerg Bornemann <joerg.bornemann@theqtcompany.com>
* Add an extendedDescription property for commands.Jake Petroules2015-08-131-1/+4
| | | | | | | | Defaults to the command line invocation for Commands, and defaults to the description for JavaScriptCommands. Change-Id: I7bddb6591b7fdbffc2c08134f35471c0d2725cf9 Reviewed-by: Christian Kandeler <christian.kandeler@theqtcompany.com>
* Always show the correct error string from the script engine.Christian Kandeler2015-07-201-1/+1
| | | | | | | | Meaning that we check whether the value is an error or the engine has an uncaught exception, and print the respective string. Change-Id: Ie2912e28a82906c7ee9323e2f0e55c6b3de96872 Reviewed-by: Joerg Bornemann <joerg.bornemann@theqtcompany.com>
* Show the full command line invocation when a command fails.Jake Petroules2015-07-161-2/+4
| | | | | Change-Id: I8f9daa08dc0822b74b9cc4d255bce2b170b55b59 Reviewed-by: Christian Kandeler <christian.kandeler@theqtcompany.com>
* Print the command that was actually executed in command echo mode.Jake Petroules2015-07-131-16/+19
| | | | | | | | Previously, it would display the command that was entered ("cmd") vs the command that was actually used ("C:\Windows\System32\cmd.exe"). Change-Id: Ia3487f2a0fee4c439d82598f3e2e81650efafa80 Reviewed-by: Christian Kandeler <christian.kandeler@theqtcompany.com>
* DRY up shell quoting code, and do it properly in all places.Jake Petroules2015-07-131-23/+4
| | | | | Change-Id: I1c12ce97d0d932a35dae5c1a99b631bfd03eef2e Reviewed-by: Christian Kandeler <christian.kandeler@theqtcompany.com>
* Try to forward "bad interpreter" errors to the user.Jake Petroules2015-03-271-1/+11
| | | | | Change-Id: If287b7480be854f4bae055ed9d575f16fd6f54df Reviewed-by: Joerg Bornemann <joerg.bornemann@theqtcompany.com>
* update license headersJoerg Bornemann2015-03-021-2/+2
| | | | | Change-Id: I36d825db3a5ef32c162c3063aab1e8a1f2696f4f Reviewed-by: Christian Kandeler <christian.kandeler@theqtcompany.com>
* Replace --show-command-lines with --command-echo-mode.Jake Petroules2015-02-251-2/+2
| | | | | | | | | | This allows users to hide command output entirely without changing the logging level. It is also used by the generator command to hide the dry run command "executions" it performs when creating the build graph for project generation. Change-Id: I27a64c8138521001f5b62473b4a3b4ff46d8ba25 Reviewed-by: Christian Kandeler <christian.kandeler@theqtcompany.com>
* Merge branch '1.3' into 'master'Christian Kandeler2015-01-281-6/+6
|\ | | | | | | | | | | | | | | | | Conflicts: doc/reference/jsextensions/jsextensions-general.qdoc src/lib/corelib/language/evaluatorscriptclass.cpp src/lib/corelib/language/evaluatorscriptclass.h Change-Id: Ic9b77b56e8fc8bf93e0a553930757de8c93fed22
| * Update LicenseEike Ziller2015-01-261-6/+6
| | | | | | | | | | Change-Id: I092fca9f950de1ba38826c33b155f86feae2770b Reviewed-by: Christian Kandeler <christian.kandeler@theqtcompany.com>
* | add option to show command lines in build outputJoerg Bornemann2014-11-071-0/+14
|/ | | | | | | | | | With "qbs build --show-command-lines" one can suppress command descriptions and show actual command lines instead. Change-Id: I98328254bad7fb85036abb0525c5ad7e734110c7 Task-number: QBS-615 Reviewed-by: Christian Kandeler <christian.kandeler@digia.com> Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@theqtcompany.com>
* Put Command properties in the filter functions' scopes.Jake Petroules2014-10-171-0/+10
| | | | | | | | This allows real functions to be used instead of unsafe construction of function objects from strings. Change-Id: Ic1b31d0f7984caaac0cf9a5b5a21d085736015b3 Reviewed-by: Joerg Bornemann <joerg.bornemann@digia.com>
* License update.Eike Ziller2014-10-161-7/+8
| | | | | | | Add LGPLv3 option. Change-Id: I8a63ad5e46a2701032b2103f791df4dec5b707e8 Reviewed-by: Joerg Bornemann <joerg.bornemann@digia.com>
* let JS extension Process find executables in PATHJoerg Bornemann2014-10-101-1/+2
| | | | | Change-Id: I9052b8e895cb638bc4b8a185959c72e6819fb628 Reviewed-by: Christian Kandeler <christian.kandeler@digia.com>
generated by cgit v1.2.3 (git 2.25.1) at 2024-06-13 18:58:09 +0000