Switch JavaScript back-end

Newer clang versions seem to expose serious bugs in QtScript, whose
complexity makes it difficult to track them down.
We therefore switch to the more light-weight QuickJS, which offers all
the features we need (most notably property access interception), as
well as good performance.

To save some porting effort, we removed the long-deprecated loadFile()
and loadExtension() functions.
During the porting procedure, we noticed and fixed thread safety issues
in artifact access from JS commands.
We consider this change important enough to bump the major version, so
the next release will be 2.0.

Detailed benchmarking data is below. In summary, we see a modest speed-
up at the cost of a similarly modest increase in memory consumption
(with the exception of project resolving on macOS, which has become a
bit slower). Importantly, the increase does not rise with project size,
as the comparison of qbs vs Qt Creator shows.

Output of qbs_benchmarker on Linux with qbs as test project:
========== Performance data for Resolving ==========
    Old instruction count: 12870602895
    New instruction count: 11923459780
    Relative change: -8 %
    Old peak memory usage: 61775848 Bytes
    New peak memory usage: 67583424 Bytes
    Relative change: +9 %
========== Performance data for Rule Execution ==========
    Old instruction count: 4074062223
    New instruction count: 3887473574
    Relative change: -5 %
    Old peak memory usage: 35123704 Bytes
    New peak memory usage: 38398392 Bytes
    Relative change: +9 %
========== Performance data for Null Build ==========
    Old instruction count: 1104417596
    New instruction count: 1011033948
    Relative change: -9 %
    Old peak memory usage: 24461824 Bytes
    New peak memory usage: 25325920 Bytes
    Relative change: +3 %

Output of qbs_benchmarker on Linux with Qt Creator as test project:
========== Performance data for Resolving ==========
    Old instruction count: 67166450352
    New instruction count: 60772791018
    Relative change: -10 %
    Old peak memory usage: 327011616 Bytes
    New peak memory usage: 343724176 Bytes
    Relative change: +5 %
========== Performance data for Rule Execution ==========
    Old instruction count: 71684351183
    New instruction count: 67051936965
    Relative change: -7 %
    Old peak memory usage: 374913688 Bytes
    New peak memory usage: 387790992 Bytes
    Relative change: +3 %
========== Performance data for Null Build ==========
    Old instruction count: 8383156078
    New instruction count: 7930705668
    Relative change: -6 %
    Old peak memory usage: 180468360 Bytes
    New peak memory usage: 182490384 Bytes
    Relative change: +1 %

Real-world data building Qt Creator (using qbs --log-time, several runs,
removing outliers):
  macOS:
    Resolving: 43s -> 47s
    Rule execution: 17s -> 14s
  Windows:
    Resolving: 18s -> 16s
    Rule execution: 22s -> 17s

Fixes: QBS-913
Fixes: QBS-1103
Fixes: QBS-1126
Fixes: QBS-1227
Fixes: QBS-1684

Change-Id: Ie5088155026e85bbd1e303f1c67addb15810a3cb
Reviewed-by: Ivan Komissarov <ABBAPOH@gmail.com>
Reviewed-by: Orgad Shaneh <orgads@gmail.com>

1 files changed, 24 insertions, 17 deletions

diff --git a/src/lib/corelib/language/projectresolver.cpp b/src/lib/corelib/language/projectresolver.cpp
index 208f2c5a6..c2c8ba134 100644
--- a/src/lib/corelib/language/projectresolver.cpp
+++ b/src/lib/corelib/language/projectresolver.cpp
@@ -67,6 +67,8 @@
 #include <tools/stlutils.h>
 #include <tools/stringconstants.h>
 
+#include <quickjs.h>
+
 #include <QtCore/qdir.h>
 #include <QtCore/qregularexpression.h>
 
@@ -326,7 +328,8 @@ void ProjectResolver::resolveProjectFully(Item *item, ProjectResolver::ProjectCo
             continue;
         const ValueConstPtr v = item->property(it.key());
         QBS_ASSERT(v && v->type() != Value::ItemValueType, continue);
-        projectProperties.insert(it.key(), m_evaluator->value(item, it.key()).toVariant());
+        const ScopedJsValue sv(m_engine->context(), m_evaluator->value(item, it.key()));
+        projectProperties.insert(it.key(), getJsVariant(m_engine->context(), sv));
     }
     projectContext->project->setProjectProperties(projectProperties);
 
@@ -475,7 +478,7 @@ void ProjectResolver::resolveProductFully(Item *item, ProjectContext *projectCon
     createProductConfig(product.get());
     product->productProperties.insert(StringConstants::destinationDirProperty(),
                                       product->destinationDirectory);
-    ModuleProperties::init(m_evaluator->scriptValue(item), product.get());
+    ModuleProperties::init(m_evaluator->engine(), m_evaluator->scriptValue(item), product.get());
 
     QList<Item *> subItems = item->children();
     const ValuePtr filesProperty = item->property(StringConstants::filesProperty());
@@ -1213,9 +1216,10 @@ void ProjectResolver::resolveRule(Item *item, ProjectContext *projectContext)
     }
 
     rule->name = m_evaluator->stringValue(item, StringConstants::nameProperty());
-    rule->prepareScript.initialize(scriptFunctionValue(item, StringConstants::prepareProperty()));
-    rule->outputArtifactsScript.initialize(scriptFunctionValue(
-                                               item, StringConstants::outputArtifactsProperty()));
+    rule->prepareScript.initialize(
+                scriptFunctionValue(item, StringConstants::prepareProperty()));
+    rule->outputArtifactsScript.initialize(
+                scriptFunctionValue(item, StringConstants::outputArtifactsProperty()));
     rule->outputFileTags = m_evaluator->fileTagsValue(
                 item, StringConstants::outputFileTagsProperty());
     if (rule->outputArtifactsScript.isValid()) {
@@ -1395,9 +1399,10 @@ void ProjectResolver::resolveScanner(Item *item, ProjectResolver::ProjectContext
     scanner->module = m_moduleContext ? m_moduleContext->module : projectContext->dummyModule;
     scanner->inputs = m_evaluator->fileTagsValue(item, StringConstants::inputsProperty());
     scanner->recursive = m_evaluator->boolValue(item, StringConstants::recursiveProperty());
-    scanner->searchPathsScript.initialize(scriptFunctionValue(
-                                              item, StringConstants::searchPathsProperty()));
-    scanner->scanScript.initialize(scriptFunctionValue(item, StringConstants::scanProperty()));
+    scanner->searchPathsScript.initialize(
+                scriptFunctionValue(item, StringConstants::searchPathsProperty()));
+    scanner->scanScript.initialize(
+                scriptFunctionValue(item, StringConstants::scanProperty()));
     m_productContext->product->scanners.push_back(scanner);
 }
 
@@ -1739,6 +1744,7 @@ QVariantMap ProjectResolver::evaluateProperties(const Item *item, const Item *pr
 void ProjectResolver::evaluateProperty(const Item *item, const QString &propName,
         const ValuePtr &propValue, QVariantMap &result, bool checkErrors)
 {
+    JSContext * const ctx = m_engine->context();
     switch (propValue->type()) {
     case Value::ItemValueType:
     {
@@ -1754,23 +1760,24 @@ void ProjectResolver::evaluateProperty(const Item *item, const QString &propName
         if (pd.flags().testFlag(PropertyDeclaration::PropertyNotAvailableInConfig)) {
             break;
         }
-        const QScriptValue scriptValue = m_evaluator->property(item, propName);
-        if (checkErrors && Q_UNLIKELY(m_evaluator->engine()->hasErrorOrException(scriptValue))) {
-            throw ErrorInfo(m_evaluator->engine()->lastError(scriptValue,
-                                                             propValue->location()));
+        const ScopedJsValue scriptValue(ctx, m_evaluator->property(item, propName));
+        if (JsException ex = m_evaluator->engine()->checkAndClearException(propValue->location())) {
+            if (checkErrors)
+                throw ex.toErrorInfo();
         }
 
         // NOTE: Loses type information if scriptValue.isUndefined == true,
         //       as such QScriptValues become invalid QVariants.
         QVariant v;
-        if (scriptValue.isFunction()) {
-            v = scriptValue.toString();
+        if (JS_IsFunction(ctx, scriptValue)) {
+            v = getJsString(ctx, scriptValue);
         } else {
-            v = scriptValue.toVariant();
+            v = getJsVariant(ctx, scriptValue);
             QVariantMap m = v.toMap();
             if (m.contains(StringConstants::importScopeNamePropertyInternal())) {
                 QVariantMap tmp = m;
-                m = scriptValue.prototype().toVariant().toMap();
+                const ScopedJsValue proto(ctx, JS_GetPrototype(ctx, scriptValue));
+                m = getJsVariant(ctx, proto).toMap();
                 for (auto it = tmp.begin(); it != tmp.end(); ++it)
                     m.insert(it.key(), it.value());
                 v = m;
@@ -1855,7 +1862,7 @@ void ProjectResolver::collectPropertiesForExportItem(const QualifiedId &moduleNa
             {
                 valueItem->setScope(moduleInstance);
                 if (!hadName) {
-                    // EvaluatorScriptClass expects a name here.
+                    // Evaluator expects a name here.
                     valueItem->setProperty(StringConstants::nameProperty(),
                                            VariantValue::create(moduleName.toString()));
                 }