qv4mm: Handle running out of native heap memory

With the current setup, we would have risked running out of native heap
space if the incremental garbage collection takes too long.
Remedy this by forcing the current gc run to complete immediately once
we overshoot the limit by a factor of 3/2.
We still need to respect that the gc must not run in a critical section:
Handle this by rechecking the condition when leaving the critical
section, and only forcing a gc run at that point if necessary.

Note that we currently update the gc limits before the gc cycle
finishes, so that the effective limit might actually not be 3/2 in that
case. This will be remedied in an upcoming patch.

Change-Id: I19c367bd1ff6ffc129293fc79e39b101f0728135
Reviewed-by: Ulf Hermann <ulf.hermann@qt.io>

2 files changed, 2 insertions, 8 deletions

diff --git a/src/qml/jsruntime/qv4engine.cpp b/src/qml/jsruntime/qv4engine.cpp
index 391ea62b50..acf6132799 100644
--- a/src/qml/jsruntime/qv4engine.cpp
+++ b/src/qml/jsruntime/qv4engine.cpp
@@ -371,8 +371,7 @@ ExecutionEngine::ExecutionEngine(QJSEngine *jsEngine)
 
     memoryManager = new QV4::MemoryManager(this);
     // we don't want to run the gc while the initial setup is not done; not even in aggressive mode
-    memoryManager->gcBlocked = MemoryManager::InCriticalSection;
-    auto cleanup = qScopeGuard([this] { memoryManager->gcBlocked = MemoryManager::Unblocked; } );
+    GCCriticalSection gcCriticalSection(this);
     // reserve space for the JS stack
     // we allow it to grow to a bit more than m_maxJSStackSize, as we can overshoot due to ScopedValues
     // allocated outside of JIT'ed methods.
diff --git a/src/qml/jsruntime/qv4executablecompilationunit.cpp b/src/qml/jsruntime/qv4executablecompilationunit.cpp
index 403b81b2b4..9e10e437a8 100644
--- a/src/qml/jsruntime/qv4executablecompilationunit.cpp
+++ b/src/qml/jsruntime/qv4executablecompilationunit.cpp
@@ -82,12 +82,7 @@ void ExecutableCompilationUnit::populate()
        gc starts marking the root set at the start of a run.
      */
     const CompiledData::Unit *data = m_compilationUnit->data;
-    auto oldState = std::exchange(engine->memoryManager->gcBlocked, MemoryManager::InCriticalSection);
-    auto cleanup = qScopeGuard([this, oldState]() {
-        engine->memoryManager->gcBlocked = oldState;
-        if (oldState != MemoryManager::Unblocked)
-            this->markObjects(engine->memoryManager->markStack());
-    });
+    GCCriticalSection<ExecutableCompilationUnit> criticalSection(engine, this);
 
     Q_ASSERT(!runtimeStrings);
     Q_ASSERT(engine);