Allow redirecting QRhi-based rendering via QQuickRenderControl

Implement the Qt 6 TODO for using an externally-provided render target
when rendering the scene via QRhi.

And say hello to QQuickRenderTarget. This class exists to allow
potentially extending later what a "render target" consists
of. Instead of hard-coding taking a single void * in the
setRenderTarget() function, it takes a (implicitly shared,
d-pointered) QQuickRenderTarget, which in turn can be created via
static factory functions - of which new ones can be added later on.

The new version of QQuickWindow::setRenderTarget() takes a
QQuickRenderTarget.

QQuickRenderControl gets a new initialize() variant, and a few extra
functions (beginFrame(), endFrame()). This allows it to, by using
QSGRhiSupport internally, create a QRhi under the hood.

As a bonus, this also fixes an existing scenegraph resource leak when
destroying the QQuickRenderControl.

The qquickrendercontrol autotest is extended, with a QRhi-based test
case that is executed for all of the QRhi backends that succeed to
initialize.  This is the internal verification. In addition, there is
a Vulkan-based one that creates its own VkDevice, VkImage, and
friends, and then uses Qt Quick with the same Vulkan device, targeting
the VkImage. This test verifies the typical application use
case. (sadly, life is too short to waste it on writing Vulkan
boilerplate for an on-screen version of this, but we have the D3D11
example instead)

What QQuickRenderControl loses, when used in combination with QRhi, is
the grab() function. This never made much sense as a public API:
QQuickWindow::grabWindow() call this when the window is associated
with a rendercontrol, so as a public API QQuickRenderControl::grab()
is redundant, because one gets the same result via the standard
QQuickWindow API. It is now made private.

More importantly, reading back the content is no longer supported,
unless the 'software' backend is in use. The reasoning here is that,
if the client of the API manages and provides the render target (as
abstracted by QQuickRenderTarget), it is then expected to be capable
of reading back the content in whatever way it sees fit, because it
owns and manages the resource (e.g. the texture) in the first
place. Providing fragile convenience functions for this is not
reasonable anymore, and was questionable even with OpenGL, given that
it is not future proof - what if the target is suddenly a floating
point texture, for instance? The software backend case makes sense
because that relies on private APIs - and has no render target concept
either - so there the same cannot be achieved by applications by
relying on public APIs only.

Another new class is QQuickGraphicsDevice. This is very similar to
QQuickRenderTarget, it is a simple container capable of holding a set
of of native objects, mostly in the form of void*s, with future
extensibility thanks to the static factory functions. (examples of
native object sets would be a ID3D11Device + ID3D11DeviceContext, or a
QOpenGLContext, or a MTLDevice + MTLCommandQueue, or a number of
Vulkan device-related objects, etc.)  This allows one to specify that
the QRhi created under the hood (either by QQuickRenderControl or by
the render loop) should use an existing graphics device (i.e. it is
basically a public wrapper for values that go into a QRhi*InitParams
under the hood).

QQuickRenderTarget and QQuickGraphicsDevice are both demonstrated in a
new example: rendercontrol_d3d11. We choose D3D11 because it is
reasonably simple to set up a renderer with a window, and, because
there is known user demand for Qt Quick - external D3D engine
interop. Passing in the custom engine's own ID3D11Device and
ID3D11DeviceContext is essential: the texture (ID3D11Texture2D) Qt
Quick is targeting would not be usable if Qt Quick's QRhi was using a
different ID3D11Device.

Task-number: QTBUG-78595
Change-Id: I5dfe7f6cf1540daffc2f11136be114a08e87202b
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Andy Nichols <andy.nichols@qt.io>

1 files changed, 283 insertions, 33 deletions

diff --git a/src/quick/items/qquickrendercontrol.cpp b/src/quick/items/qquickrendercontrol.cpp
index 190e0942ab..b14a3f35f7 100644
--- a/src/quick/items/qquickrendercontrol.cpp
+++ b/src/quick/items/qquickrendercontrol.cpp
@@ -43,29 +43,36 @@
 #include <QtCore/QCoreApplication>
 #include <QtCore/QTime>
 #include <QtQuick/private/qquickanimatorcontroller_p.h>
+#include <QtQuick/private/qsgdefaultrendercontext_p.h>
+#include <QtQuick/private/qsgrhisupport_p.h>
 
 #if QT_CONFIG(opengl)
 # include <QOpenGLContext>
-# include <QtQuick/private/qsgdefaultrendercontext_p.h>
 #if QT_CONFIG(quick_shadereffect)
 # include <QtQuick/private/qquickopenglshadereffectnode_p.h>
 #endif
 #endif
 #include <QtGui/private/qguiapplication_p.h>
 #include <qpa/qplatformintegration.h>
+#include <QtGui/qoffscreensurface.h>
 
 #include <QtQml/private/qqmlglobal_p.h>
 
 #include <QtQuick/QQuickWindow>
+#include <QtQuick/QQuickRenderTarget>
 #include <QtQuick/private/qquickwindow_p.h>
 #include <QtQuick/private/qquickitem_p.h>
 #include <QtQuick/private/qsgsoftwarerenderer_p.h>
 #include <QtCore/private/qobject_p.h>
 
+#include <QtQuick/private/qquickwindow_p.h>
+#include <QtGui/private/qrhi_p.h>
+
 QT_BEGIN_NAMESPACE
 #if QT_CONFIG(opengl)
 extern Q_GUI_EXPORT QImage qt_gl_read_framebuffer(const QSize &size, bool alpha_format, bool include_alpha);
 #endif
+
 /*!
   \class QQuickRenderControl
 
@@ -136,9 +143,14 @@ extern Q_GUI_EXPORT QImage qt_gl_read_framebuffer(const QSize &size, bool alpha_
 
 QSGContext *QQuickRenderControlPrivate::sg = nullptr;
 
-QQuickRenderControlPrivate::QQuickRenderControlPrivate()
-    : initialized(0),
-      window(nullptr)
+QQuickRenderControlPrivate::QQuickRenderControlPrivate(QQuickRenderControl *renderControl)
+    : q(renderControl),
+      initialized(false),
+      window(nullptr),
+      rhi(nullptr),
+      cb(nullptr),
+      offscreenSurface(nullptr),
+      sampleCount(1)
 {
     if (!sg) {
         qAddPostRoutine(cleanup);
@@ -158,7 +170,7 @@ void QQuickRenderControlPrivate::cleanup()
    object \a parent.
 */
 QQuickRenderControl::QQuickRenderControl(QObject *parent)
-    : QObject(*(new QQuickRenderControlPrivate), parent)
+    : QObject(*(new QQuickRenderControlPrivate(this)), parent)
 {
 }
 
@@ -182,11 +194,16 @@ QQuickRenderControl::~QQuickRenderControl()
     d->windowDestroyed();
 
     delete d->rc;
+
+    d->resetRhi();
 }
 
 void QQuickRenderControlPrivate::windowDestroyed()
 {
     if (window) {
+        QQuickWindowPrivate *cd = QQuickWindowPrivate::get(window);
+        cd->cleanupNodesOnShutdown();
+
         rc->invalidate();
 
         QQuickWindowPrivate::get(window)->animationController.reset();
@@ -215,6 +232,90 @@ void QQuickRenderControl::prepareThread(QThread *targetThread)
 }
 
 /*!
+    Sets the number of samples to use for multisampling. When \a sampleCount is
+    0 or 1, multisampling is disabled.
+
+    \note This function is always used in combination with a multisample render
+    target, which means \a sampleCount must match the sample count passed to
+    QQuickRenderTarget::fromNativeTexture(), which in turn must match the
+    sample count of the native texture.
+
+    \since 6.0
+
+    \sa initialize(), QQuickRenderTarget
+ */
+void QQuickRenderControl::setSamples(int sampleCount)
+{
+    Q_D(QQuickRenderControl);
+    d->sampleCount = qMax(1, sampleCount);
+}
+
+/*!
+    \return the current sample count. 1 or 0 means no multisampling.
+
+    \since 6.0
+ */
+int QQuickRenderControl::samples() const
+{
+    Q_D(const QQuickRenderControl);
+    return d->sampleCount;
+}
+
+/*!
+    Initializes the scene graph resources. When using a graphics API, such as
+    Vulkan, Metal, OpenGL, or Direct3D, for Qt Quick rendering,
+    QQuickRenderControl will set up an appropriate rendering engine when this
+    function is called. This rendering infrastructure exists as long as the
+    QQuickRenderControl exists.
+
+    To control what graphics API Qt Quick uses, call
+    QQuickWindow::setSceneGraphBackend() with one of the
+    QSGRendererInterface:GraphicsApi constants. That must be done before
+    calling this function.
+
+    To prevent the scenegraph from creating its own device and context objects,
+    specify an appropriate QQuickGraphicsDevice, wrapping existing graphics
+    objects, by calling QQuickWindow::setGraphicsDevice().
+
+    \note This function does not need to be, and must not be, called when using
+    the \c software adaptation of Qt Quick.
+
+    \since 6.0
+
+    \sa QQuickRenderTarget, QQuickGraphicsDevice
+ */
+bool QQuickRenderControl::initialize()
+{
+    Q_D(QQuickRenderControl);
+
+    if (!d->window) {
+        qWarning("QQuickRenderControl::initialize called with no associated window");
+        return false;
+    }
+
+    if (!d->initRhi())
+        return false;
+
+    QQuickWindowPrivate *wd = QQuickWindowPrivate::get(d->window);
+    wd->rhi = d->rhi;
+
+    QSGDefaultRenderContext *renderContext = qobject_cast<QSGDefaultRenderContext *>(d->rc);
+    if (renderContext) {
+        QSGDefaultRenderContext::InitParams params;
+        params.rhi = d->rhi;
+        params.sampleCount = d->sampleCount;
+        params.initialSurfacePixelSize = d->window->size() * d->window->effectiveDevicePixelRatio();
+        params.maybeSurface = d->window;
+        renderContext->initialize(&params);
+    } else {
+        qWarning("QRhi is only compatible with default adaptation");
+        return false;
+    }
+
+    return true;
+}
+
+/*!
   Initializes the scene graph resources. The context \a gl has to be the
   current OpenGL context or null if it is not relevant because a Qt Quick
   backend other than OpenGL is in use.
@@ -295,24 +396,33 @@ bool QQuickRenderControl::sync()
         return false;
 
     QQuickWindowPrivate *cd = QQuickWindowPrivate::get(d->window);
+    // we may not have a d->rhi (software backend) hence the check is important
+    if (d->rhi) {
+        if (!d->rhi->isRecordingFrame()) {
+            qWarning("QQuickRenderControl can only sync when beginFrame() has been called");
+            return false;
+        }
+        if (!d->cb) {
+            qWarning("QQuickRenderControl cannot be used with QRhi when no QRhiCommandBuffer is provided");
+            return false;
+        }
+        cd->setCustomCommandBuffer(d->cb);
+    }
+
     cd->syncSceneGraph();
     d->rc->endSync();
 
-    // TODO: find out if the sync actually caused a scenegraph update.
     return true;
 }
 
 /*!
-  Stop rendering and release resources. Requires a current context.
+  Stop rendering and release resources.
 
   This is the equivalent of the cleanup operations that happen with a
   real QQuickWindow when the window becomes hidden.
 
   This function is called from the destructor. Therefore there will
-  typically be no need to call it directly. Pay attention however to
-  the fact that this requires the context, that was passed to
-  initialize(), to be the current one at the time of destroying the
-  QQuickRenderControl instance.
+  typically be no need to call it directly.
 
   Once invalidate() has been called, it is possible to reuse the
   QQuickRenderControl instance by calling initialize() again.
@@ -353,6 +463,19 @@ void QQuickRenderControl::render()
         return;
 
     QQuickWindowPrivate *cd = QQuickWindowPrivate::get(d->window);
+    // we may not have a d->rhi (software backend) hence the check is important
+    if (d->rhi) {
+        if (!d->rhi->isRecordingFrame()) {
+            qWarning("QQuickRenderControl can only render when beginFrame() has been called");
+            return;
+        }
+        if (!d->cb) {
+            qWarning("QQuickRenderControl cannot be used with QRhi when no QRhiCommandBuffer is provided");
+            return;
+        }
+        cd->setCustomCommandBuffer(d->cb);
+    }
+
     cd->renderSceneGraph(d->window->size());
 }
 
@@ -378,48 +501,50 @@ void QQuickRenderControl::render()
     for example. This will lead to better performance.
 */
 
-/*!
-  Grabs the contents of the scene and returns it as an image.
-
-  \note Requires the context to be current.
- */
-QImage QQuickRenderControl::grab()
+QImage QQuickRenderControlPrivate::grab()
 {
-    Q_D(QQuickRenderControl);
-    if (!d->window)
+    if (!window)
         return QImage();
 
     QImage grabContent;
 
-    if (d->window->rendererInterface()->graphicsApi() == QSGRendererInterface::OpenGL) {
+    if (rhi) {
+
+        // As documented by QQuickWindow::grabWindow(): Nothing to do here, we
+        // do not support "grabbing" with an application-provided render target
+        // in Qt 6. (with the exception of the software backend because that
+        // does not support custom render targets, so the grab implementation
+        // here is still valuable)
+
+    } else if (window->rendererInterface()->graphicsApi() == QSGRendererInterface::OpenGL) {
 #if QT_CONFIG(opengl)
-        QQuickWindowPrivate *cd = QQuickWindowPrivate::get(d->window);
+        QQuickWindowPrivate *cd = QQuickWindowPrivate::get(window);
         cd->polishItems();
         cd->syncSceneGraph();
-        d->rc->endSync();
-        render();
-        const bool alpha = d->window->format().alphaBufferSize() > 0 && d->window->color().alpha() < 255;
-        grabContent = qt_gl_read_framebuffer(d->window->size() * d->window->effectiveDevicePixelRatio(), alpha, alpha);
-        if (QQuickRenderControl::renderWindowFor(d->window)) {
-            grabContent.setDevicePixelRatio(d->window->effectiveDevicePixelRatio());
+        rc->endSync();
+        q->render();
+        const bool alpha = window->format().alphaBufferSize() > 0 && window->color().alpha() < 255;
+        grabContent = qt_gl_read_framebuffer(window->size() * window->effectiveDevicePixelRatio(), alpha, alpha);
+        if (QQuickRenderControl::renderWindowFor(window)) {
+            grabContent.setDevicePixelRatio(window->effectiveDevicePixelRatio());
         }
 #endif
 #if QT_CONFIG(thread)
-    } else if (d->window->rendererInterface()->graphicsApi() == QSGRendererInterface::Software) {
-        QQuickWindowPrivate *cd = QQuickWindowPrivate::get(d->window);
+    } else if (window->rendererInterface()->graphicsApi() == QSGRendererInterface::Software) {
+        QQuickWindowPrivate *cd = QQuickWindowPrivate::get(window);
         cd->polishItems();
         cd->syncSceneGraph();
         QSGSoftwareRenderer *softwareRenderer = static_cast<QSGSoftwareRenderer *>(cd->renderer);
         if (softwareRenderer) {
-            const qreal dpr = d->window->effectiveDevicePixelRatio();
-            const QSize imageSize = d->window->size() * dpr;
+            const qreal dpr = window->effectiveDevicePixelRatio();
+            const QSize imageSize = window->size() * dpr;
             grabContent = QImage(imageSize, QImage::Format_ARGB32_Premultiplied);
             grabContent.setDevicePixelRatio(dpr);
             QPaintDevice *prevDev = softwareRenderer->currentPaintDevice();
             softwareRenderer->setCurrentPaintDevice(&grabContent);
             softwareRenderer->markDirty();
-            d->rc->endSync();
-            render();
+            rc->endSync();
+            q->render();
             softwareRenderer->setCurrentPaintDevice(prevDev);
         }
 #endif
@@ -474,6 +599,131 @@ QWindow *QQuickRenderControl::renderWindowFor(QQuickWindow *win, QPoint *offset)
     return nullptr;
 }
 
+/*!
+    \return the QQuickWindow this QQuickRenderControl is associated with.
+
+    \note A QQuickRenderControl gets associated with a QQuickWindow when
+    constructing the QQuickWindow. The return value from this function is null
+    before that point.
+
+    \since 6.0
+ */
+QQuickWindow *QQuickRenderControl::window() const
+{
+    Q_D(const QQuickRenderControl);
+    return d->window;
+}
+
+/*!
+    Specifies the start of a graphics frame. Calls to sync() or render() must
+    be enclosed by calls to beginFrame() and endFrame().
+
+    Unlike the earlier OpenGL-only world of Qt 5, rendering with other graphics
+    APIs requires more well-defined points of starting and ending a frame. When
+    manually driving the rendering loop via QQuickRenderControl, it now falls
+    to the user of QQuickRenderControl to specify these points.
+
+    A typical update step, including initialization of rendering into an
+    existing texture, could like like the following. The example snippet
+    assumes Direct3D 11 but the same concepts apply other graphics APIs as
+    well.
+
+    \badcode
+        if (!m_quickInitialized) {
+            m_quickWindow->setGraphicsDevice(QQuickGraphicsDevice::fromDeviceAndContext(m_engine->device(), m_engine->context()));
+
+            if (!m_renderControl->initialize())
+                qWarning("Failed to initialize redirected Qt Quick rendering");
+
+            m_quickWindow->setRenderTarget(QQuickRenderTarget::fromNativeTexture({ &m_res.texture, 0 },
+                                                                                 QSize(QML_WIDTH, QML_HEIGHT),
+                                                                                 SAMPLE_COUNT));
+
+            m_quickInitialized = true;
+        }
+
+        m_renderControl->polishItems();
+
+        m_renderControl->beginFrame();
+        m_renderControl->sync();
+        m_renderControl->render();
+        m_renderControl->endFrame(); // Qt Quick's rendering commands are submitted to the device context here
+    \endcode
+
+    \since 6.0
+
+    \sa endFrame(), initialize(), sync(), render(), QQuickGraphicsDevice, QQuickRenderTarget
+ */
+void QQuickRenderControl::beginFrame()
+{
+    Q_D(QQuickRenderControl);
+    if (!d->rhi || d->rhi->isRecordingFrame())
+        return;
+
+    d->rhi->beginOffscreenFrame(&d->cb, QRhi::ExternalContentsInPass); // must specify ExternalContents since we do not know in advance
+}
+
+/*!
+    Specifies the end of a graphics frame. Calls to sync() or render() must be
+    enclosed by calls to beginFrame() and endFrame().
+
+    When this function is called, any graphics commands enqueued by the
+    scenegraph are submitted to the context or command queue, whichever is
+    applicable.
+
+    \since 6.0
+
+    \sa beginFrame(), initialize(), sync(), render(), QQuickGraphicsDevice, QQuickRenderTarget
+ */
+void QQuickRenderControl::endFrame()
+{
+    Q_D(QQuickRenderControl);
+    if (!d->rhi || !d->rhi->isRecordingFrame())
+        return;
+
+    d->rhi->endOffscreenFrame();
+    d->cb = nullptr;
+}
+
+bool QQuickRenderControlPrivate::initRhi()
+{
+    // initialize() - invalidate() - initialize() uses the QRhi the first
+    // initialize() created, so if already exists, we are done
+    if (rhi)
+        return true;
+
+    QSGRhiSupport *rhiSupport = QSGRhiSupport::instance();
+
+    // sanity check for Vulkan
+#if QT_CONFIG(vulkan)
+    if (rhiSupport->rhiBackend() == QRhi::Vulkan && !window->vulkanInstance()) {
+        qWarning("QQuickRenderControl: No QVulkanInstance set for QQuickWindow, cannot initialize");
+        return false;
+    }
+#endif
+
+    // for OpenGL
+    if (!offscreenSurface)
+        offscreenSurface = rhiSupport->maybeCreateOffscreenSurface(window);
+
+    rhi = rhiSupport->createRhi(window, offscreenSurface);
+    if (!rhi) {
+        qWarning("QQuickRenderControl: Failed to initialize QRhi");
+        return false;
+    }
+
+    return true;
+}
+
+void QQuickRenderControlPrivate::resetRhi()
+{
+    delete rhi;
+    rhi = nullptr;
+
+    delete offscreenSurface;
+    offscreenSurface = nullptr;
+}
+
 QT_END_NAMESPACE
 
 #include "moc_qquickrendercontrol.cpp"