1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
|
// Copyright (C) 2023 The Qt Company Ltd.
// SPDX-License-Identifier: LicenseRef-Qt-Commercial OR GFDL-1.3-no-invariants-only
/*!
\example scenegraph/rhiunderqml
\title Scene Graph - RHI Under QML
\examplecategory {Graphics}
\ingroup qtquickexamples
\brief Shows how to render directly with \l QRhi under a Qt Quick scene.
\image rhiunderqml-example.jpg
\section1 Introduction
The RHI Under QML example shows how an application can make use of the \l
QQuickWindow::beforeRendering() and \l
QQuickWindow::beforeRenderPassRecording() signals to draw custom \l{QRhi}-based
content under a Qt Quick scene.
Applications that wish to render \l QRhi content on top of the Qt Quick scene,
can do so by connecting to the \l QQuickWindow::afterRendering() and \l
QQuickWindow::afterRenderPassRecording() signals.
In this example, we will also see how it is possible to have values that
are exposed to QML which affect the QRhi-based rendering. We animate the
threshold value using a NumberAnimation in the QML file and this float
value is then passed on in a uniform buffer to the fragment shader.
The example is equivalent in most ways to the \l{Scene Graph - OpenGL Under
QML}{OpenGL Under QML}, \l{Scene Graph - Direct3D 11 Under QML}{Direct3D 11
Under QML}, \l{Scene Graph - Metal Under QML}{Metal Under QML}, and \l{Scene
Graph - Vulkan Under QML}{Vulkan Under QML} examples. Those examples render
the same content by directly using a 3D API. This example on the other hand
is fully cross-platform and portable, as it inherently supports operating
with all the 3D APIs supported by QRhi (such as, OpenGL, Vulkan, Metal,
Direct 3D 11 and 12).
\note This example demonstrates advanced, low-level functionality performing
portable, cross-platform 3D rendering, while relying on APIs with limited
compatibility guarantee from the Qt Gui module. To be able to use the QRhi
APIs, the application links to \c{Qt::GuiPrivate} and includes
\c{<rhi/qrhi.h>}.
Adding custom rendering as an underlay/overlay is one of the three ways to integrate
custom 2D/3D rendering into a Qt Quick scene. The other two options are to perform
the rendering "inline" with the Qt Quick scene's own rendering using QSGRenderNode,
or to generate a whole separate render pass targeting a dedicated render target
(a texture) and then have an item in the scene display the texture.
Refer to the \l{Scene Graph - RHI Texture Item} and the
\l{Scene Graph - Custom QSGRenderNode} examples regarding those approaches.
\section1 Core Concepts
The beforeRendering() signal is emitted at the start of every frame, before
the scene graph starts its rendering, thus any \l QRhi draw calls that are
made as a response to this signal, will stack under the Qt Quick items.
However, there are two signals that are relevant here: the application's own
\l QRhi commands should be recorded onto the same command buffer that is
used by the scene graph, and what's more, the commands should belong to the
same render pass. beforeRendering() on its own is not sufficient for this
because it gets emitted at the start of the frame, before starting to record
a render pass via \l QRhiCommandBuffer::beginPass(). By also connecting to
beforeRenderPassRecording(), the application's own commands and the scene
graph's own rendering will end up in the right order:
\list
\li The scene graph's render loop calls \l QRhi::beginFrame()
\li \l QQuickWindow::beforeRendering() is emitted - the application prepares resources for its custom rendering
\li The scene graph calls \l QRhiCommandBuffer::beginPass()
\li \l QQuickWindow::beforeRenderPassRecording() is emitted - the application records draw calls
\li The scene graph records draw calls
\endlist
\section1 Walkthrough
The custom rendering is encapsulated within a custom QQuickItem. \c
RhiSquircle derives from \l QQuickItem, and is exposed to QML (note the
\c{QML_ELEMENT}). The QML scene instantiates \c RhiSquircle. Note however
that this is not a visual item: the \l QQuickItem::ItemHasContents flag is
not set. Thus the item's position and size has no relevance and it does not
reimplement \l{QQuickItem::updatePaintNode()}{updatePaintNode()}.
\snippet scenegraph/rhiunderqml/rhisquircle.h 0
Instead, when the item gets associated with a \l QQuickWindow, it connects
to the \l{QQuickWindow::beforeSynchronizing()} signal. Using
Qt::DirectConnection is important since this signal is emitted on the Qt
Quick render thread, if there is one. We want the connected slot to be
invoked on this same thread.
\snippet scenegraph/rhiunderqml/rhisquircle.cpp init
In the scene graph's synchronizing phase, the rendering infrastructure is
created, if not yet done, and the data relevant for rendering is
synchronized, i.e. copied from the \c RhiSquircle item, that lives on the
main thread, to the \c SquircleRenderer object that lives on the render
thread. (if there is no render thread, then both objects live on the main
thread) Accessing data is safe because the main thread is blocked while the
render thread is executing its synchronize phase. See \l{Qt Quick Scene
Graph} for more information on the scene graph threading and rendering
model.
In addition to the value of \c t, the associated QQuickWindow pointer is
copied as well. While the \c SquircleRenderer could query
\l{QQuickItem::window()}{window()} on the \c RhiSquircle item even when
operating on the render thread, that is, in theory, not entirely safe. Hence
making a copy.
When setting up the \c SquircleRenderer, connections to the
\l{QQuickWindow::beforeRendering()}{beforeRendering()} and
\l{QQuickWindow::beforeRenderPassRecording()}{beforeRenderPassRecording()}
are made, which are the key to be able to act and inject the application's
custom 3D rendering commands at the appropriate time.
\snippet scenegraph/rhiunderqml/rhisquircle.cpp sync
When \l{QQuickWindow::beforeRendering()}{beforeRendering()} is emitted, the
QRhi resources needed for our custom rendering, such as \l QRhiBuffer, \l
QRhiGraphicsPipeline, and related objects, are created if not yet done.
The data in the buffers is updated (more precisely, the data update
operations are enqueued) using \l QRhiResourceUpdateBatch and \l
QRhiCommandBuffer::resourceUpdate(). The vertex buffer does not change its
contents once the initial set of vertices are uploaded to it. The uniform
buffer however is a \l{QRhiBuffer::Dynamic}{dynamic} buffer, as is typical
for such buffers. Its content, some regions at least, is updated for every
frame. Hence the unconditional call to
\l{QRhiResourceUpdateBatch::updateDynamicBuffer()}{updateDynamicBuffer()}
for offset 0 and a byte size of 4 (which is \c{sizeof(float)} since the C++
\c float type happens to match GLSL's 32-bit \c float). What is stored at
that position is the value of \c t, and that is updated in every frame,
meaning in every invocation of frameStart().
There is an additional float value in the buffer, starting at offset 4. This
is used to cater to the coordinate system differences of the 3D APIs: when
\l{QRhi::isYUpInNDC()}{isYUpInNDC()} returns \c false, which is the case
with Vulkan in particular, the value is set to -1.0 which leads to flipping
the Y value in the 2 component vector that is passed on (with interpolation)
to the fragment shader based on which the color is calculated. This way the
output on the screen is identical (i.e. the top-left corner is green-ish,
the bottom-left is red-ish), regardless of which 3D API is in use. This
value is updated only once in the uniform buffer, similarly to the vertex
buffer. This highlights an issue low-level rendering code that aims to be
portable often needs to deal with: the coordinate system differences in
normalized device coordinates (NDC) and in images and framebuffers. For
example, the NDC uses a origin-at-bottom-left system everywhere except
Vulkan. Whereas framebuffers use an origin-at-top-left system everywhere
except OpenGL. Typical renderers that work with a perspective projection can
often be oblivious to this problem by conveniently relying on
\l{QRhi::clipSpaceCorrMatrix()}, which is a matrix that can be multiplied in
to the projection matrix, and applies both an Y flip when needed, and also
caters to the fact that clip space depth runs \c{-1..1} with OpenGL but
\c{0..1} everywhere else. However, in some cases, such as in this example,
this is not applicable. Rather, the application and shader logic needs to
perform the necessary adjustment of vertex and UV positions as appropriate
based on querying \l QRhi::isYUpInNDC() and \l QRhi::isYUpInFramebuffer().
To gain access to the \l QRhi and \l QRhiSwapChain objects Qt Quick uses,
they can simply be queried from the \l QQuickWindow. Note that this assumes
that the QQuickWindow is a regular, on-screen window. If it used \l
QQuickRenderControl instead, e.g. to perform off-screen rendering into a
texture, querying the swapchain would be wrong since there is no swapchain
then.
Due to the signal being emitted after Qt Quick calls \l QRhi::beginFrame(),
it is already possible to query the command buffer and render target from
the swapchain. This is what allows to conveniently issue a \l
QRhiCommandBuffer::resourceUpdate() on the object returned from \l
QRhiSwapChain::currentFrameCommandBuffer(). When creating a graphics
pipeline, a QRhiRenderPassDescriptor can be retrieved from the
QRhiRenderTarget returned from \l QRhiSwapChain::currentFrameRenderTarget().
(note that this means the graphics pipeline built here is suitable only for
rendering to the swapchain, or at best another render target that is
\l{QRhiRenderPassDescriptor::isCompatible()}{compatible} with it; it is
likely that if we wanted to render to a texture, then a different
QRhiRenderPassDescriptor, and so a different graphics pipeline, would be
needed since the texture and swapchain formats may differ)
\snippet scenegraph/rhiunderqml/rhisquircle.cpp frame-start
Finally, upon \l QQuickWindow::beforeRenderPassRecording(), a draw call for
a triangle strip with 4 vertices is recorded. This example simply draws a
quad in practice, and calculates the pixel colors using the logic in the
fragment shaders, but applications are free to do more complicated drawing:
creating multiple graphics pipelines and recording multiple draw calls is
perfectly fine as well. The important thing to keep in mind is that whatever
is recorded on the \l QRhiCommandBuffer retrieved from the window's
\l{QRhiSwapChain}{swapchain}, it is effectively prepended before the Qt Quick
scene graph's own rendering within the main render pass.
\note This means that if depth buffer usage with depth testing and writing
out depth values is involved, then the Qt Quick content may be affected by
the values written to the depth buffer. See \l{Qt Quick Scene Graph Default
Renderer} for details on the scene graph's renderer, in particular the
sections about the handling of \e opaque and \e{alpha blended} primitives.
To get the window size in pixels, \l QRhiRenderTarget::pixelSize() is used.
This is convenient because this way the example does not need to calculate
the viewport size by other means and does not have to worry about applying
the \l{QWindow::devicePixelRatio()}{high DPI scale factor}, if there is any.
\snippet scenegraph/rhiunderqml/rhisquircle.cpp frame-render
The vertex and fragment shaders go through the standard QRhi shader
conditioning pipeline. Initially written as Vulkan-compatible GLSL, they get
compiled to SPIR-V and then transpiled to other shading languages by Qt's
tools. When using CMake, the example relies on the \c qt_add_shaders command
that makes it simple and convenient to bundle the shaders with the
application and perform the necessary processing at build time. See \l{Qt
Shader Tools Build System Integration} for details.
Specifying \c BASE helps removing the \c{../shared} prefix, while \c PREFIX
adds the intended \c{/scenegraph/rhiunderqml} prefix. Thus the final path is
\c{:/scenegraph/rhiunderqml/squircle_rhi.vert.qsb}.
\badcode
qt_add_shaders(rhiunderqml "rhiunderqml_shaders"
PRECOMPILE
OPTIMIZED
PREFIX
/scenegraph/rhiunderqml
BASE
../shared
FILES
../shared/squircle_rhi.vert
../shared/squircle_rhi.frag
)
\endcode
To support qmake, the example still ships the \c{.qsb} files that would
normally be generated at build time, and lists them in the qrc file. This
approach is however not recommended for new applications that use CMake as
the build system.
\sa {Scene Graph - RHI Texture Item}, {Scene Graph - Custom QSGRenderNode}
*/
|
