Do full overlap checks within the alpha render list when rebuilding

Not applicable to Qt 6.1 and newer because those already have this as
part of another patch (9aa3db2e19b2e1622b878cf34b1978f4fdbcac39), which
was meant to be a performance optimization.

With the example provided it is clear however that there is an actual
batching problem in there, with certain scenes, where the problem of
incorrectly batching two elements together becomes visible only upon
a set of conditions such as:

- there is a set of mergeable and non-mergeable nodes (e.g. non-atlased
Images + Rectangles or non-atlased Images + Texts)

- all in the alpha render list (semi-transparency, e.g. Image with
alpha, Text, Rectangle with opacity, etc.)

- laid out in a certain way that certain overlaps occur, e.g. in
a column with the Images under each other, with Rectangles or Texts
on top of the last two.

- then trigger certain scene changes, e.g. swap the source of the
second and third Image.

Certain changes will end up with the incorrect merging described
in the render_AlphaOverlapRebuild.qml test case. If the two Rectangles
or Texts get merged in one batch, rendered before the third Image,
the result is incorrect since the "background" Image is then on top.

This can be difficult to reproduce as there are a number of
preconditions for the scene and the changes in the scene. But when it
happens, it is a critical rendering error.

Fixes: QTBUG-92984
Change-Id: I016614d8465f6917ebf6c7d9b2460a31decdda34
Reviewed-by: Eskil Abrahamsen Blomfeldt <eskil.abrahamsen-blomfeldt@qt.io>

1 files changed, 4 insertions, 2 deletions

diff --git a/src/quick/scenegraph/coreapi/qsgbatchrenderer.cpp b/src/quick/scenegraph/coreapi/qsgbatchrenderer.cpp
index 024c0d68ef..e6c2c1b573 100644
--- a/src/quick/scenegraph/coreapi/qsgbatchrenderer.cpp
+++ b/src/quick/scenegraph/coreapi/qsgbatchrenderer.cpp
@@ -1878,7 +1878,7 @@ bool Renderer::checkOverlap(int first, int last, const Rect &bounds)
 {
     for (int i=first; i<=last; ++i) {
         Element *e = m_alphaRenderList.at(i);
-        if (!e || e->batch)
+        if (!e)
             continue;
         Q_ASSERT(e->boundsComputed);
         if (e->bounds.intersects(bounds))
@@ -1949,8 +1949,10 @@ void Renderer::prepareAlphaBatches()
                 continue;
             if (ej->root != ei->root || ej->isRenderNode)
                 break;
-            if (ej->batch)
+            if (ej->batch) {
+                overlapBounds |= ej->bounds;
                 continue;
+            }
 
             QSGGeometryNode *gnj = ej->node;
             if (gnj->geometry()->vertexCount() == 0)