rhi: Attempt to fix up the logic around beginExternal()

Mainly for Vulkan where it lacked the recording of the still queued
commands. Uncovered by Qt Quick examples that integrate custom Vulkan
rendering.

This still has an issue that needs to be tackled separately. (we probably
will switch to using a dedicated secondary command buffer with
RENDER_PASS_CONTINUE_BIT for the external commands, and then just have
a vkCmdExecuteCommands in our own queue instead of recording everything in
beginExternal).

The possibility of losing glMemoryBarrier() calls due to begin/endExternal()
with the OpenGL backend is fixed too. The logic here mirrors Vulkan
to some extent except that we do not have a concept of (and so the trouble
with) renderpass instances.

Clean up around the implementations of finish() as well. Attempting to share
code via a "flushCommandBuffer" function is admirable but is not worth it
since some semantics are different. (finish() cannot be called within a
begin/endPass, unlike begin/endExternal).

Change-Id: I5137db598d6a40d484e53678f5c919abf750d9ed
Reviewed-by: Andy Nichols <andy.nichols@qt.io>

1 files changed, 3 insertions, 2 deletions

diff --git a/src/gui/rhi/qrhivulkan_p_p.h b/src/gui/rhi/qrhivulkan_p_p.h
index e2ea8fec2a..dd9a7d4216 100644
--- a/src/gui/rhi/qrhivulkan_p_p.h
+++ b/src/gui/rhi/qrhivulkan_p_p.h
@@ -324,9 +324,9 @@ struct QVkCommandBuffer : public QRhiCommandBuffer
     };
 
     void resetState() {
-        resetCommands();
         recordingPass = NoPass;
         currentTarget = nullptr;
+        resetCommands();
         resetCachedState();
     }
 
@@ -510,9 +510,10 @@ struct QVkCommandBuffer : public QRhiCommandBuffer
 
     void resetCommands() {
         commands.clear();
+        resetPools();
+
         passResTrackers.clear();
         currentPassResTrackerIndex = -1;
-        resetPools();
     }
 
     void resetPools() {