rhi: Pipeline cache load/save

Add QRhi APIs to retrieve and reload the contents of the "pipeline
cache".

The only API where there is a true pipeline cache is object is Vulkan
(VkPipelineCache). For OpenGL, the other backend where we support this,
it is simulated with program binaries. The Qt 5 style OpenGL program
binary disk cache continues to work like before, but one has now the
option to do things in a more modern, graphics API agnostic way, that
leads to generating a single blob instead of a large set of files in
some system location, allowing easier "pre-baking" of the cache content.

It is expected that Qt Quick exposes the two new functions in form
if QSG_RHI_ environment variables, thus allowing easy testing and
cache file generation.

As an example for the performance improvements this can give, consider
Vulkan, where we do not have any existing persistent caching mechanism
in place:

Running BenchmarkDemoQt6.exe --scene flythrough --mode demo creates 18
QRhiGraphicsPipeline objects from Qt Quick and Qt Quick 3D.

The total time spent in QRhiGraphicsPipeline::create() during application
startup for these 18 pipelines is 35-40 ms on a given Windows (NVIDIA)
system.

When exporting the pipeline cache contents to a file, and then, in a
subsequent run, reloading the cache contents, this is reduced to 5-7 ms
on the same system, meaning we get a 6-7x improvement.

The generated data is always specific to a given Qt version, RHI
backend, graphics device, and driver version. Much of the implementation
consists of adding and verifying the appropriate header to the blobs
retrieved from the driver, to allow gracefully ignoring data that was
generated with a device or driver that differs from the one used at
run time. This should provide robustness, even if the Vulkan or OpenGL
implementation is for some reason not prepared to identity and reject
incompatible cache/program blobs.

Fixes: QTBUG-90398
Change-Id: I67b197f393562434f372c7b7377f638abab85cb3
Reviewed-by: Andy Nichols <andy.nichols@qt.io>

13 files changed, 574 insertions, 54 deletions

diff --git a/src/gui/rhi/qrhi.cpp b/src/gui/rhi/qrhi.cpp
index 52ceb38f3d..4b07f4c346 100644
--- a/src/gui/rhi/qrhi.cpp
+++ b/src/gui/rhi/qrhi.cpp
@@ -453,6 +453,19 @@ Q_LOGGING_CATEGORY(QRHI_LOG_INFO, "qt.rhi.general")
     possible to decide if an adapter/device is software-based, this flag is
     ignored. It may also be ignored with graphics APIs that have no concept and
     means of enumerating adapters/devices.
+
+    \value EnablePipelineCacheDataSave Enables retrieving the pipeline cache
+    contents, where applicable. When not set, pipelineCacheData() will return
+    an empty blob always. Opting in is relevant in particular with backends
+    where additional, potentially time consuming work is needed to maintain the
+    data structures with the serialized, binary versions of shader programs. An
+    example is OpenGL, where the "pipeline cache" is simulated by retrieving
+    and loading shader program binaries. With backends where retrieving and
+    restoring the pipeline cache contents is not supported, the flag has no
+    effect. With some backends (such as, OpenGL) there are additional,
+    disk-based caching mechanisms for shader binaries. Writing to those may get
+    disabled whenever this flag is set since storing program binaries (OpenGL)
+    to multiple caches is not sensible.
  */
 
 /*!
@@ -612,6 +625,12 @@ Q_LOGGING_CATEGORY(QRHI_LOG_INFO, "qt.rhi.general")
     the 1 byte per component formats QRhiTexture::R8 and
     QRhiTexture::RED_OR_ALPHA8 are supported as well. Backends other than
     OpenGL can be expected to return true for this feature.
+
+    \value PipelineCacheDataLoadSave Indicates that the pipelineCacheData() and
+    setPipelineCacheData() functions are functional. When not supported, the
+    functions will not perform any action, the retrieved blob is always empty,
+    and thus no benefits can be expected from retrieving and, during a
+    subsequent run of the application, reloading the pipeline cache content.
  */
 
 /*!
@@ -6089,6 +6108,70 @@ bool QRhi::isDeviceLost() const
 }
 
 /*!
+    \return a binary \a data blob with data collected from the
+    QRhiGraphicsPipeline and QRhiComputePipeline successfully created during
+    the lifetime of this QRhi.
+
+    By saving and then, in subsequent runs of the same application, reloading
+    the cache data, pipeline and shader creation times can potentially be
+    accelerated.
+
+    When the PipelineCacheDataLoadSave is reported as unsupported, the returned
+    QByteArray is empty.
+
+    When the EnablePipelineCacheDataSave flag was not specified when calling
+    create(), the returned QByteArray may be empty, even when the
+    PipelineCacheDataLoadSave feature is supported.
+
+    When the returned data is non-empty, it is always specific to the QRhi
+    backend, the graphics device, and the driver implementation in use. QRhi
+    takes care of adding the appropriate header and safeguards that ensure that
+    the data can always be passed safely to setPipelineCacheData().
+
+    \note Calling releaseCachedResources() may, depending on the backend, clear
+    the pipeline data collected. A subsequent call to this function may then
+    not return any data.
+
+    \sa setPipelineCacheData(), create(), isFeatureSupported()
+ */
+QByteArray QRhi::pipelineCacheData()
+{
+    return d->pipelineCacheData();
+}
+
+/*!
+    Loads \a data into the pipeline cache, when applicable.
+
+    When the PipelineCacheDataLoadSave is reported as unsupported, the function
+    is safe to call, but has no effect.
+
+    The blob returned by pipelineCacheData() is always specific to a QRhi
+    backend, a graphics device, and a given version of the graphics driver.
+    QRhi takes care of adding the appropriate header and safeguards that ensure
+    that the data can always be passed safely to this function. If there is a
+    mismatch, e.g. because the driver has been upgraded to a newer version, or
+    because the data was generated from a different QRhi backend, a warning is
+    printed and \a data is safely ignored.
+
+    With Vulkan, this maps directly to VkPipelineCache. Calling this function
+    creates a new Vulkan pipeline cache object, with its initial data sourced
+    from \a data. The pipeline cache object is then used by all subsequently
+    created QRhiGraphicsPipeline and QRhiComputePipeline objects, thus
+    accelerating, potentially, the pipeline creation.
+
+    \note QRhi cannot give any guarantees that \a data has an effect on the
+    pipeline and shader creation performance. With APIs like Vulkan, it is up
+    to the driver to decide if \a data is used for some purpose, or if it is
+    ignored.
+
+    \sa pipelineCacheData(), isFeatureSupported()
+ */
+void QRhi::setPipelineCacheData(const QByteArray &data)
+{
+    d->setPipelineCacheData(data);
+}
+
+/*!
     \return a new graphics pipeline resource.
 
     \sa QRhiResource::destroy()
diff --git a/src/gui/rhi/qrhi_p.h b/src/gui/rhi/qrhi_p.h
index 3c7f9b6bc4..f3255eca5b 100644
--- a/src/gui/rhi/qrhi_p.h
+++ b/src/gui/rhi/qrhi_p.h
@@ -1488,7 +1488,8 @@ public:
     enum Flag {
         EnableProfiling = 1 << 0,
         EnableDebugMarkers = 1 << 1,
-        PreferSoftwareRenderer = 1 << 2
+        PreferSoftwareRenderer = 1 << 2,
+        EnablePipelineCacheDataSave = 1 << 3
     };
     Q_DECLARE_FLAGS(Flags, Flag)
 
@@ -1524,7 +1525,8 @@ public:
         RenderToNonBaseMipLevel,
         IntAttributes,
         ScreenSpaceDerivatives,
-        ReadBackAnyTextureFormat
+        ReadBackAnyTextureFormat,
+        PipelineCacheDataLoadSave
     };
 
     enum BeginFrameFlag {
@@ -1637,6 +1639,9 @@ public:
 
     bool isDeviceLost() const;
 
+    QByteArray pipelineCacheData();
+    void setPipelineCacheData(const QByteArray &data);
+
 protected:
     QRhi();
 
diff --git a/src/gui/rhi/qrhi_p_p.h b/src/gui/rhi/qrhi_p_p.h
index 44f10587ea..fc9c99af7e 100644
--- a/src/gui/rhi/qrhi_p_p.h
+++ b/src/gui/rhi/qrhi_p_p.h
@@ -171,6 +171,9 @@ public:
     virtual void releaseCachedResources() = 0;
     virtual bool isDeviceLost() const = 0;
 
+    virtual QByteArray pipelineCacheData() = 0;
+    virtual void setPipelineCacheData(const QByteArray &data) = 0;
+
     bool isCompressedFormat(QRhiTexture::Format format) const;
     void compressedFormatInfo(QRhiTexture::Format format, const QSize &size,
                               quint32 *bpl, quint32 *byteSize,
@@ -220,6 +223,12 @@ public:
     bool sanityCheckShaderResourceBindings(QRhiShaderResourceBindings *srb);
     void updateLayoutDesc(QRhiShaderResourceBindings *srb);
 
+    quint32 pipelineCacheRhiId() const
+    {
+        const quint32 ver = (QT_VERSION_MAJOR << 16) | (QT_VERSION_MINOR << 8) | (QT_VERSION_PATCH);
+        return (quint32(implType) << 24) | ver;
+    }
+
     QRhi *q;
 
     static const int MAX_SHADER_CACHE_ENTRIES = 128;
diff --git a/src/gui/rhi/qrhid3d11.cpp b/src/gui/rhi/qrhid3d11.cpp
index 207043fadd..fef911c22a 100644
--- a/src/gui/rhi/qrhid3d11.cpp
+++ b/src/gui/rhi/qrhid3d11.cpp
@@ -540,6 +540,8 @@ bool QRhiD3D11::isFeatureSupported(QRhi::Feature feature) const
         return true;
     case QRhi::ReadBackAnyTextureFormat:
         return true;
+    case QRhi::PipelineCacheDataLoadSave:
+        return false;
     default:
         Q_UNREACHABLE();
         return false;
@@ -610,6 +612,16 @@ bool QRhiD3D11::isDeviceLost() const
     return deviceLost;
 }
 
+QByteArray QRhiD3D11::pipelineCacheData()
+{
+    return QByteArray();
+}
+
+void QRhiD3D11::setPipelineCacheData(const QByteArray &data)
+{
+    Q_UNUSED(data);
+}
+
 QRhiRenderBuffer *QRhiD3D11::createRenderBuffer(QRhiRenderBuffer::Type type, const QSize &pixelSize,
                                                 int sampleCount, QRhiRenderBuffer::Flags flags,
                                                 QRhiTexture::Format backingFormatHint)
diff --git a/src/gui/rhi/qrhid3d11_p_p.h b/src/gui/rhi/qrhid3d11_p_p.h
index 115729a130..5eb866119f 100644
--- a/src/gui/rhi/qrhid3d11_p_p.h
+++ b/src/gui/rhi/qrhid3d11_p_p.h
@@ -680,6 +680,9 @@ public:
     void releaseCachedResources() override;
     bool isDeviceLost() const override;
 
+    QByteArray pipelineCacheData() override;
+    void setPipelineCacheData(const QByteArray &data) override;
+
     void enqueueSubresUpload(QD3D11Texture *texD, QD3D11CommandBuffer *cbD,
                              int layer, int level, const QRhiTextureSubresourceUploadDescription &subresDesc);
     void enqueueResourceUpdates(QRhiCommandBuffer *cb, QRhiResourceUpdateBatch *resourceUpdates);
diff --git a/src/gui/rhi/qrhigles2.cpp b/src/gui/rhi/qrhigles2.cpp
index 3259912c74..fac05f4947 100644
--- a/src/gui/rhi/qrhigles2.cpp
+++ b/src/gui/rhi/qrhigles2.cpp
@@ -328,6 +328,18 @@ QT_BEGIN_NAMESPACE
 #define GL_TEXTURE_CUBE_MAP_SEAMLESS      0x884F
 #endif
 
+#ifndef GL_CONTEXT_LOST
+#define GL_CONTEXT_LOST                   0x0507
+#endif
+
+#ifndef GL_PROGRAM_BINARY_LENGTH
+#define GL_PROGRAM_BINARY_LENGTH          0x8741
+#endif
+
+#ifndef GL_NUM_PROGRAM_BINARY_FORMATS
+#define GL_NUM_PROGRAM_BINARY_FORMATS     0x87FE
+#endif
+
 /*!
     Constructs a new QRhiGles2InitParams.
 
@@ -438,8 +450,8 @@ bool QRhiGles2::ensureContext(QSurface *surface) const
 
 bool QRhiGles2::create(QRhi::Flags flags)
 {
-    Q_UNUSED(flags);
     Q_ASSERT(fallbackSurface);
+    rhiFlags = flags;
 
     if (!importedContext) {
         ctx = new QOpenGLContext;
@@ -582,12 +594,24 @@ bool QRhiGles2::create(QRhi::Flags flags)
     caps.intAttributes = caps.ctxMajor >= 3; // 3.0 or ES 3.0
     caps.screenSpaceDerivatives = f->hasOpenGLExtension(QOpenGLExtensions::StandardDerivatives);
 
-    // TO DO: We could also check for ARB_texture_multisample but it is not
-    // currently in QOpenGLExtensions
-    // 3.0 or ES 3.1
-    caps.multisampledTexture = caps.gles
-            ? (caps.ctxMajor > 3 || (caps.ctxMajor >= 3 && caps.ctxMinor >= 1))
-            : (caps.ctxMajor >= 3);
+    if (caps.gles)
+        caps.multisampledTexture = caps.ctxMajor > 3 || (caps.ctxMajor == 3 && caps.ctxMinor >= 1); // ES 3.1
+    else
+        caps.multisampledTexture = caps.ctxMajor >= 3; // 3.0
+
+    // Program binary support: only the core stuff, do not bother with the old
+    // extensions like GL_OES_get_program_binary
+    if (caps.gles)
+        caps.programBinary = caps.ctxMajor >= 3; // ES 3.0
+    else
+        caps.programBinary = caps.ctxMajor > 4 || (caps.ctxMajor == 4 && caps.ctxMinor >= 1); // 4.1
+
+    if (caps.programBinary) {
+        GLint fmtCount = 0;
+        f->glGetIntegerv(GL_NUM_PROGRAM_BINARY_FORMATS, &fmtCount);
+        if (fmtCount < 1)
+            caps.programBinary = false;
+    }
 
     if (!caps.gles) {
         f->glEnable(GL_VERTEX_PROGRAM_POINT_SIZE);
@@ -966,6 +990,8 @@ bool QRhiGles2::isFeatureSupported(QRhi::Feature feature) const
         return caps.screenSpaceDerivatives;
     case QRhi::ReadBackAnyTextureFormat:
         return false;
+    case QRhi::PipelineCacheDataLoadSave:
+        return caps.programBinary;
     default:
         Q_UNREACHABLE();
         return false;
@@ -1035,6 +1061,8 @@ void QRhiGles2::releaseCachedResources()
         f->glDeleteShader(shader);
 
     m_shaderCache.clear();
+
+    m_pipelineCache.clear();
 }
 
 bool QRhiGles2::isDeviceLost() const
@@ -1042,6 +1070,137 @@ bool QRhiGles2::isDeviceLost() const
     return contextLost;
 }
 
+struct QGles2PipelineCacheDataHeader
+{
+    quint32 rhiId;
+    quint32 arch;
+    quint32 programBinaryCount;
+    quint32 dataSize;
+    char driver[240];
+};
+
+QByteArray QRhiGles2::pipelineCacheData()
+{
+    Q_STATIC_ASSERT(sizeof(QGles2PipelineCacheDataHeader) == 256);
+
+    if (m_pipelineCache.isEmpty())
+        return QByteArray();
+
+    QGles2PipelineCacheDataHeader header;
+    memset(&header, 0, sizeof(header));
+    header.rhiId = pipelineCacheRhiId();
+    header.arch = quint32(sizeof(void*));
+    header.programBinaryCount = m_pipelineCache.count();
+    const size_t driverStrLen = qMin(sizeof(header.driver) - 1, size_t(driverInfoStruct.deviceName.count()));
+    if (driverStrLen)
+        memcpy(header.driver, driverInfoStruct.deviceName.constData(), driverStrLen);
+    header.driver[driverStrLen] = '\0';
+
+    const size_t dataOffset = sizeof(header);
+    size_t dataSize = 0;
+    for (auto it = m_pipelineCache.cbegin(), end = m_pipelineCache.cend(); it != end; ++it) {
+        dataSize += sizeof(quint32) + it.key().size()
+                  + sizeof(quint32) + it->data.size()
+                  + sizeof(quint32);
+    }
+
+    QByteArray buf(dataOffset + dataSize, Qt::Uninitialized);
+    char *p = buf.data() + dataOffset;
+    for (auto it = m_pipelineCache.cbegin(), end = m_pipelineCache.cend(); it != end; ++it) {
+        const QByteArray key = it.key();
+        const QByteArray data = it->data;
+        const quint32 format = it->format;
+
+        quint32 i = key.size();
+        memcpy(p, &i, 4);
+        p += 4;
+        memcpy(p, key.constData(), key.size());
+        p += key.size();
+
+        i = data.size();
+        memcpy(p, &i, 4);
+        p += 4;
+        memcpy(p, data.constData(), data.size());
+        p += data.size();
+
+        memcpy(p, &format, 4);
+        p += 4;
+    }
+    Q_ASSERT(p == buf.data() + dataOffset + dataSize);
+
+    header.dataSize = quint32(dataSize);
+    memcpy(buf.data(), &header, sizeof(header));
+
+    return buf;
+}
+
+void QRhiGles2::setPipelineCacheData(const QByteArray &data)
+{
+    if (data.isEmpty())
+        return;
+
+    const size_t headerSize = sizeof(QGles2PipelineCacheDataHeader);
+    if (data.size() < qsizetype(headerSize)) {
+        qWarning("setPipelineCacheData: Invalid blob size (header incomplete)");
+        return;
+    }
+    const size_t dataOffset = headerSize;
+    QGles2PipelineCacheDataHeader header;
+    memcpy(&header, data.constData(), headerSize);
+
+    const quint32 rhiId = pipelineCacheRhiId();
+    if (header.rhiId != rhiId) {
+        qWarning("setPipelineCacheData: The data is for a different QRhi version or backend (%u, %u)",
+                 rhiId, header.rhiId);
+        return;
+    }
+    const quint32 arch = quint32(sizeof(void*));
+    if (header.arch != arch) {
+        qWarning("setPipelineCacheData: Architecture does not match (%u, %u)",
+                 arch, header.arch);
+        return;
+    }
+    if (header.programBinaryCount == 0)
+        return;
+
+    const size_t driverStrLen = qMin(sizeof(header.driver) - 1, size_t(driverInfoStruct.deviceName.count()));
+    if (strncmp(header.driver, driverInfoStruct.deviceName.constData(), driverStrLen)) {
+        qWarning("setPipelineCacheData: OpenGL vendor/renderer/version does not match");
+        return;
+    }
+
+    if (data.size() < qsizetype(dataOffset + header.dataSize)) {
+        qWarning("setPipelineCacheData: Invalid blob size (data incomplete)");
+        return;
+    }
+
+    m_pipelineCache.clear();
+
+    const char *p = data.constData() + dataOffset;
+    for (quint32 i = 0; i < header.programBinaryCount; ++i) {
+        quint32 len = 0;
+        memcpy(&len, p, 4);
+        p += 4;
+        QByteArray key(len, Qt::Uninitialized);
+        memcpy(key.data(), p, len);
+        p += len;
+
+        memcpy(&len, p, 4);
+        p += 4;
+        QByteArray data(len, Qt::Uninitialized);
+        memcpy(data.data(), p, len);
+        p += len;
+
+        quint32 format;
+        memcpy(&format, p, 4);
+        p += 4;
+
+        m_pipelineCache.insert(key, { format, data });
+    }
+
+    qCDebug(QRHI_LOG_INFO, "Seeded pipeline cache with %d program binaries", int(m_pipelineCache.count()));
+}
+
 QRhiRenderBuffer *QRhiGles2::createRenderBuffer(QRhiRenderBuffer::Type type, const QSize &pixelSize,
                                                 int sampleCount, QRhiRenderBuffer::Flags flags,
                                                 QRhiTexture::Format backingFormatHint)
@@ -3784,22 +3943,28 @@ static inline QShader::Stage toShaderStage(QRhiShaderStage::Type type)
     }
 }
 
-QRhiGles2::DiskCacheResult QRhiGles2::tryLoadFromDiskCache(const QRhiShaderStage *stages,
-                                                           int stageCount,
-                                                           GLuint program,
-                                                           const QVector<QShaderDescription::InOutVariable> &inputVars,
-                                                           QByteArray *cacheKey)
+QRhiGles2::ProgramCacheResult QRhiGles2::tryLoadFromDiskOrPipelineCache(const QRhiShaderStage *stages,
+                                                                        int stageCount,
+                                                                        GLuint program,
+                                                                        const QVector<QShaderDescription::InOutVariable> &inputVars,
+                                                                        QByteArray *cacheKey)
 {
-    QRhiGles2::DiskCacheResult result = QRhiGles2::DiskCacheMiss;
-    QByteArray diskCacheKey;
+    Q_ASSERT(cacheKey);
 
-    if (isProgramBinaryDiskCacheEnabled()) {
+    // the traditional QOpenGL disk cache since Qt 5.9
+    const bool legacyDiskCacheEnabled = isProgramBinaryDiskCacheEnabled();
+
+    // QRhi's own (set)PipelineCacheData()
+    const bool pipelineCacheEnabled = caps.programBinary && !m_pipelineCache.isEmpty();
+
+    // calculating the cache key based on the source code is common for both types of caches
+    if (legacyDiskCacheEnabled || pipelineCacheEnabled) {
         QOpenGLProgramBinaryCache::ProgramDesc binaryProgram;
         for (int i = 0; i < stageCount; ++i) {
             const QRhiShaderStage &stage(stages[i]);
             QByteArray source = shaderSource(stage, nullptr);
             if (source.isEmpty())
-                return QRhiGles2::DiskCacheError;
+                return QRhiGles2::ProgramCacheError;
 
             if (stage.type() == QRhiShaderStage::Vertex) {
                 // Now add something to the key that indicates the vertex input locations.
@@ -3832,30 +3997,71 @@ QRhiGles2::DiskCacheResult QRhiGles2::tryLoadFromDiskCache(const QRhiShaderStage
             binaryProgram.shaders.append(QOpenGLProgramBinaryCache::ShaderDesc(toShaderStage(stage.type()), source));
         }
 
-        diskCacheKey = binaryProgram.cacheKey();
+        *cacheKey = binaryProgram.cacheKey();
+
+        // Try our pipeline cache simulation first, if it got seeded with
+        // setPipelineCacheData and there's a hit, then no need to go to the
+        // filesystem at all.
+        if (pipelineCacheEnabled) {
+            auto it = m_pipelineCache.constFind(*cacheKey);
+            if (it != m_pipelineCache.constEnd()) {
+                GLenum err;
+                for ( ; ; ) {
+                    err = f->glGetError();
+                    if (err == GL_NO_ERROR || err == GL_CONTEXT_LOST)
+                        break;
+                }
+                f->glProgramBinary(program, it->format, it->data.constData(), it->data.size());
+                err = f->glGetError();
+                if (err == GL_NO_ERROR) {
+                    GLint linkStatus = 0;
+                    f->glGetProgramiv(program, GL_LINK_STATUS, &linkStatus);
+                    if (linkStatus == GL_TRUE)
+                        return QRhiGles2::ProgramCacheHit;
+                }
+            }
+        }
 
-        if (qrhi_programBinaryCache()->load(diskCacheKey, program)) {
+        if (legacyDiskCacheEnabled && qrhi_programBinaryCache()->load(*cacheKey, program)) {
+            // use the logging category QOpenGLShaderProgram would
             qCDebug(lcOpenGLProgramDiskCache, "Program binary received from cache, program %u, key %s",
-                    program, diskCacheKey.constData());
-            result = QRhiGles2::DiskCacheHit;
+                    program, cacheKey->constData());
+            return QRhiGles2::ProgramCacheHit;
         }
     }
 
-    if (cacheKey)
-        *cacheKey = diskCacheKey;
-
-    return result;
+    return QRhiGles2::ProgramCacheMiss;
 }
 
 void QRhiGles2::trySaveToDiskCache(GLuint program, const QByteArray &cacheKey)
 {
+    // This is only for the traditional QOpenGL disk cache since Qt 5.9.
+
     if (isProgramBinaryDiskCacheEnabled()) {
+        // use the logging category QOpenGLShaderProgram would
         qCDebug(lcOpenGLProgramDiskCache, "Saving program binary, program %u, key %s",
                 program, cacheKey.constData());
         qrhi_programBinaryCache()->save(cacheKey, program);
     }
 }
 
+void QRhiGles2::trySaveToPipelineCache(GLuint program, const QByteArray &cacheKey, bool force)
+{
+    // This handles our own simulated "pipeline cache". (specific to QRhi, not
+    // shared with legacy QOpenGL* stuff)
+
+    if (caps.programBinary && (force || !m_pipelineCache.contains(cacheKey))) {
+        GLint blobSize = 0;
+        f->glGetProgramiv(program, GL_PROGRAM_BINARY_LENGTH, &blobSize);
+        QByteArray blob(blobSize, Qt::Uninitialized);
+        GLint outSize = 0;
+        GLenum binaryFormat = 0;
+        f->glGetProgramBinary(program, blobSize, &outSize, &binaryFormat, blob.data());
+        if (blobSize == outSize)
+            m_pipelineCache.insert(cacheKey, { binaryFormat, blob });
+    }
+}
+
 QGles2Buffer::QGles2Buffer(QRhiImplementation *rhi, Type type, UsageFlags usage, int size)
     : QRhiBuffer(rhi, type, usage, size)
 {
@@ -4585,16 +4791,16 @@ bool QGles2GraphicsPipeline::create()
             fsDesc = shaderStage.shader().description();
     }
 
-    QByteArray diskCacheKey;
-    QRhiGles2::DiskCacheResult diskCacheResult = rhiD->tryLoadFromDiskCache(m_shaderStages.constData(),
-                                                                            m_shaderStages.count(),
-                                                                            program,
-                                                                            vsDesc.inputVariables(),
-                                                                            &diskCacheKey);
-    if (diskCacheResult == QRhiGles2::DiskCacheError)
+    QByteArray cacheKey;
+    QRhiGles2::ProgramCacheResult cacheResult = rhiD->tryLoadFromDiskOrPipelineCache(m_shaderStages.constData(),
+                                                                                     m_shaderStages.count(),
+                                                                                     program,
+                                                                                     vsDesc.inputVariables(),
+                                                                                     &cacheKey);
+    if (cacheResult == QRhiGles2::ProgramCacheError)
         return false;
 
-    if (diskCacheResult == QRhiGles2::DiskCacheMiss) {
+    if (cacheResult == QRhiGles2::ProgramCacheMiss) {
         for (const QRhiShaderStage &shaderStage : qAsConst(m_shaderStages)) {
             if (shaderStage.type() == QRhiShaderStage::Vertex) {
                 if (!rhiD->compileShader(program, shaderStage, nullptr))
@@ -4612,7 +4818,22 @@ bool QGles2GraphicsPipeline::create()
         if (!rhiD->linkProgram(program))
             return false;
 
-        rhiD->trySaveToDiskCache(program, diskCacheKey);
+        if (rhiD->rhiFlags.testFlag(QRhi::EnablePipelineCacheDataSave)) {
+            // force replacing existing cache entry (if there is one, then
+            // something is wrong with it, as there was no hit)
+            rhiD->trySaveToPipelineCache(program, cacheKey, true);
+        } else {
+            // legacy QOpenGLShaderProgram style behavior: the "pipeline cache"
+            // was not enabled, so instead store to the Qt 5 disk cache
+            rhiD->trySaveToDiskCache(program, cacheKey);
+        }
+    } else {
+        Q_ASSERT(cacheResult == QRhiGles2::ProgramCacheHit);
+        if (rhiD->rhiFlags.testFlag(QRhi::EnablePipelineCacheDataSave)) {
+            // just so that it ends up in the pipeline cache also when the hit was
+            // from the disk cache
+            rhiD->trySaveToPipelineCache(program, cacheKey);
+        }
     }
 
     // Use the same work area for the vertex & fragment stages, thus ensuring
@@ -4688,19 +4909,34 @@ bool QGles2ComputePipeline::create()
     const QShaderDescription csDesc = m_shaderStage.shader().description();
     program = rhiD->f->glCreateProgram();
 
-    QByteArray diskCacheKey;
-    QRhiGles2::DiskCacheResult diskCacheResult = rhiD->tryLoadFromDiskCache(&m_shaderStage, 1, program, {}, &diskCacheKey);
-    if (diskCacheResult == QRhiGles2::DiskCacheError)
+    QByteArray cacheKey;
+    QRhiGles2::ProgramCacheResult cacheResult = rhiD->tryLoadFromDiskOrPipelineCache(&m_shaderStage, 1, program, {}, &cacheKey);
+    if (cacheResult == QRhiGles2::ProgramCacheError)
         return false;
 
-    if (diskCacheResult == QRhiGles2::DiskCacheMiss) {
+    if (cacheResult == QRhiGles2::ProgramCacheMiss) {
         if (!rhiD->compileShader(program, m_shaderStage, nullptr))
             return false;
 
         if (!rhiD->linkProgram(program))
             return false;
 
-        rhiD->trySaveToDiskCache(program, diskCacheKey);
+        if (rhiD->rhiFlags.testFlag(QRhi::EnablePipelineCacheDataSave)) {
+            // force replacing existing cache entry (if there is one, then
+            // something is wrong with it, as there was no hit)
+            rhiD->trySaveToPipelineCache(program, cacheKey, true);
+        } else {
+            // legacy QOpenGLShaderProgram style behavior: the "pipeline cache"
+            // was not enabled, so instead store to the Qt 5 disk cache
+            rhiD->trySaveToDiskCache(program, cacheKey);
+        }
+    } else {
+        Q_ASSERT(cacheResult == QRhiGles2::ProgramCacheHit);
+        if (rhiD->rhiFlags.testFlag(QRhi::EnablePipelineCacheDataSave)) {
+            // just so that it ends up in the pipeline cache also when the hit was
+            // from the disk cache
+            rhiD->trySaveToPipelineCache(program, cacheKey);
+        }
     }
 
     QSet<int> activeUniformLocations;
diff --git a/src/gui/rhi/qrhigles2_p_p.h b/src/gui/rhi/qrhigles2_p_p.h
index a29dfa289a..dff2de3038 100644
--- a/src/gui/rhi/qrhigles2_p_p.h
+++ b/src/gui/rhi/qrhigles2_p_p.h
@@ -816,6 +816,9 @@ public:
     void releaseCachedResources() override;
     bool isDeviceLost() const override;
 
+    QByteArray pipelineCacheData() override;
+    void setPipelineCacheData(const QByteArray &data) override;
+
     bool ensureContext(QSurface *surface = nullptr) const;
     void executeDeferredReleases();
     void trackedBufferBarrier(QGles2CommandBuffer *cbD, QGles2Buffer *bufD, QGles2Buffer::Access access);
@@ -855,18 +858,20 @@ public:
                         QGles2SamplerDescriptionVector *dst);
     bool isProgramBinaryDiskCacheEnabled() const;
 
-    enum DiskCacheResult {
-        DiskCacheHit,
-        DiskCacheMiss,
-        DiskCacheError
+    enum ProgramCacheResult {
+        ProgramCacheHit,
+        ProgramCacheMiss,
+        ProgramCacheError
     };
-    DiskCacheResult tryLoadFromDiskCache(const QRhiShaderStage *stages,
-                                         int stageCount,
-                                         GLuint program,
-                                         const QVector<QShaderDescription::InOutVariable> &inputVars,
-                                         QByteArray *cacheKey);
+    ProgramCacheResult tryLoadFromDiskOrPipelineCache(const QRhiShaderStage *stages,
+                                                      int stageCount,
+                                                      GLuint program,
+                                                      const QVector<QShaderDescription::InOutVariable> &inputVars,
+                                                      QByteArray *cacheKey);
     void trySaveToDiskCache(GLuint program, const QByteArray &cacheKey);
+    void trySaveToPipelineCache(GLuint program, const QByteArray &cacheKey, bool force = false);
 
+    QRhi::Flags rhiFlags;
     QOpenGLContext *ctx = nullptr;
     bool importedContext = false;
     QSurfaceFormat requestedFormat;
@@ -914,7 +919,8 @@ public:
               nonBaseLevelFramebufferTexture(false),
               texelFetch(false),
               intAttributes(true),
-              screenSpaceDerivatives(false)
+              screenSpaceDerivatives(false),
+              programBinary(false)
         { }
         int ctxMajor;
         int ctxMinor;
@@ -956,6 +962,7 @@ public:
         uint texelFetch : 1;
         uint intAttributes : 1;
         uint screenSpaceDerivatives : 1;
+        uint programBinary : 1;
     } caps;
     QGles2SwapChain *currentSwapChain = nullptr;
     QList<GLint> supportedCompressedFormats;
@@ -1001,6 +1008,12 @@ public:
     } ofr;
 
     QHash<QRhiShaderStage, uint> m_shaderCache;
+
+    struct PipelineCacheData {
+        quint32 format;
+        QByteArray data;
+    };
+    QHash<QByteArray, PipelineCacheData> m_pipelineCache;
 };
 
 Q_DECLARE_TYPEINFO(QRhiGles2::DeferredReleaseEntry, Q_RELOCATABLE_TYPE);
diff --git a/src/gui/rhi/qrhimetal.mm b/src/gui/rhi/qrhimetal.mm
index 63c09b0173..9eb0ba75b5 100644
--- a/src/gui/rhi/qrhimetal.mm
+++ b/src/gui/rhi/qrhimetal.mm
@@ -597,6 +597,8 @@ bool QRhiMetal::isFeatureSupported(QRhi::Feature feature) const
         return true;
     case QRhi::ReadBackAnyTextureFormat:
         return true;
+    case QRhi::PipelineCacheDataLoadSave:
+        return false;
     default:
         Q_UNREACHABLE();
         return false;
@@ -670,6 +672,16 @@ bool QRhiMetal::isDeviceLost() const
     return false;
 }
 
+QByteArray QRhiMetal::pipelineCacheData()
+{
+    return QByteArray();
+}
+
+void QRhiMetal::setPipelineCacheData(const QByteArray &data)
+{
+    Q_UNUSED(data);
+}
+
 QRhiRenderBuffer *QRhiMetal::createRenderBuffer(QRhiRenderBuffer::Type type, const QSize &pixelSize,
                                                 int sampleCount, QRhiRenderBuffer::Flags flags,
                                                 QRhiTexture::Format backingFormatHint)
diff --git a/src/gui/rhi/qrhimetal_p_p.h b/src/gui/rhi/qrhimetal_p_p.h
index dc4eb0c3fa..4971ecc18c 100644
--- a/src/gui/rhi/qrhimetal_p_p.h
+++ b/src/gui/rhi/qrhimetal_p_p.h
@@ -444,6 +444,9 @@ public:
     void releaseCachedResources() override;
     bool isDeviceLost() const override;
 
+    QByteArray pipelineCacheData() override;
+    void setPipelineCacheData(const QByteArray &data) override;
+
     void executeDeferredReleases(bool forced = false);
     void finishActiveReadbacks(bool forced = false);
     qsizetype subresUploadByteSize(const QRhiTextureSubresourceUploadDescription &subresDesc) const;
diff --git a/src/gui/rhi/qrhinull.cpp b/src/gui/rhi/qrhinull.cpp
index ef584439b4..32ba27fcf8 100644
--- a/src/gui/rhi/qrhinull.cpp
+++ b/src/gui/rhi/qrhinull.cpp
@@ -201,6 +201,16 @@ bool QRhiNull::isDeviceLost() const
     return false;
 }
 
+QByteArray QRhiNull::pipelineCacheData()
+{
+    return QByteArray();
+}
+
+void QRhiNull::setPipelineCacheData(const QByteArray &data)
+{
+    Q_UNUSED(data);
+}
+
 QRhiRenderBuffer *QRhiNull::createRenderBuffer(QRhiRenderBuffer::Type type, const QSize &pixelSize,
                                                int sampleCount, QRhiRenderBuffer::Flags flags,
                                                QRhiTexture::Format backingFormatHint)
diff --git a/src/gui/rhi/qrhinull_p_p.h b/src/gui/rhi/qrhinull_p_p.h
index 2ceabc803e..273aee9c3e 100644
--- a/src/gui/rhi/qrhinull_p_p.h
+++ b/src/gui/rhi/qrhinull_p_p.h
@@ -301,6 +301,9 @@ public:
     void releaseCachedResources() override;
     bool isDeviceLost() const override;
 
+    QByteArray pipelineCacheData() override;
+    void setPipelineCacheData(const QByteArray &data) override;
+
     void simulateTextureUpload(const QRhiResourceUpdateBatchPrivate::TextureOp &u);
     void simulateTextureCopy(const QRhiResourceUpdateBatchPrivate::TextureOp &u);
     void simulateTextureGenMips(const QRhiResourceUpdateBatchPrivate::TextureOp &u);
diff --git a/src/gui/rhi/qrhivulkan.cpp b/src/gui/rhi/qrhivulkan.cpp
index a2e5579165..34b81671ca 100644
--- a/src/gui/rhi/qrhivulkan.cpp
+++ b/src/gui/rhi/qrhivulkan.cpp
@@ -412,9 +412,7 @@ static inline QRhiDriverInfo::DeviceType toRhiDeviceType(VkPhysicalDeviceType ty
 
 bool QRhiVulkan::create(QRhi::Flags flags)
 {
-    Q_UNUSED(flags);
     Q_ASSERT(inst);
-
     if (!inst->isValid()) {
         qWarning("Vulkan instance is not valid");
         return false;
@@ -424,6 +422,8 @@ bool QRhiVulkan::create(QRhi::Flags flags)
 
     f = inst->functions();
 
+    rhiFlags = flags;
+
     QList<VkQueueFamilyProperties> queueFamilyProps;
     auto queryQueueFamilyProps = [this, &queueFamilyProps] {
         uint32_t queueCount = 0;
@@ -2632,7 +2632,7 @@ VkShaderModule QRhiVulkan::createShader(const QByteArray &spirv)
     return shaderModule;
 }
 
-bool QRhiVulkan::ensurePipelineCache()
+bool QRhiVulkan::ensurePipelineCache(const void *initialData, size_t initialDataSize)
 {
     if (pipelineCache)
         return true;
@@ -2640,6 +2640,8 @@ bool QRhiVulkan::ensurePipelineCache()
     VkPipelineCacheCreateInfo pipelineCacheInfo;
     memset(&pipelineCacheInfo, 0, sizeof(pipelineCacheInfo));
     pipelineCacheInfo.sType = VK_STRUCTURE_TYPE_PIPELINE_CACHE_CREATE_INFO;
+    pipelineCacheInfo.initialDataSize = initialDataSize;
+    pipelineCacheInfo.pInitialData = initialData;
     VkResult err = df->vkCreatePipelineCache(dev, &pipelineCacheInfo, nullptr, &pipelineCache);
     if (err != VK_SUCCESS) {
         qWarning("Failed to create pipeline cache: %d", err);
@@ -4236,6 +4238,8 @@ bool QRhiVulkan::isFeatureSupported(QRhi::Feature feature) const
         return true;
     case QRhi::ReadBackAnyTextureFormat:
         return true;
+    case QRhi::PipelineCacheDataLoadSave:
+        return true;
     default:
         Q_UNREACHABLE();
         return false;
@@ -4311,6 +4315,129 @@ bool QRhiVulkan::isDeviceLost() const
     return deviceLost;
 }
 
+struct QVkPipelineCacheDataHeader
+{
+    quint32 rhiId;
+    quint32 arch;
+    quint32 driverVersion;
+    quint32 vendorId;
+    quint32 deviceId;
+    quint32 dataSize;
+    quint32 uuidSize;
+    quint32 reserved;
+};
+
+QByteArray QRhiVulkan::pipelineCacheData()
+{
+    Q_STATIC_ASSERT(sizeof(QVkPipelineCacheDataHeader) == 32);
+
+    QByteArray data;
+    if (!pipelineCache || !rhiFlags.testFlag(QRhi::EnablePipelineCacheDataSave))
+        return data;
+
+    size_t dataSize = 0;
+    VkResult err = df->vkGetPipelineCacheData(dev, pipelineCache, &dataSize, nullptr);
+    if (err != VK_SUCCESS) {
+        qWarning("Failed to get pipeline cache data size: %d", err);
+        return QByteArray();
+    }
+    const size_t headerSize = sizeof(QVkPipelineCacheDataHeader);
+    const size_t dataOffset = headerSize + VK_UUID_SIZE;
+    data.resize(dataOffset + dataSize);
+    err = df->vkGetPipelineCacheData(dev, pipelineCache, &dataSize, data.data() + dataOffset);
+    if (err != VK_SUCCESS) {
+        qWarning("Failed to get pipeline cache data of %d bytes: %d", int(dataSize), err);
+        return QByteArray();
+    }
+
+    QVkPipelineCacheDataHeader header;
+    header.rhiId = pipelineCacheRhiId();
+    header.arch = quint32(sizeof(void*));
+    header.driverVersion = physDevProperties.driverVersion;
+    header.vendorId = physDevProperties.vendorID;
+    header.deviceId = physDevProperties.deviceID;
+    header.dataSize = quint32(dataSize);
+    header.uuidSize = VK_UUID_SIZE;
+    memcpy(data.data(), &header, headerSize);
+    memcpy(data.data() + headerSize, physDevProperties.pipelineCacheUUID, VK_UUID_SIZE);
+
+    return data;
+}
+
+void QRhiVulkan::setPipelineCacheData(const QByteArray &data)
+{
+    if (data.isEmpty())
+        return;
+
+    const size_t headerSize = sizeof(QVkPipelineCacheDataHeader);
+    if (data.size() < qsizetype(headerSize)) {
+        qWarning("setPipelineCacheData: Invalid blob size");
+        return;
+    }
+    QVkPipelineCacheDataHeader header;
+    memcpy(&header, data.constData(), headerSize);
+
+    const quint32 rhiId = pipelineCacheRhiId();
+    if (header.rhiId != rhiId) {
+        qWarning("setPipelineCacheData: The data is for a different QRhi version or backend (%u, %u)",
+                 rhiId, header.rhiId);
+        return;
+    }
+    const quint32 arch = quint32(sizeof(void*));
+    if (header.arch != arch) {
+        qWarning("setPipelineCacheData: Architecture does not match (%u, %u)",
+                 arch, header.arch);
+        return;
+    }
+    if (header.driverVersion != physDevProperties.driverVersion) {
+        qWarning("setPipelineCacheData: driverVersion does not match (%u, %u)",
+                 physDevProperties.driverVersion, header.driverVersion);
+        return;
+    }
+    if (header.vendorId != physDevProperties.vendorID) {
+        qWarning("setPipelineCacheData: vendorID does not match (%u, %u)",
+                 physDevProperties.vendorID, header.vendorId);
+        return;
+    }
+    if (header.deviceId != physDevProperties.deviceID) {
+        qWarning("setPipelineCacheData: deviceID does not match (%u, %u)",
+                 physDevProperties.deviceID, header.deviceId);
+        return;
+    }
+    if (header.uuidSize != VK_UUID_SIZE) {
+        qWarning("setPipelineCacheData: VK_UUID_SIZE does not match (%u, %u)",
+                 quint32(VK_UUID_SIZE), header.uuidSize);
+        return;
+    }
+
+    if (data.size() < qsizetype(headerSize + VK_UUID_SIZE)) {
+        qWarning("setPipelineCacheData: Invalid blob, no uuid");
+        return;
+    }
+    if (memcmp(data.constData() + headerSize, physDevProperties.pipelineCacheUUID, VK_UUID_SIZE)) {
+        qWarning("setPipelineCacheData: pipelineCacheUUID does not match");
+        return;
+    }
+
+    const size_t dataOffset = headerSize + VK_UUID_SIZE;
+    if (data.size() < qsizetype(dataOffset + header.dataSize)) {
+        qWarning("setPipelineCacheData: Invalid blob, data missing");
+        return;
+    }
+
+    if (pipelineCache) {
+        df->vkDestroyPipelineCache(dev, pipelineCache, nullptr);
+        pipelineCache = VK_NULL_HANDLE;
+    }
+
+    if (ensurePipelineCache(data.constData() + dataOffset, header.dataSize)) {
+        qCDebug(QRHI_LOG_INFO, "Created pipeline cache with initial data of %d bytes",
+                int(header.dataSize));
+    } else {
+        qWarning("Failed to create pipeline cache with initial data specified");
+    }
+}
+
 QRhiRenderBuffer *QRhiVulkan::createRenderBuffer(QRhiRenderBuffer::Type type, const QSize &pixelSize,
                                                  int sampleCount, QRhiRenderBuffer::Flags flags,
                                                  QRhiTexture::Format backingFormatHint)
diff --git a/src/gui/rhi/qrhivulkan_p_p.h b/src/gui/rhi/qrhivulkan_p_p.h
index bee6e98d61..f54439b5f2 100644
--- a/src/gui/rhi/qrhivulkan_p_p.h
+++ b/src/gui/rhi/qrhivulkan_p_p.h
@@ -759,6 +759,9 @@ public:
     void releaseCachedResources() override;
     bool isDeviceLost() const override;
 
+    QByteArray pipelineCacheData() override;
+    void setPipelineCacheData(const QByteArray &data) override;
+
     VkResult createDescriptorPool(VkDescriptorPool *pool);
     bool allocateDescriptorSet(VkDescriptorSetAllocateInfo *allocInfo, VkDescriptorSet *result, int *resultPoolIndex);
     uint32_t chooseTransientImageMemType(VkImage img, uint32_t startIndex);
@@ -782,7 +785,7 @@ public:
                                    bool preserveDs,
                                    QRhiRenderBuffer *depthStencilBuffer,
                                    QRhiTexture *depthTexture);
-    bool ensurePipelineCache();
+    bool ensurePipelineCache(const void *initialData = nullptr, size_t initialDataSize = 0);
     VkShaderModule createShader(const QByteArray &spirv);
 
     void prepareNewFrame(QRhiCommandBuffer *cb);
@@ -847,6 +850,7 @@ public:
     QVkAllocator allocator = nullptr;
     QVulkanFunctions *f = nullptr;
     QVulkanDeviceFunctions *df = nullptr;
+    QRhi::Flags rhiFlags;
     VkPhysicalDeviceFeatures physDevFeatures;
     VkPhysicalDeviceProperties physDevProperties;
     VkDeviceSize ubufAlign;