rhi: Add the basic infrastructure for tessellation support

...but this will only be supported with Vulkan and OpenGL 4.0+ and
OpenGL ES 3.2+ for the time being.

Taking the Vulkan model as our standard, the situation is the
following:

- Vulkan is ok, qsb secretly accepts .tesc and .tese files as input
  already (plus QShader already has the necessary plumbing when it
  comes to enums and such) To switch the tessellation domain origin to
  bottom left we require Vulkan 1.1 (don't bother with
  VK_KHR_maintenance2 on top of 1.0 at this point since 1.1 or 1.2
  implementations should be common by now). The change is essential to
  allow the same evaluation shader to work with both OpenGL and
  Vulkan: this way we can use the same shader source, declaring the
  tessellation winding order as CCW, with both APIs.

- OpenGL 4.0 and OpenGL ES 3.2 (or ES 3.1 with the Android extension
  pack, but we won't bother with checking that for now) can be made
  working without much complications, though we need to be careful
  when it comes to gathering and setting uniforms so that we do not
  leave the new tessellation stages out. We will stick to the Vulkan
  model in the sense that the inner and outer tessellation levels must
  be specified from the control shader, and cannot be specified from
  the host side, even though OpenGL would allow this. (basically the
  same story as with point size in vertex shaders)

- D3D11 would be no problem API-wise, and we could likely implement
  the support for hull and domain shader stages in the backend, but
  SPIRV-Cross does not support translating tessellation shaders to
  HLSL.  Attempting to feed in a .tesc or .tese file to qsb with
  --hlsl specified will always fail. One issue here is how hull
  shaders are structured, with the patchconstantfunc attribute
  specifying a separate function computing the patch constant
  data. With GLSL there is a single entry point in the tessellation
  control shader, which then performs both the calculations on the
  control points as well as the constant data (such as, the inner and
  outer tessellation factors).  One option here is to inject
  handwritten HLSL shaders in the .qsb files using qsb's replace (-r)
  mode, but this is not exactly a viable universal solution.

- Metal uses a different tessellation pipeline involving compute
  shaders. This needs more investigation but probably not something we
  can prioritize in practice. SPIRV-Cross does support this,
  generating a compute shader for control and a (post-)vertex shader
  for evaluation, presumably in order to enable MoltenVK to function
  when it comes to tessellation, but it is not clear yet how usable
  this is for us.

Change-Id: Ic953c63850bda5bc912c7ac354425041b43157ef
Reviewed-by: Andy Nichols <andy.nichols@qt.io>

12 files changed, 245 insertions, 0 deletions

diff --git a/tests/manual/rhi/CMakeLists.txt b/tests/manual/rhi/CMakeLists.txt
index 9e53e3eedd..5b756dbee3 100644
--- a/tests/manual/rhi/CMakeLists.txt
+++ b/tests/manual/rhi/CMakeLists.txt
@@ -23,3 +23,4 @@ add_subdirectory(instancing)
 add_subdirectory(noninstanced)
 add_subdirectory(tex3d)
 add_subdirectory(texturearray)
+add_subdirectory(tessellation)
diff --git a/tests/manual/rhi/tessellation/CMakeLists.txt b/tests/manual/rhi/tessellation/CMakeLists.txt
new file mode 100644
index 0000000000..74752dbd96
--- /dev/null
+++ b/tests/manual/rhi/tessellation/CMakeLists.txt
@@ -0,0 +1,22 @@
+qt_internal_add_manual_test(tessellation
+    GUI
+    SOURCES
+        tessellation.cpp
+    PUBLIC_LIBRARIES
+        Qt::Gui
+        Qt::GuiPrivate
+)
+
+set(tessellation_resource_files
+    "test.vert.qsb"
+    "test.tesc.qsb"
+    "test.tese.qsb"
+    "test.frag.qsb"
+)
+
+qt_internal_add_resource(tessellation "tessellation"
+    PREFIX
+        "/"
+    FILES
+        ${tessellation_resource_files}
+)
diff --git a/tests/manual/rhi/tessellation/buildshaders.bat b/tests/manual/rhi/tessellation/buildshaders.bat
new file mode 100644
index 0000000000..c44916067d
--- /dev/null
+++ b/tests/manual/rhi/tessellation/buildshaders.bat
@@ -0,0 +1,4 @@
+qsb --glsl 320es,410 test.vert -o test.vert.qsb
+qsb --glsl 320es,410 test.tesc -o test.tesc.qsb
+qsb --glsl 320es,410 test.tese -o test.tese.qsb
+qsb --glsl 320es,410 test.frag -o test.frag.qsb
diff --git a/tests/manual/rhi/tessellation/tessellation.cpp b/tests/manual/rhi/tessellation/tessellation.cpp
new file mode 100644
index 0000000000..e8a031a1cf
--- /dev/null
+++ b/tests/manual/rhi/tessellation/tessellation.cpp
@@ -0,0 +1,155 @@
+/****************************************************************************
+**
+** Copyright (C) 2021 The Qt Company Ltd.
+** Contact: https://www.qt.io/licensing/
+**
+** This file is part of the examples of the Qt Toolkit.
+**
+** $QT_BEGIN_LICENSE:BSD$
+** Commercial License Usage
+** Licensees holding valid commercial Qt licenses may use this file in
+** accordance with the commercial license agreement provided with the
+** Software or, alternatively, in accordance with the terms contained in
+** a written agreement between you and The Qt Company. For licensing terms
+** and conditions see https://www.qt.io/terms-conditions. For further
+** information use the contact form at https://www.qt.io/contact-us.
+**
+** BSD License Usage
+** Alternatively, you may use this file under the terms of the BSD license
+** as follows:
+**
+** "Redistribution and use in source and binary forms, with or without
+** modification, are permitted provided that the following conditions are
+** met:
+**   * Redistributions of source code must retain the above copyright
+**     notice, this list of conditions and the following disclaimer.
+**   * Redistributions in binary form must reproduce the above copyright
+**     notice, this list of conditions and the following disclaimer in
+**     the documentation and/or other materials provided with the
+**     distribution.
+**   * Neither the name of The Qt Company Ltd nor the names of its
+**     contributors may be used to endorse or promote products derived
+**     from this software without specific prior written permission.
+**
+**
+** THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+** "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+** LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+** A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+** OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+** SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+** LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+** DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+** THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+** (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+** OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE."
+**
+** $QT_END_LICENSE$
+**
+****************************************************************************/
+
+#include "../shared/examplefw.h"
+
+static const float tri[] = {
+    0.0f, 0.5f, 0.0f,     0.0f, 0.0f, 1.0f,
+    -0.5f, -0.5f, 0.0f,   1.0f, 0.0f, 0.0f,
+    0.5f, -0.5f, 0.0f,    0.0f, 1.0f, 0.0f,
+};
+
+struct {
+    QVector<QRhiResource *> releasePool;
+    QRhiBuffer *vbuf = nullptr;
+    QRhiBuffer *ubuf = nullptr;
+    QRhiShaderResourceBindings *srb = nullptr;
+    QRhiGraphicsPipeline *ps = nullptr;
+    QRhiResourceUpdateBatch *initialUpdates = nullptr;
+    QMatrix4x4 winProj;
+    float time = 0.0f;
+} d;
+
+void Window::customInit()
+{
+    if (!m_r->isFeatureSupported(QRhi::Tessellation))
+        qFatal("Tessellation is not supported");
+
+    d.vbuf = m_r->newBuffer(QRhiBuffer::Immutable, QRhiBuffer::VertexBuffer, sizeof(tri));
+    d.vbuf->create();
+    d.releasePool << d.vbuf;
+
+    d.ubuf = m_r->newBuffer(QRhiBuffer::Dynamic, QRhiBuffer::UniformBuffer, 64 + 4 + 4);
+    d.ubuf->create();
+    d.releasePool << d.ubuf;
+
+    d.srb = m_r->newShaderResourceBindings();
+    d.releasePool << d.srb;
+    const QRhiShaderResourceBinding::StageFlags tese = QRhiShaderResourceBinding::TessellationEvaluationStage;
+    d.srb->setBindings({ QRhiShaderResourceBinding::uniformBuffer(0, tese, d.ubuf) });
+    d.srb->create();
+
+    d.ps = m_r->newGraphicsPipeline();
+    d.releasePool << d.ps;
+
+    d.ps->setTopology(QRhiGraphicsPipeline::Patches);
+    d.ps->setPatchControlPointCount(3);
+
+    d.ps->setShaderStages({
+        { QRhiShaderStage::Vertex, getShader(QLatin1String(":/test.vert.qsb")) },
+        { QRhiShaderStage::TessellationControl, getShader(QLatin1String(":/test.tesc.qsb")) },
+        { QRhiShaderStage::TessellationEvaluation, getShader(QLatin1String(":/test.tese.qsb")) },
+        { QRhiShaderStage::Fragment, getShader(QLatin1String(":/test.frag.qsb")) }
+    });
+
+    d.ps->setCullMode(QRhiGraphicsPipeline::Back);
+    d.ps->setDepthTest(true);
+    d.ps->setDepthWrite(true);
+    QRhiVertexInputLayout inputLayout;
+    inputLayout.setBindings({
+        { 6 * sizeof(float) }
+    });
+    inputLayout.setAttributes({
+        { 0, 0, QRhiVertexInputAttribute::Float3, 0 },
+        { 0, 1, QRhiVertexInputAttribute::Float3, 3 * sizeof(float) }
+    });
+    d.ps->setVertexInputLayout(inputLayout);
+    d.ps->setShaderResourceBindings(d.srb);
+    d.ps->setRenderPassDescriptor(m_rp);
+    d.ps->create();
+
+    d.initialUpdates = m_r->nextResourceUpdateBatch();
+    d.initialUpdates->uploadStaticBuffer(d.vbuf, tri);
+    const float amplitude = 0.5f;
+    d.initialUpdates->updateDynamicBuffer(d.ubuf, 68, 4, &amplitude);
+}
+
+void Window::customRelease()
+{
+    qDeleteAll(d.releasePool);
+    d.releasePool.clear();
+}
+
+void Window::customRender()
+{
+    const QSize outputSizeInPixels = m_sc->currentPixelSize();
+    QRhiCommandBuffer *cb = m_sc->currentFrameCommandBuffer();
+    QRhiResourceUpdateBatch *u = m_r->nextResourceUpdateBatch();
+    if (d.initialUpdates) {
+        u->merge(d.initialUpdates);
+        d.initialUpdates->release();
+        d.initialUpdates = nullptr;
+    }
+    if (d.winProj != m_proj) {
+        d.winProj = m_proj;
+        u->updateDynamicBuffer(d.ubuf, 0, 64, d.winProj.constData());
+    }
+    u->updateDynamicBuffer(d.ubuf, 64, 4, &d.time);
+    d.time += 0.1f;
+
+    cb->beginPass(m_sc->currentFrameRenderTarget(), m_clearColor, { 1.0f, 0 }, u);
+    cb->setGraphicsPipeline(d.ps);
+    cb->setViewport({ 0, 0, float(outputSizeInPixels.width()), float(outputSizeInPixels.height()) });
+    cb->setShaderResources();
+    QRhiCommandBuffer::VertexInput vbufBinding(d.vbuf, 0);
+    cb->setVertexInput(0, 1, &vbufBinding);
+    cb->draw(3);
+    cb->endPass();
+}
diff --git a/tests/manual/rhi/tessellation/test.frag b/tests/manual/rhi/tessellation/test.frag
new file mode 100644
index 0000000000..375587662f
--- /dev/null
+++ b/tests/manual/rhi/tessellation/test.frag
@@ -0,0 +1,10 @@
+#version 440
+
+layout(location = 0) in vec3 v_color;
+
+layout(location = 0) out vec4 fragColor;
+
+void main()
+{
+    fragColor = vec4(v_color, 1.0);
+}
diff --git a/tests/manual/rhi/tessellation/test.frag.qsb b/tests/manual/rhi/tessellation/test.frag.qsb
new file mode 100644
index 0000000000..d680373221
--- /dev/null
+++ b/tests/manual/rhi/tessellation/test.frag.qsb
diff --git a/tests/manual/rhi/tessellation/test.tesc b/tests/manual/rhi/tessellation/test.tesc
new file mode 100644
index 0000000000..9cbf9c12c7
--- /dev/null
+++ b/tests/manual/rhi/tessellation/test.tesc
@@ -0,0 +1,21 @@
+#version 440
+
+layout(vertices = 3) out;
+
+layout(location = 0) in vec3 inColor[];
+
+layout(location = 0) out vec3 outColor[];
+
+void main()
+{
+    if (gl_InvocationID == 0) {
+        gl_TessLevelOuter[0] = 4.0;
+        gl_TessLevelOuter[1] = 4.0;
+        gl_TessLevelOuter[2] = 4.0;
+
+        gl_TessLevelInner[0] = 4.0;
+    }
+
+    gl_out[gl_InvocationID].gl_Position = gl_in[gl_InvocationID].gl_Position;
+    outColor[gl_InvocationID] = inColor[gl_InvocationID];
+}
diff --git a/tests/manual/rhi/tessellation/test.tesc.qsb b/tests/manual/rhi/tessellation/test.tesc.qsb
new file mode 100644
index 0000000000..fc9dfbaea1
--- /dev/null
+++ b/tests/manual/rhi/tessellation/test.tesc.qsb
diff --git a/tests/manual/rhi/tessellation/test.tese b/tests/manual/rhi/tessellation/test.tese
new file mode 100644
index 0000000000..c82344af8e
--- /dev/null
+++ b/tests/manual/rhi/tessellation/test.tese
@@ -0,0 +1,20 @@
+#version 440
+
+layout(triangles, fractional_odd_spacing, ccw) in;
+
+layout(location = 0) in vec3 inColor[];
+
+layout(location = 0) out vec3 outColor;
+
+layout(std140, binding = 0) uniform buf {
+    mat4 mvp;
+    float time;
+    float amplitude;
+};
+
+void main()
+{
+    gl_Position = mvp * ((gl_TessCoord.x * gl_in[0].gl_Position) + (gl_TessCoord.y * gl_in[1].gl_Position) + (gl_TessCoord.z * gl_in[2].gl_Position));
+    gl_Position.x += sin(time + gl_Position.y) * amplitude;
+    outColor = gl_TessCoord.x * inColor[0] + gl_TessCoord.y * inColor[1] + gl_TessCoord.z * inColor[2];
+}
diff --git a/tests/manual/rhi/tessellation/test.tese.qsb b/tests/manual/rhi/tessellation/test.tese.qsb
new file mode 100644
index 0000000000..fe99cd5001
--- /dev/null
+++ b/tests/manual/rhi/tessellation/test.tese.qsb
diff --git a/tests/manual/rhi/tessellation/test.vert b/tests/manual/rhi/tessellation/test.vert
new file mode 100644
index 0000000000..3838d2f3bb
--- /dev/null
+++ b/tests/manual/rhi/tessellation/test.vert
@@ -0,0 +1,12 @@
+#version 440
+
+layout(location = 0) in vec3 position;
+layout(location = 1) in vec3 color;
+
+layout(location = 0) out vec3 v_color;
+
+void main()
+{
+    gl_Position = vec4(position, 1.0);
+    v_color = color;
+}
diff --git a/tests/manual/rhi/tessellation/test.vert.qsb b/tests/manual/rhi/tessellation/test.vert.qsb
new file mode 100644
index 0000000000..18f2e7b297
--- /dev/null
+++ b/tests/manual/rhi/tessellation/test.vert.qsb