macOS: Experimental Vulkan support via MoltenVK

Add support for QSurface::VulkanSurface and QVulkanWindow.

Usage:
  1) Build MoltenVK according to instructions
  2) Configure Qt: ./configure -I /path/to/MoltenVK/Package/Release/MoltenVK/include
  3) export QT_VULKAN_LIB=/path/to/MoltenVK/Package/Release/MoltenVK/macOS/libMoltenVK.

Implement support for QSurface::VulkanSurface by enabling
layer mode for QNSView and then creating a CAMetalLayer,
which the MoltenVK translation layer can run on.

MoltenVK provides an implementation of the Vulcan API,
which means that the platform integration is similar
to other platforms: implement a QCocoaVulkanInstance
where we pass the QNSView instance to the vkCreateMacOSSurfaceMVK
Vulkan surface constructor function.

Using Vulkan directly without QVulkanWindow is possible, but not
tested.

We currently load libMoltenVK at run-time and use the
existing QT_VULKAN_LIB environment variable to set its
path. For deployment purposes it would be better to
link against MoltenVK.frameworkm, but this

Task-number: QTBUG-66966
Change-Id: I04ec6289c40b199dca9fed32902b5d2ad4e9c030
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>

1 files changed, 8 insertions, 2 deletions

diff --git a/src/plugins/platforms/cocoa/qnsview_drawing.mm b/src/plugins/platforms/cocoa/qnsview_drawing.mm
index 3bba68b0cf..daa1a2e250 100644
--- a/src/plugins/platforms/cocoa/qnsview_drawing.mm
+++ b/src/plugins/platforms/cocoa/qnsview_drawing.mm
@@ -141,13 +141,19 @@
     // on and off is not a supported use-case, so this code is effectively
     // returning a constant for the lifetime of our QSNSView, which means
     // we don't care about emitting KVO signals for @"wantsLayer".
-    return qt_mac_resolveOption(false, m_platformWindow->window(),
+    bool layerRequested = qt_mac_resolveOption(false, m_platformWindow->window(),
         "_q_mac_wantsLayer", "QT_MAC_WANTS_LAYER");
+
+    // Support Vulkan via MoltenVK, which requires a Metal layer
+    bool layerForVulkan = (m_platformWindow->window()->surfaceType() == QWindow::VulkanSurface);
+
+    return layerRequested || layerForVulkan;
 }
 
 - (CALayer *)makeBackingLayer
 {
-    bool makeMetalLayer = false; // TODO: Add/implement enabling API
+    // Support Vulkan via MoltenVK, which requires a Metal layer
+    bool makeMetalLayer = (m_platformWindow->window()->surfaceType() == QWindow::VulkanSurface);
     if (makeMetalLayer) {
         // Check if Metal is supported. If it isn't then it's most likely
         // too late at this point and the QWindow will be non-functional,