Gstreamer: Introduce support of OpenGL plugin

Since uploading to gl texture is quite performance-wise operation
introduced integration of opengl plugin which can provide a way
to avoid uploading to texture on each draw on qt's side.
Which potentially fixes some performance issues especially on embedded devices.

Added glupload and glcolorconvert gst elements to pipeline.
For qml apps current opengl context is provided to the gst plugin to share opengl data with.
It allows at the end to just use gl textures that created and filled inside the gst.

For overlay apps, glimagesink element is added.

Requires gstreamer-gl-1.0 to be installed.
Since this plugin is almost always available, so enabling it by default will break behavior of current pipeline
and might introduce some side effects.
Thus also provided QT_GSTREAMER_USE_OPENGL_PLUGIN env var to explicitly enable OpenGL features from gst.

After this change video frames will be texture based.
But it is still possible to map frames to download their data.

In case if the video surface does not support gl textures,
glmemory will be anyway requested, which will lead to uploading data, downloading
and uploading it again to textures in scene graph video node.

Task-number: QTBUG-66162
Change-Id: I32044ba0bf0c0cf90434d72f8991ad00927e1380
Reviewed-by: Christian Strømme <christian.stromme@qt.io>

1 files changed, 14 insertions, 0 deletions

diff --git a/src/gsttools/qgstvideorenderersink_p.h b/src/gsttools/qgstvideorenderersink_p.h
index d2417a7c9..38854291a 100644
--- a/src/gsttools/qgstvideorenderersink_p.h
+++ b/src/gsttools/qgstvideorenderersink_p.h
@@ -51,6 +51,7 @@
 // We mean it.
 //
 
+#include <QtMultimedia/private/qtmultimediaglobal_p.h>
 #include <gst/video/gstvideosink.h>
 #include <gst/video/video.h>
 
@@ -67,6 +68,13 @@
 
 #include "qgstvideorendererplugin_p.h"
 
+#if QT_CONFIG(gstreamer_gl)
+#ifndef GST_USE_UNSTABLE_API
+#define GST_USE_UNSTABLE_API
+#endif
+#include <gst/gl/gl.h>
+#endif
+
 QT_BEGIN_NAMESPACE
 class QAbstractVideoSurface;
 
@@ -89,6 +97,7 @@ private:
     QVideoSurfaceFormat m_format;
     GstVideoInfo m_videoInfo;
     bool m_flushed;
+    QAbstractVideoBuffer::HandleType m_handleType;
 };
 
 class QVideoSurfaceGstDelegate : public QObject
@@ -110,6 +119,7 @@ public:
     GstFlowReturn render(GstBuffer *buffer);
 
     bool event(QEvent *event) override;
+    bool query(GstQuery *query);
 
 private slots:
     bool handleEvent(QMutexLocker *locker);
@@ -132,6 +142,9 @@ private:
     GstCaps *m_surfaceCaps;
     GstCaps *m_startCaps;
     GstBuffer *m_renderBuffer;
+#if QT_CONFIG(gstreamer_gl)
+    GstGLContext *m_gstGLDisplayContext = nullptr;
+#endif
 
     bool m_notified;
     bool m_stop;
@@ -168,6 +181,7 @@ private:
     static gboolean unlock(GstBaseSink *sink);
 
     static GstFlowReturn show_frame(GstVideoSink *sink, GstBuffer *buffer);
+    static gboolean query(GstBaseSink *element, GstQuery *query);
 
 private:
     QVideoSurfaceGstDelegate *delegate;