summaryrefslogtreecommitdiffstats
path: root/src/plugins/multimedia/ffmpeg/playbackengine/qffmpegaudiorenderer_p.h
Commit message (Collapse)AuthorAgeFilesLines
* Fix startTime of created audio buffers upon playbackArtem Dyomin2024-05-301-2/+2
| | | | | | | | | | | The resampler should consider a time point upon the initialization in order to get proper startTime to users. Change-Id: Ia7e58794dd540c1cc1b7a3d2b8631ef4571a287b Reviewed-by: Pavel Dubsky <pavel.dubsky@qt.io> Reviewed-by: Mikko Hallamaa <mikko.hallamaa@qt.io> Reviewed-by: Tim Blechmann <tim@klingt.org> Reviewed-by: Jøger Hansegård <joger.hansegard@qt.io>
* Harden the logic of emitting frames by QAudioBufferOutputArtem Dyomin2024-05-301-1/+7
| | | | | | | | | | | | | We need to define explicitly that only the first (not repeated) frame should be pushed to QAudioBufferOutput to get round potential duplications. Task-number: QTBUG-111094 Change-Id: I0f50e59154decc83905ef12af40a7975bba43eb5 Reviewed-by: Mikko Hallamaa <mikko.hallamaa@qt.io> Reviewed-by: Tim Blechmann <tim@klingt.org> Reviewed-by: Pavel Dubsky <pavel.dubsky@qt.io> Reviewed-by: Jøger Hansegård <joger.hansegard@qt.io>
* Implement QAudioBufferOutput, as a custom output for QMediaPlayerArtem Dyomin2024-05-301-3/+9
| | | | | | | | | | | | | | | | | | | | | | Users want to be able to visualize output audio data from QMediaPlayer. The current patch implements a mechnism allowing getting decoded audio data, so that users could visualize what they want. The class is a part of programmatic IO that we add in 6.8. The functionality will allow us to harden QtMM integration tests. [ChangeLog] Added QAudioBufferOutput allowing to get the decoded audio data from QMediaPlayer Task-number: QTBUG-111094 Change-Id: I5f83e7835965f1e7e47b138da0ce82f650abaa4b Reviewed-by: Mikko Hallamaa <mikko.hallamaa@qt.io> Reviewed-by: Pavel Dubsky <pavel.dubsky@qt.io> Reviewed-by: Tim Blechmann <tim@klingt.org> Reviewed-by: Jøger Hansegård <joger.hansegard@qt.io>
* Implement audio resampler in the ffmpeg pluginArtem Dyomin2024-01-231-5/+2
| | | | | | | | | | | | | | | QFFmpeg::Resampler has been renamed and reworked for inheritance from QPlatformAudioResampler. The main purpose of the set of commits is to align the sound played by QMediaPlayer and QSoundEffect. The classes have different volume levels and channel mixing factors if we use ffmpeg resampling in one and doesn't use in another one. Pick-to: 6.7 6.6 6.5 Task-number: QTBUG-118099 Change-Id: I01cb909de756d5e7cd0f1c40e749f186efd2f649 Reviewed-by: Artem Dyomin <artem.dyomin@qt.io> Reviewed-by: Jøger Hansegård <joger.hansegard@qt.io>
* Minor cleanup: move QAudioBuffer + offset to a separate structureArtem Dyomin2023-11-171-2/+11
| | | | | | | | Pick-to: 6.6 6.5 Change-Id: I9f85934b7a7206364a6a3c96c256bc6ba7f56273 Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org> Reviewed-by: Pavel Dubsky <pavel.dubsky@qt.io> Reviewed-by: Artem Dyomin <artem.dyomin@qt.io>
* Reduce audio stutterings of audio sink underflow reasonArtem Dyomin2023-11-161-3/+38
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | We should rely on AudioSink state change to address buffer underflows. The patch removes the audio sample compensation as it appears that the sample compensation doesn't ensure good results on all platforms, also, it has side effects, e.g. it may increase the audio volume with white noise sound, even cause some clippings. The side effects are to be investigated; sample compensation might be reverted back in the future. Some details: - The main reason for the refactoring is the pulse audio sink which may consume sound bytes faster than expected in some random moments, that makes crackling. - in theory, it may need more improvements in the future, e.g. faster handling of the idle event (a few ms delay may cause a little crackling) - even though the patch removes soft sample compensation, the synchronization with video is still soft due to the logic in the time controller of the playback engine. - the implemented algorithm has been tested on macOS, Linux, Windows, and the acceptable result has been achieved. Task-number: QTBUG-116020 Pick-to: 6.6 6.5 Change-Id: I3230be9740542c8548e5e7a5071a55ccb338d5c5 Reviewed-by: Artem Dyomin <artem.dyomin@qt.io>
* Add draining delay of audiosink buffer in mediaplayerArtem Dyomin2023-10-261-0/+1
| | | | | | | | | | After finishing of audio frames play we should wait for the rest of sound in the buffer. Pick-to: 6.6 6.5 Change-Id: I874e2a70b2b9bfaa5e0842a5f347ba4f09870d92 Reviewed-by: Artem Dyomin <artem.dyomin@qt.io> Reviewed-by: Pavel Dubsky <pavel.dubsky@qt.io>
* Fix audioplayback of large frames and improve synchronizationArtem Dyomin2023-07-071-1/+3
| | | | | | | | | | | | | In some cases audioframes are larger than buffer size, we should be able to handle them. As a result, the audio playback and audio synchronization have been improved. Task-number: QTBUG-113211 Pick-to: 6.5 6.6 Change-Id: I6cd2650f9b66e924e152919a0974a9687b2d3bf4 Reviewed-by: Artem Dyomin <artem.dyomin@qt.io> Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
* Handle QMediaPlayer outputs corner casesArtem Dyomin2023-06-021-0/+2
| | | | | | | | | | | | | | | | | | | | | - Clean-up the video sink frame after stop, QMediaPlayer deleting, and after setting new media. - ensure no frames lost if change QVideoSink - ensure no frames sent after changing of outputs. The patch fixes problems with tests on Android, summarizing of the related commits: codereview.qt-project.org/c/qt/qtmultimedia/+/472214 codereview.qt-project.org/c/qt/qtmultimedia/+/470890 codereview.qt-project.org/c/qt/qtmultimedia/+/472052 Users want to have flush mode customization as it was it Qt5. It might be added in 6.6 afterwards. Fixes: QTBUG-112173 Task-number: QTBUG-111912 Change-Id: I0e4d34de06fcf23adf8a5736cbff4119478e9baf Reviewed-by: Lars Knoll <lars@knoll.priv.no>
* Reduce choppy sound on audio playback if audio sink is almost emptyArtem Dyomin2023-05-051-0/+2
| | | | | | | | | | | | | | If audio sink buffer is almost empty, that happens on pause/play, the sound pretty often has little gaps. The solution is using some little samples compensation (slows down the playback on 1% for 1-2sec what allows to increase buffer loadig from 0-10% to 20%). In other words, the patch implements slight "soft" audio samples compensation in order to avoid having empty adio sink buffer. Pick-to: 6.5 Change-Id: I6b963996eab8f8b8f610fcd4a566405aec3d13cf Reviewed-by: Lars Knoll <lars@knoll.priv.no>
* Code cleanup: remove old ffmpeg decoderArtem Dyomin2022-12-221-2/+2
| | | | | | | | | | | | | | What's done: - remove old decoder - namespace PlaybackEngineInternal seems to be not needed, so it's removed - some minor clean up with unique_ptr usage - logic is not changed Pick-to: 6.5 Change-Id: I5089c87ef4c424930bca96d5f2935bfd88c20f5f Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org> Reviewed-by: Lars Knoll <lars@knoll.priv.no>
* Redesign of ffmpeg decoderArtem Dyomin2022-10-171-0/+72
Main principles of the new design are described in documentation in the file qffmpegplaybackengine_p.h The idea is to make the ffmpeg playback more stable, faster, and easier for support. For this reason, a new design has been implemented. Currently, the implementation has similar interface with Decoder, so it's possible not to remove the old one while testing and aligning of the functionality. Read main principles of the new design in documentation in the file qffmpegplaybackengine_p.h Problems with the previous design (in general: not stable and hard for support). - No clear mutex-locking policy: cross-calls under mutexes potentially cause deadlocks; not all data is strictly protected by mutexes. - a lot of mutex locking causes reducing of performance - Not transparent protocol of interractions between threads. Hence, further extension and fixes are complicated. - Too weak encapsulation of internal objects - Explicit managing of queues with which might be implemented via threading event loops and slots/signals mechanizm. - Hard to unit test separate objects What has been done: - The common business logic of playing is the same as in previous impl. - Renamings and decomposition - Use QThread event loop instead of custom ones. - Don't use mutexes at all. All synchronization is based on Queued connections. - Introduce mechanizm of "soft" synchronization to avoid bumpy playing. - Decouple working objects from threads to make it more flexible. E.g., it's possible to customize usage different threads by different jobs. Tested on linux, macos, windows: - play/pause/stop - OK - seek in playing and paused states - OK - playback rate change - OK - av track change - OK - sound volume, muting - OK - playing with subtitles - OK - time synchromization (via artificial sound delays) - OK - UT tst_mediaplayerbackend using new backend: linux, windows, macos - OK To do: - Integrate the engine with AudioPlayer - Align minor details with old implementation (Decoder) and delete it - Add unit tests for PlaybackEngine and for its objects as well - Improve errors handling Task-number: QTBUG-106259 Change-Id: I05481df84be3fd5f17ec66348cb1ab1f8261775a Reviewed-by: Lars Knoll <lars@knoll.priv.no>
generated by cgit v1.2.3 (git 2.25.1) at 2024-06-15 09:32:13 +0000