diff options
18 files changed, 276 insertions, 18 deletions
diff --git a/doc/config/qtmultimedia_doc.pri b/doc/config/qtmultimedia_doc.pri index 92fd45e13..1055a1911 100644 --- a/doc/config/qtmultimedia_doc.pri +++ b/doc/config/qtmultimedia_doc.pri @@ -43,4 +43,5 @@ OTHER_FILES += \ doc/src/audiooverview.qdoc \ doc/src/radiooverview.qdoc \ doc/src/videooverview.qdoc \ + doc/src/audioengineoverview.qdoc \ doc/src/plugins/qml-multimedia.qdoc diff --git a/doc/src/audioengineoverview.qdoc b/doc/src/audioengineoverview.qdoc new file mode 100644 index 000000000..aa988f497 --- /dev/null +++ b/doc/src/audioengineoverview.qdoc @@ -0,0 +1,71 @@ +/**************************************************************************** +** +** Copyright (C) 2011 Nokia Corporation and/or its subsidiary(-ies). +** Contact: http://www.qt-project.org/ +** +** This file is part of the documentation of the Qt Toolkit. +** +** $QT_BEGIN_LICENSE:FDL$ +** GNU Free Documentation License +** Alternatively, this file may be used under the terms of the GNU Free +** Documentation License version 1.3 as published by the Free Software +** Foundation and appearing in the file included in the packaging of +** this file. +** +** Other Usage +** Alternatively, this file may be used in accordance with the terms +** and conditions contained in a signed written agreement between you +** and Nokia. +** +** +** +** +** +** $QT_END_LICENSE$ +** +****************************************************************************/ + +/*! +\page audioengineoverview.html +\title QtAudioEngine Overview +\brief 3D Positional Audio + +\section1 QtAudioEngine features + +Qt Multimedia includes the \c QtAudioEngine QML module for +providing 3D positional audio. + +These element can be accessed through importing the +\bold{QtAudioEngine 1.0} module. + +\qml +import QtQuick 2.0 +import QtAudioEngine 1.0 + +AudioEngine { +// ... +\endqml + +\section1 Examples +\list + \o \l {AudioEngine Example}{Audio Engine} +\endlist + +\section1 Reference Documentation + +\section2 QML Elements + +\list + \o \l AudioEngine + \o \l AudioSample + \o \l AudioCategory + \o \l Sound + \o \l SoundInstance + \o \l PlayVariation + \o \l AudioListener + \o \l AttenuationModelLinear + \o \l AttenuationModelInverse + +\endlist + +*/ diff --git a/doc/src/audiooverview.qdoc b/doc/src/audiooverview.qdoc index 6f774b519..d9872ffcc 100644 --- a/doc/src/audiooverview.qdoc +++ b/doc/src/audiooverview.qdoc @@ -28,12 +28,15 @@ /*! \page audiooverview.html \title Audio Overview -\brief Audio playback and recording +\brief Audio playback, recording and processing \section1 Audio Features Qt Multimedia offers a range of audio classes, covering both low and -high level approaches to audio input and output. +high level approaches to audio input, output and processing. In +addition to traditional audio usage, the \l {QtAudioEngine}{Qt AudioEngine} +QML classes offers high level 3D positional audio for QML applications. +See that documentation for more information. \section1 Audio Implementation Details @@ -51,7 +54,16 @@ or VOIP) and high latency (like music playback). The available hardware determines what audio outputs and inputs are available. \section3 Push and Pull -[TBD] - description of push vs. pull mode. +The low level audio classes can operate in two modes - \c push and \c pull. +In \c pull mode, the audio device is started by giving it a QIODevice. For +an output device, the QAudioOutput class will pull data from the QIODevice +(using \l QIODevice::read()) when more audio data is required. Conversely, +for \c pull mode with QAudioInput, when audio data is available then the +data will be written directly to the QIODevice. + +In \c push mode, the audio device provides a QIODevice instance that +can be written or read to as needed. Typically this results in simpler +code but more buffering, which may affect latency. \section2 Low latency sound effects @@ -82,12 +94,43 @@ Here is how you play a local file using C++: You can also put files (even remote URLs) into a playlist: \snippet doc/src/snippets/multimedia-snippets/media.cpp Audio playlist +\section2 Decoding compressed audio to memory +In some cases you may want to decode a compressed audio file and do further +processing yourself (like mix multiple samples, or some custom digital signal +processing algorithms). Qt Multimedia 5.0 offers a preliminary API for this +case - the \l QAudioDecoder class. QAudioDecoder supports decoding local files +or from a QIODevice instances. + +Here's an example of decoding a local file: + + \snippet doc/src/snippets/multimedia-snippets/audio.cpp Local audio decoding + +Note: This API is preliminary at this time - the API may change or be +removed before the final 5.0 release. In addition, it is necessary to +add "multimedia-private" to the QT variable in your .pro file to use this class. + + \snippet doc/src/snippets/multimedia-snippets/audio.cpp Audio decoder header + \section2 Recording audio to a file For recording audio to a file, the \l {QAudioRecorder} class allows you to compress audio data from an input device and record it. \snippet doc/src/snippets/multimedia-snippets/media.cpp Audio recorder +\section2 Monitoring audio data during playback or recording + +The \l QAudioProbe class allows you to monitor audio data being played or +recorded in the higher level classes like \l QMediaPlayer, \l QCamera and +\l QAudioRecorder. After creating your high level class, you can simply +set the source of the probe to your class, and receive audio buffers as they +are processed. This is useful for several audio processing tasks, particularly +for visualization or adjusting gain. You cannot modify the buffers, and +they may arrive at a slightly different time than the media pipeline +processes them. + +Here's an example of installing a probe during recording: + \snippet doc/src/snippets/multimedia-snippets/media.cpp Audio probe + \section1 Examples There are both C++ and QML examples available. diff --git a/doc/src/multimedia.qdoc b/doc/src/multimedia.qdoc index 38fe8c509..3bd92eb0a 100644 --- a/doc/src/multimedia.qdoc +++ b/doc/src/multimedia.qdoc @@ -47,6 +47,9 @@ Qt Multimedia offers APIs for doing many multimedia related tasks: \o Record audio and compress it \o Tune and listen to radio stations, and receive radio program information \o Use a camera, including viewfinder, image capture, and movie recording +\o Play 3D positional audio +\o Decode audio media files into memory for processing +\o Accessing video frames or audio buffers as they are played or recorded \endlist \section2 Multimedia Components diff --git a/doc/src/snippets/multimedia-snippets/audio.cpp b/doc/src/snippets/multimedia-snippets/audio.cpp index f9d9ca1ae..eb329a8d8 100644 --- a/doc/src/snippets/multimedia-snippets/audio.cpp +++ b/doc/src/snippets/multimedia-snippets/audio.cpp @@ -47,6 +47,11 @@ #include "qaudiodeviceinfo.h" #include "qaudioinput.h" #include "qaudiooutput.h" +#include "qaudioprobe.h" + +//! [Audio decoder header] +#include "qaudiodecoder_p.h" +//! [Audio decoder header] class AudioInputExample : public QObject { Q_OBJECT @@ -208,3 +213,34 @@ void AudioDeviceInfo() qDebug() << "Device name: " << deviceInfo.deviceName(); //! [Dumping audio formats] } + +class AudioDecodingExample : public QObject { + Q_OBJECT +public: + void decode(); + +public Q_SLOTS: + void stateChanged(QAudio::State newState); + void readBuffer(); +}; + +void AudioDecodingExample::decode() +{ + //! [Local audio decoding] + QAudioFormat desiredFormat; + desiredFormat.setChannelCount(2); + desiredFormat.setCodec("audio/x-raw"); + desiredFormat.setSampleType(QAudioFormat::UnSignedInt); + desiredFormat.setSampleRate(48000); + desiredFormat.setSampleSize(16); + + QAudioDecoder *decoder = new QAudioDecoder(this); + decoder->setAudioFormat(desiredFormat); + decoder->setSourceFilename("level1.mp3"); + + connect(decoder, SIGNAL(bufferReady()), this, SLOT(readBuffer())); + decoder->start(); + + // Now wait for bufferReady() signal and call decoder->read() + //! [Local audio decoding] +} diff --git a/doc/src/snippets/multimedia-snippets/media.cpp b/doc/src/snippets/multimedia-snippets/media.cpp index 4fc38b4d4..7e7a95f69 100644 --- a/doc/src/snippets/multimedia-snippets/media.cpp +++ b/doc/src/snippets/multimedia-snippets/media.cpp @@ -52,6 +52,10 @@ #include "qvideowidget.h" #include "qcameraimagecapture.h" #include "qcamera.h" +#include "qcameraviewfinder.h" +#include "qaudioprobe.h" +#include "qaudiorecorder.h" +#include "qvideoprobe.h" class MediaExample : public QObject { Q_OBJECT @@ -60,8 +64,11 @@ class MediaExample : public QObject { void MediaPlayer(); void RadioTuna(); void MediaRecorder(); + void AudioRecorder(); void EncoderSettings(); void ImageEncoderSettings(); + void AudioProbe(); + void VideoProbe(); private: // Common naming @@ -73,9 +80,14 @@ private: QMediaContent video; QMediaRecorder *recorder; QCamera *camera; + QCameraViewfinder *viewfinder; QCameraImageCapture *imageCapture; QString fileName; QRadioTuner *radio; + QAudioRecorder *audioRecorder; + QAudioProbe *audioProbe; + QVideoProbe *videoProbe; + QMediaContent image1; QMediaContent image2; QMediaContent image3; @@ -182,7 +194,6 @@ void MediaExample::MediaPlayer() void MediaExample::MediaRecorder() { //! [Media recorder] - // Audio only recording recorder = new QMediaRecorder(camera); QAudioEncoderSettings audioSettings; @@ -194,8 +205,10 @@ void MediaExample::MediaRecorder() recorder->setOutputLocation(QUrl::fromLocalFile(fileName)); recorder->record(); //! [Media recorder] +} -#if 0 +void MediaExample::AudioRecorder() +{ //! [Audio recorder] audioRecorder = new QAudioRecorder; @@ -221,7 +234,6 @@ void MediaExample::MediaRecorder() audioRecorder->setAudioInput(selectedInput); //! [Audio recorder endpoints] -#endif } void MediaExample::RadioTuna() @@ -238,4 +250,59 @@ void MediaExample::RadioTuna() //! [Radio tuner] } +void MediaExample::AudioProbe() +{ + //! [Audio probe] + audioRecorder = new QAudioRecorder; + + QAudioEncoderSettings audioSettings; + audioSettings.setCodec("audio/amr"); + audioSettings.setQuality(QtMultimedia::HighQuality); + + audioRecorder->setEncodingSettings(audioSettings); + + audioRecorder->setOutputLocation(QUrl::fromLocalFile("test.amr")); + + audioProbe = new QAudioProbe(this); + if (audioProbe->setSource(audioRecorder)) { + // Probing succeeded, audioProbe->isValid() should be true. + connect(audioProbe, SIGNAL(audioBufferProbed(QAudioBuffer)), + this, SLOT(calculateLevel(QAudioBuffer))); + } + + audioRecorder->record(); + // Now audio buffers being recorded should be signaled + // by the probe, so we can do things like calculating the + // audio power level, or performing a frequency transform + //! [Audio probe] +} + +void MediaExample::VideoProbe() +{ + //! [Video probe] + camera = new QCamera; + viewfinder = new QCameraViewfinder(); + camera->setViewfinder(viewfinder); + + camera->setCaptureMode(QCamera::CaptureVideo); + + videoProbe = new QVideoProbe(this); + + if (videoProbe->setSource(camera)) { + // Probing succeeded, videoProbe->isValid() should be true. + connect(videoProbe, SIGNAL(videoFrameProbed(QVideoFrame)), + this, SLOT(detectBarcodes(QVideoFrame))); + } + + camera->start(); + // Viewfinder frames should now also be emitted by + // the video probe, even in still image capture mode. + // Another alternative is to install the probe on a + // QMediaRecorder connected to the camera to get the + // recorded frames, if they are different from the + // viewfinder frames. + + //! [Video probe] +} + diff --git a/doc/src/snippets/multimedia-snippets/multimedia-snippets.pro b/doc/src/snippets/multimedia-snippets/multimedia-snippets.pro index 24e8735b5..9571b026e 100644 --- a/doc/src/snippets/multimedia-snippets/multimedia-snippets.pro +++ b/doc/src/snippets/multimedia-snippets/multimedia-snippets.pro @@ -11,7 +11,7 @@ INCLUDEPATH += ../../../../src/global \ CONFIG += console -QT += multimedia multimediawidgets widgets +QT += multimedia multimediawidgets widgets multimedia-private SOURCES += \ audio.cpp \ diff --git a/doc/src/videooverview.qdoc b/doc/src/videooverview.qdoc index ca5c7bd64..8a8347fc8 100644 --- a/doc/src/videooverview.qdoc +++ b/doc/src/videooverview.qdoc @@ -90,6 +90,20 @@ You can use the \l QMediaRecorder class in conjunction with other classes to record video to disk. Primarily this is used with the camera, so consult the \l {Camera Overview} for more information. +\section2 Monitoring video frames +You can use the \l QVideoProbe class to access video frames as they +flow through different parts of a media pipeline when using other +classes like \l QMediaPlayer, \l QMediaRecorder or \l QCamera. After +creating the high level media class, you can set the source of the +video probe to that instance. This can be useful for performing +some video processing tasks (like barcode recognition, or object +detection) while the video is rendered normally. You can not affect +the video frames using this class, and they may arrive at a slightly +different time than they are being rendered. + +Here's an example of installing a video probe while recording the camera: + \snippet doc/src/snippets/multimedia-snippets/media.cpp Video probe + \section1 Examples There are both C++ and QML examples available. diff --git a/examples/audioengine/audioengine.pro b/examples/audioengine/audioengine.pro index d054115c6..80fe1a8c6 100644 --- a/examples/audioengine/audioengine.pro +++ b/examples/audioengine/audioengine.pro @@ -5,3 +5,6 @@ SUBDIRS = sources.files = qml/* sources.path = $$[QT_INSTALL_EXAMPLES]/qtmultimedia/declarative/audioengine INSTALLS += sources + +OTHER_FILES += qml/*.qml qml/*.qmlproject qml/content/* + diff --git a/src/imports/audioengine/qdeclarative_attenuationmodel_p.cpp b/src/imports/audioengine/qdeclarative_attenuationmodel_p.cpp index 6624773b6..0f359e12e 100644 --- a/src/imports/audioengine/qdeclarative_attenuationmodel_p.cpp +++ b/src/imports/audioengine/qdeclarative_attenuationmodel_p.cpp @@ -94,8 +94,10 @@ void QDeclarativeAttenuationModel::setName(const QString& name) \since 5.0 \brief The AttenuationModelLinear element allows you to define a linear attenuation curve for Sound element. - \ingroup qml-multimedia + \inmodule QtMultimedia + \ingroup multimedia_audioengine \inherits Item + \preliminary This element is part of the \bold{QtAudioEngine 1.0} module. @@ -224,8 +226,10 @@ qreal QDeclarativeAttenuationModelLinear::calculateGain(const QVector3D &listene \since 5.0 \brief The AttenuationModelInverse element allows you to define a non-linear attenuation curve for Sound element. - \ingroup qml-multimedia + \inmodule QtMultimedia + \ingroup multimedia_audioengine \inherits Item + \preliminary This element is part of the \bold{QtAudioEngine 1.0} module. diff --git a/src/imports/audioengine/qdeclarative_audiocategory_p.cpp b/src/imports/audioengine/qdeclarative_audiocategory_p.cpp index a7379c24b..7b56daf5f 100644 --- a/src/imports/audioengine/qdeclarative_audiocategory_p.cpp +++ b/src/imports/audioengine/qdeclarative_audiocategory_p.cpp @@ -50,8 +50,10 @@ QT_USE_NAMESPACE \qmlclass AudioCategory QDeclarativeAudioCategory \since 5.0 \brief The AudioCategory element allows you to control all active sound instances by group - \ingroup qml-multimedia + \inmodule QtMultimedia + \ingroup multimedia_audioengine \inherits Item + \preliminary This element is part of the \bold{QtAudioEngine 1.0} module. diff --git a/src/imports/audioengine/qdeclarative_audioengine_p.cpp b/src/imports/audioengine/qdeclarative_audioengine_p.cpp index a2763e021..11870d53b 100644 --- a/src/imports/audioengine/qdeclarative_audioengine_p.cpp +++ b/src/imports/audioengine/qdeclarative_audioengine_p.cpp @@ -59,8 +59,10 @@ QT_USE_NAMESPACE \qmlclass AudioEngine QDeclarativeAudioEngine \since 5.0 \brief The AudioEngine element allows you to organize all your 3d audio content in one place. - \ingroup qml-multimedia + \inmodule QtMultimedia + \ingroup multimedia_audioengine \inherits Item + \preliminary This element is part of the \bold{QtAudioEngine 1.0} module. diff --git a/src/imports/audioengine/qdeclarative_audiolistener_p.cpp b/src/imports/audioengine/qdeclarative_audiolistener_p.cpp index 97c424f97..274e48913 100644 --- a/src/imports/audioengine/qdeclarative_audiolistener_p.cpp +++ b/src/imports/audioengine/qdeclarative_audiolistener_p.cpp @@ -51,8 +51,10 @@ QT_USE_NAMESPACE \qmlclass AudioListener QDeclarativeAudioListener \since 5.0 \brief The AudioListener element allows you to control global listener parameters. - \ingroup qml-multimedia + \inmodule QtMultimedia + \ingroup multimedia_audioengine \inherits Item + \preliminary This element is part of the \bold{QtAudioEngine 1.0} module. diff --git a/src/imports/audioengine/qdeclarative_audiosample_p.cpp b/src/imports/audioengine/qdeclarative_audiosample_p.cpp index 24a88ae76..b731b5dce 100644 --- a/src/imports/audioengine/qdeclarative_audiosample_p.cpp +++ b/src/imports/audioengine/qdeclarative_audiosample_p.cpp @@ -53,8 +53,10 @@ QT_USE_NAMESPACE \qmlclass AudioSample QDeclarativeAudioSample \since 5.0 \brief The AudioSample element allows you to load audio samples, mostly wav file. - \ingroup qml-multimedia + \inmodule QtMultimedia + \ingroup multimedia_audioengine \inherits Item + \preliminary This element is part of the \bold{QtAudioEngine 1.0} module. diff --git a/src/imports/audioengine/qdeclarative_playvariation_p.cpp b/src/imports/audioengine/qdeclarative_playvariation_p.cpp index ec0682723..4aab02317 100644 --- a/src/imports/audioengine/qdeclarative_playvariation_p.cpp +++ b/src/imports/audioengine/qdeclarative_playvariation_p.cpp @@ -55,8 +55,10 @@ QT_USE_NAMESPACE So each time the playback of the same sound can be a slightly different even with the same AudioSample. - \ingroup qml-multimedia + \inmodule QtMultimedia + \ingroup multimedia_audioengine \inherits Item + \preliminary This element is part of the \bold{QtAudioEngine 1.0} module. diff --git a/src/imports/audioengine/qdeclarative_sound_p.cpp b/src/imports/audioengine/qdeclarative_sound_p.cpp index 37912732b..12d4f84d1 100644 --- a/src/imports/audioengine/qdeclarative_sound_p.cpp +++ b/src/imports/audioengine/qdeclarative_sound_p.cpp @@ -142,8 +142,10 @@ void QDeclarativeSoundCone::componentComplete() \since 5.0 \brief The Sound element allows you to define a variety of samples and parameters to be used for SoundInstance. - \ingroup qml-multimedia + \inmodule QtMultimedia + \ingroup multimedia_audioengine \inherits Item + \preliminary This element is part of the \bold{QtAudioEngine 1.0} module. diff --git a/src/imports/audioengine/qdeclarative_soundinstance_p.cpp b/src/imports/audioengine/qdeclarative_soundinstance_p.cpp index 4f3ade9ea..689d8255f 100644 --- a/src/imports/audioengine/qdeclarative_soundinstance_p.cpp +++ b/src/imports/audioengine/qdeclarative_soundinstance_p.cpp @@ -54,8 +54,10 @@ QT_USE_NAMESPACE \qmlclass SoundInstance QDeclarativeSoundInstance \since 5.0 \brief The SoundInstance element allows you to play 3d audio content. - \ingroup qml-multimedia + \inmodule QtMultimedia + \ingroup multimedia_audioengine \inherits Item + \preliminary This element is part of the \bold{QtAudioEngine 1.0} module. diff --git a/src/multimedia/audio/qaudioprobe.cpp b/src/multimedia/audio/qaudioprobe.cpp index e51c56a7c..9a71b65d6 100644 --- a/src/multimedia/audio/qaudioprobe.cpp +++ b/src/multimedia/audio/qaudioprobe.cpp @@ -151,14 +151,16 @@ bool QAudioProbe::setSource(QMediaObject *source) /*! Starts monitoring the given \a mediaRecorder. + Returns true on success. + If \a source is zero, this probe will be deactivated - and this function wil return true. + and this function will return true. If the media recorder instance does not support monitoring audio, this function will return false. Any previously monitored objects will no longer be monitored. - Passing in the same object will be ignored, but + Passing in the same (valid) object will be ignored, but monitoring will continue. */ bool QAudioProbe::setSource(QMediaRecorder *mediaRecorder) |