diff options
Diffstat (limited to 'src/corelib/doc/src/ipc.qdoc')
| -rw-r--r-- | src/corelib/doc/src/ipc.qdoc | 492 |
1 files changed, 492 insertions, 0 deletions
diff --git a/src/corelib/doc/src/ipc.qdoc b/src/corelib/doc/src/ipc.qdoc new file mode 100644 index 0000000000..7ec6f5fa3b --- /dev/null +++ b/src/corelib/doc/src/ipc.qdoc @@ -0,0 +1,492 @@ +// Copyright (C) 2022 Intel Corporation. +// SPDX-License-Identifier: LicenseRef-Qt-Commercial OR GFDL-1.3-no-invariants-only + +/*! + \page ipc.html + \title Inter-Process Communication + \ingroup groups + \ingroup frameworks-technologies + \keyword ipc + \ingroup explanations-networkingandconnectivity + + \brief An overview of Qt's inter-process communication functionality + + Qt supports many ways of communicating with other processes running in the + same system or in different systems. There are basically three types of + inter-process communication mechanisms: + + \list 1 + \li Synchronization primitives + \li Exchanging of arbitrary byte-level data + \li Passing structured messages + \endlist + + \section1 Synchronization primitives + + Qt only provides one class for explicit inter-process synchronization: + \l{QSystemSemaphore}. A QSystemSemaphore is like a \l{QSemaphore} that is + accessible by multiple processes in the same system. It is globally + identified by a "key", which in Qt is represented by the \l{QNativeIpcKey} + class. Additionally, depending on the OS, Qt may support multiple different + backends for sharing memory; see the \l{Native IPC Keys} documentation for + more information and limitations. + + It is possible to use regular thread-synchronization primitives such as + mutexes, wait conditions, and read-write locks, located in memory that is + shared between processes. Qt does not provide any class to support this, + but applications can use low-level operations on certain operating systems. + + Other Qt classes may be used to provide higher-level locking, like + \l{QLockFile}, or by acquiring a unique, system-wide resource. Such + techniques include \l{QTcpServer}{TCP} or \l{QUdpSocket}{UDP} ports or + well-known names in \l{QDBusConnection::registerService}{D-Bus}. + + \section1 Byte-level data sharing + + Using byte-level data, applications can implement any communication + protocol they may choose. Sharing of byte data can be stream-oriented + (serialized) or can allow random access (a similar condition to + QFileDevice::isSequential()). + + For serial communication, Qt provides a number of different classes and + even full modules: + \list + \li Pipes and FIFOs: \l QFile + \li Child processes: \l QProcess + \li Sockets: \l QTcpSocket, \l QUdpSocket (in \l{Qt Network}) + \li HTTP(S): \l QNetworkAccessManager (in \l{Qt Network}) and + \l QHttpServer (in \l{Qt HTTP Server}) + \li CoAP(S): \l QCoapClient (in \l{Qt CoAP}) + \endlist + + For random-access data sharing within the same system, Qt provides + \l{QSharedMemory}. See the \l{Shared Memory} documentation for detailed + information. + + \section1 Structured message passing + + Qt also provides a number of techniques to exchange structured messages + with other processes. Applications can build on top of the byte-level + solutions above, such as by using \l QJsonDocument or \l QXmlStreamReader / + \l QXmlStreamWriter over HTTP to perform JSONRPC or XMLRPC, respectively, + or \l QCborValue with QtCoAP. + + Dedicated Qt modules for structured messages and remote procedure-calling + include: + \list + \li \l{Qt D-Bus} + \li \l{Qt Remote Objects} + \li \l{Qt WebSockets} + \endlist +*/ + +/*! + \page shared-memory.html + \title Shared Memory + \keyword ipc + \keyword shared memory + + \brief Overview of the techniques for sharing memory between processes. + + Qt provides two techniques to share memory with other processes in the same + system: \l{QSharedMemory} and memory-mapped files using \l{QFile}. Memory + that is shared with other processes is often referred to as a "segment", + and although it may have been implemented as specific segments on + processors with segmented memory models in the past, this is not the case + in any modern operating system. Shared memory segments are simply regions + of memory that the operating system will ensure are available to all + processes participating. + + \note The address at which the segment is located in memory will almost + always be different for each process that is participating in the sharing. + Therefore, applications must take care to share only position-independent + data, such as primitive C++ types or arrays of such types. + + \section1 Sharing memory using QSharedMemory + + QSharedMemory provides a simple API to create a shared memory segment of a + given size or attach to one that was created by another process. + Additionally, it provides a pair of methods to \l{QSharedMemory::}{lock} + and \l{QSharedMemory::}{unlock} the whole segment, using an internal + \l{QSystemSemaphore}. + + Shared memory segments and system semaphores are globally identified in the + system through a "key", which in Qt is represented by the \l{QNativeIpcKey} + class. Additionally, depending on the OS, Qt may support multiple different + backends for sharing memory; see the \l{Native IPC Keys} documentation for + more information and limitations. + + QSharedMemory is designed to share memory only within the same privilege + level (that is, not with untrusted other processes, such as those started + by other users). For backends that support it, QSharedMemory will create + segments such that only processes with the same privilege level can attach. + + \section1 Sharing memory via memory-mapped files + + Most files can be mapped to memory using QFile::map() and, if the + \l{QFileDevice::MapPrivateOption}{MapPrivateOption} option is not specified, + any writes to the mapped segment will be observed by all other processes + that have mapped the same file. Exceptions to files that can be mapped to + memory include remote files found in network shares or those located in + certain filesystems. Even if the operating system does allow mapping remote + files to memory, I/O operations on the file will likely be cached and + delayed, thus making true memory sharing impossible. + + This solution has the major advantages of being independent of any backend + API and of being simpler to interoperate with from non-Qt applications. + Since \l{QTemporaryFile} is a \l{QFile}, applications can use that class to + achieve clean-up semantics and to create unique shared memory segments too. + + To achieve locking of the shared memory segment, applications will need to + deploy their own mechanisms. One way may be to use \l QLockFile. Another + and less costly solution is to use QBasicAtomicInteger or \c{std::atomic} in + a pre-determined offset in the segment itself. Higher-level locking + primitives may be available on some operating systems; for example, on + Linux, applications can set the "pshared" flag in the mutex attribute + passed to \c{pthread_mutex_create()} to indicate that the mutex resides in + a shared memory segment. + + Be aware that the operating system will likely attempt to commit to + permanent storage any writes made to the shared memory. This may be desired + or it may be a performance penalty if the file itself was meant to be + temporary. In that case, applications should locate a RAM-backed + filesystem, such as \c{tmpfs} on Linux (see + QStorageInfo::fileSystemType()), or pass a flag to the native file-opening + function to inform the OS to avoid committing the contents to storage. + + It is possible to use file-backed shared memory to communicate with + untrusted processes, in which case the application should exercise great + care. The files may be truncated/shrunk and cause applications accessing + memory beyond the file's size to crash. + + \section2 Linux hints on memory-mapped files + + On modern Linux systems, while the \c{/tmp} directory is often a \c{tmpfs} + mount point, that is not a requirement. However, the \c{/dev/shm} directory + is required to be a \c{tmpfs} and exists for the very purpose of sharing + memory. Do note that it is world-readable and writable (like \c{/tmp} and + \c{/var/tmp}), so applications must be careful of the contents revealed + there. Another alternative is to use the XDG Runtime Directory (see + QStandardPaths::writableLocation() and \l{QStandardPaths::RuntimeLocation}), + which on Linux systems using systemd is a user-specific \c{tmpfs}. + + An even more secure solution is to create a "memfd" using \c{memfd_create(2)} + and use interprocess communication to pass the file descriptor, like + \l{QDBusUnixFileDescriptor} or by letting the child process of a \l{QProcess} + inherit it. "memfds" can also be sealed against being shrunk, so they are + safe to be used when communicating with processes with a different privilege + level. + + \section2 FreeBSD hints on memory-mapped files + + FreeBSD also has \c{memfd_create(2)} and can pass file descriptors to other + processes using the same techniques as Linux. It does not have temporary + filesystems mounted by default. + + \section2 Windows hints on memory-mapped files + + On Windows, the application can request the operating system avoid saving + the file's contents on permanent storage. This request is performed by + passing the \c{FILE_ATTRIBUTE_TEMPORARY} flag in the + \c{dwFlagsAndAttributes} parameter to the \c{CreateFile} Win32 function, + the \c{_O_SHORT_LIVED} flag to \c{_open()} low-level function, or by + including the modifier "T" to the + \c{fopen()} C runtime function. + + There's also a flag to inform the operating system to delete the file when + the last handle to it is closed (\c{FILE_FLAG_DELETE_ON_CLOSE}, + \c{_O_TEMPORARY}, and the "D" modifier), but do note that all processes + attempting to open the file must agree on using this flag or not using it. A + mismatch will likely cause a sharing violation and failure to open the file. +*/ + +/*! + \page native-ipc-keys.html + \title Native IPC Keys + \keyword ipc + \keyword shared memory + \keyword system semaphore + + \brief An overview of keys for QSharedMemory and QSystemSemaphore + + The \l QSharedMemory and \l QSystemSemaphore classes identify their + resource using a system-wide identifier known as a "key". The low-level key + value as well as the key type are encapsulated in Qt using the \l + QNativeIpcKey class. That class also provides the proper means of + exchanging the key with other processes, by way of + QNativeIpcKey::toString() and QNativeIpcKey::fromString(). + + Qt currently supports three distinct backends for those two classes, which + match the values available in the \l{QNativeIpcKey::Type} enumeration. + \list + \li POSIX Realtime extensions (IEEE 1003.1b, POSIX.1b) + \li X/Open System Interfaces (XSI) or System V (SVr4), though also now part of POSIX + \li Windows primitives + \endlist + + As the name indicates, the Windows primitives are only available on the + Windows operating system, where they are the default backend. The other two + are usually both available on Unix operating systems. The following table + provides an overview of typical availability since Qt 6.6: + + \table + \header \li Operating system \li POSIX \li System V \li Windows + \row \li Android \li \li \li + \row \li INTEGRITY \li \li \li + \row \li QNX \li Yes \li \li + \row \li \macos \li Yes \li Usually (1) \li + \row \li Other Apple OSes \li Yes \li \li + \row \li Other Unix systems \li Yes \li Yes \li + \row \li Windows \li Rarely (2) \li \li Yes + \endtable + + \note 1 Sandboxed \macos applications, which include all applications + distributed via the Apple App Store, may not use System V objects. + + \note 2 Some GCC-compatible C runtimes on Windows provide POSIX-compatible + shared memory support, but this is rare. It is always absent with the + Microsoft compiler. + + To determine whether a given key type is supported, applications should + call QSharedMemory::isKeyTypeSupported() and + QSystemSemaphore::isKeyTypeSupported(). + + QNativeIpcKey also provides support for compatibility with Qt applications + prior to its introduction. The following sections detail the limitations of + the backends, the contents of the string keys themselves, and + compatibility. + + \section1 Cross-platform safe key format + + QNativeIpcKey::setNativeKey() and QNativeIpcKey::nativeKey() handle the + low-level native key, which may be used with the native APIs and shared + with other, non-Qt processes (see below for the API). This format is not + usually cross-platform, so both QSharedMemory and QSystemSemaphore provide + a function to translate a cross-platform identifier string to the native + key: QSharedMemory::platformSafeKey() and + QSystemSemaphore::platformSafeKey(). + + The length of the cross-platform key on most platforms is the same as that + of a file name, but is severely limited on Apple platforms to only 30 + usable bytes (be mindful of UTF-8 encoding if using characters outside the + US-ASCII range). The format of the key is also similar to that of a file + path component, meaning it should not contain any characters not allowed in + file names, in particular those that separate path components (slash and + backslash), with the exception of sandboxed applications on Apple operating + systems. The following are good examples of cross-platform keys: "myapp", + "org.example.myapp", "org.example.myapp-12345". Note that it is up to the + caller to prevent oversized keys, and to ensure that the key contains legal + characters on the respective platform. Qt will silently truncate keys that + are too long. + + \b{Apple sandbox limitations:} if the application is running inside of a + sandbox in an Apple operating system, the key must be in a very specific + format: \c {<application group identifier>/<custom identifier>}. Sandboxing + is implied for all applications distributed through the Apple App Store. + See Apple's documentation + \l{https://developer.apple.com/library/archive/documentation/Security/Conceptual/AppSandboxDesignGuide/AppSandboxInDepth/AppSandboxInDepth.html#//apple_ref/doc/uid/TP40011183-CH3-SW24} + {here} and \l{https://developer.apple.com/documentation/bundleresources/entitlements/com_apple_security_application-groups} + {here} for more information, including how to obtain the application's group identifier. + + \section1 Native key format + + This section details the format of the native keys of the supported + backends. + + \section3 POSIX Realtime + Native keys resemble file names and may contain any character that file + names do, except for a slash. POSIX requires the first character in the key + name to be a slash and leaves undetermined whether any additional slashes + are permitted. On most operating systems, the key length is the same as a + file name, but it is limited to 32 characters on Apple operating systems + (this includes the first slash and the terminating null, so only 30 usable + characters are possible). + + The following are good examples of native POSIX keys: "/myapp", + "/org.example.myapp", "/org.example.myapp-12345". + + QSharedMemory::platformSafeKey() and QSystemSemaphore::platformSafeKey() + simply prepend the slash. On Apple operating systems, they also truncate + the result to the available size. + + \section3 Windows + Windows key types are NT + \l{https://learn.microsoft.com/en-us/windows/win32/sync/object-namespaces}{kernel + object names} and may be up to \c{MAX_PATH} (260) characters in length. + They look like relative paths (that is, they don't start with a backslash + or a drive letter), but unlike file names on Windows, they are + case-sensitive. + + The following are good examples of native Windows keys: "myapp", + "org.example.myapp", "org.example.myapp-12345". + + QSharedMemory::platformSafeKey() and QSystemSemaphore::platformSafeKey() + insert a prefix to disambiguate shared memory and system semaphores, + respectively. + + \section3 X/Open System Interfaces (XSI) / System V + System V keys take the form of the name of a file in the system, and thus + have the exact same limitations as file paths do. Both QSharedMemory and + QSystemSemaphore will create this file if it does not exist when creating + the object. If auto-removal is disabled, it may also be shared between + QSharedMemory and QSystemSemaphore without conflict and can be any extant + file (for example, it can be the process executable itself, see + QCoreApplication::applicationFilePath()). The path should be an absolute + one to avoid mistakes due to different current directories. + + QSharedMemory::platformSafeKey() and QSystemSemaphore::platformSafeKey() + always return an absolute path. If the input was already absolute, they + will return their input unchanged. Otherwise, they will prepend a suitable + path where the application usually has permission to create files in. + + \section1 Ownership + + Shared memory and system semaphore objects need to be created before use, + which is accomplished with QSharedMemory::create() or by passing + QSystemSemaphore::Create to the constructor, respectively. + + On Unix systems, the Qt classes that created the object will be responsible + for cleaning up the object in question. Therefore, if the application with + that C++ object exits uncleanly (a crash, qFatal(), etc.), the object may + be left behind. If that happens, applications may fail to create the + object again and should instead attach to an existing one. For example, for + QSharedMemory: + + \code + if (!shm.create(4096) && shm.error() == QSharedMemory::AlreadyExists) + shm.attach(); + \endcode + + Re-attaching to a QSystemSemaphore is probably unwise, as the token counter + in it is probably in an unknown state and therefore may lead to deadlocks. + + \section3 POSIX Realtime + POSIX Realtime object ownership is patterned after files, in the sense that + they exist independent of any process using them or not. Qt is unable to + determine if the object is still in use, so auto-removal will remove it + even then, which will make attaching to the same object impossible but + otherwise not affecting existing attachments. + + Prior to Qt 6.6, Qt never cleaned up POSIX Realtime objects, except on QNX. + + \section3 X/Open System Interfaces (XSI) / System V + There are two resources managed by the Qt classes: the file the key refers + to and the object itself. QSharedMemory manages the object cooperatively: + the last attachment is responsible for removing the object itself and then + removing the key file. QSystemSemaphore will remove the object if and only + if it was passed QSystemSemaphore::Create; additionally, if it created the + key file, it will remove that too. + + Since Qt 6.6, it is possible to ask either class not to clean up. + + \section3 Windows + The operating system owns the object and will clean up after the last + handle to the object is closed. + + \section1 Interoperability with old Qt applications + + The QNativeIpcKey class was introduced in Qt 6.6. Prior to this version, + QSharedMemory and QSystemSemaphore backends were determined at the time of + Qt's own build. For Windows systems, it was always the Windows backend. For + Unix systems, it defaulted to the System V backend if the configuration + script determined it was available. If it was not available, it fell back + to the POSIX one. The POSIX backend could be explicitly + selected using the \c{-feature-ipc_posix} option to the Qt configure + script; if it was enabled, the \c{QT_POSIX_IPC} macro would be defined. + + Qt 6.6 retains the configure script option but it no longer controls the + availability of the backends. Instead, it changes what + QNativeIpcKey::legacyDefaultTypeForOs() will return. Applications that need + to retain compatibility must use this key type exclusively to guarantee + interoperability. + + The API in both QSharedMemory and QSystemSemaphore had the concept of a + cross-platform key, which is now deprecated in favor of using + QSharedMemory::legacyNativeKey() and QSystemSemaphore::legacyNativeKey(). + Those two functions produce the same native key as the deprecated functions + did in prior versions. If the old code was for example: + + \code + QSharedMemory shm("org.example.myapplication"); + QSystemSemaphore sem("org.example.myapplication"); + \endcode + + It can be updated to be: + \code + QSharedMemory shm(QSharedMemory::legacyNativeKey("org.example.myapplication")); + QSystemSemaphore sem(QSystemSemaphore::legacyNativeKey("org.example.myapplication")); + \endcode + + If the two applications exchanged native keys, there is no need to update + code such as: + + \code + QSharedMemory shm; + shm.setNativeKey(key); + \endcode + + Though if the older application did accept a native key, the new one may + opt to use \c{platformSafeKey()} with a second argument of + QNativeIpcKey::legacyDefaultTypeForOs(). + + \section3 X/Open System Interfaces (XSI) / System V + Never use existing files for QSharedMemory keys, as the old Qt application + may attempt to remove it. Instead, let QSharedMemory create it. + + \section1 Interoperability with non-Qt applications + + Interoperability with non-Qt applications is possible, with some limitations: + \list + \li Creation of shared memory segments must not race + \li QSharedMemory support for locking the segment is unavailable + \endlist + + Communication with non-Qt applications must always be through the native + key. + + QSharedMemory always maps the entire segment to memory. The non-Qt + application may choose to only map a subset of it to memory, with no ill + effects. + + \section3 POSIX Realtime + POSIX shared memory can be opened using + \l{https://pubs.opengroup.org/onlinepubs/9699919799/functions/shm_open.html}{shm_open()} + and POSIX system semaphores can be opened using + \l{https://pubs.opengroup.org/onlinepubs/9699919799/functions/sem_open.html}{sem_open()}. + + Both of those functions take a \c name parameter that is the result of + QNativeIpcKey::nativeKey(), encoded for file names using + QFile::encodeName() / QFile::decodeName(). + + \section3 Windows + Windows shared memory objects can be opened using + \l{https://learn.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-createfilemappingw}{CreateFileMappingW} + and Windows system semaphore objects can be opened using + \l{https://learn.microsoft.com/en-us/windows/win32/api/synchapi/nf-synchapi-createsemaphorew}{CreateSemaphoreW}. + Despite the name of both functions starting with "Create", they are able + to attach to existing objects. + + The \c lpName parameter to those functions is the result of + QNativeIpcKey::nativeKey(), without transformation. + + If the foreign application uses the non-Unicode version of those functions + (ending in "A"), the name can be converted to and from 8-bit using QString. + + \section3 X/Open System Interfaces (XSI) / System V + System V shared memory can be obtained using + \l{https://pubs.opengroup.org/onlinepubs/9699919799/functions/shmget.html}{shmget()} + and System V system semaphores can be obtained using + \l{https://pubs.opengroup.org/onlinepubs/9699919799/functions/semget.html}{semget()}. + + The \c{key} parameter to either of those functions is the result of the + \l{https://pubs.opengroup.org/onlinepubs/9699919799/functions/ftok.html}{ftok()} + function when passed the file name obtained from QNativeIpcKey::nativeKey() + with an \c id of 81 or 0x51 (the ASCII capital letter 'Q'). + + System V semaphore objects may contain multiple semaphores, but + QSystemSemaphore only uses the first one (number 0 for \c{sem_num}). + + Both QSharedMemory and QSystemSemaphore default to removing the object + using the \c{IPC_RMID} operation to \c{shmctl()} and \c{semctl()} + respectively if they are the last attachment. +*/ |