summaryrefslogtreecommitdiffstats
path: root/src/corelib/codecs
Commit message (Collapse)AuthorAgeFilesLines
* Move QTextCodec support out of QtCoreKarsten Heimrich2020-06-2044-44658/+0
| | | | | | | | | | * Assume UTF-8 on all Unix like systems * Export some functions to be able to compile QTextCodec once moved to Qt5Compat. Task-number: QTBUG-75665 Change-Id: I52ec47a848bc0ba72e9c7689668b1bcc5d736c29 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
* Document the string converter classesLars Knoll2020-05-141-1/+1
| | | | | | | | | | Document QStringConverter, QStringDecoder and QStringEncoder. In addition, do some touches to the API, renaming one enum value, add a flags argument to one constructor and make some members private. Change-Id: I8f99dc3d98fb8860cf6fa46301e34b7eb400511b Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* Remove QTextCodec dependency from QCoreGobalDataLars Knoll2020-05-144-16/+52
| | | | | | | | | As we want to move text codecs out of Qt Core, disentangle the dependency, but moving the global codec data into qtextcodec.*. Change-Id: Id7498423c7c4f9f42fd00c450947305d2af8c4be Reviewed-by: Thiago Macieira <thiago.macieira@intel.com> Reviewed-by: Alex Blasche <alexander.blasche@qt.io>
* Ensure the conversion methods in qstringconverter always get a valid stateLars Knoll2020-05-142-0/+27
| | | | | | | | | | | | Make sure that the conversion methods always get a valid state. This is already the ecase then using the new QStringConverter API, ensure the old QTextCodec API also passes in a valid state. This helps simplify the logic inside those methods. Change-Id: I1945e98cdefd46bf1427e11984337f1d62abcaa2 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com> Reviewed-by: Alex Blasche <alexander.blasche@qt.io>
* Clean up the Flag handling in QStringConverterLars Knoll2020-05-141-0/+4
| | | | | | | | | | | | | | | IgnoreHeader was a rather badly defined enum, in addition the utf8 and utf16 codecs where handling BOMs somewhat different for stateless decoding. Fix this by introducing explicit flags for writing a bom when encoding and not skipping the initial bom when decoding. Source compatibility for QTextCodec is done with a couple of static constexpr variables. Change-Id: I0b2d94f84c937cec1e0494c16ef448c00382691d Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* Move local8bit conversion over to qutfsupportLars Knoll2020-05-142-178/+4
| | | | | | | | Local8Bit is always UTF-8 except for Windows platforms. Also add a Locale encoding to QStringConverter. Change-Id: I8d729931fd4c1d7fc6857696b6442a44def3fd9d Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* Move the UTF conversion methods to qstringconverterLars Knoll2020-05-142-1201/+1
| | | | | | | | | | | | | | Separate them from the qutfcodec, so that the codec can later on be moved out of Qt Core. Fix the QUtf methods to take qsizetype instead of int for length arguments. This also makes it possible to not build QTextCodec into the bootstrap lib anymore. Change-Id: I0b4f83139d61b19c651520a2f3a5012aa7e85cb8 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* Start work on a new API to replace QTextCodecLars Knoll2020-05-142-48/+5
| | | | | | | | | | | | | | | The new QStringEncoder and QStringDecoder classes (with a common QStringConverter base class) are there to replace QTextCodec in Qt 6. It currently uses a trivial wrapper around the utf encoding functionality. Added some autotests, mostly copied from the text codec tests. Change-Id: Ib6eeee55fba918b9424be244cbda9dfd5096f7eb Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* QtCore: use new QChar::fromUcs{2,4}()Marc Mutz2020-05-121-7/+3
| | | | | | | | | | Also replace one case of QChar(0) with QChar::Null. These were errors in my local tree, which means they're included in bootstrap builds (incl. qmake). Change-Id: I3dffa9383fd1a30aa43fe2491ad95bb2b1869b40 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
* Port qt_from_latin1() from ushort to char16_tMarc Mutz2020-05-101-2/+2
| | | | | Change-Id: I054e2f604be7253d3322751d55d03b5bac09aefc Reviewed-by: Lars Knoll <lars.knoll@qt.io>
* Clean up state handling for ICU and iconv based codecsLars Knoll2020-05-105-38/+35
| | | | | | | | Get rid of the hack for the FreeFunction and instead add a proper function pointer to clear the data to the ConverterState struct. Change-Id: I104aae1a4381c69f1a254713ec76e1aeaa862cdc Reviewed-by: Lars Knoll <lars.knoll@qt.io>
* Correctly flag remaining chars for the ICU codecLars Knoll2020-05-011-0/+7
| | | | | | | | This will help us get rid of an extremely ugly hack in QTextStream, where we copy the state. Change-Id: If3e15eeb4e35c8607be54dd11c09ddfed87a4d93 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* QTextCodecs: fix a few more ushort/uint -> QChar conversionsMarc Mutz2020-04-302-2/+2
| | | | | | | | | They leaked through since QChar(char) was explicit and therefore QChar(char16_t) was the only viable constructor left once we made all non-character-type QChar ctors explicit, too. Change-Id: I6d774fc4b0d3b7eb480bdb93afd17c2ee9260b1c Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* Move Qt::codecForHtml() to qtextcodec.*Lars Knoll2020-04-242-0/+16
| | | | | | | | The method should move out of Qt together with QTextCodec in Qt 6. Qt 6 will in any case always assume utf8 for 8bit encoded HTML. Change-Id: I68adbc1f0df7e36a82ef6b434ee3771f23e3b790 Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
* Remove QTextCodec dependency from QClipboardLars Knoll2020-04-212-0/+27
| | | | | | | | | | | | | QClipboard used QTextCodec to convert the war clipboard data to a QString. HTML is nowadays always encoded as utf8, and we were only supporting utf based encodings for other text. Add a qFromUtfEncoded() to our UTF helpers that auto detects utf16 and utf32 byte order marks, and assumes utf8 otherwise, to keep this compatible with what we have been doing in Qt 5. Change-Id: I5a9fccb67a88dff27cbbdecff9bd548d31aa1c6c Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
* Merge remote-tracking branch 'origin/5.14' into 5.15Liang Qi2020-02-181-0/+1
|\ | | | | | | | | | | | | | | | | Conflicts: src/corelib/tools/qlinkedlist.h src/plugins/platforms/wasm/qwasmintegration.cpp src/plugins/platforms/wasm/qwasmscreen.cpp Change-Id: Iefca7f9f4966bdc20e7052aca736874861055738
| * Only read the first BOM as a BOM; the rest are ZWNBS !Edward Welbourne2020-02-141-0/+1
| | | | | | | | | | | | | | | | | | | | QUtf32::convertToUnicode() was forgetting to set headerdone when it dealt with the header (for contrast, Utf16::convertToUnicode() does). Fixes: QTBUG-62011 Change-Id: Ia254782ce0967a6cf9ce0e81eb06d41521150eed Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org> Reviewed-by: Lars Knoll <lars.knoll@qt.io>
* | Tidy nullptr usageAllan Sandfeld Jensen2019-12-063-42/+42
|/ | | | | | | | | | | Move away from using 0 as pointer literal. Done using clang-tidy. This is not complete as run-clang-tidy can't handle all of qtbase in one go. Change-Id: I1076a21f32aac0dab078af6f175f7508145eece0 Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io> Reviewed-by: Lars Knoll <lars.knoll@qt.io>
* QGb18030Codec: fix out-of-bounds access when decoding surrogate pairsThiago Macieira2019-11-261-1/+0
| | | | | | | | | This is already the second surrogate of the pair, there's no need to + +i. Fixes: QTBUG-80268 Change-Id: Ia2aa807ffa8a4c798425fffd15d9a48ee0ca8ed7 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
* QTextCodec: try to work around an ICC 19 bugMarc Mutz2019-10-071-1/+4
| | | | | | | | | | | | | | | | | | | | | | | | ICC 19 barfs on the TextCodecsMutexLocker class because it doesn't have a user-provided default ctor: ../../corelib/codecs/qtextcodec.cpp(543): error #854: const variable locker requires an initializer -- class TextCodecsMutexLocker has no user-provided default constructor [...] But the class doesn't have members that would delete the implictly-declared default ctor, so no user-provided default ctor should be necessary: The only member is the result of qt_unique_lock(), which is std::unique_lock, which does have a default ctor. We conclude that this is a compiler bug, and work around it with the introduction of a user-provided default ctor. Fix brace placement as a drive-by. Fixes: QTBUG-78844 Change-Id: I1f5a326afd68138fbebad506ba9aa1926f1afb85 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* Merge remote-tracking branch 'origin/5.13' into 5.14Liang Qi2019-09-041-4/+5
|\ | | | | | | | | | | | | | | | | | | | | Conflicts: src/corelib/codecs/qicucodec.cpp src/dbus/qdbusserver.cpp src/gui/painting/qbezier.cpp src/plugins/platforms/eglfs/deviceintegration/eglfs_kms/qeglfskmsgbmscreen.cpp src/plugins/printsupport/cups/qppdprintdevice.cpp Change-Id: I2703128bb64baf5580fbc2c2061b55b0f0611d2a
| * Fix building with -no-feature-codecsAllan Sandfeld Jensen2019-08-201-2/+5
| | | | | | | | | | Change-Id: Ifce31a5ef07101f5ef8aabbc81b42eee47153961 Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
| * Merge remote-tracking branch 'origin/5.12' into 5.13Qt Forward Merge Bot2019-07-121-3/+3
| |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: configure.pri Also required s/solid\.color/solidColor/ in a couple of places in: src/gui/painting/qpaintengine_raster.cpp Change-Id: I29937f63e9779deb6dac7ae77e2948d06ebc0319
| | * QSimpleTextCodec: fix load memory order of atomic pointerMarc Mutz2019-07-081-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The pointer value is not the only data we're interested in, but instead points to indirect data, so we need a release fence on store (present) and a corresponding acquire fence on load (was missing). Change-Id: I51f8251c0c7f4056192880430f2be5e0836dbed6 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com> (cherry picked from commit 6f84829031f318bfda1deff5f409b5ea6c6a5c5f) (cherry picked from commit 4cc6e1419294a729e53d698bace2254903c1429b) Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
* | | Fix build without feature.codecsTasuku Suzuki2019-08-201-0/+2
| | | | | | | | | | | | | | | Change-Id: I7113928c686319f132e6306f6925da009e8f41b1 Reviewed-by: Ulf Hermann <ulf.hermann@qt.io>
* | | QTextCodec: port to QRecursiveMutexMarc Mutz2019-08-131-12/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Write a local mutex locker for DRY. Use qt_unique_lock(Mutex*) because we do not intend to add a QRecursiveMutexLocker and by Qt 6, the current work-around where QMutexLocker works for QRecursiveMutex, too, will be gone. Could have used qt_scoped_lock, because no premature unlock() calls are present, but it turns out the code attempts to take the mutex after it's destroyed by Q_GLOBAL_STATIC, so we need QMutexLocker compat, thus qt_unique_lock(Mutex*) instead of qt_scoped_lock(Mutex&). Need to use decltype and using because auto doesn't work for NSDMs. Change-Id: I8b0f8a3067f0a412279d075cb959105e485789cd Reviewed-by: Lars Knoll <lars.knoll@qt.io>
* | | Remove useless null pointer checksThiago Macieira2019-07-161-7/+4
| | | | | | | | | | | | | | | | | | | | | Cache can't be null, since it's a member of an extant object. Change-Id: Id98140e1c2f0426cabbefffd157ed3cdd62a8bba Reviewed-by: Lars Knoll <lars.knoll@qt.io>
* | | Be less laissez-faire with implicit conversions to QCharMarc Mutz2019-07-092-12/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | QChar currently is convertible from nearly every integral type. This is bad code hygiene and should be fixed come Qt 6. The present patch is the result of compile fixes from marking these constructors explicit. As is clear from the distribution of fixes, only low-level string handling code used these implicit conversions, an indication that they're not in widespread use elsewhere. Change-Id: Ief5336f21e6d181e03ab92893b3d13a14adc7cb0 Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
* | | QSimpleTextCodec: fix load memory order of atomic pointerMarc Mutz2019-06-211-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The pointer value is not the only data we're interested in, but instead points to indirect data, so we need a release fence on store (present) and a corresponding acquire fence on load (was missing). Change-Id: I51f8251c0c7f4056192880430f2be5e0836dbed6 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* | | Port from QAtomic::load() to loadRelaxed()Giuseppe D'Angelo2019-06-201-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Semi-automated, just needed ~20 manual fixes: $ find \( -iname \*.cpp -or -iname \*.h \) -exec perl -pe 's/(\.|->)load\(\)/$1loadRelaxed\(\)/g' -i \{\} + $ find \( -iname \*.cpp -or -iname \*.h \) -exec perl -pe 's/(\.|->)store\(/$1storeRelaxed\(/g' -i \{\} + It can be easily improved (e.g. for store check that there are no commas after the opening parens). The most common offender is QLibrary::load, and some code using std::atomic directly. Change-Id: I07c38a3c8ed32c924ef4999e85c7e45cf48f0f6c Reviewed-by: Marc Mutz <marc.mutz@kdab.com>
* | | Move away from using 0 as a pointer constantAllan Sandfeld Jensen2019-06-072-14/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | Cleans up most of corelib to use nullptr or default enums where appropriate. Change-Id: Ifcaac14ecdaaee730f87f10941db3ce407d71ef9 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* | | Merge remote-tracking branch 'origin/5.13' into devLiang Qi2019-05-131-1/+1
|\| | | | | | | | | | | | | | | | | | | | Conflicts: src/corelib/tools/qstring.cpp Change-Id: I81dbf90fc936c9bf08197baefa071117bddb1c63
| * | Doc: replace even more null/0/nullptr with \nullptr macroChristian Ehrlicher2019-05-081-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | Try to replace all wordings like '.. to 0' with '.. to \nullptr'. Also checked for 'null pointer' and similar. Change-Id: I73341f59ba51e0798e816a8b1a532c7c7374b74a Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
* | | Merge remote-tracking branch 'origin/5.13' into devQt Forward Merge Bot2019-04-051-1/+1
|\| | | | | | | | | | | Change-Id: If4974bbf0a166de244dd57cb71b05fa28bcc34ce
| * | Merge remote-tracking branch 'origin/5.12' into 5.13Qt Forward Merge Bot2019-04-041-1/+1
| |\| | | | | | | | | | Change-Id: Ia7328524f2cd9d5995ac8705f0fe0bf570b2e831
| | * Doc: Use the \nullptr macro instead of 0Venugopal Shivashankar2019-04-021-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | This enables overriding the macro so that it translates to 'None' in the Qt for Python context. Change-Id: Ib3cecf57eeb0405a1929309b71e9f012a07f11cf Reviewed-by: Christian Ehrlicher <ch.ehrlicher@gmx.de>
* | | Replace Q_DECL_NOEXCEPT with noexcept in corelibAllan Sandfeld Jensen2019-04-033-4/+4
| | | | | | | | | | | | | | | | | | | | | In preparation of Qt6 move away from pre-C++11 macros. Change-Id: I44126693c20c18eca5620caab4f7e746218e0ce3 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* | | Merge "Merge remote-tracking branch 'origin/5.13' into dev" into ↵Liang Qi2019-03-151-2/+2
|\| | | | | | | | | | | refs/staging/dev
| * | Doc: replace 0 with \nullptr in documentationChristian Ehrlicher2019-03-131-2/+2
| | | | | | | | | | | | | | | | | | | | | Replace some more 0 with \nullptr. Change-Id: I2af91bf3712eef5161b11da0c44614bc039ade03 Reviewed-by: Paul Wicking <paul.wicking@qt.io>
* | | More nullptr usage in headersKevin Funk2019-03-141-1/+1
|/ / | | | | | | | | | | | | | | | | | | | | Diff generated by running clang-tidy's modernize-use-nullptr checker on the CMake-based Qt version. Skipping src/3rdparty, examples/, tests/ Change-Id: Ib182074e2e2fd52f63093f73b3e2e4c0cb7af188 Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io> Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
* | Add missing "override" specifiersAlessandro Portale2019-02-131-10/+10
| | | | | | | | | | | | | | | | Annotate this function with 'override' or (rarely) 'final' [modernize-use-override] Change-Id: If31ab03b46f885e4984ba3aaaf1ad8d46aae9d9c Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
* | Merge remote-tracking branch 'origin/5.12' into devLiang Qi2019-01-261-1/+5
|\| | | | | | | | | | | | | | | Conflicts: src/android/templates/AndroidManifest.xml tests/auto/widgets/styles/qstylesheetstyle/tst_qstylesheetstyle.cpp Change-Id: I4c9679e3a8ebba118fbf4772301ff8fde60455b9
| * Merge remote-tracking branch 'origin/5.12' into 5.12.1Liang Qi2019-01-083-7/+4
| |\ | | | | | | | | | | | | | | | | | | Conflicts: src/widgets/kernel/qtooltip.cpp Change-Id: Ic2f9a425359050eb56b3a4e5162cf5e3447058c8
| * | QWindowsLocalCodec::convertToUnicode(): Fix remaining char conversionDavid Schulz2018-12-131-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Prevent passing a zero argument to the cbMultiByte parameter of MultiByteToWideChar. According to the documentation this will always fail. This is triggered when calling convertToUnicode with a state having remainingChars of one and a length of one. The remaining chars will be completed and the length will be reduced to zero before the actual convert. Adding an early return after the completion to prevent calling MultiByteToWideChar with the invalid zero argument. Fixes: QTCREATORBUG-21622 Change-Id: Ic6a433dd929e97b1681583cb1b9a347afaa9a09f Reviewed-by: Edward Welbourne <edward.welbourne@qt.io> Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* | | Merge remote-tracking branch 'origin/5.12' into devLiang Qi2019-01-042-6/+3
|\ \ \ | | |/ | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Also blacklist tst_QRawFont::unsupportedWritingSystem() and tst_QGlyphRun::mixedScripts() on windows for now. Conflicts: qmake/generators/makefile.cpp src/corelib/itemmodels/qstringlistmodel.cpp src/platformsupport/fontdatabases/windows/qwindowsfontengine_p.h tests/auto/corelib/itemmodels/qstringlistmodel/tst_qstringlistmodel.cpp tests/auto/gui/text/qglyphrun/BLACKLIST tests/auto/gui/text/qrawfont/BLACKLIST Task-number: QTBUG-72836 Change-Id: I10fea1493f0ae1a5708e1e48d0a4d7d6b76258b9
| * | Doc: Update page Tamil Script Code for Information InterchangeNico Vertriest2019-01-021-4/+2
| | | | | | | | | | | | | | | | | | Task-number: QTBUG-56668 Change-Id: I173d73ef5e9b08e865cb75fa03849665fcda652e Reviewed-by: Venugopal Shivashankar <Venugopal.Shivashankar@qt.io>
| * | configure: modernize iconv useOswald Buddenhagen2018-12-191-2/+1
| | | | | | | | | | | | | | | | | | | | | use library objects for all variants, and inline the tests. Change-Id: I029f9a6655a783dab4a22abf601aadbb484c03af Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
* | | Merge remote-tracking branch 'origin/5.12' into devQt Forward Merge Bot2018-12-131-1/+1
|\| | | | | | | | | | | Change-Id: I0fe623517af28e408b642c879efd59f633ab63ac
| * | Fix tautological compare in error checkingThiago Macieira2018-12-121-1/+1
| |/ | | | | | | | | | | | | | | | | size is size_t, so it's never less than zero. Fixes: QTBUG-72286 Change-Id: Idd0c85a4e7b64f9c9c7dfffd156d404d0de5ed8d Reviewed-by: Oliver Wolff <oliver.wolff@qt.io> Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
* | QTextCodec::codecForLocale(): clarify and update documentationEdward Welbourne2018-12-061-5/+3
| | | | | | | | | | Change-Id: If3eec8cfc90cadcbd418807891ccf19b796f4006 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
generated by cgit v1.2.3 (git 2.25.1) at 2024-04-19 07:15:31 +0000