summaryrefslogtreecommitdiffstats
path: root/src/corelib/codecs/qtextcodec.cpp
Commit message (Collapse)AuthorAgeFilesLines
* Move QTextCodec support out of QtCoreKarsten Heimrich2020-06-201-1299/+0
| | | | | | | | | | * Assume UTF-8 on all Unix like systems * Export some functions to be able to compile QTextCodec once moved to Qt5Compat. Task-number: QTBUG-75665 Change-Id: I52ec47a848bc0ba72e9c7689668b1bcc5d736c29 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
* Remove QTextCodec dependency from QCoreGobalDataLars Knoll2020-05-141-10/+31
| | | | | | | | | As we want to move text codecs out of Qt Core, disentangle the dependency, but moving the global codec data into qtextcodec.*. Change-Id: Id7498423c7c4f9f42fd00c450947305d2af8c4be Reviewed-by: Thiago Macieira <thiago.macieira@intel.com> Reviewed-by: Alex Blasche <alexander.blasche@qt.io>
* Start work on a new API to replace QTextCodecLars Knoll2020-05-141-20/+0
| | | | | | | | | | | | | | | The new QStringEncoder and QStringDecoder classes (with a common QStringConverter base class) are there to replace QTextCodec in Qt 6. It currently uses a trivial wrapper around the utf encoding functionality. Added some autotests, mostly copied from the text codec tests. Change-Id: Ib6eeee55fba918b9424be244cbda9dfd5096f7eb Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* Port qt_from_latin1() from ushort to char16_tMarc Mutz2020-05-101-2/+2
| | | | | Change-Id: I054e2f604be7253d3322751d55d03b5bac09aefc Reviewed-by: Lars Knoll <lars.knoll@qt.io>
* Clean up state handling for ICU and iconv based codecsLars Knoll2020-05-101-4/+13
| | | | | | | | Get rid of the hack for the FreeFunction and instead add a proper function pointer to clear the data to the ConverterState struct. Change-Id: I104aae1a4381c69f1a254713ec76e1aeaa862cdc Reviewed-by: Lars Knoll <lars.knoll@qt.io>
* Move Qt::codecForHtml() to qtextcodec.*Lars Knoll2020-04-241-0/+11
| | | | | | | | The method should move out of Qt together with QTextCodec in Qt 6. Qt 6 will in any case always assume utf8 for 8bit encoded HTML. Change-Id: I68adbc1f0df7e36a82ef6b434ee3771f23e3b790 Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
* QTextCodec: try to work around an ICC 19 bugMarc Mutz2019-10-071-1/+4
| | | | | | | | | | | | | | | | | | | | | | | | ICC 19 barfs on the TextCodecsMutexLocker class because it doesn't have a user-provided default ctor: ../../corelib/codecs/qtextcodec.cpp(543): error #854: const variable locker requires an initializer -- class TextCodecsMutexLocker has no user-provided default constructor [...] But the class doesn't have members that would delete the implictly-declared default ctor, so no user-provided default ctor should be necessary: The only member is the result of qt_unique_lock(), which is std::unique_lock, which does have a default ctor. We conclude that this is a compiler bug, and work around it with the introduction of a user-provided default ctor. Fix brace placement as a drive-by. Fixes: QTBUG-78844 Change-Id: I1f5a326afd68138fbebad506ba9aa1926f1afb85 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* QTextCodec: port to QRecursiveMutexMarc Mutz2019-08-131-12/+17
| | | | | | | | | | | | | | | | | Write a local mutex locker for DRY. Use qt_unique_lock(Mutex*) because we do not intend to add a QRecursiveMutexLocker and by Qt 6, the current work-around where QMutexLocker works for QRecursiveMutex, too, will be gone. Could have used qt_scoped_lock, because no premature unlock() calls are present, but it turns out the code attempts to take the mutex after it's destroyed by Q_GLOBAL_STATIC, so we need QMutexLocker compat, thus qt_unique_lock(Mutex*) instead of qt_scoped_lock(Mutex&). Need to use decltype and using because auto doesn't work for NSDMs. Change-Id: I8b0f8a3067f0a412279d075cb959105e485789cd Reviewed-by: Lars Knoll <lars.knoll@qt.io>
* Remove useless null pointer checksThiago Macieira2019-07-161-7/+4
| | | | | | | Cache can't be null, since it's a member of an extant object. Change-Id: Id98140e1c2f0426cabbefffd157ed3cdd62a8bba Reviewed-by: Lars Knoll <lars.knoll@qt.io>
* Move away from using 0 as a pointer constantAllan Sandfeld Jensen2019-06-071-12/+12
| | | | | | | | | Cleans up most of corelib to use nullptr or default enums where appropriate. Change-Id: Ifcaac14ecdaaee730f87f10941db3ce407d71ef9 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* Merge remote-tracking branch 'origin/5.13' into devLiang Qi2019-05-131-1/+1
|\ | | | | | | | | | | | | Conflicts: src/corelib/tools/qstring.cpp Change-Id: I81dbf90fc936c9bf08197baefa071117bddb1c63
| * Doc: replace even more null/0/nullptr with \nullptr macroChristian Ehrlicher2019-05-081-1/+1
| | | | | | | | | | | | | | | | Try to replace all wordings like '.. to 0' with '.. to \nullptr'. Also checked for 'null pointer' and similar. Change-Id: I73341f59ba51e0798e816a8b1a532c7c7374b74a Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
* | Merge remote-tracking branch 'origin/5.13' into devQt Forward Merge Bot2019-04-051-1/+1
|\| | | | | | | Change-Id: If4974bbf0a166de244dd57cb71b05fa28bcc34ce
| * Merge remote-tracking branch 'origin/5.12' into 5.13Qt Forward Merge Bot2019-04-041-1/+1
| |\ | | | | | | | | | Change-Id: Ia7328524f2cd9d5995ac8705f0fe0bf570b2e831
| | * Doc: Use the \nullptr macro instead of 0Venugopal Shivashankar2019-04-021-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | This enables overriding the macro so that it translates to 'None' in the Qt for Python context. Change-Id: Ib3cecf57eeb0405a1929309b71e9f012a07f11cf Reviewed-by: Christian Ehrlicher <ch.ehrlicher@gmx.de>
* | | Replace Q_DECL_NOEXCEPT with noexcept in corelibAllan Sandfeld Jensen2019-04-031-1/+1
|/ / | | | | | | | | | | | | In preparation of Qt6 move away from pre-C++11 macros. Change-Id: I44126693c20c18eca5620caab4f7e746218e0ce3 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* | Doc: replace 0 with \nullptr in documentationChristian Ehrlicher2019-03-131-2/+2
| | | | | | | | | | | | | | Replace some more 0 with \nullptr. Change-Id: I2af91bf3712eef5161b11da0c44614bc039ade03 Reviewed-by: Paul Wicking <paul.wicking@qt.io>
* | QTextCodec::codecForLocale(): clarify and update documentationEdward Welbourne2018-12-061-5/+3
| | | | | | | | | | Change-Id: If3eec8cfc90cadcbd418807891ccf19b796f4006 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* | Compile with QT_STRICT_ITERATORS definedLars Knoll2018-11-261-2/+2
|/ | | | | | | This will be the only options for Qt 6, so make sure the code compiles now. Change-Id: I23f791d1efcbd0bd33805bb4563d40460954db43 Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@qt.io>
* Merge remote-tracking branch 'origin/5.11' into 5.12Liang Qi2018-11-091-4/+1
|\ | | | | | | | | | | | | | | | | | | Conflicts: .qmake.conf qmake/Makefile.unix src/gui/text/qtextdocument.cpp src/gui/text/qtextdocument.h Change-Id: Iba26da0ecbf2aa4ff4b956391cfb373f977f88c9
| * Modernize the "textcodec" featureLiang Qi2018-11-071-4/+1
| | | | | | | | | | | | | | | | | | Also clean up QTextCodec usage in qmake build and some includes of qtextcodec.h. Change-Id: I0475b82690024054add4e85a8724c8ea3adcf62a Reviewed-by: Edward Welbourne <edward.welbourne@qt.io> Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@qt.io>
* | Optimize QTextCodec::codecForUtfText a littleThiago Macieira2018-11-081-14/+14
| | | | | | | | | | | | | | | | | | Instead of doing byte comparisons, let the compiler do 16- and 32-bit comparisons on its own. Change-Id: If7e743cf8476463880ccfffd155f8629991b0b87 Reviewed-by: Olivier Goffart (Woboq GmbH) <ogoffart@woboq.com> Reviewed-by: Tobias Hunger <tobias.hunger@qt.io>
* | QTextCodec: update the docs for codecForUtfTextThiago Macieira2018-11-081-5/+26
| | | | | | | | | | | | | | | | | | | | | | | | It can detect the standard UTF codecs, but not non-standard like UTF-7[1], UTF-9 or UTF-18[2]. [1] https://tools.ietf.org/html/rfc2152 [2] https://tools.ietf.org/html/rfc4042 Fixes: QTBUG-67188 Change-Id: If7e743cf8476463880ccfffd155f853dc947421a Reviewed-by: Olivier Goffart (Woboq GmbH) <ogoffart@woboq.com>
* | Merge remote-tracking branch 'origin/5.11' into 5.12Liang Qi2018-10-171-6/+8
|\| | | | | | | | | | | | | | | Conflicts: src/plugins/platformthemes/platformthemes.pro src/printsupport/kernel/qplatformprintdevice.cpp Change-Id: Iac01729ad954bb1c7af5867d982eb243b2139ee6
| * Modernize the "codecs" featureLiang Qi2018-10-121-2/+4
| | | | | | | | | | | | Change-Id: Idee19112581bff64a2e0b8e331dd3d779aca165b Reviewed-by: Edward Welbourne <edward.welbourne@qt.io> Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@qt.io>
| * Modernize the "big_codecs" featureLiang Qi2018-10-121-4/+4
| | | | | | | | | | | | Change-Id: Ic23f4a1f81a21711cd81aaa2942b493aca5b38b8 Reviewed-by: Edward Welbourne <edward.welbourne@qt.io> Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@qt.io>
* | Add tests for decoding too-short UTF-8 sequencesThiago Macieira2018-09-251-1/+15
| | | | | | | | | | | | | | | | | | | | | | | | We were handling this properly, but not testing them. I guess we weren't testing because the condition is a valid intermediate state, so hasFailure() is correct it returning false. Testing inspired by the bug reported in https://github.com/intel/tinycbor/issues/137 Change-Id: Ib47c56818178458a88b4fffd1554ecfdd0af637e Reviewed-by: Lars Knoll <lars.knoll@qt.io>
* | Deregister QTextCodec on destructionIgor Mironchik2018-05-261-0/+18
|/ | | | | | | | | | | QTextCodec automatically deregisters on destruction now. [ChangeLog][QtCore][QTextCodec] QTextCodec automatically deregisters on destruction now. Task-number: QTBUG-56203 Change-Id: Ic9a66c512642c9913aa27ea5167b9f7341e7f0fe Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* Remove references to obsolete platformsJake Petroules2017-11-051-2/+2
| | | | | | | | | | | | | | ultrix and reliant have not seen a release since 1995. dgux not since 2001. bsdi not since 2003. irix not since 2006. osf not since 2010. dynix... unclear, but no later than 2002. symbian needs no mention. All considered obsolete, all gone. sco and unixware are effectively obsolete. Remove them until someone expresses a real need. Change-Id: Ia3d9d370016adce9213ae5ad0ef965ef8de2a3ff Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@qt.io> Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* QtBase: use new QStaticByteArrayMatcher where applicableMarc Mutz2017-04-241-2/+5
| | | | | | | | | | | | | | | | | Even for compilers that don't yet support C++14 constexpr, this should improve performance of searches a lot, if, indeed, Boyer-Moore still is an optimization over linear searching at all in these days of hardware prefetchers and deep CPU pipelines, because the setup cost is only incurred once. As function-statics, we also don't care about startup ordering and cost. It's a pity that the platform that would benefit the most - Windows - doesn't have constexpr support, yet. Change-Id: I827df135854fd6fbd6546e248dc37ef0fbaf1792 Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io> Reviewed-by: Milian Wolff <milian.wolff@kdab.com>
* QTextCodec: add QStringView overloadsMarc Mutz2017-04-061-0/+42
| | | | | | | | | | [ChangeLog][QtCore][QTextCodec] Added fromUnicode() and canEncode() overloads taking QStringView. [ChangeLog][QtCore][QTextEncoder] Added fromUnicode() overload taking QStringView. Change-Id: I2599f25570480d967921ccd4414e092bfc90d821 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
* Properly use QT_CONFIG macro to check for ICULars Knoll2016-11-291-9/+9
| | | | | | | And remove the QT_USE_ICU define. Change-Id: I8134ee18af7c90ed7070926ca31b3a57b3ec37dd Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@qt.io>
* Clean up iconv configurationLars Knoll2016-11-231-3/+3
| | | | | | | | Turn iconv off if ICU is being used (in line with codecs.pri) and get rid of the DEFINES += GNU_LIBICONV in the pri file. Change-Id: I6fbca975498adbb3e67f913ae9b1dd5cc53ee8da Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@qt.io>
* QtCore: Remove Windows CE.Friedemann Kleint2016-03-301-3/+3
| | | | | | | | | Remove QSysInfo::WV_CE_5/6 enumeration values, #ifdef sections for Q_OS_WINCE and wince .pro file clauses in library, examples and tests. Task-number: QTBUG-51673 Change-Id: Ib63463445f3a26e04d018b193e4655030002f5f9 Reviewed-by: Lars Knoll <lars.knoll@theqtcompany.com>
* Updated license headersJani Heikkinen2016-01-151-14/+20
| | | | | | | | | | | From Qt 5.7 -> LGPL v2.1 isn't an option anymore, see http://blog.qt.io/blog/2016/01/13/new-agreement-with-the-kde-free-qt-foundation/ Updated license headers to use new LGPL header instead of LGPL21 one (in those files which will be under LGPL v3) Change-Id: I046ec3e47b1876cd7b4b0353a576b352e3a946d9 Reviewed-by: Lars Knoll <lars.knoll@theqtcompany.com>
* Mark qt_from_latin1() as nothrowMarc Mutz2015-11-051-1/+1
| | | | | | | | | | | | | | It's used from a different TU, too, so we can't rely on the compiler to inspect it to draw this conclusion itself. This function has technically not a wide contract, because the output buffer needs to be large enough to hold the result. However, this precondition cannot be checked from within the function, therefore no assertion can ever be added and the nothrow marker becomes acceptable (even desireable). Change-Id: I2dc6c4f3d9d8147c6483865c5c4bbc8e9af291b7 Reviewed-by: Olivier Goffart (Woboq GmbH) <ogoffart@woboq.com>
* Remove documentation of QTextCodec::setCodecForTr().Friedemann Kleint2015-07-101-10/+0
| | | | | | | | | This function has been removed. Fixes documentation warning: qtbase/src/corelib/codecs/qtextcodec.cpp:1160: warning: Cannot find 'setCodecForTr(...)' in '\fn' QTextCodec::setCodecForTr ( QTextCodec * c ) Change-Id: I8d17705291b414d9c27827af4248bbf9e090962e Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* Doc: added doc to undocumented functionsNico Vertriest2015-05-181-3/+20
| | | | | | | Task-number: QTBUG-36985 Change-Id: Ia98654f88cf5da77245b3fcd903b860d12862fc2 Reviewed-by: Martin Smith <martin.smith@digia.com> Reviewed-by: Topi Reiniö <topi.reinio@digia.com>
* QTextCodec: micro-optimize loops.Friedemann Kleint2015-02-121-12/+15
| | | | | | | | Don't use index-based iteration, but use iterators. Change-Id: If8dabd56ff880191bb53861e31bfa6a007802c9c Reviewed-by: Marc Mutz <marc.mutz@kdab.com> Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* Update copyright headersJani Heikkinen2015-02-111-7/+7
| | | | | | | | | | | | | | | | | | Qt copyrights are now in The Qt Company, so we could update the source code headers accordingly. In the same go we should also fix the links to point to qt.io. Outdated header.LGPL removed (use header.LGPL21 instead) Old header.LGPL3 renamed to header.LGPL3-COMM to match actual licensing combination. New header.LGPL-COMM taken in the use file which were using old header.LGPL3 (src/plugins/platforms/android/extract.cpp) Added new header.LGPL3 containing Commercial + LGPLv3 + GPLv2 license combination Change-Id: I6f49b819a8a20cc4f88b794a8f6726d975e8ffbe Reviewed-by: Matti Paaso <matti.paaso@theqtcompany.com>
* Doc: corrected autolink errors corelib/ioNico Vertriest2014-11-261-2/+2
| | | | | | | Task-number: QTBUG-40362 Change-Id: I1cdbde1f6b003556ba4b5e97a49c6d918518da0d Reviewed-by: Venugopal Shivashankar <venugopal.shivashankar@digia.com> Reviewed-by: Jerome Pasion <jerome.pasion@digia.com>
* Load default codecs even if custom QTextCodec has been registeredKai Koehne2014-10-301-3/+8
| | | | | | Task-number: QTBUG-40378 Change-Id: I33f1e92127972e1346993aa4e07731bf4b697667 Reviewed-by: Lars Knoll <lars.knoll@digia.com>
* QTextCodec: Fix source code indentationKai Koehne2014-10-291-1/+1
| | | | | Change-Id: Ia9a79e659e028eb6173a7adef12d4973f78c32e9 Reviewed-by: Liang Qi <liang.qi@theqtcompany.com>
* Hardcode UTF-8 for "unicode" in QTextCodec::codecForHtml().Friedemann Kleint2014-10-281-1/+4
| | | | | | | | | | ICU would return a utf-16 (endian dependent) codec for unicode which is very rarely what people want. In most cases, unicode is encoded in utf8 these days, so return a utf8 codec for it. Task-number: QTBUG-41998 Change-Id: I51ee758d520702b263a8b2011787eb1f3455ed96 Reviewed-by: Lars Knoll <lars.knoll@digia.com>
* Doc: link to MIBenumNico Vertriest2014-10-271-4/+4
| | | | | | Task-number: QTBUG- 40362 Change-Id: I0261117d8aef8383ef77887a201d61ed0bd0ba52 Reviewed-by: Mitch Curtis <mitch.curtis@digia.com>
* Fix doc about the list of supported codecsAlexander Volkov2014-10-241-4/+3
| | | | | | | | | | | | | Replace "Apple Roman" by "Macintosh" which is registered by IANA: http://tools.ietf.org/html/rfc1345. Replace unsupported "GB18030-0" by "GB18030". Remove "JIS X 0201" and "JIS X 0208" as they are supported as parts of other Japanese encodings but not directly. Add "HP-ROMAN8" which is supported by both Qt and ICU. Also clean the codecs test. Change-Id: Iaf8e8ff1900d3f92ea0e0df75c60fe1534de23ac Reviewed-by: Lars Knoll <lars.knoll@digia.com>
* Update license headers and add new license filesMatti Paaso2014-09-241-19/+11
| | | | | | | | | - Renamed LICENSE.LGPL to LICENSE.LGPLv21 - Added LICENSE.LGPLv3 - Removed LICENSE.GPL Change-Id: Iec3406e3eb3f133be549092015cefe33d259a3f2 Reviewed-by: Iikka Eklund <iikka.eklund@digia.com>
* QTextCodec::codecForHtml looks at the first 1024 bytesShawn Rutledge2014-08-211-1/+1
| | | | | | | | The HTML 5 spec requires it, instead of only looking at the first 512. Task-number: QTBUG-40383 Change-Id: Ie10cf8c745ed1a3402914e126a02bc43d5035fff Reviewed-by: Mitch Curtis <mitch.curtis@digia.com>
* Make QTextDecoder use our qt_from_latin1 codeThiago Macieira2014-02-221-5/+5
| | | | | | | | | | | | Disassembly shows the Intel compiler does expand to SIMD, but a much worse code than ours. Clang 3.4 does generate a compact SIMD version, probably of the same quality as our hand-written code. And GCC 4.7 through 4.9 don't generate SIMD at all. So let's use the most efficient version. Change-Id: I418e201a774ac0df1fb2b7a7d9589df7c9b655db Reviewed-by: Lars Knoll <lars.knoll@digia.com>
* Doc: Adding mark-up to boolean default values.Jerome Pasion2013-10-081-2/+2
| | | | | | | | | | | | | | | | | Default values should have mark-up to denote that they are code. This commit changes: -"property is true" to "property is \c true". -"Returns true" to "Returns \c true". -"property is false" to "property is \c false". -"returns true" to "returns \c true". -"returns false" to "returns \c false". src/3rdparty and non-documentation instances were ignored. Task-number: QTBUG-33360 Change-Id: Ie87eaa57af947caa1230602b61c5c46292a4cf4e Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@digia.com> Reviewed-by: Jerome Pasion <jerome.pasion@digia.com>
generated by cgit v1.2.3 (git 2.25.1) at 2024-05-08 19:58:55 +0000