summaryrefslogtreecommitdiffstats
path: root/src/corelib/text/qstringconverter.cpp
Commit message (Collapse)AuthorAgeFilesLines
* Be explicit in a uint->QChar conversionGiuseppe D'Angelo2020-11-081-1/+3
| | | | | | | | And check that the result fits. Change-Id: Iaee1085315559bdffea9400b94b29869621ab7ff Reviewed-by: Edward Welbourne <edward.welbourne@qt.io> Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* Fix two missing conversions from in to qsizetypeAllan Sandfeld Jensen2020-10-291-1/+1
| | | | | Change-Id: I08ac9a0eb5dfb17f2b6cfaca9e3f3d7375c02bd4 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
* Doc: Fix documentation warnings for string classesTopi Reinio2020-10-291-43/+14
| | | | | | Task-number: QTBUG-86295 Change-Id: I13033635eecd495b32c98c7ec9318a0102b6f23e Reviewed-by: Paul Wicking <paul.wicking@qt.io>
* Convert a couple of APIs to use viewsLars Knoll2020-10-061-85/+106
| | | | | | | | | | | Try to get rid of APIs that use raw 'const {char, QChar} *, length' pairs. Instead, use QByteArrayView or QStringView. As QStringConverter is a new class, simply change the API to what we'd like to have. Also adjust hidden API in QStringBuilder and friends. Change-Id: I897d47f63a7b965f5574a1e51da64147f9e981f6 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
* Fix some MSVC conversion warningsFriedemann Kleint2020-09-221-1/+1
| | | | | Change-Id: Ib2c1fdb7b84f89201136438362ab5962126ec929 Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
* Only include <bit> if using post C++17Allan Sandfeld Jensen2020-09-151-1/+1
| | | | | | | Otherwise we hit an #error statement in MSVC standard library. Change-Id: Ib029edf0be8513a80f2640fd9ca75541615a0448 Reviewed-by: Giuseppe D'Angelo <giuseppe.dangelo@kdab.com>
* Use universal C++20 bit operations when availableAllan Sandfeld Jensen2020-09-131-0/+8
| | | | | | | Avoids using compiler builtins, and can in future replace them. Change-Id: I3f0afe7d28b6ba05bcd1c1132b44a8db7b182d8a Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* Initialize "uc1" variableWang Wei2020-09-081-1/+1
| | | | | | | remove warning for -Wmaybe-uninitialized. Change-Id: Ib17200eb86ee443875a9059b62762b2dc2100634 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
* Add x86 vector implementation of UTF-8 comparison to UTF-16Thiago Macieira2020-08-061-17/+121
| | | | | Change-Id: Ied637aece2a7427b8a2dfffd161181f75b738532 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
* Optimize non-vectorized UTF-8 to UTF-16 comparison to US-ASCIIThiago Macieira2020-08-051-13/+21
| | | | | | | | This allows us to skip the surrogate pair decoding too, since it can't match anyway. Change-Id: Ied637aece2a7427b8a2dfffd16118183e5d76794 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
* Implement UTF-16 to UTF-8 case-insensitive compare and make publicThiago Macieira2020-08-051-1/+1
| | | | | Change-Id: Ied637aece2a7427b8a2dfffd16116cf3645c6359 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
* Rename local variable hiding parameterDimitrios Apostolou2020-07-101-2/+2
| | | | | | | As found by LGTM.com. Change-Id: If3024bdaad06cafcd401b4ee04524ad03bd69a97 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* Remove local variable masking a global oneDimitrios Apostolou2020-07-101-2/+1
| | | | | | | As found by LGTM.com. Change-Id: Ied926a9eace9044ec9abc859557cd28178e5defb Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* Fix a crash in QClipBoard when copying utf16 dataLars Knoll2020-07-061-20/+0
| | | | | | | | | | | | Properly use the new QStringConverter API and not an internal qFromUtfEncoded method that was buggy after the changes. Take the oppportunity to clean up and remove qFromUtfEncoded, as QClipboard was its only user. Fixes: QTBUG-85417 Change-Id: I8540d12056bf3f448c1f628ce0bd0ad462a6447d Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
* Add explicit encode()/decode() methods to QStringConverterLars Knoll2020-06-261-0/+6
| | | | | | | | | The functional style interface is nice, but does feel alien in some contexts, so better also have explicit encode and decode methods. Change-Id: Ic07ced15f65cdb3a7f1cf044041e341d2ef87f79 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com> Reviewed-by: Alex Blasche <alexander.blasche@qt.io>
* Fix QString{Encoder|Decoder} documentationKarsten Heimrich2020-06-181-7/+9
| | | | | | Change-Id: I8e29846db77581953d90c818060950744cb9f521 Reviewed-by: Leena Miettinen <riitta-leena.miettinen@qt.io> Reviewed-by: Oliver Wolff <oliver.wolff@qt.io>
* Remove winrtOliver Wolff2020-06-061-16/+0
| | | | | | | | | Macros and the await helper function from qfunctions_winrt(_p).h are needed in other Qt modules which use UWP APIs on desktop windows. Task-number: QTBUG-84434 Change-Id: Ice09c11436ad151c17bdccd2c7defadd08c13925 Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
* Streamline the code in the conversion to and from utf8Lars Knoll2020-05-141-90/+82
| | | | | | | | | | | | | Move pre/and post condition handling out of the main loop to make that one as fast as possible. Remove special handling of a corner case when the input length is zero, where the utf8 decoder did something else than all other decoders. Change-Id: I94992767ea15405b38f7953adadaa6ff98b20b6f Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org> Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* Document the string converter classesLars Knoll2020-05-141-25/+390
| | | | | | | | | | Document QStringConverter, QStringDecoder and QStringEncoder. In addition, do some touches to the API, renaming one enum value, add a flags argument to one constructor and make some members private. Change-Id: I8f99dc3d98fb8860cf6fa46301e34b7eb400511b Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* Add a method to determine the encoding for encoded HTML dataLars Knoll2020-05-141-0/+48
| | | | | | | This is a replacement for Qt::codecForHtml(). Change-Id: I31f03518fd9c70507cbd210a8bcf405b6a0106b1 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* Add QStringConverter::encodingForData()Lars Knoll2020-05-141-0/+40
| | | | | | | | Add method that tries to determine the encoding of the data from an initial byte order mark. Change-Id: I348c51a3d4db9b434af53359b739a7e17acfc760 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* Add methods to convert between encoding and name to QStringConverterLars Knoll2020-05-141-4/+17
| | | | | | | | Add static methods that allow converting between a name for an encoding and the Encoding enum. Change-Id: I12bc503cf757ea31d3ca8d5e1f1216efddcb16d4 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* Construct a string converter by nameLars Knoll2020-05-141-9/+40
| | | | | | | | | | | Add a constructor, that allows constructing a string converter by name. This is required in some cases and also makes it possible to (in the future) extend the API to 3rd party encodings. Also add a name() accessor. Change-Id: I606d6ce9405ee967f76197b803615e27c5b001cf Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* Cleanup QUtf32::convertToUnicodeLars Knoll2020-05-141-64/+86
| | | | | | | | | | Cleanup the implementation and improve performance by handling the first char outside of the main loop. Also avoid one copy of the data when using QStringConverter. Change-Id: Ie698e62de1864352612a4dddc907cb139e7e6407 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* Refactor QUtf32::convertFromUnicodeLars Knoll2020-05-141-40/+64
| | | | | | | | Implement proper state handling, and avoid a copy when using it through QStringConverter. Change-Id: I201fe966601c424c337e452e359a2e71f76354ad Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* Clean up QUtf16::convertTo/FromUnicodeLars Knoll2020-05-141-97/+94
| | | | | | | | | | | Clean up the method, and refactor it so we can avoid one copy of the data when using QStringConverter. Make the conversion to unicode more by avoiding conditions in the inner loop and doing a memcpy if endianness matches. Change-Id: I869daf861f886d69b67a1b223ac2238498b609ac Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* Cleanup state handling in QUtf8::convertFromUnicodeLars Knoll2020-05-141-109/+106
| | | | | | | | And optimize the method so we can avoid a copy of the data. Change-Id: Ic267150db80358dbc4010bb1db2af5c0eb97dc65 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* Ensure the conversion methods in qstringconverter always get a valid stateLars Knoll2020-05-141-1/+41
| | | | | | | | | | | | Make sure that the conversion methods always get a valid state. This is already the ecase then using the new QStringConverter API, ensure the old QTextCodec API also passes in a valid state. This helps simplify the logic inside those methods. Change-Id: I1945e98cdefd46bf1427e11984337f1d62abcaa2 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com> Reviewed-by: Alex Blasche <alexander.blasche@qt.io>
* Clean up the Flag handling in QStringConverterLars Knoll2020-05-141-42/+51
| | | | | | | | | | | | | | | IgnoreHeader was a rather badly defined enum, in addition the utf8 and utf16 codecs where handling BOMs somewhat different for stateless decoding. Fix this by introducing explicit flags for writing a bom when encoding and not skipping the initial bom when decoding. Source compatibility for QTextCodec is done with a couple of static constexpr variables. Change-Id: I0b2d94f84c937cec1e0494c16ef448c00382691d Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* Add Latin1 to the set of supported encodings in QStringConverterLars Knoll2020-05-141-0/+31
| | | | | | | | | | | | | Latin1 is the only non Unicode encoding that is still being used to some extent. Current web site statistics show that it is being used in ~2% of all web sites. An additional 1% of web sites use Windows1251 (which is almost the same as latin1). As it's trivial to support this encoding, we keep it supported in QStringConverter. Change-Id: I0eff53a490b6c43d3e474107e7823be245d1715a Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* Move local8bit conversion over to qutfsupportLars Knoll2020-05-141-1/+205
| | | | | | | | Local8Bit is always UTF-8 except for Windows platforms. Also add a Locale encoding to QStringConverter. Change-Id: I8d729931fd4c1d7fc6857696b6442a44def3fd9d Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* Move the UTF conversion methods to qstringconverterLars Knoll2020-05-141-2/+947
| | | | | | | | | | | | | | Separate them from the qutfcodec, so that the codec can later on be moved out of Qt Core. Fix the QUtf methods to take qsizetype instead of int for length arguments. This also makes it possible to not build QTextCodec into the bootstrap lib anymore. Change-Id: I0b4f83139d61b19c651520a2f3a5012aa7e85cb8 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* Start work on a new API to replace QTextCodecLars Knoll2020-05-141-0/+186
The new QStringEncoder and QStringDecoder classes (with a common QStringConverter base class) are there to replace QTextCodec in Qt 6. It currently uses a trivial wrapper around the utf encoding functionality. Added some autotests, mostly copied from the text codec tests. Change-Id: Ib6eeee55fba918b9424be244cbda9dfd5096f7eb Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
generated by cgit v1.2.3 (git 2.25.1) at 2024-06-03 16:22:12 +0000