summaryrefslogtreecommitdiffstats
path: root/src/corelib/text/qlocale_p.h
Commit message (Collapse)AuthorAgeFilesLines
* Use char16_t in favor of ushort for locale data tablesEdward Welbourne2020-02-171-7/+7
| | | | | | Change-Id: I890dd2b52c1b786db1081744c8ca343baba93de4 Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org> Reviewed-by: Lars Knoll <lars.knoll@qt.io>
* Allow surrogate pairs for various "single character" locale dataEdward Welbourne2020-02-171-43/+54
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Extract the character in its proper unicode form and encode it in a new single_character_data table of locale data. Record each entry as the range within that table that encodes it. Also added an assertion in the generator script to check that the digits CLDR gives us are a contiguous sequence in increasing order, as has been assumed by the C++ code for some time. Lots of number-formatting code now has to take account of how wide the digits are. This leaves nowhere for updateSystemPrivate() to record values read from sys_locale->query(), so we must always consult that function when accessing these members of the systemData() object. Various internal users of these single-character fields need the system-or-CLDR value rather than the raw CLDR value, so move QLocalePrivate's methods to supply them down to QLocaleData and ensure they check for system values, where appropriate first. This allows us to finally support the Chakma language and script, for whose number system UTF-16 needs surrogate pairs. Costs 10.8 kB in added data, much of it due to adding two new locales that need surrogates to represent digits. [ChangeLog][QtCore][QLocale] Various QLocale methods that returned single QChar values now return QString values to accommodate those locales which need a surrogate pair to represent the (single character) return value. Fixes: QTBUG-69324 Fixes: QTBUG-81053 Change-Id: I481722d6f5ee266164f09031679a851dfa6e7839 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* Merge remote-tracking branch 'origin/5.15' into devLiang Qi2020-02-131-1/+1
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: examples/widgets/graphicsview/boxes/scene.h src/corelib/Qt5CoreMacros.cmake src/corelib/Qt6CoreMacros.cmake src/network/ssl/qsslsocket.cpp src/network/ssl/qsslsocket.h src/platformsupport/fontdatabases/windows/qwindowsfontenginedirectwrite.cpp src/testlib/CMakeLists.txt src/testlib/.prev_CMakeLists.txt tests/auto/corelib/tools/qscopeguard/tst_qscopeguard.cpp Disabled building manual tests with CMake for now, because qmake doesn't do it, and it confuses people. Done-With: Alexandru Croitor <alexandru.croitor@qt.io> Done-With: Volker Hilsheimer <volker.hilsheimer@qt.io> Change-Id: I865ae347bd01f4e59f16d007b66d175a52f1f152
| * Enable system locale to skip digit-grouping if configured to do soEdward Welbourne2020-02-031-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | On macOS it's possible to configure the system locale to not do digit grouping (separating "thousands", in most western locales); it then returns an empty string when asked for the grouping character, which QLocale's system-configuration then ignored, falling back on using the base UI locale's grouping separator. This could lead to the same separator being used for decimal and grouping, which should never happen, least of all when configured to not group at all. In order to notice when this happens, query() must take care to return an empty QString (as a QVariant, which is then non-null) when it *has* a value for the locale property, and that value is empty, as opposed to a null QVariant when it doesn't find a configured value. The caller can then distinguish the two cases. Furthermore, the group and decimal separators need to be distinct, so we need to take care to avoid cases where the system overrides one with what the CLDR has given for the other and doesn't over-ride that other. Only presently implemented for macOS and MS-Win, since the (other) Unix implementation of the system locale returns single QChar values for the numeric tokens - see QTBUG-69324, QTBUG-81053. Fixes: QTBUG-80459 Change-Id: Ic3fbb0fb86e974604a60781378b09abc13bab15d Reviewed-by: Ulf Hermann <ulf.hermann@qt.io>
* | Separate offsets from sizes in QLocale's dataEdward Welbourne2020-01-301-54/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This enables us to make the sizes quint8 and benefit from the resulting packing, making the locale data smaller. The sizes for long month-name lists (which concatenate twelve names with semicolon as separator) can overflow an 8-bit member, so use quint16 where needed. Re-ordered the data in QLocaleData and QCalendarLocale. Now all long-short(-narrow) families arise in that order; and any standalone is grouped with the one of the same length. (This cost 20 bytes in the date-format table, which optimises out more duplication if short is before long, but the saving in the (smaller) time-format table more than make up for it; and 20 bytes isn't worth the confusion that being inconsistent in ordering might cause.) At the same time, drop trailing semicolons from list entries (which join various names with semicolon) as they're not needed: we know where the end of the list is, because we know the size of the string that results from concatenation. The code that parses such lists can even correctly handle empty entries at the end. Saves 26 kB of data in the compiled binaries. Task-number: QTBUG-81053 Change-Id: If6ccc96a6910828817aa605d10fd814f567ae1e8 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com> Reviewed-by: Lars Knoll <lars.knoll@qt.io>
* | Refactor QLocale's data access to be less verboseEdward Welbourne2020-01-301-1/+70
| | | | | | | | | | | | | | | | | | | | | | | | | | Add QLocaleData::DataRange and methods returning it, to package each of the m_*_idx, m_*_size pairs of data members, to simplify access to these data. This extends the experiment started in QCalendarLocale, which is now adapted to use the new DataRange also. Two static functions of qlocale.cpp are replaced by methods of DataRange, saving considerable duplication of long member names in callers. Change-Id: Iad9899ba72f00522594b55a0402baec47491999c Reviewed-by: Thiago Macieira <thiago.macieira@intel.com> Reviewed-by: Lars Knoll <lars.knoll@qt.io>
* | Preserve the case of the exponent separator CLDR suppliesEdward Welbourne2020-01-301-1/+1
|/ | | | | | | | | | | | | | | | | | | | | | We have long (since 4.5.1) coerced it to lower-case, for no readily apparent, much less documented, reason. CLDR says most locales use an upper-case E for this - let's actually use what CLDR says we should use. The code that matches the exponent separator was doing so case-insensitively in any case; that needed adaptation now that the separator's case isn't pre-determined; and, in any case, should have been done using case-folding rather than upper-casing. In the process, removed some spurious checks for "'e' or 'E'" in the result, since the exponent separator is always represented by 'e' (and an 'e' might also be present for the separate reason of its use as a beyond-decimal digit representing fourteen). [ChangeLog][QtCore][QLocale] QLocale::exponential() now preserves the case of the CLDR source, where previously it was lower-cased. Change-Id: Ic9ac02136cff79cb9f136d72141b5dbf54d9e0a6 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
* QLocale: Support Indian number formattingTuomas Heimonen2019-12-301-1/+2
| | | | | | | | | | | When QLocale::Country is set to QLocale::India numbers are written so that after first three from the right and then after every second will be comma. E.g. 10000000 is written as 1,00,00,000 Task-number: QTBUG-24301 Change-Id: Ic06241c127b0af1824104f94f7e2ce6e2058a070 Reviewed-by: Venugopal Shivashankar <Venugopal.Shivashankar@qt.io> Reviewed-by: Teemu Holappa <teemu.holappa@qt.io>
* Include likely-adjusted uiLanguages for the system localeEdward Welbourne2019-10-041-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | QLocale::uiLanguages() on the system locale uses whatever the system locale's query(QSystemLocale::UILanguages,...) returns. On Android, this is just a list of locales. However, for non-system locales, we also include some results of removing likely sub-tags from the locale name, where equivalent. Thus zh-CN would also get zh and zh-Hans-CN added to it; however, if the system locale is zh-Hans-CN, the shorter forms are omitted. So post-process the system locale list in the same way, albeit tweaked to avoid duplicates and rearranged so that we can insert likely-adjusted entries between what they adjust and what followed it. Added QLocalePrivate::rawName() in the process, since it looks likely to be useful in other contexts (and I needed its value): it just joins such tags as are non-Any. This, however, uses QByteArrayList, so added that (it's small) to the bootstrap library and qmake. This follows up on commit 8796e3016fae1672e727e2fa4e48f671a0c667ba. [ChangeLog][QtCore][QLocale] The system locale's UI languages list now includes, as for that of an ordinary locale, the results of adding likely sub-tags from each locale name, and of removing some, where this doesn't change which locale is specified. This gives searches for translation files a better chance of finding a suitable file. Fixes: QTBUG-75413 Change-Id: Iaafd79aac6a0fdd5f44aed16e445e84a2267c9da Reviewed-by: Paul Wicking <paul.wicking@qt.io> Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org> Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* Add support for calendars beside GregorianSoroush Rabiei2019-08-201-16/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add QCalendarBackend as a base class for calendar implementations and QCalendar as a facade via which to access it. QDate's implicit implementation of the Gregorian calendar becomes QGregorianCalendar and QDate methods now support choice of calendar. Convert QLocale's CLDR data for month names to a locale-data component of each supported calendar and relevant QLocale methods now support choice of calendar. Adapt Python scripts for locale data generation to extract month name data from CLDR (keeping on version v35.1) into the new calendar-locale files. The locale data for the Gregorian calendar is held in a Roman calendar base, for sharing with other calendars. Add tests for basic uses of the new API. [ChangeLog][QtCore][QCalendar] Added QCalendar to support diverse calendars, supported by implementing QCalendarBackend. [ChangeLog][QtCore][QDate] Allow choice of calendar in various operations, with Gregorian remaining the default. Done-with: Lars Knoll <lars.knoll@qt.io> Done-with: Edward Welbourne <edward.welbourne@qt.io> Fixes: QTBUG-17110 Fixes: QTBUG-950 Change-Id: I9d6278f394269a183aee8156e990cec4d5198ab8 Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
* Move text-related code out of corelib/tools/ to corelib/text/Edward Welbourne2019-07-101-0/+471
This includes byte array, string, char, unicode, locale, collation and regular expressions. Change-Id: I8b125fa52c8c513eb57a0f1298b91910e5a0d786 Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>