summaryrefslogtreecommitdiffstats
path: root/util
Commit message (Collapse)AuthorAgeFilesLines
* Update CLDR to v37, adding Nigerian Pidgin as a new languageEdward Welbourne2020-10-292-1/+18
| | | | | | | | | | | | | | | | | | | Routine update by running scripts, ignoring clang-format's extensive grumbles. Added notes to util/locale_database/'s README, on the need for that, and enumdata.py, on when to add entries. As usual, several new locales are also added, for existing languages, territories and scripts. [ChangeLog][QtCore][QLocale] Updated to new version of CLDR (the Unicode Consortium's Common Locale Data Repository) v37. Various Adlam-script locales are dropped due to its use of a number system unsupported by 5.15's QLocale. Support for these locales shall be restored in Qt 6. Task-number: QTBUG-84669 Change-Id: Ib76848bf4bd1219180faf46820077e8d8049a4e3 Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io> (cherry picked from commit cb23d50f38ec9d1daed570aa2b89e6d4b3f45c3f)
* Revert "WebGradients: redo implementation"Giuseppe D'Angelo2020-05-1510-1261/+59
| | | | | | | | | | | | | | | | | | | Q_ARRAY_LITERAL does not work properly in Qt 5, and no fix is planned for it, so this implementation (that relies on it) must be removed as well; it's causing regressions from Qt 5.14. More in detail: a QVector built using Q_ARRAY_LITERAL will not copy the existing contents when it detaches. This corrupts the preset gradients. A followup commit will also revert the corresponding QVector constructor so that no one will try to use it. This reverts commit 5b4b437b30b320e2cd7c9a566999a39772e5d431. Task-number: QTBUG-84069 Change-Id: Ia828f3e8a124331b58e0322e322707f4d54769f0 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* Fix parameter order in cldr2qlocalexml.py's usage()Edward Welbourne2020-04-061-1/+1
| | | | | | | Callers and definition were out of sync. Change-Id: Icda26887cb64c61c7e373766f25559b0d450d112 Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
* Ensure we use UTF-8 for the emitted QLocaleXML data fileEdward Welbourne2020-04-021-1/+5
| | | | | | | | | | | | | | | Python helpfully uses a sensible locale when stdout is a tty but uses the system (not the filesystem) default encoding, which may be ascii and unable to encode some of the data we need to save. So brute force kludge it to ensure emit.encoding is UTF-8 when writing the output we'll read as UTF-8 anyway. (This matches dev's commit 0ef79d94f6dcf276ca55b084d27f980b1f260473 for the reworked version of the script.) Task-number: QTBUG-79902 Change-Id: I60ddc896a308c06e01fa87e8e18e112faa17d601 Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
* Purge a stray space from calendar locale dataEdward Welbourne2020-04-021-1/+1
| | | | | | | | It was causing all lines after the first, in each calendar's locale_data[], to be over-indented. This only changes spacing. Change-Id: Ibfc4986548eecbfdba2902cc18f44a2af669bc6d Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
* Convert the qlocale2cpp's last few %-formats to modern format() styleEdward Welbourne2020-04-021-3/+3
| | | | | | | | | I've taken care of all the others in the course of other changes already ... Task-number: QTBUG-81344 Change-Id: I44e40a0d1c9f1e1a540a5f4cd252369fdc9b2698 Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
* Check all matches for each XPath when searchingEdward Welbourne2020-04-021-63/+67
| | | | | | | | | | | | | | | | | | | Previously, if we found one element with required attributes, we would search into it and ignore any later elements also with those required attributes. This meant that, if the first didn't contain the child elements we were looking for, we'd fail to find what we sought, if it was in a later matching element (e.g. with some ignored attributes). We would then go on to look for a match in a later file, where there might have been a match we should have found in the earlier file. Check all matches, rather than only the first match in each file. Do the search in each file "in parallel" to save reparsing the XPath. This clears the search code of rather hard-to-follow break/else handling in loops; and currently makes no change to the generated data. Change-Id: I86b010e65b9a1fc1b79e5fdd45a5aeff1ed5d5d5 Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
* Change QLocale to use CLDR's accounting formats for currenciesEdward Welbourne2020-04-021-1/+1
| | | | | | | | | | | | | | | | In particular, this changed the US currency formats for negative amounts to be parenthesised versions of the positive amount forms, rather than having a minus sign after the $ sign. Test updated. [ChangeLog][QtCore][QLocale] Currency formats are now based on CLDR's accounting formats, where they were previously mostly based (more or less by accident) on standard formats. In particular, this now means negative currency formats are specified, where available, where they (mostly) were not previously. Task-number: QTBUG-79902 Change-Id: Ie0c07515ece8bd518a74a6956bf97ca85e9894eb Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
* Take CLDR's distinguished attributes into accountEdward Welbourne2020-04-022-21/+97
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When doing XPATH searches, child nodes that have distinguished attributes that were not asked for should be skipped. This is part of the LDML spec and matters when resolving locale inheritance. Scan the LDML DTD (previously only scanned for the CLDR version) to find which attributes of which tags are ignorable - all others are distinguished - and take the result into account when performing XPATH searches. The XPath we were using for currency formats wasn't excluding currencyFormatLength elements with type="short" and patterns specific to thousands (and larger multiples); this is fixed by taking distinguished attributes into account. However, the XPATH also wasn't specifying the always distinguished attribute type="standard" that was, in practice, used for nearly all locales that weren't (wrongly) using short-forms for thousands; so type="standard" is now made explicit, so as to minimize the diff. This leaves only twenty-one locales with a negative currency formats. A later commit shall switch to using accounting by default (it falls back via an alias to standard, in any case), thereby restoring the two mentioned below that were using it by accident, but the present change gives the minimal diff here. Thousands-specific formats replaced with sensible ones: * zh_Hant_{HK,MO} (Traditional Mandarin, Hong Kong and Macau) * eo_001 (Esperanto) * fr_CA (Canadian French) * ha_* (Hausa, when not written in Arabic) * es_{GT,MX,US} (Spanish - Guatemala, Mexico, USA) * sw_KE (Swahili, Kenya) * yi_001 (Yiddish) * mfe_MU (Morisyen, Mauritius) * lag_TZ (Langi, Tanzania) * mgh_MZ (Makhuwa Meetto, Mozambique) * wae_CH (Walser, Switzerland) * kkj_CM (Kako, Cameroon) * lkt_US (Lakota, USA) * pa_Arab_PK (Punjabi, in Arabic script, as used in Pakistan; uses arabext number system, whose currency falls back to latn's, for which pa_Arab over-rides the thousands-format). Format changed from an over-ridden type="accounting" to standard (so these lost a negative-specific form) in: * en_SI (English, Slovenia) * es_DO (Spanish, Dominican Republic; same) For some locales we were picking up over-rides of narrow or short list formats, or formats for or-lists or unit-lists rather than and-lists, in place of the standard list format, that these locales don't over-ride, provided by a parent locale. This changed list formats for: * en_CA, en_IN (dropped "Oxford" comma before "and") * qu_* (Quechua; dropped "utaq", presumably meaning "and") * ur_IN (Urdu, India; was using unit-list formats) [ChangeLog][QtCore][QLocale] Data used for currency formats in several locales and list patterns in some locales have changed due to now parsing the CLDR data more faithfully. Fixes: QTBUG-81344 Change-Id: I6b95c6c37db92df167153767c1b103becfb0ac98 Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
* Take number system into account in currency format look-upEdward Welbourne2020-04-021-1/+6
| | | | | | | | | | | | | | | | | | | | | | | | CLDR's currency formats do have number system variation, so take it into account. (The old xpathlite code clearly intended to do this, but failed at it due to looking for the wrong component of an XPATH to fix.) This changes the currency formats in use for * all Dutch locales (because nl.xml lists a currency format for arab before the one for latn, and they differ), * Punjabi, Urdu - specifically pa_Guru_IN, ur_Arab_PK (both like Dutch, arabext before latn; which is correct for pa_Arab_PK and ur_Arab_IN), * Sindi (whose over-ride of latn currency format we were using, where we should be using arab's format, supplied by root's default), * Tatar (which specifies a generic currency format, which we were using, before one specific to latn, which we now use), * Tongan (same as Dutch), * Konkani (like Dutch, deva before latn) and * several North African Arabic locales (whose default number system is latn, rather than arab, but previously used arab's formats). Task-number: QTBUG-79902 Change-Id: I18d8ec16bfd3a516d1bcd2f63bc7f7f15179a3f4 Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
* Rework cldr2qlocalexml.py's reading of CLDR dataEdward Welbourne2020-04-025-898/+972
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Move the code out to a CldrReader class in cldr.py, expand CldrAccess with facilities that needs, expand ldml.py to include support for more features, finally making xpathlite.py redundant. This initial commit aims, though, to be bug-for-bug compatible with xpathlite in its reading of the CLDR data. It turns out we've been using draftier data than we were aware of (which might not be a bad thing). The xpathlite code appeared to check for draft attributes, but these only appear on leaf nodes and most data were fetched by finding a parent and then scanning its children without the draft check; only am/pm data was actually being excluded based on draft values. (We allowed contributed, for am/pm, in addition to approved, which is all the xpathlite code allows otherwise.) There are also some less equivocal bugs; I'll deal with these in later commits. Simplified number-system data look-ups; the old get_number_in_system() was taking care of old LDML versions' placement of the number system attribute; this is no longer needed. (It was also being used for a currency value to which it was not appropriate, which is now handled separately; this is one of the bugs mentioned above.) Ditched a fall-back to nativeZeroDigit, which no longer exists in CLDR. Change the command-line to take the root of the CLDR data tree, rather than its common/main/ sub-directory. Support naming the file to which to write output, as a second command-line argument, instead of always writing to stdout (which remains the default) and leaving whoever runs the script to redirect stdout. Support (internally for now, while adding TODOs to give main() more command-line options) separating the stderr output into its more and less interesting parts; for now, continue producing both, but suppress the least interesting entirely. Task-number: QTBUG-81344 Change-Id: Ie611b47403a9452b51feaeeaaa0fbc8f7e84dc71 Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
* Move cldr2qtimezone.py's CLDR-reading to a CldrAccess classEdward Welbourne2020-04-023-71/+339
| | | | | | | | | | | | This begins the process of replacing xpathlite.py, adding low-level DOM-access classes to ldml.py and the CldrAccess class to cldr.py Moved a format comment from cldr2qtimezone.py's doc-string to the method of CldrAccess that does the actual reading. Task-number: QTBUG-81344 Change-Id: I46ae3f402f8207ced6d30a1de5cedaeef47b2bcf Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
* Rework qlocalexml2cpp.py to use writers based on TranscriberEdward Welbourne2020-04-021-508/+438
| | | | | | | | | | | | | | | | | | This saves repetition of temporary-file manipulation code. In the process, ensure that we tidy away temporary files on failure. Moved a comment in qlocale.h to *outside* the re-written portion, to save having to rewrite it every time. Added blank lines to separate script data from country data in the generated output. Changed 0s in one comment to zeros, to match another comment. Isolated use of sys to the __main__ block. Isolated use of enumdata to the new LocaleHeaderWriter class. Modernised all the string-formatting I touched. Task-number: QTBUG-81344 Change-Id: I5768e45d9a8ea23facc303b3dd8af8b3ccbf7ff2 Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
* Rework cldr2qtimezone.py into more maintainable formEdward Welbourne2020-04-021-165/+160
| | | | | | | | | | | | | | | | | | | | | | | | | Broke out the updating of a source file to a ZoneIdWriter helper class, which enables tidying away the temporary file if we fail. Collected up the rest of the script into a main() that's now called from a __name__ == '__main__' block. Rationalized the imports. Eliminated an inefficient lookup function by constructing a suitable dict() before entering the loop that needed it. Separated the "data you might need to update" tables from the code that does the work, to make it easier for those adding support for new zones to see what they're doing. Removed the spurious $Revision$ from the output and reworded the premable of the generated file. (It would seem CLDR no longer uses an RCS-based version-control system.) Generated output is otherwise unchanged. Task-number: QTBUG-81344 Change-Id: I7d9de8357ebcb599d154de9f862e25f7ade00390 Reviewed-by: Lars Knoll <lars.knoll@qt.io> Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
* Add tools to localetools to facilitate source file recreationEdward Welbourne2020-04-021-0/+99
| | | | | | | | | | For now unused; later commits shall put them to use. Transcriber -- base, takes care of tempfile and renaming. SourceFileEditor -- handles copying parts before and after a common delimiter. Task-number: QTBUG-81344 Change-Id: I28cf977d0a08825fbb873fb330da6823b88ad3ed Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
* Move some shared code to a localetools moduleEdward Welbourne2020-04-026-75/+105
| | | | | | | | | | | | | | | The time-zone script was importing two functions from the locale data generation script. Move them to a separate module, to which I'll shortly add some more shared utilities. Cleaned up some imports in the process. Combined qlocalexml2cpp's and xpathlit's error classes into a new Error class in the new module and made it a bit more like a proper python error class. Task-number: QTBUG-81344 Change-Id: Idbe0139ba9aaa2f823b8f7216dee1d2539c18b75 Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
* Move qlocalexml2cpp.py's XML-reading to QLocaleXmlReaderEdward Welbourne2020-04-022-277/+223
| | | | | | | | | | | | | | | | | | | | This new class mirrors the existing QLocaleXmlWriter and places the two side-by-side in qlocalexml.py, rather than having the writing and reading in separate places. Made judicious use of transformed versions of mappings to save repeated iteration of a mapping's entries to do lookups on fist entries of pair-values; several (id, name, code) data-sets are sometimes indexed by id, sometimes by name. Reworked the default_map, that the complicated compareLocaleKeys() used in sorting locale keys, to map IDs instead of names; the function also needed the locale_map so that it could convert IDs to names, which we can skip by going directly with IDs. Task-number: QTBUG-81344 Change-Id: Iff6a97f7f0755b56dda70d8a6796ec074c558910 Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
* Rework cldr2qlocalexml.py in terms of a QLocaleXmlWriter classEdward Welbourne2020-04-022-197/+330
| | | | | | | | | | | | | Delegate the output of XML to a helper class provided by qlocalexml.py and restructure the driver script so that it can be imported without running anything. It now has a minimal __name__ == '__main__' block that calls a main() function. This, for the moment, requires a global via which it shares the CLDR directory with various other functions; that shall go away in a later commit. Task-number: QTBUG-81344 Change-Id: Ica2d3ec09f2d38ba42fd930258cc765283f29a71 Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
* Deduplicate day-name data in QLocaleXML filesEdward Welbourne2020-03-161-2/+0
| | | | | | | | | | This is a follow-up to commit ebb0212133bd91f1da4931b29eb1d33fb77b1444. The day name data appeared twice in the XML files. Skip the second copy, saving 8.8% of the intermediate file-size. This makes no change to generated QLocale data. Change-Id: Ic2cc543a2a85cbb1d2d47ebac7df4fa9ad6ee0a7 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
* Update UCD to Revision 26Edward Welbourne2020-03-1416-190/+1503
| | | | | | | | | | | | | | Include WordBreakTest.html, since a test uses sample strings from it, albeit without actually reading the file. Had to comment out more of the new tests, as at Revision 24, pending an update to harfbuzz and the text boundary detection code. Task-number: QTBUG-79631 Task-number: QTBUG-79418 Task-number: QTBUG-82747 Change-Id: I0082294b09d67ffdc6a9b5c15acf77ad3b86f65f Reviewed-by: Lars Knoll <lars.knoll@qt.io>
* Rename the localexml module to qlocalexmlEdward Welbourne2020-03-033-4/+4
| | | | | | | | | It implements interaction with the QLocaleXML file format type, so rename it to match. Task-number: QTBUG-81344 Change-Id: I46302d4ac1038cdfc5929e73b554b6d793814c56 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
* Rename the endonym members of the Locale typeEdward Welbourne2020-03-032-7/+6
| | | | | | | | | | | | | All other members had camelCase names, but the endonyms had prefix_endonym names, requiring munging where they were emitted to XML. So just do that munging upstream in the attribute name of the Locale objects. Makes no change to the data output by the scripts, not even to the intermediate QLocaleXML file. Task-number: QTBUG-81344 Change-Id: I01c15a822216281dc669b3e7ebda096d18b04f9b Reviewed-by: Lars Knoll <lars.knoll@qt.io> Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
* Use Qt::SplitBehavior in preference to QString::SplitBehaviorEdward Welbourne2020-02-281-1/+1
| | | | | | | | The Qt version was added in 5.14 "for use as eventual replacement for QString::SplitBehavior." Move another step closer to that goal. Change-Id: I399b5ea56e9255e775ca1746632f7421519a6616 Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
* Merge remote-tracking branch 'origin/5.14' into 5.15Qt Forward Merge Bot2020-02-042-1/+2
|\ | | | | | | | | | | | | | | | | | | Conflicts: .qmake.conf examples/widgets/widgets/imageviewer/imageviewer.cpp src/corelib/text/qchar.cpp src/corelib/time/qdatetime.cpp Change-Id: I9762f5c4ff650799219729d6aee79ac07ce9024a
| * Enable system locale to skip digit-grouping if configured to do soEdward Welbourne2020-02-031-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | On macOS it's possible to configure the system locale to not do digit grouping (separating "thousands", in most western locales); it then returns an empty string when asked for the grouping character, which QLocale's system-configuration then ignored, falling back on using the base UI locale's grouping separator. This could lead to the same separator being used for decimal and grouping, which should never happen, least of all when configured to not group at all. In order to notice when this happens, query() must take care to return an empty QString (as a QVariant, which is then non-null) when it *has* a value for the locale property, and that value is empty, as opposed to a null QVariant when it doesn't find a configured value. The caller can then distinguish the two cases. Furthermore, the group and decimal separators need to be distinct, so we need to take care to avoid cases where the system overrides one with what the CLDR has given for the other and doesn't over-ride that other. Only presently implemented for macOS and MS-Win, since the (other) Unix implementation of the system locale returns single QChar values for the numeric tokens - see QTBUG-69324, QTBUG-81053. Fixes: QTBUG-80459 Change-Id: Ic3fbb0fb86e974604a60781378b09abc13bab15d Reviewed-by: Ulf Hermann <ulf.hermann@qt.io>
| * glgen: Fix compile errorJohan Klokkhammer Helsing2020-01-311-1/+1
| | | | | | | | | | | | | | Fixes compile error introduced in ff2b2032a08. Change-Id: Iee1016f7cf05638db575475c5c0d632f8edb3a8c Reviewed-by: Sona Kurazyan <sona.kurazyan@qt.io>
* | WebGradients: redo implementationGiuseppe D'Angelo2019-12-1410-59/+1261
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The previous implementation was *extremely* expensive. It relied on loading a binary JSON file from resources (which involved decompressing it), then extracting information out of it to build a gradient. Already-loaded gradients were kept in a local cache, which had to be mutex protected. Instead, this patch extends the gradient generator to build static arrays filled with the web gradient data, sitting in .rodata. These arrays are used when building QGradient objects with a web gradient. No explicit mutex protection is necessary, since accesses will just read from the arrays. As benefits, this patch removes: * the binary json representation from QtGui's resources (~4KB compressed, ~50KB uncompressed) * the overhead of reading from the JSON for each used web gradient; * the startup costs of registering the webgradients in the resources; * all the overhead of mutex locking when building such gradients; * all the runtime memory allocations to load, parse and cache the web gradients (including the memory + CPU spike on first load due to the uncompression of the JSON data, as well as a couple of deep copies). Change-Id: If5c3d704430df76ce8faf55ee75ebd4639ba09c4 Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io> Reviewed-by: Ulf Hermann <ulf.hermann@qt.io> Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org> Reviewed-by: Thiago Macieira <thiago.macieira@intel.com> Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
* | Unicode tables: minor prettificationEdward Welbourne2019-11-281-6/+10
| | | | | | | | | | | | | | | | | | | | | | Put blank lines before the final Num*Classes entries in enums, to set them off visibly from the "real" members. Moved some oddly placed commas to the ends of preceding lines, so that later additions can just add lines (with comma on end) without having to modify the preceding line while doing so. Change-Id: I5188dc25af9e4c17a1882fd9dab070e88013060b Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* | Add missing docs for UCD additions at 5.15Edward Welbourne2019-11-282-2/+5
| | | | | | | | | | | | | | | | | | Also remove two stray commas pointed out in code-review and some others noticed on checking for similar. This amends commit c3eb521a0f10112df6b61d2592351c4eef2e1f9b. Change-Id: If20c5146b740defe8d25ff61d399031b5c66ded1 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
* | Merge remote-tracking branch 'origin/5.14' into 5.15Qt Forward Merge Bot2019-11-081-2/+2
|\| | | | | | | Change-Id: I963b5f48b5d6f3500ca379fbd7f1a4290b570175
| * Merge remote-tracking branch 'origin/5.13' into 5.14Qt Forward Merge Bot2019-11-061-2/+2
| |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: .qmake.conf mkspecs/features/mac/default_post.prf src/corelib/tools/qsimd_p.h src/platformsupport/fontdatabases/mac/qcoretextfontdatabase.mm util/qfloat16-tables/gen_qfloat16_tables.cpp Change-Id: If48fa8a3bc3c983706b609a6d3822cb67c1352a4
* | | Update UCD data to Unicode 12.1.0's Revision 24Edward Welbourne2019-10-3016-611/+2614
|/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Had to teach the update program to accept category Lm as for Joining_Transparent, for the sake of a new ArabicShaping.txt entry. Added three new Unicode versions, several new scripts and a new word-break class. Updated UCD's test data for tst_QTextBoundaryFinder. This left 57 tests failing; I have commented out the data rows for those tests, pending someone with more knowledge addressing this. Task-number: QTBUG-79631 Task-number: QTBUG-79418 Change-Id: Ic33d3b3551195d47a84d98e84020f57a68f0b201 Reviewed-by: Eskil Abrahamsen Blomfeldt <eskil.abrahamsen-blomfeldt@qt.io>
* | Update CLDR to v36Edward Welbourne2019-10-252-3/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Released on October 4th. Adds Windows names for two time zones, Qyzylorda and Volgograd. Added languages Chickasaw (cic), Muscogee (mus) and Silesian (szl). Norwegian number formatting has flipped back to using colon rather than dot as time separator; it's flipped back and forth over the last several CLDR releases. The dot form is present as a variant, the colon form was long given as the normal pattern, then went away; but now it's back as a contributed draft and that's what we pick up. The MS-Win time-zone ID script was iterating a dict, causing random reshuffling when new entries are added. Fixed that by doing the critical iteration in sorted order. Omitted locales ccp_BD and ccp_IN due to QTBUG-69324. Task-number: QTBUG-79418 Change-Id: I43869ee1810ecc1fe876523947ddcbcddf4e550a Reviewed-by: Lars Knoll <lars.knoll@qt.io>
* | Correct some references to corelib/tools/ to say corelib/text/Edward Welbourne2019-10-252-3/+3
| | | | | | | | | | | | | | | | | | | | The Unicode data tables moved with QString and friends. So did the locale data generated from CLDR. This amends commit a9aa206b7b8ac4e69f8c46233b4080e00e845ff5. Change-Id: If12f0420b559dcb78993adc00e9f39751bca684a Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
* | Compile-fixes in the generator for qurltld_p.hEdward Welbourne2019-10-251-3/+3
| | | | | | | | | | | | | | This is a follow-up to c0ab2ad98f5c3907be032dc4de938ba5d88e697d. Change-Id: Ic05d80fa0561f7609703407cc58a0caccbcb1061 Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
* | QUnicodeTables: use array for case folding tablesMarc Mutz2019-09-041-60/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Instead of four pairs of :1 :15 bit fields, use an array of four :1, :15 structs. This allows to replace the case folding traits classes with a simple enum that indexes into said array. I don't know what the WASM #ifdef'ed code is supposed to effect (a :0 bit-field is only useful to separate adjacent bit-field into separate memory locations for multi-threading), but I thought it safer to leave it in, and that means the array must be a 64-bit block of its own, so I had to move two fields around. Saves ~4.5KiB in text size on optimized GCC 10 LTO Linux AMD64 builds. Change-Id: Ib52cd7706342d5227b50b57545d073829c45da9a Reviewed-by: Lars Knoll <lars.knoll@qt.io>
* | QUnicodeTables: pack Properties structMarc Mutz2019-09-041-4/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | GCC doesn't like the sequence : 5 : 5 : 8 : 6 : 8 and inserts a :6 padding between the :5 and the :8 and a :2 padding between the :6 and the :8, growing the bitfield by 8 bits of embedded padding and another byte to bring the struct back to sizeof % 2 == 0. Fix by reshuffling the elements and adding a static_assert for the next round. Saves ~5KiB in QtCore executable size. Change-Id: I4758a6f48ba389abc2aee92f60997d42ebb0e5b8 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* | Add support for the Islamic Civil calendarSoroush Rabiei2019-08-223-3/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This has its own locale data, extracted from CLDR. This data may potentially be shared with other variants on the Islamic calendar, so is handled by a separate base-class, QHijriCalendar, on which such variants may base their implementations. [ChangeLog][QtCore][QCalendar] Added support for the Islamic Civil calendar, controlled by feature islamiccivilcalendar, with locale data that can be shared with other implementations, controlled by feature hijricalendar. Fixes: QTBUG-56675 Change-Id: Idf32d3da7034baa8ec5e66ef847e59a8a2f31cbd Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
* | Add support for the Jalali (Solar Hijri or Persian) calendarSoroush Rabiei2019-08-213-2/+7
| | | | | | | | | | | | | | | | | | | | | | | | This has its own locale data, extracted from CLDR. [ChangeLog][QtCore][QCalendar] Added support for the Jalali (Persian or Solar Hijri) calendar, controlled by feature jalalicalendar. Fixes: QTBUG-58404 Change-Id: Id5c56a10db05a4fd612aafc01615273db81ec743 Reviewed-by: Paul Wicking <paul.wicking@qt.io> Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
* | Add support for calendars beside GregorianSoroush Rabiei2019-08-203-54/+138
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add QCalendarBackend as a base class for calendar implementations and QCalendar as a facade via which to access it. QDate's implicit implementation of the Gregorian calendar becomes QGregorianCalendar and QDate methods now support choice of calendar. Convert QLocale's CLDR data for month names to a locale-data component of each supported calendar and relevant QLocale methods now support choice of calendar. Adapt Python scripts for locale data generation to extract month name data from CLDR (keeping on version v35.1) into the new calendar-locale files. The locale data for the Gregorian calendar is held in a Roman calendar base, for sharing with other calendars. Add tests for basic uses of the new API. [ChangeLog][QtCore][QCalendar] Added QCalendar to support diverse calendars, supported by implementing QCalendarBackend. [ChangeLog][QtCore][QDate] Allow choice of calendar in various operations, with Gregorian remaining the default. Done-with: Lars Knoll <lars.knoll@qt.io> Done-with: Edward Welbourne <edward.welbourne@qt.io> Fixes: QTBUG-17110 Fixes: QTBUG-950 Change-Id: I9d6278f394269a183aee8156e990cec4d5198ab8 Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
* | Extract a large format string as a module constant valueSoroush Rabiei2019-08-081-14/+15
| | | | | | | | | | | | | | | | | | The template for the "This is a generated file" notice made a clumsy intrusion in the code in which it appeared, so split it out as a constant of the module and access it by name where it's used. Change-Id: Ic4dfb8e873078c54410b191654d6c21d082c9016 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
* | Move text-related code out of corelib/tools/ to corelib/text/Edward Welbourne2019-07-104-14/+14
| | | | | | | | | | | | | | | | This includes byte array, string, char, unicode, locale, collation and regular expressions. Change-Id: I8b125fa52c8c513eb57a0f1298b91910e5a0d786 Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
* | Add data for Windows Time-Zone IDs added in the last two yearsEdward Welbourne2019-07-011-0/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We've not run util/locale_database/cldr2qtimezone.py for a while, so CLDR has had time to add several more zones. Catch up, inserting the new entries in order. Change-Id: I8625548b0f7775958230eccbd89b897d7afed9e9 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* | Tidy up in cldr2qtimezone.py and document the need to run itEdward Welbourne2019-07-012-12/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It wasn't mentioned in cldr2qlocalexml.py's instructions, so I didn't know to run it. The data it used in an illustration was out of date. Two tests could be combined with no loss. Change-Id: I26e619e6210ea5b1258326fc4bc2b6aee9d6a999 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
* | cldr2qtimezone.py: report all missing zones, rather than just the firstEdward Welbourne2019-07-011-1/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When scanning the CLDR data, the script raised an exception if it didn't recognize a zone ID. Instead, collect up such unrecognized IDs in a list and report them all at the end, so that whoever runs this can do them all in one go, rather than doing one, running the script, doing the next, running the script, ad nauseam. Change-Id: Ia659f1d1c7e1c1b4ccb87cc23828a0588a5bf958 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
* | Use simpler data structures in cldr2qtimezone.pyEdward Welbourne2019-07-011-169/+165
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Use tuples for the fixed data. The numbering of rows in the data tables isn't part of any public API, so we can change it freely; it is thus unnecessary, as we can just enumerate a tuple of the data values to generate sequential indices on the fly. (Updates to the data shall no longer need to renumber in order to insert entries.) Restore ordering of the data tables, and remove wanton spacing from inside parens, in the process. Change-Id: I59956cfb6191fe729300b57070671b7e66bd0379 Reviewed-by: Konstantin Ritt <ritt.ks@gmail.com> Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
* | Remove usages of deprecated APIs from QtAlgorithmsSona Kurazyan2019-06-296-10/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Task-number: QTBUG-76491 Change-Id: I9dab736a0cbd2e86588919640c26e8ce6b3674d0 Reviewed-by: Alex Blasche <alexander.blasche@qt.io> Reviewed-by: Leena Miettinen <riitta-leena.miettinen@qt.io>
* | gen_qfloat16_tables: add a comment explaining what the tool isThiago Macieira2019-06-031-1/+13
| | | | | | | | | | | | | | | | For our future selves. Change-Id: I2b1955a995ad40f3b89afffd15a4c127686662bc Reviewed-by: Allan Sandfeld Jensen <allan.jensen@qt.io>
* | qfloat16: Check in the tables to GitThiago Macieira2019-06-032-0/+153
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It's a 38k source file, which makes it MUCH smaller than other generated files like qlocale_data_p.h (982k) and qunicodetables.cpp (718k). The constants are platform-independent, since they are defined by IEEE 754, so they will never change. The generator tool is moved to util/ and removed from the build. That's one fewer bootstrapped tool to have to worry about. The output file is committed as .cpp so it won't get installed. Fixes: QTBUG-76165 Change-Id: I2b1955a995ad40f3b89afffd15a3ded58dc3e35f Reviewed-by: Allan Sandfeld Jensen <allan.jensen@qt.io>
* | Separate out the time, zone, date code from corelib/tools/Edward Welbourne2019-06-061-2/+2
| | | | | | | | | | | | | | | | | | | | We'll be adding calendar code here as well, and tools/ was getting rather crowded, so it looks like time to move out a reasonably coherent sub-bundle of it all. Change-Id: I7e8030f38c31aa307f519dd918a43fc44baa6aa1 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
generated by cgit v1.2.3 (git 2.25.1) at 2024-04-30 22:58:55 +0000