Allow surrogate pairs for various "single character" locale data

Extract the character in its proper unicode form and encode it in a
new single_character_data table of locale data. Record each entry as
the range within that table that encodes it. Also added an assertion
in the generator script to check that the digits CLDR gives us are a
contiguous sequence in increasing order, as has been assumed by the
C++ code for some time. Lots of number-formatting code now has to take
account of how wide the digits are.

This leaves nowhere for updateSystemPrivate() to record values read
from sys_locale->query(), so we must always consult that function when
accessing these members of the systemData() object. Various internal
users of these single-character fields need the system-or-CLDR value
rather than the raw CLDR value, so move QLocalePrivate's methods to
supply them down to QLocaleData and ensure they check for system
values, where appropriate first.

This allows us to finally support the Chakma language and script, for
whose number system UTF-16 needs surrogate pairs.

Costs 10.8 kB in added data, much of it due to adding two new locales
that need surrogates to represent digits.

[ChangeLog][QtCore][QLocale] Various QLocale methods that returned
single QChar values now return QString values to accommodate those
locales which need a surrogate pair to represent the (single
character) return value.

Fixes: QTBUG-69324
Fixes: QTBUG-81053
Change-Id: I481722d6f5ee266164f09031679a851dfa6e7839
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>

1 files changed, 4 insertions, 4 deletions

diff --git a/src/corelib/text/qlocale_tools_p.h b/src/corelib/text/qlocale_tools_p.h
index e2142bf545..f04c4512c0 100644
--- a/src/corelib/text/qlocale_tools_p.h
+++ b/src/corelib/text/qlocale_tools_p.h
@@ -67,7 +67,7 @@ double qt_asciiToDouble(const char *num, int numLen, bool &ok, int &processed,
 void qt_doubleToAscii(double d, QLocaleData::DoubleForm form, int precision, char *buf, int bufSize,
                       bool &sign, int &length, int &decpt);
 
-QString qulltoa(qulonglong l, int base, const QChar _zero);
+QString qulltoa(qulonglong l, int base, const QStringView zero);
 Q_CORE_EXPORT QString qdtoa(qreal d, int *decpt, int *sign);
 
 enum PrecisionMode {
@@ -76,13 +76,13 @@ enum PrecisionMode {
     PMChopTrailingZeros =   0x03
 };
 
-QString &decimalForm(QChar zero, QChar decimal, QChar group,
+QString &decimalForm(const QString &zero, const QString &decimal, const QString &group,
                      QString &digits, int decpt, int precision,
                      PrecisionMode pm,
                      bool always_show_decpt,
                      bool thousands_group);
-QString &exponentForm(QChar zero, QChar decimal, QChar exponential,
-                      QChar group, QChar plus, QChar minus,
+QString &exponentForm(const QString &zero, const QString &decimal, const QString &exponential,
+                      const QString &group, const QString &plus, const QString &minus,
                       QString &digits, int decpt, int precision,
                       PrecisionMode pm,
                       bool always_show_decpt,