Harfbuzz-thai: Hide ZWJ and ZWNJ characters and show Inherited characters

Thai is not supposed to have ZWJ and ZWNJ characters or any other of the
Inherited Unicode Scripts
(http://www.verisigninc.com/assets/idn-inherited-unicode-script.pdf)
- they don't have a mapping to the thai encoding tis620 which libthai
requires.  However it is an unfortunate fact that there are many websites
etc that liberally place these ZWJ and ZWNJ characters throughout thai text
to force word boundaries, so we must also deal with them.

We deal with all Inherited characters by mapping them to the invalid code ~0
in tis620 encoding, following what libthai does internally in its own tis620
encoding functions, and then replacing this character with the original
unicode and setting dontPrint to true to hide the ZWJ and ZWNJ
characters.

Includes a unit test to check the behaviour.

Change-Id: I1ee8388b650cb5fc61bcb265efb9843c73f327ac
Reviewed-by: Adrian Yanes <ext-adrian.yanes@nokia.com>
Reviewed-by: Lars Knoll <lars.knoll@nokia.com>
Reviewed-by: Eskil Abrahamsen Blomfeldt <eskil.abrahamsen-blomfeldt@nokia.com>

1 files changed, 8 insertions, 3 deletions

diff --git a/src/3rdparty/harfbuzz/src/harfbuzz-thai.c b/src/3rdparty/harfbuzz/src/harfbuzz-thai.c
index 386fd5c252..bf6c35b19c 100644
--- a/src/3rdparty/harfbuzz/src/harfbuzz-thai.c
+++ b/src/3rdparty/harfbuzz/src/harfbuzz-thai.c
@@ -77,7 +77,7 @@ static void to_tis620(const HB_UChar16 *string, hb_uint32 len, const char *cstr)
         else if (string[i] >= 0xe01 && string[i] <= 0xe5b)
             result[i] = (unsigned char)(string[i] - 0xe00 + 0xa0);
         else
-            result[i] = '?';
+            result[i] = (unsigned char)~0; // Same encoding as libthai uses for invalid chars
     }
 
     result[len] = 0;
@@ -259,8 +259,13 @@ static HB_Bool HB_ThaiConvertStringToGlyphIndices (HB_ShaperItem *item)
         for (int lgi = 0; lgi < lgn; lgi++) {
             if ( rglyphs[lgi] == 0xdd/*TH_BLANK_BASE_GLYPH*/ ) {
                 glyphString[slen++] = C_DOTTED_CIRCLE;
-            }
-            else {
+            } else if (cstr[i] == (signed char)~0) {
+                // The only glyphs that should be passed to this function that cannot be mapped to
+                // tis620 are the ones of type Inherited class.  Pass these glyphs untouched.
+                glyphString[slen++] = string[i];
+                if (string[i] == 0x200D || string[i] == 0x200C)
+                    item->attributes[slen-1].dontPrint = true; // Hide ZWJ and ZWNJ characters
+            } else {
                 glyphString[slen++] = (HB_UChar16) thai_get_glyph_index (font_type, rglyphs[lgi]);
             }
         }