diff options
Diffstat (limited to 'src/3rdparty/freetype/src/gxvalid/README')
| -rw-r--r-- | src/3rdparty/freetype/src/gxvalid/README | 532 |
1 files changed, 532 insertions, 0 deletions
diff --git a/src/3rdparty/freetype/src/gxvalid/README b/src/3rdparty/freetype/src/gxvalid/README new file mode 100644 index 0000000000..28e535b0bc --- /dev/null +++ b/src/3rdparty/freetype/src/gxvalid/README @@ -0,0 +1,532 @@ +gxvalid: TrueType GX validator +============================== + + +1. What is this +--------------- + + `gxvalid' is a module to validate TrueType GX tables: a collection of + additional tables in TrueType font which are used by `QuickDraw GX + Text', Apple Advanced Typography (AAT). In addition, gxvalid can + validates `kern' tables which have been extended for AAT. Like the + otvalid module, gxvalid uses Freetype 2's validator framework + (ftvalid). + + You can link gxvalid with your program; before running your own layout + engine, gxvalid validates a font file. As the result, you can remove + error-checking code from the layout engine. It is also possible to + use gxvalid as a stand-alone font validator; the `ftvalid' test + program included in the ft2demo bundle calls gxvalid internally. + A stand-alone font validator may be useful for font developers. + + This documents documents the following issues. + + - supported TrueType GX tables + - fundamental validation limitations + - permissive error handling of broken GX tables + - `kern' table issue. + + +2. Supported tables +------------------- + + The following GX tables are currently supported. + + bsln + feat + just + kern(*) + lcar + mort + morx + opbd + prop + trak + + The following GX tables are currently unsupported. + + cvar + fdsc + fmtx + fvar + gvar + Zapf + + The following GX tables won't be supported. + + acnt(**) + hsty(***) + + The following undocumented tables in TrueType fonts designed for Apple + platform aren't handled either. + + addg + CVTM + TPNM + umif + + + *) The `kern' validator handles both the classic and the new kern + formats; the former is supported on both Microsoft and Apple + platforms, while the latter is supported on Apple platforms. + + **) `acnt' tables are not supported by currently available Apple font + tools. + + ***) There is one more Apple extension, `hsty', but it is for + Newton-OS, not GX (Newton-OS is a platform by Apple, but it can + use sfnt- housed bitmap fonts only). Therefore, it should be + excluded from `Apple platform' in the context of TrueType. + gxvalid ignores it as Apple font tools do so. + + + We have checked 183 fonts bundled with MacOS 9.1, MacOS 9.2, MacOS + 10.0, MacOS X 10.1, MSIE for MacOS, and AppleWorks 6.0. In addition, + we have checked 67 Dynalab fonts (designed for MacOS) and 189 Ricoh + fonts (designed for Windows and MacOS dual platforms). The number of + fonts including TrueType GX tables are as follows. + + bsln: 76 + feat: 191 + just: 84 + kern: 59 + lcar: 4 + mort: 326 + morx: 19 + opbd: 4 + prop: 114 + trak: 16 + + Dynalab and Ricoh fonts don't have GX tables except of `feat' and + `mort'. + + +3. Fundamental validation limitations +------------------------------------- + + TrueType GX provides layout information to libraries for font + rasterizers and text layout. gxvalid can check whether the layout + data in a font is conformant to the TrueType GX format specified by + Apple. But gxvalid cannot check a how QuickDraw GX/AAT renderer uses + the stored information. + + 3-1. Validation of State Machine activity + ----------------------------------------- + + QuickDraw GX/AAT uses a `State Machine' to provide `stateful' layout + features, and TrueType GX stores the state transition diagram of + this `State Machine' in a `StateTable' data structure. While the + State Machine receives a series of glyph IDs, the State Machine + starts with `start of text' state, walks around various states and + generates various layout information to the renderer, and finally + reaches the `end of text' state. + + gxvalid can check essential errors like: + + - possibility of state transitions to undefined states + - existence of glyph IDs that the State Machine doesn't know how + to handle + - the State Machine cannot compute the layout information from + given diagram + + These errors can be checked within finite steps, and without the + State Machine itself, because these are `expression' errors of state + transition diagram. + + There is no limitation about how long the State Machine walks + around, so validation of the algorithm in the state transition + diagram requires infinite steps, even if we had a State Machine in + gxvalid. Therefore, the following errors and problems cannot be + checked. + + - existence of states which the State Machine never transits to + - the possibility that the State Machine never reaches `end of + text' + - the possibility of stack underflow/overflow in the State Machine + (in ligature and contextual glyph substitutions, the State + Machine can store 16 glyphs onto its stack) + + In addition, gxvalid doesn't check `temporary glyph IDs' used in the + chained State Machines (in `mort' and `morx' tables). If a layout + feature is implemented by a single State Machine, a glyph ID + converted by the State Machine is passed to the glyph renderer, thus + it should not point to an undefined glyph ID. But if a layout + feature is implemented by chained State Machines, a component State + Machine (if it is not the final one) is permitted to generate + undefined glyph IDs for temporary use, because it is handled by next + component State Machine and not by the glyph renderer. To validate + such temporary glyph IDs, gxvalid must stack all undefined glyph IDs + which can occur in the output of the previous State Machine and + search them in the `ClassTable' structure of the current State + Machine. It is too complex to list all possible glyph IDs from the + StateTable, especially from a ligature substitution table. + + 3-2. Validation of relationship between multiple layout features + ---------------------------------------------------------------- + + gxvalid does not validate the relationship between multiple layout + features at all. + + If multiple layout features are defined in TrueType GX tables, + possible interactions, overrides, and conflicts between layout + features are implicitly given in the font too. For example, there + are several predefined spacing control features: + + - Text Spacing (Proportional/Monospace/Half-width/Normal) + - Number Spacing (Monospaced-numbers/Proportional-numbers) + - Kana Spacing (Full-width/Proportional) + - Ideographic Spacing (Full-width/Proportional) + - CJK Roman Spacing (Half-width/Proportional/Default-roman + /Full-width-roman/Proportional) + + If all layout features are independently managed, we can activate + inconsistent typographic rules like `Text Spacing=Monospace' and + `Ideographic Spacing=Proportional' at the same time. + + The combinations of layout features is managed by a 32bit integer + (one bit each for selector setting), so we can define relationships + between up to 32 features, theoretically. But if one feature + setting affects another feature setting, we need typographic + priority rules to validate the relationship. Unfortunately, the + TrueType GX format specification does not give such information even + for predefined features. + + +4. Permissive error handling of broken GX tables +------------------------------------------------ + + When Apple's font rendering system finds an inconsistency, like a + specification violation or an unspecified value in a TrueType GX + table, it does not always return error. In most cases, the rendering + engine silently ignores such wrong values or even whole tables. In + fact, MacOS is shipped with fonts including broken GX/AAT tables, but + no harmful effects due to `officially broken' fonts are observed by + end-users. + + gxvalid is designed to continue the validation process as long as + possible. When gxvalid find wrong values, gxvalid warns it at least, + and takes a fallback procedure if possible. The fallback procedure + depends on the debug level. + + We used the following three tools to investigate Apple's error handling. + + - FontValidator (for MacOS 8.5 - 9.2) resource fork font + - ftxvalidator (for MacOS X 10.1 -) dfont or naked-sfnt + - ftxdumperfuser (for MacOS X 10.1 -) dfont or naked-sfnt + + However, all tests were done on a PowerPC based Macintosh; at present, + we have not checked those tools on a m68k-based Macintosh. + + In total, we checked 183 fonts bundled to MacOS 9.1, MacOS 9.2, MacOS + 10.0, MacOS X 10.1, MSIE for MacOS, and AppleWorks 6.0. These fonts + are distributed officially, but many broken GX/AAT tables were found + by Apple's font tools. In the following, we list typical violation of + the GX specification, in fonts officially distributed with those Apple + systems. + + 4-1. broken BinSrchHeader (19/183) + ---------------------------------- + + `BinSrchHeader' is a header of a data array for m68k platforms to + access memory efficiently. Although there are only two independent + parameters for real (`unitSize' and `nUnits'), BinSrchHeader has + three additional parameters which can be calculated from `unitSize' + and `nUnits', for fast setup. Apple font tools ignore them + silently, so gxvalid warns if it finds and inconsistency, and always + continues validation. The additional parameters are ignored + regardless of the consistency. + + 19 fonts include such inconsistencies; all breaks are in the + BinSrchHeader structure of the `kern' table. + + 4-2. too-short LookupTable (5/183) + ---------------------------------- + + LookupTable format 0 is a simple array to get a value from a given + GID (glyph ID); the index of this array is a GID too. Therefore, + the length of the array is expected to be same as the maximum GID + value defined in the `maxp' table, but there are some fonts whose + LookupTable format 0 is too short to cover all GIDs. FontValidator + ignores this error silently, ftxvalidator and ftxdumperfuser both + warn and continue. Similar problems are found in format 3 subtables + of `kern'. gxvalid warns always and abort if the validation level + is set to FT_VALIDATE_PARANOID. + + 5 fonts include too-short kern format 0 subtables. + 1 font includes too-short kern format 3 subtable. + + 4-3. broken LookupTable format 2 (1/183) + ---------------------------------------- + + LookupTable format 2, subformat 4 covers the GID space by a + collection of segments which are specified by `firstGlyph' and + `lastGlyph'. Some fonts store `firstGlyph' and `lastGlyph' in + reverse order, so the segment specification is broken. Apple font + tools ignore this error silently; a broken segment is ignored as if + it did not exist. gxvalid warns and normalize the segment at + FT_VALIDATE_DEFAULT, or ignore the segment at FT_VALIDATE_TIGHT, or + abort at FT_VALIDATE_PARANOID. + + 1 font includes broken LookupTable format 2, in the `just' table. + + *) It seems that all fonts manufactured by ITC for AppleWorks have + this error. + + 4-4. bad bracketing in glyph property (14/183) + ---------------------------------------------- + + GX/AAT defines a `bracketing' property of the glyphs in the `prop' + table, to control layout features of strings enclosed inside and + outside of brackets. Some fonts give inappropriate bracket + properties to glyphs. Apple font tools warn about this error; + gxvalid warns too and aborts at FT_VALIDATE_PARANOID. + + 14 fonts include wrong bracket properties. + + + 4-5. invalid feature number (117/183) + ------------------------------------- + + The GX/AAT extension can include 255 different layout features, but + popular layout features are predefined (see + http://developer.apple.com/fonts/Registry/index.html). Some fonts + include feature numbers which are incompatible with the predefined + feature registry. + + In our survey, there are 140 fonts including `feat' table. + + a) 67 fonts use a feature number which should not be used. + b) 117 fonts set the wrong feature range (nSetting). This is mostly + found in the `mort' and `morx' tables. + + Apple font tools give no warning, although they cannot recognize + what the feature is. At FT_VALIDATE_DEFAULT, gxvalid warns but + continues in both cases (a, b). At FT_VALIDATE_TIGHT, gxvalid warns + and aborts for (a), but continues for (b). At FT_VALIDATE_PARANOID, + gxvalid warns and aborts in both cases (a, b). + + 4-6. invalid prop version (10/183) + ---------------------------------- + + As most TrueType GX tables, the `prop' table must start with a 32bit + version identifier: 0x00010000, 0x00020000 or 0x00030000. But some + fonts store nonsense binary data instead. When Apple font tools + find them, they abort the processing immediately, and the data which + follows is unhandled. gxvalid does the same. + + 10 fonts include broken `prop' version. + + All of these fonts are classic TrueType fonts for the Japanese + script, manufactured by Apple. + + 4-7. unknown resource name (2/183) + ------------------------------------ + + NOTE: THIS IS NOT A TRUETYPE GX ERROR. + + If a TrueType font is stored in the resource fork or in dfont + format, the data must be tagged as `sfnt' in the resource fork index + to invoke TrueType font handler for the data. But the TrueType font + data in `Keyboard.dfont' is tagged as `kbd', and that in + `LastResort.dfont' is tagged as `lst'. Apple font tools can detect + that the data is in TrueType format and successfully validate them. + Maybe this is possible because they are known to be dfont. The + current implementation of the resource fork driver of FreeType + cannot do that, thus gxvalid cannot validate them. + + 2 fonts use an unknown tag for the TrueType font resource. + +5. `kern' table issues +---------------------- + + In common terminology of TrueType, `kern' is classified as a basic and + platform-independent table. But there are Apple extensions of `kern', + and there is an extension which requires a GX state machine for + contextual kerning. Therefore, gxvalid includes a special validator + for `kern' tables. Unfortunately, there is no exact algorithm to + check Apple's extension, so gxvalid includes a heuristic algorithm to + find the proper validation routines for all possible data formats, + including the data format for Microsoft. By calling + classic_kern_validate() instead of gxv_validate(), you can specify the + `kern' format explicitly. However, current FreeType2 uses Microsoft + `kern' format only, others are ignored (and should be handled in a + library one level higher than FreeType). + + 5-1. History + ------------ + + The original 16bit version of `kern' was designed by Apple in the + pre-GX era, and it was also approved by Microsoft. Afterwards, + Apple designed a new 32bit version of the `kern' table. According + to the documentation, the difference between the 16bit and 32bit + version is only the size of variables in the `kern' header. In the + following, we call the original 16bit version as `classic', and + 32bit version as `new'. + + 5-2. Versions and dialects which should be differentiated + --------------------------------------------------------- + + The `kern' table consists of a table header and several subtables. + The version number which identifies a `classic' or a `new' version + is explicitly written in the table header, but there are + undocumented differences between Microsoft's and Apple's formats. + It is called a `dialect' in the following. There are three cases + which should be handled: the new Apple-dialect, the classic + Apple-dialect, and the classic Microsoft-dialect. An analysis of + the formats and the auto detection algorithm of gxvalid is described + in the following. + + 5-2-1. Version detection: classic and new kern + ---------------------------------------------- + + According to Apple TrueType specification, there are only two + differences between the classic and the new: + + - The `kern' table header starts with the version number. + The classic version starts with 0x0000 (16bit), + the new version starts with 0x00010000 (32bit). + + - In the `kern' table header, the number of subtables follows + the version number. + In the classic version, it is stored as a 16bit value. + In the new version, it is stored as a 32bit value. + + From Apple font tool's output (DumpKERN is also tested in addition + to the three Apple font tools in above), there is another + undocumented difference. In the new version, the subtable header + includes a 16bit variable named `tupleIndex' which does not exist + in the classic version. + + The new version can store all subtable formats (0, 1, 2, and 3), + but the Apple TrueType specification does not mention the subtable + formats available in the classic version. + + 5-2-2. Available subtable formats in classic version + ---------------------------------------------------- + + Although the Apple TrueType specification recommends to use the + classic version in the case if the font is designed for both the + Apple and Microsoft platforms, it does not document the available + subtable formats in the classic version. + + According to the Microsoft TrueType specification, the subtable + format assured for Windows and OS/2 support is only subtable + format 0. The Microsoft TrueType specification also describes + subtable format 2, but does not mention which platforms support + it. Aubtable formats 1, 3, and higher are documented as reserved + for future use. Therefore, the classic version can store subtable + formats 0 and 2, at least. `ttfdump.exe', a font tool provided by + Microsoft, ignores the subtable format written in the subtable + header, and parses the table as if all subtables are in format 0. + + `kern' subtable format 1 uses a StateTable, so it cannot be + utilized without a GX State Machine. Therefore, it is reasonable + to assume that format 1 (and 3) were introduced after Apple had + introduced GX and moved to the new 32bit version. + + 5-2-3. Apple and Microsoft dialects + ----------------------------------- + + The `kern' subtable has a 16bit `coverage' field to describe + kerning attributes, but bit interpretations by Apple and Microsoft + are different: For example, Apple uses bits 0-7 to identify the + subtable, while Microsoft uses bits 8-15. + + In addition, due to the output of DumpKERN and FontValidator, + Apple's bit interpretations of coverage in classic and new version + are incompatible also. In summary, there are three dialects: + classic Apple dialect, classic Microsoft dialect, and new Apple + dialect. The classic Microsoft dialect and the new Apple dialect + are documented by each vendors' TrueType font specification, but + the documentation for classic Apple dialect is not available. + + For example, in the new Apple dialect, bit 15 is documented as + `set to 1 if the kerning is vertical'. On the other hand, in + classic Microsoft dialect, bit 1 is documented as `set to 1 if the + kerning is horizontal'. From the outputs of DumpKERN and + FontValidator, classic Apple dialect recognizes 15 as `set to 1 + when the kerning is horizontal'. From the results of similar + experiments, classic Apple dialect seems to be the Endian reverse + of the classic Microsoft dialect. + + As a conclusion it must be noted that no font tool can identify + classic Apple dialect or classic Microsoft dialect automatically. + + 5-2-4. gxvalid auto dialect detection algorithm + ----------------------------------------------- + + The first 16 bits of the `kern' table are enough to identify the + version: + + - if the first 16 bits are 0x0000, the `kern' table is in + classic Apple dialect or classic Microsoft dialect + - if the first 16 bits are 0x0001, and next 16 bits are 0x0000, + the kern table is in new Apple dialect. + + If the `kern' table is a classic one, the 16bit `coverage' field + is checked next. Firstly, the coverage bits are decoded for the + classic Apple dialect using the following bit masks (this is based + on DumpKERN output): + + 0x8000: 1=horizontal, 0=vertical + 0x4000: not used + 0x2000: 1=cross-stream, 0=normal + 0x1FF0: reserved + 0x000F: subtable format + + If any of reserved bits are set or the subtable bits is + interpreted as format 1 or 3, we take it as `impossible in classic + Apple dialect' and retry, using the classic Microsoft dialect. + + The most popular coverage in new Apple-dialect: 0x8000, + The most popular coverage in classic Apple-dialect: 0x0000, + The most popular coverage in classic Microsoft dialect: 0x0001. + + 5-3. Tested fonts + ----------------- + + We checked 59 fonts bundled with MacOS and 38 fonts bundled with + Windows, where all font include a `kern' table. + + - fonts bundled with MacOS + * new Apple dialect + format 0: 18 + format 2: 1 + format 3: 1 + * classic Apple dialect + format 0: 14 + * classic Microsoft dialect + format 0: 15 + + - fonts bundled with Windows + * classic Microsoft dialect + format 0: 38 + + It looks strange that classic Microsoft-dialect fonts are bundled to + MacOS: they come from MSIE for MacOS, except of MarkerFelt.dfont. + + + ACKNOWLEDGEMENT + --------------- + + Some parts of gxvalid are derived from both the `gxlayout' module and + the `otvalid' module. Development of gxlayout was supported by the + Information-technology Promotion Agency(IPA), Japan. + + The detailed analysis of undefined glyph ID utilization in `mort' and + `morx' tables is provided by George Williams. + +------------------------------------------------------------------------ + +Copyright 2004, 2005, 2007 by +suzuki toshiya, Masatake YAMATO, Red hat K.K., +David Turner, Robert Wilhelm, and Werner Lemberg. + +This file is part of the FreeType project, and may only be used, +modified, and distributed under the terms of the FreeType project +license, LICENSE.TXT. By continuing to use, modify, or distribute this +file you indicate that you have read the license and understand and +accept it fully. + + +--- end of README --- |