path: root/src/3rdparty/freetype/src/gxvalid/README
diff options
Diffstat (limited to 'src/3rdparty/freetype/src/gxvalid/README')
1 files changed, 532 insertions, 0 deletions
diff --git a/src/3rdparty/freetype/src/gxvalid/README b/src/3rdparty/freetype/src/gxvalid/README
new file mode 100644
index 0000000000..28e535b0bc
--- /dev/null
+++ b/src/3rdparty/freetype/src/gxvalid/README
@@ -0,0 +1,532 @@
+gxvalid: TrueType GX validator
+1. What is this
+ `gxvalid' is a module to validate TrueType GX tables: a collection of
+ additional tables in TrueType font which are used by `QuickDraw GX
+ Text', Apple Advanced Typography (AAT). In addition, gxvalid can
+ validates `kern' tables which have been extended for AAT. Like the
+ otvalid module, gxvalid uses Freetype 2's validator framework
+ (ftvalid).
+ You can link gxvalid with your program; before running your own layout
+ engine, gxvalid validates a font file. As the result, you can remove
+ error-checking code from the layout engine. It is also possible to
+ use gxvalid as a stand-alone font validator; the `ftvalid' test
+ program included in the ft2demo bundle calls gxvalid internally.
+ A stand-alone font validator may be useful for font developers.
+ This documents documents the following issues.
+ - supported TrueType GX tables
+ - fundamental validation limitations
+ - permissive error handling of broken GX tables
+ - `kern' table issue.
+2. Supported tables
+ The following GX tables are currently supported.
+ bsln
+ feat
+ just
+ kern(*)
+ lcar
+ mort
+ morx
+ opbd
+ prop
+ trak
+ The following GX tables are currently unsupported.
+ cvar
+ fdsc
+ fmtx
+ fvar
+ gvar
+ Zapf
+ The following GX tables won't be supported.
+ acnt(**)
+ hsty(***)
+ The following undocumented tables in TrueType fonts designed for Apple
+ platform aren't handled either.
+ addg
+ umif
+ *) The `kern' validator handles both the classic and the new kern
+ formats; the former is supported on both Microsoft and Apple
+ platforms, while the latter is supported on Apple platforms.
+ **) `acnt' tables are not supported by currently available Apple font
+ tools.
+ ***) There is one more Apple extension, `hsty', but it is for
+ Newton-OS, not GX (Newton-OS is a platform by Apple, but it can
+ use sfnt- housed bitmap fonts only). Therefore, it should be
+ excluded from `Apple platform' in the context of TrueType.
+ gxvalid ignores it as Apple font tools do so.
+ We have checked 183 fonts bundled with MacOS 9.1, MacOS 9.2, MacOS
+ 10.0, MacOS X 10.1, MSIE for MacOS, and AppleWorks 6.0. In addition,
+ we have checked 67 Dynalab fonts (designed for MacOS) and 189 Ricoh
+ fonts (designed for Windows and MacOS dual platforms). The number of
+ fonts including TrueType GX tables are as follows.
+ bsln: 76
+ feat: 191
+ just: 84
+ kern: 59
+ lcar: 4
+ mort: 326
+ morx: 19
+ opbd: 4
+ prop: 114
+ trak: 16
+ Dynalab and Ricoh fonts don't have GX tables except of `feat' and
+ `mort'.
+3. Fundamental validation limitations
+ TrueType GX provides layout information to libraries for font
+ rasterizers and text layout. gxvalid can check whether the layout
+ data in a font is conformant to the TrueType GX format specified by
+ Apple. But gxvalid cannot check a how QuickDraw GX/AAT renderer uses
+ the stored information.
+ 3-1. Validation of State Machine activity
+ -----------------------------------------
+ QuickDraw GX/AAT uses a `State Machine' to provide `stateful' layout
+ features, and TrueType GX stores the state transition diagram of
+ this `State Machine' in a `StateTable' data structure. While the
+ State Machine receives a series of glyph IDs, the State Machine
+ starts with `start of text' state, walks around various states and
+ generates various layout information to the renderer, and finally
+ reaches the `end of text' state.
+ gxvalid can check essential errors like:
+ - possibility of state transitions to undefined states
+ - existence of glyph IDs that the State Machine doesn't know how
+ to handle
+ - the State Machine cannot compute the layout information from
+ given diagram
+ These errors can be checked within finite steps, and without the
+ State Machine itself, because these are `expression' errors of state
+ transition diagram.
+ There is no limitation about how long the State Machine walks
+ around, so validation of the algorithm in the state transition
+ diagram requires infinite steps, even if we had a State Machine in
+ gxvalid. Therefore, the following errors and problems cannot be
+ checked.
+ - existence of states which the State Machine never transits to
+ - the possibility that the State Machine never reaches `end of
+ text'
+ - the possibility of stack underflow/overflow in the State Machine
+ (in ligature and contextual glyph substitutions, the State
+ Machine can store 16 glyphs onto its stack)
+ In addition, gxvalid doesn't check `temporary glyph IDs' used in the
+ chained State Machines (in `mort' and `morx' tables). If a layout
+ feature is implemented by a single State Machine, a glyph ID
+ converted by the State Machine is passed to the glyph renderer, thus
+ it should not point to an undefined glyph ID. But if a layout
+ feature is implemented by chained State Machines, a component State
+ Machine (if it is not the final one) is permitted to generate
+ undefined glyph IDs for temporary use, because it is handled by next
+ component State Machine and not by the glyph renderer. To validate
+ such temporary glyph IDs, gxvalid must stack all undefined glyph IDs
+ which can occur in the output of the previous State Machine and
+ search them in the `ClassTable' structure of the current State
+ Machine. It is too complex to list all possible glyph IDs from the
+ StateTable, especially from a ligature substitution table.
+ 3-2. Validation of relationship between multiple layout features
+ ----------------------------------------------------------------
+ gxvalid does not validate the relationship between multiple layout
+ features at all.
+ If multiple layout features are defined in TrueType GX tables,
+ possible interactions, overrides, and conflicts between layout
+ features are implicitly given in the font too. For example, there
+ are several predefined spacing control features:
+ - Text Spacing (Proportional/Monospace/Half-width/Normal)
+ - Number Spacing (Monospaced-numbers/Proportional-numbers)
+ - Kana Spacing (Full-width/Proportional)
+ - Ideographic Spacing (Full-width/Proportional)
+ - CJK Roman Spacing (Half-width/Proportional/Default-roman
+ /Full-width-roman/Proportional)
+ If all layout features are independently managed, we can activate
+ inconsistent typographic rules like `Text Spacing=Monospace' and
+ `Ideographic Spacing=Proportional' at the same time.
+ The combinations of layout features is managed by a 32bit integer
+ (one bit each for selector setting), so we can define relationships
+ between up to 32 features, theoretically. But if one feature
+ setting affects another feature setting, we need typographic
+ priority rules to validate the relationship. Unfortunately, the
+ TrueType GX format specification does not give such information even
+ for predefined features.
+4. Permissive error handling of broken GX tables
+ When Apple's font rendering system finds an inconsistency, like a
+ specification violation or an unspecified value in a TrueType GX
+ table, it does not always return error. In most cases, the rendering
+ engine silently ignores such wrong values or even whole tables. In
+ fact, MacOS is shipped with fonts including broken GX/AAT tables, but
+ no harmful effects due to `officially broken' fonts are observed by
+ end-users.
+ gxvalid is designed to continue the validation process as long as
+ possible. When gxvalid find wrong values, gxvalid warns it at least,
+ and takes a fallback procedure if possible. The fallback procedure
+ depends on the debug level.
+ We used the following three tools to investigate Apple's error handling.
+ - FontValidator (for MacOS 8.5 - 9.2) resource fork font
+ - ftxvalidator (for MacOS X 10.1 -) dfont or naked-sfnt
+ - ftxdumperfuser (for MacOS X 10.1 -) dfont or naked-sfnt
+ However, all tests were done on a PowerPC based Macintosh; at present,
+ we have not checked those tools on a m68k-based Macintosh.
+ In total, we checked 183 fonts bundled to MacOS 9.1, MacOS 9.2, MacOS
+ 10.0, MacOS X 10.1, MSIE for MacOS, and AppleWorks 6.0. These fonts
+ are distributed officially, but many broken GX/AAT tables were found
+ by Apple's font tools. In the following, we list typical violation of
+ the GX specification, in fonts officially distributed with those Apple
+ systems.
+ 4-1. broken BinSrchHeader (19/183)
+ ----------------------------------
+ `BinSrchHeader' is a header of a data array for m68k platforms to
+ access memory efficiently. Although there are only two independent
+ parameters for real (`unitSize' and `nUnits'), BinSrchHeader has
+ three additional parameters which can be calculated from `unitSize'
+ and `nUnits', for fast setup. Apple font tools ignore them
+ silently, so gxvalid warns if it finds and inconsistency, and always
+ continues validation. The additional parameters are ignored
+ regardless of the consistency.
+ 19 fonts include such inconsistencies; all breaks are in the
+ BinSrchHeader structure of the `kern' table.
+ 4-2. too-short LookupTable (5/183)
+ ----------------------------------
+ LookupTable format 0 is a simple array to get a value from a given
+ GID (glyph ID); the index of this array is a GID too. Therefore,
+ the length of the array is expected to be same as the maximum GID
+ value defined in the `maxp' table, but there are some fonts whose
+ LookupTable format 0 is too short to cover all GIDs. FontValidator
+ ignores this error silently, ftxvalidator and ftxdumperfuser both
+ warn and continue. Similar problems are found in format 3 subtables
+ of `kern'. gxvalid warns always and abort if the validation level
+ 5 fonts include too-short kern format 0 subtables.
+ 1 font includes too-short kern format 3 subtable.
+ 4-3. broken LookupTable format 2 (1/183)
+ ----------------------------------------
+ LookupTable format 2, subformat 4 covers the GID space by a
+ collection of segments which are specified by `firstGlyph' and
+ `lastGlyph'. Some fonts store `firstGlyph' and `lastGlyph' in
+ reverse order, so the segment specification is broken. Apple font
+ tools ignore this error silently; a broken segment is ignored as if
+ it did not exist. gxvalid warns and normalize the segment at
+ FT_VALIDATE_DEFAULT, or ignore the segment at FT_VALIDATE_TIGHT, or
+ 1 font includes broken LookupTable format 2, in the `just' table.
+ *) It seems that all fonts manufactured by ITC for AppleWorks have
+ this error.
+ 4-4. bad bracketing in glyph property (14/183)
+ ----------------------------------------------
+ GX/AAT defines a `bracketing' property of the glyphs in the `prop'
+ table, to control layout features of strings enclosed inside and
+ outside of brackets. Some fonts give inappropriate bracket
+ properties to glyphs. Apple font tools warn about this error;
+ gxvalid warns too and aborts at FT_VALIDATE_PARANOID.
+ 14 fonts include wrong bracket properties.
+ 4-5. invalid feature number (117/183)
+ -------------------------------------
+ The GX/AAT extension can include 255 different layout features, but
+ popular layout features are predefined (see
+ Some fonts
+ include feature numbers which are incompatible with the predefined
+ feature registry.
+ In our survey, there are 140 fonts including `feat' table.
+ a) 67 fonts use a feature number which should not be used.
+ b) 117 fonts set the wrong feature range (nSetting). This is mostly
+ found in the `mort' and `morx' tables.
+ Apple font tools give no warning, although they cannot recognize
+ what the feature is. At FT_VALIDATE_DEFAULT, gxvalid warns but
+ continues in both cases (a, b). At FT_VALIDATE_TIGHT, gxvalid warns
+ and aborts for (a), but continues for (b). At FT_VALIDATE_PARANOID,
+ gxvalid warns and aborts in both cases (a, b).
+ 4-6. invalid prop version (10/183)
+ ----------------------------------
+ As most TrueType GX tables, the `prop' table must start with a 32bit
+ version identifier: 0x00010000, 0x00020000 or 0x00030000. But some
+ fonts store nonsense binary data instead. When Apple font tools
+ find them, they abort the processing immediately, and the data which
+ follows is unhandled. gxvalid does the same.
+ 10 fonts include broken `prop' version.
+ All of these fonts are classic TrueType fonts for the Japanese
+ script, manufactured by Apple.
+ 4-7. unknown resource name (2/183)
+ ------------------------------------
+ If a TrueType font is stored in the resource fork or in dfont
+ format, the data must be tagged as `sfnt' in the resource fork index
+ to invoke TrueType font handler for the data. But the TrueType font
+ data in `Keyboard.dfont' is tagged as `kbd', and that in
+ `LastResort.dfont' is tagged as `lst'. Apple font tools can detect
+ that the data is in TrueType format and successfully validate them.
+ Maybe this is possible because they are known to be dfont. The
+ current implementation of the resource fork driver of FreeType
+ cannot do that, thus gxvalid cannot validate them.
+ 2 fonts use an unknown tag for the TrueType font resource.
+5. `kern' table issues
+ In common terminology of TrueType, `kern' is classified as a basic and
+ platform-independent table. But there are Apple extensions of `kern',
+ and there is an extension which requires a GX state machine for
+ contextual kerning. Therefore, gxvalid includes a special validator
+ for `kern' tables. Unfortunately, there is no exact algorithm to
+ check Apple's extension, so gxvalid includes a heuristic algorithm to
+ find the proper validation routines for all possible data formats,
+ including the data format for Microsoft. By calling
+ classic_kern_validate() instead of gxv_validate(), you can specify the
+ `kern' format explicitly. However, current FreeType2 uses Microsoft
+ `kern' format only, others are ignored (and should be handled in a
+ library one level higher than FreeType).
+ 5-1. History
+ ------------
+ The original 16bit version of `kern' was designed by Apple in the
+ pre-GX era, and it was also approved by Microsoft. Afterwards,
+ Apple designed a new 32bit version of the `kern' table. According
+ to the documentation, the difference between the 16bit and 32bit
+ version is only the size of variables in the `kern' header. In the
+ following, we call the original 16bit version as `classic', and
+ 32bit version as `new'.
+ 5-2. Versions and dialects which should be differentiated
+ ---------------------------------------------------------
+ The `kern' table consists of a table header and several subtables.
+ The version number which identifies a `classic' or a `new' version
+ is explicitly written in the table header, but there are
+ undocumented differences between Microsoft's and Apple's formats.
+ It is called a `dialect' in the following. There are three cases
+ which should be handled: the new Apple-dialect, the classic
+ Apple-dialect, and the classic Microsoft-dialect. An analysis of
+ the formats and the auto detection algorithm of gxvalid is described
+ in the following.
+ 5-2-1. Version detection: classic and new kern
+ ----------------------------------------------
+ According to Apple TrueType specification, there are only two
+ differences between the classic and the new:
+ - The `kern' table header starts with the version number.
+ The classic version starts with 0x0000 (16bit),
+ the new version starts with 0x00010000 (32bit).
+ - In the `kern' table header, the number of subtables follows
+ the version number.
+ In the classic version, it is stored as a 16bit value.
+ In the new version, it is stored as a 32bit value.
+ From Apple font tool's output (DumpKERN is also tested in addition
+ to the three Apple font tools in above), there is another
+ undocumented difference. In the new version, the subtable header
+ includes a 16bit variable named `tupleIndex' which does not exist
+ in the classic version.
+ The new version can store all subtable formats (0, 1, 2, and 3),
+ but the Apple TrueType specification does not mention the subtable
+ formats available in the classic version.
+ 5-2-2. Available subtable formats in classic version
+ ----------------------------------------------------
+ Although the Apple TrueType specification recommends to use the
+ classic version in the case if the font is designed for both the
+ Apple and Microsoft platforms, it does not document the available
+ subtable formats in the classic version.
+ According to the Microsoft TrueType specification, the subtable
+ format assured for Windows and OS/2 support is only subtable
+ format 0. The Microsoft TrueType specification also describes
+ subtable format 2, but does not mention which platforms support
+ it. Aubtable formats 1, 3, and higher are documented as reserved
+ for future use. Therefore, the classic version can store subtable
+ formats 0 and 2, at least. `ttfdump.exe', a font tool provided by
+ Microsoft, ignores the subtable format written in the subtable
+ header, and parses the table as if all subtables are in format 0.
+ `kern' subtable format 1 uses a StateTable, so it cannot be
+ utilized without a GX State Machine. Therefore, it is reasonable
+ to assume that format 1 (and 3) were introduced after Apple had
+ introduced GX and moved to the new 32bit version.
+ 5-2-3. Apple and Microsoft dialects
+ -----------------------------------
+ The `kern' subtable has a 16bit `coverage' field to describe
+ kerning attributes, but bit interpretations by Apple and Microsoft
+ are different: For example, Apple uses bits 0-7 to identify the
+ subtable, while Microsoft uses bits 8-15.
+ In addition, due to the output of DumpKERN and FontValidator,
+ Apple's bit interpretations of coverage in classic and new version
+ are incompatible also. In summary, there are three dialects:
+ classic Apple dialect, classic Microsoft dialect, and new Apple
+ dialect. The classic Microsoft dialect and the new Apple dialect
+ are documented by each vendors' TrueType font specification, but
+ the documentation for classic Apple dialect is not available.
+ For example, in the new Apple dialect, bit 15 is documented as
+ `set to 1 if the kerning is vertical'. On the other hand, in
+ classic Microsoft dialect, bit 1 is documented as `set to 1 if the
+ kerning is horizontal'. From the outputs of DumpKERN and
+ FontValidator, classic Apple dialect recognizes 15 as `set to 1
+ when the kerning is horizontal'. From the results of similar
+ experiments, classic Apple dialect seems to be the Endian reverse
+ of the classic Microsoft dialect.
+ As a conclusion it must be noted that no font tool can identify
+ classic Apple dialect or classic Microsoft dialect automatically.
+ 5-2-4. gxvalid auto dialect detection algorithm
+ -----------------------------------------------
+ The first 16 bits of the `kern' table are enough to identify the
+ version:
+ - if the first 16 bits are 0x0000, the `kern' table is in
+ classic Apple dialect or classic Microsoft dialect
+ - if the first 16 bits are 0x0001, and next 16 bits are 0x0000,
+ the kern table is in new Apple dialect.
+ If the `kern' table is a classic one, the 16bit `coverage' field
+ is checked next. Firstly, the coverage bits are decoded for the
+ classic Apple dialect using the following bit masks (this is based
+ on DumpKERN output):
+ 0x8000: 1=horizontal, 0=vertical
+ 0x4000: not used
+ 0x2000: 1=cross-stream, 0=normal
+ 0x1FF0: reserved
+ 0x000F: subtable format
+ If any of reserved bits are set or the subtable bits is
+ interpreted as format 1 or 3, we take it as `impossible in classic
+ Apple dialect' and retry, using the classic Microsoft dialect.
+ The most popular coverage in new Apple-dialect: 0x8000,
+ The most popular coverage in classic Apple-dialect: 0x0000,
+ The most popular coverage in classic Microsoft dialect: 0x0001.
+ 5-3. Tested fonts
+ -----------------
+ We checked 59 fonts bundled with MacOS and 38 fonts bundled with
+ Windows, where all font include a `kern' table.
+ - fonts bundled with MacOS
+ * new Apple dialect
+ format 0: 18
+ format 2: 1
+ format 3: 1
+ * classic Apple dialect
+ format 0: 14
+ * classic Microsoft dialect
+ format 0: 15
+ - fonts bundled with Windows
+ * classic Microsoft dialect
+ format 0: 38
+ It looks strange that classic Microsoft-dialect fonts are bundled to
+ MacOS: they come from MSIE for MacOS, except of MarkerFelt.dfont.
+ ---------------
+ Some parts of gxvalid are derived from both the `gxlayout' module and
+ the `otvalid' module. Development of gxlayout was supported by the
+ Information-technology Promotion Agency(IPA), Japan.
+ The detailed analysis of undefined glyph ID utilization in `mort' and
+ `morx' tables is provided by George Williams.
+Copyright 2004, 2005, 2007 by
+suzuki toshiya, Masatake YAMATO, Red hat K.K.,
+David Turner, Robert Wilhelm, and Werner Lemberg.
+This file is part of the FreeType project, and may only be used,
+modified, and distributed under the terms of the FreeType project
+license, LICENSE.TXT. By continuing to use, modify, or distribute this
+file you indicate that you have read the license and understand and
+accept it fully.
+--- end of README ---