qmldom: line by line indentation/reformatting

Introduces support for line by line indentation and reformatting
This is done with a simplified restartable lexer and parser.

* qqmldomscanner: a tokenizer for a single line, built on the top of
the normal lexer. One of the key property is that the state can be
stored and restarted.
Thus after an edit, re-indenting or split of a line one can restart
the lexer with the state at the end of the previous line, and
re-evaluate from there on.
* qqmlcodeformatter: contains a stack based parser that is used for
syntax highlighting and indentation. The state is a stack of integers,
and along with it also the indent level is stored
and the accepted syntax is a superset of the grammar (typically
repetitions are allowed and order is more permissive).
This builds on the top of the scanner, and also allows restarts.
The public interface focuses on having the stored state of the parser
and lexer, and being able to update it after a line of text, or give
the correct indentation for a line.
* qqmldomindentinglinewriter: has a LineWriter that uses the
CodeFormatter internally to re-indent every line.

Change-Id: Ifce9ed14ecd157bec65fb740b2c7ee8a5fc0729a
Reviewed-by: Ulf Hermann <ulf.hermann@qt.io>

1 files changed, 3 insertions, 0 deletions

diff --git a/src/qml/parser/qqmljs.g b/src/qml/parser/qqmljs.g
index 48785109a3..afda0f27aa 100644
--- a/src/qml/parser/qqmljs.g
+++ b/src/qml/parser/qqmljs.g
@@ -73,6 +73,9 @@
 %token T_GET "get"
 %token T_SET "set"
 
+-- token representing no token
+%token T_NONE
+
 %token T_ERROR
 
 -- states for line by line parsing