Long live QStringTokenizer!

This class is designed as C++20-style generator / lazy sequence, and
the new return value of QString{,View}::tokenize().

It thus is more similar to a hand-coded loop around indexOf() than
QString::split(), which returns a container (the filling of which
allocates memory).

The template arguments of QStringTokenizer intricately depend on the
arguments with which it is constructed, so QStringTokenizer cannot be used
directly without C++17 CTAD. To work around this issue, add a factory
function, qTokenize().

LATER:
- ~Optimize QLatin1String needles (avoid repeated L1->UTF16 conversion)~
  (out of scope for QStringTokenizer, should be solved in the respective
  indexOf())
- Keep per-instantiation state:
  * Boyer-Moore table

[ChangeLog][QtCore][QStringTokenizer] New class.

[ChangeLog][QtCore][qTokenize] New function.

Change-Id: I7a7a02e9175cdd3887778f29f2f91933329be759
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>

0 files changed, 0 insertions, 0 deletions