Markdown: blockquotes, code blocks, and generalized nesting

Can now detect nested quotes and code blocks inside quotes, and can
rewrite the markdown too.

QTextHtmlParser sets hard-coded left and right margins, so we need to do
the same to be able to read HTML and write markdown, or vice-versa,
and to ensure that all views (QTextEdit, QTextBrowser, QML Text etc.)
will render it with margins.  But now we add a semantic memory too:
BlockQuoteLevel is similar to HeadingLevel, which was added in
310daae53926628f80c08e4415b94b90ad525c8f to preserve H1..H6 heading
levels, because detecting it via font size didn't make sense in
QTextMarkdownWriter.  Likewise detecting quote level by its margins
didn't make sense; markdown supports nesting quotes; and indenting
nested quotes via 40 pixels may be a bit too much, so we should consider
it subject to change (and perhaps be able to change it via CSS later on).
Since we're adding BlockQuoteLevel and depending on it in QTextMarkdownWriter,
it's necessary to set it in QTextHtmlParser to enable HTML->markdown
conversion.  (But so far, nested blockquotes in HTML are not supported.)

Quotes (and nested quotes) can contain indented code blocks, but it seems
the reverse is not true (according to https://spec.commonmark.org/0.29/#example-201 )

Quotes can contain fenced code blocks.

Quotes can contain lists.  Nested lists can be interrupted with
nested code blocks and nested quotes.

So far the writer assumes all code blocks are the indented type.
It will be necessary to add another attribute to remember whether the
code block is indented or fenced (assuming that's necessary).
Fenced code blocks would work better for writing inside block quotes
and list items because the fence is less ambiguous than the indent.

Postponing cursor->insertBlock() as long as possible helps with nesting.
cursor->insertBlock() needs to be done "just in time" before inserting
text that will go in the block.  The block and char formats aren't
necessarily known until that time.  When a nested block (such as a
nested quote) ends, the context reverts to the previous block format,
which then needs to be re-determined and set before we insert text
into the outer block; but if no text will be inserted, no new block
is necessary.  But we can't use QTextBlockFormat itself as storage,
because for some reason bullets become very "sticky" and it becomes
impossible to have plain continuation paragraphs inside list items:
they all get bullets.  Somehow QTextBlockFormat remembers, if we copy it.
But we can create a new one each time and it's OK.

Change-Id: Icd0529eb90d2b6a3cb57f0104bf78a7be81ede52
Reviewed-by: Gatis Paeglis <gatis.paeglis@qt.io>

4 files changed, 58 insertions, 3 deletions

diff --git a/tests/auto/gui/text/qtextmarkdownimporter/qtextmarkdownimporter.pro b/tests/auto/gui/text/qtextmarkdownimporter/qtextmarkdownimporter.pro
index 7afc807c9b..7b7fb61244 100644
--- a/tests/auto/gui/text/qtextmarkdownimporter/qtextmarkdownimporter.pro
+++ b/tests/auto/gui/text/qtextmarkdownimporter/qtextmarkdownimporter.pro
@@ -4,5 +4,6 @@ QT += core-private gui-private testlib
 SOURCES += tst_qtextmarkdownimporter.cpp
 TESTDATA += \
     data/thematicBreaks.md \
+    data/headingBulletsContinuations.md \
 
 DEFINES += SRCDIR=\\\"$$PWD\\\"
diff --git a/tests/auto/gui/text/qtextmarkdownwriter/data/blockquotes.md b/tests/auto/gui/text/qtextmarkdownwriter/data/blockquotes.md
new file mode 100644
index 0000000000..f429fcc21b
--- /dev/null
+++ b/tests/auto/gui/text/qtextmarkdownwriter/data/blockquotes.md
@@ -0,0 +1,39 @@
+In 1958, Mahatma Gandhi was quoted as follows:
+
+> The Earth provides enough to satisfy every man's need but not for every man's
+> greed.
+
+In [The CommonMark Specification](https://spec.commonmark.org/0.29/) John
+MacFarlane writes:
+
+> What distinguishes Markdown from many other lightweight markup syntaxes,
+> which are often easier to write, is its readability. As Gruber writes:
+
+> > The overriding design goal for Markdown's formatting syntax is to make it
+> > as readable as possible. The idea is that a Markdown-formatted document should
+> > be publishable as-is, as plain text, without looking like it's been marked up
+> > with tags or formatting instructions. ( 
+> > [http://daringfireball.net/projects/markdown/](http://daringfireball.net/projects/markdown/))
+
+> The point can be illustrated by comparing a sample of AsciiDoc with an
+> equivalent sample of Markdown. Here is a sample of AsciiDoc from the AsciiDoc
+> manual:
+
+>     1. List item one.
+>     +
+>     List item one continued with a second paragraph followed by an
+>     Indented block.
+>     +
+>     .................
+>     $ ls *.sh
+>     $ mv *.sh ~/tmp
+>     .................
+>     +
+>     List item continued with a third paragraph.
+>     
+>     2. List item two continued with an open block.
+>     ...
+>     
+The quotation includes an embedded quotation and a code quotation and ends with
+an ellipsis due to being incomplete.
+
diff --git a/tests/auto/gui/text/qtextmarkdownwriter/qtextmarkdownwriter.pro b/tests/auto/gui/text/qtextmarkdownwriter/qtextmarkdownwriter.pro
index 04cf7ef5dd..6144710b99 100644
--- a/tests/auto/gui/text/qtextmarkdownwriter/qtextmarkdownwriter.pro
+++ b/tests/auto/gui/text/qtextmarkdownwriter/qtextmarkdownwriter.pro
@@ -2,6 +2,8 @@ CONFIG += testcase
 TARGET = tst_qtextmarkdownwriter
 QT += core-private gui-private testlib
 SOURCES += tst_qtextmarkdownwriter.cpp
-TESTDATA += data/example.md
+TESTDATA += \
+    data/example.md \
+    data/blockquotes.md \
 
 DEFINES += SRCDIR=\\\"$$PWD\\\"
diff --git a/tests/auto/gui/text/qtextmarkdownwriter/tst_qtextmarkdownwriter.cpp b/tests/auto/gui/text/qtextmarkdownwriter/tst_qtextmarkdownwriter.cpp
index 3d77ccd9d8..9998794762 100644
--- a/tests/auto/gui/text/qtextmarkdownwriter/tst_qtextmarkdownwriter.cpp
+++ b/tests/auto/gui/text/qtextmarkdownwriter/tst_qtextmarkdownwriter.cpp
@@ -54,6 +54,7 @@ private slots:
     void testWriteNestedBulletLists();
     void testWriteNestedNumericLists();
     void testWriteTable();
+    void rewriteDocument_data();
     void rewriteDocument();
     void fromHtml_data();
     void fromHtml();
@@ -350,10 +351,19 @@ void tst_QTextMarkdownWriter::testWriteTable()
     QCOMPARE(md, QString::fromLatin1("\n|a ||b|\n|-|-|-|\n|c|d ||\n|e|f| |\n\n"));
 }
 
+void tst_QTextMarkdownWriter::rewriteDocument_data()
+{
+    QTest::addColumn<QString>("inputFile");
+
+    QTest::newRow("block quotes") << "blockquotes.md";
+    QTest::newRow("example") << "example.md";
+}
+
 void tst_QTextMarkdownWriter::rewriteDocument()
 {
+    QFETCH(QString, inputFile);
     QTextDocument doc;
-    QFile f(QFINDTESTDATA("data/example.md"));
+    QFile f(QFINDTESTDATA("data/" + inputFile));
     QVERIFY(f.open(QFile::ReadOnly | QIODevice::Text));
     QString orig = QString::fromUtf8(f.readAll());
     f.close();
@@ -361,7 +371,7 @@ void tst_QTextMarkdownWriter::rewriteDocument()
     QString md = doc.toMarkdown();
 
 #ifdef DEBUG_WRITE_OUTPUT
-    QFile out("/tmp/rewrite.md");
+    QFile out("/tmp/rewrite-" + inputFile);
     out.open(QFile::WriteOnly);
     out.write(md.toUtf8());
     out.close();
@@ -401,6 +411,9 @@ void tst_QTextMarkdownWriter::fromHtml_data()
     QTest::newRow("thematic break") <<
         "something<hr/>something else" <<
         "something\n\n- - -\nsomething else\n\n";
+    QTest::newRow("block quote") <<
+        "<p>In 1958, Mahatma Gandhi was quoted as follows:</p><blockquote>The Earth provides enough to satisfy every man's need but not for every man's greed.</blockquote>" <<
+        "In 1958, Mahatma Gandhi was quoted as follows:\n\n> The Earth provides enough to satisfy every man's need but not for every man's\n> greed.\n\n";
     // TODO
 //    QTest::newRow("escaped number and paren after double newline") <<
 //        "<p>(The first sentence of this paragraph is a line, the next paragraph has a number</p>13) but that's not part of an ordered list" <<