QDoc: Move parseFnArg out of ClangCodeParser

When QDoc processes a project, it parses its code-base to obtain and
sanitize the user-provided documentation for the project.

For projects that makes use of C++ code, QDoc uses Clang's APIs to
obtain the required information about the code-base.

Profiling shows that calls to Clang are the major bottleneck in QDoc at
the current time.

To reduce the time spent by Clang, QDoc is being moved to parallelize
and batch certain operations that are related to Clang calls.

`ClangCodeParser`, the component responsible for all of the calls to
Clang libraries, is currently architectured in a way that is not usable
for those optimization purposes.

Hence, it is slowly being modified to be better suited for those
purposes.
As part of those modification, we expect `ClangCodeParser` to be split
into multiple components.

Currently, `ClangCodeParser` is the entry point to a series of
operations that happen in different phases and have very different
restriction on their scopes and their dependencies.

To fullfill and simplify the parallelization efforts, those operations
are being split up to so that we can reorganize their order and
restrictions.

When QDoc parses a project, it extracts the user-provided block-comments
that comprise the documentation.

Later, it processes them in multiple ways.
For example, it ties a documentation block to a `Node`, an internal
representation for documentable elements, by examining the usage of "topic
commands" in the block.

For "\fn" commands, which are used to document a C++ callable, QDoc
makes calls to Clang's APIs to parse the argument of the command so that
it can later compare it to the internal representation that QDoc built
of the code-base.

This is performed by `ClangCodeParser::parseFnArg` which is called by
`CppCodeParser`, which is the component responsible for processing
documentation blocks.

`parseFnArg` calls are intended, later on, to be batched for performance
reasons, albeit they are not expected to be parallelized at this point.

Hence, we move `parseFnArg` out of `ClangCodeParser` to simplify those
feature developments by having more control over which parts of
`ClangCodeParser` receive which changes.

`parseFnArg` was removed and replaced with the call operator of a new
class, `FnCommandParser`, a component that will only take care of
processing "\fn" commands.

`CppCodeParser`, which depended implicitly upon `ClangCodeParser` to
call `parseFnArg`, was modified to depend on `FnCommandParser`
explicitly.

The main loop of QDoc was modified to create and pass around an instance
of `FnCommandParser` as needed.

Task-number: QTBUG-111686
Change-Id: I517bd0d62b348ca39c4a8c10c4c2d78ab7c5ae15
Reviewed-by: Topi Reiniö <topi.reinio@qt.io>

5 files changed, 47 insertions, 21 deletions

diff --git a/src/qdoc/qdoc/src/qdoc/clangcodeparser.cpp b/src/qdoc/qdoc/src/qdoc/clangcodeparser.cpp
index 38e79cf42..98c6fe44d 100644
--- a/src/qdoc/qdoc/src/qdoc/clangcodeparser.cpp
+++ b/src/qdoc/qdoc/src/qdoc/clangcodeparser.cpp
@@ -2,6 +2,7 @@
 // SPDX-License-Identifier: LicenseRef-Qt-Commercial OR GPL-3.0-only WITH Qt-GPL-exception-1.0
 
 #include "clangcodeparser.h"
+#include "cppcodeparser.h"
 
 #include "access.h"
 #include "classnode.h"
@@ -14,6 +15,7 @@
 #include "qdocdatabase.h"
 #include "typedefnode.h"
 #include "variablenode.h"
+#include "utilities.h"
 
 #include <QtCore/qdebug.h>
 #include <QtCore/qelapsedtimer.h>
@@ -1779,8 +1781,6 @@ ParsedCppFileIR ClangCodeParser::parse_cpp_file(const QString &filePath)
         } else {
             parse_result.untied.emplace_back(UntiedDocumentation{doc, QStringList()});
 
-            // Store the namespace scope from lexical parents of the comment
-            m_namespaceScope.clear();
             CXCursor cur = clang_getCursor(tu, commentLoc);
             while (true) {
                 CXCursorKind kind = clang_getCursorKind(cur);
@@ -1806,7 +1806,7 @@ ParsedCppFileIR ClangCodeParser::parse_cpp_file(const QString &filePath)
   command. \a location is used for reporting errors. \a fnSignature
   is the string to parse. It is always a function decl.
  */
-Node *ClangCodeParser::parseFnArg(const Location &location, const QString &fnSignature, const QString &idTag, QStringList context)
+Node *FnCommandParser::operator()(const Location &location, const QString &fnSignature, const QString &idTag, QStringList context)
 {
     Node *fnNode = nullptr;
     /*
@@ -1869,13 +1869,13 @@ Node *ClangCodeParser::parseFnArg(const Location &location, const QString &fnSig
     }
 
     TranslationUnit tu;
-    s_fn.clear();
+    QByteArray s_fn{};
     for (const auto &ns : std::as_const(context))
         s_fn.prepend("namespace " + ns.toUtf8() + " {");
     s_fn += fnSignature.toUtf8();
     if (!s_fn.endsWith(";"))
         s_fn += "{ }";
-    s_fn.append(m_namespaceScope.size(), '}');
+    s_fn.append(context.size(), '}');
 
     const char *dummyFileName = fnDummyFileName;
     CXUnsavedFile unsavedFile { dummyFileName, s_fn.constData(),
diff --git a/src/qdoc/qdoc/src/qdoc/clangcodeparser.h b/src/qdoc/qdoc/src/qdoc/clangcodeparser.h
index 2318a048d..e6d78c719 100644
--- a/src/qdoc/qdoc/src/qdoc/clangcodeparser.h
+++ b/src/qdoc/qdoc/src/qdoc/clangcodeparser.h
@@ -4,8 +4,7 @@
 #ifndef CLANGCODEPARSER_H
 #define CLANGCODEPARSER_H
 
-#include "cppcodeparser.h"
-
+#include "codeparser.h"
 #include "config.h"
 
 #include <QtCore/qtemporarydir.h>
@@ -15,6 +14,8 @@
 
 typedef struct CXTranslationUnitImpl *CXTranslationUnit;
 
+class CppCodeParser;
+
 QT_BEGIN_NAMESPACE
 
 struct ParsedCppFileIR {
@@ -35,6 +36,34 @@ std::optional<PCHFile> buildPCH(
     const QList<QByteArray>& defines
 );
 
+struct FnCommandParser {
+    FnCommandParser(
+        QDocDatabase* qdb,
+        const std::set<Config::HeaderFilePath>& all_headers,
+        const QList<QByteArray>& defines,
+        std::optional<std::reference_wrapper<const PCHFile>> pch
+    ) : m_qdb{qdb},
+        m_allHeaders{all_headers},
+        m_defines{defines},
+        m_args{},
+        m_pch{pch}
+    {}
+
+    Node *operator()(
+        const Location &location,
+        const QString &fnSignature,
+        const QString &idTag,
+        QStringList context
+   );
+
+private:
+    QDocDatabase* m_qdb;
+    const std::set<Config::HeaderFilePath>& m_allHeaders; // file name->path
+    QList<QByteArray> m_defines {};
+    std::vector<const char *> m_args {};
+    std::optional<std::reference_wrapper<const PCHFile>> m_pch;
+};
+
 class ClangCodeParser : public CodeParser
 {
 public:
@@ -52,7 +81,6 @@ public:
     QStringList sourceFileNameFilter() override;
     void parseSourceFile(const Location &, const QString &, CppCodeParser&) override {}
     ParsedCppFileIR parse_cpp_file(const QString &filePath);
-    Node *parseFnArg(const Location &location, const QString &fnSignature, const QString &idTag, QStringList context);
 
 private:
     std::set<Config::HeaderFilePath> m_allHeaders {}; // file name->path
diff --git a/src/qdoc/qdoc/src/qdoc/cppcodeparser.cpp b/src/qdoc/qdoc/src/qdoc/cppcodeparser.cpp
index 14297c0fd..3523538ac 100644
--- a/src/qdoc/qdoc/src/qdoc/cppcodeparser.cpp
+++ b/src/qdoc/qdoc/src/qdoc/cppcodeparser.cpp
@@ -50,7 +50,8 @@ static const QMap<QString, NodeTypeTestFunc> s_nodeTypeTestFuncMap{
     { COMMAND_VARIABLE, &Node::isVariable },
 };
 
-CppCodeParser::CppCodeParser()
+CppCodeParser::CppCodeParser(FnCommandParser&& parser)
+    : fn_parser{parser}
 {
     Config &config = Config::instance();
     QStringList exampleFilePatterns{config.get(CONFIG_EXAMPLES
@@ -891,9 +892,7 @@ std::vector<TiedDocumentation> CppCodeParser::processTopicArgs(const UntiedDocum
         if (args.size() == 1) {
             if (topic == COMMAND_FN) {
                 if (Config::instance().showInternal() || !doc.isInternal())
-                    node = static_cast<ClangCodeParser *>(CodeParser::parserForLanguage("Clang"))
-                                   ->parseFnArg(doc.location(), args[0].first, args[0].second,
-                                                untied.context);
+                    node = fn_parser(doc.location(), args[0].first, args[0].second, untied.context);
             } else if (topic == COMMAND_MACRO) {
                 node = parseMacroArg(doc.location(), args[0].first);
             } else if (isQMLMethodTopic(topic)) {
@@ -912,10 +911,7 @@ std::vector<TiedDocumentation> CppCodeParser::processTopicArgs(const UntiedDocum
                 node = nullptr;
                 if (topic == COMMAND_FN) {
                     if (Config::instance().showInternal() || !doc.isInternal())
-                        node = static_cast<ClangCodeParser *>(
-                                       CodeParser::parserForLanguage("Clang"))
-                                       ->parseFnArg(doc.location(), arg.first, arg.second,
-                                                    untied.context);
+                        node = fn_parser(doc.location(), arg.first, arg.second, untied.context);
                 } else if (topic == COMMAND_MACRO) {
                     node = parseMacroArg(doc.location(), arg.first);
                 } else if (isQMLMethodTopic(topic)) {
diff --git a/src/qdoc/qdoc/src/qdoc/cppcodeparser.h b/src/qdoc/qdoc/src/qdoc/cppcodeparser.h
index edcc014c8..c4c2c6fff 100644
--- a/src/qdoc/qdoc/src/qdoc/cppcodeparser.h
+++ b/src/qdoc/qdoc/src/qdoc/cppcodeparser.h
@@ -4,6 +4,7 @@
 #ifndef CPPCODEPARSER_H
 #define CPPCODEPARSER_H
 
+#include "clangcodeparser.h"
 #include "codeparser.h"
 #include "utilities.h"
 
@@ -34,7 +35,7 @@ public:
         << COMMAND_QMLINSTANTIATES << COMMAND_REIMP << COMMAND_RELATES;
 
 public:
-    CppCodeParser();
+    CppCodeParser(FnCommandParser&& parser);
 
     FunctionNode *parseMacroArg(const Location &location, const QString &macroArg);
     FunctionNode *parseOtherFuncArg(const QString &topic, const Location &location,
@@ -61,6 +62,7 @@ private:
     static void processComparesCommand(Node *node, const QString &arg, const Location &loc);
 
 private:
+    FnCommandParser fn_parser;
     QString m_exampleNameFilter;
     QString m_exampleImageFilter;
     bool m_showLinkErrors { false };
diff --git a/src/qdoc/qdoc/src/qdoc/main.cpp b/src/qdoc/qdoc/src/qdoc/main.cpp
index 92646898f..59ab0f7f3 100644
--- a/src/qdoc/qdoc/src/qdoc/main.cpp
+++ b/src/qdoc/qdoc/src/qdoc/main.cpp
@@ -60,10 +60,8 @@ bool creationTimeBefore(const QFileInfo &fi1, const QFileInfo &fi2)
 
     \sa CodeParser::parserForSourceFile, CodeParser::sourceFileNameFilter
 */
-static void parseSourceFiles(std::vector<QString>&& sources)
+static void parseSourceFiles(std::vector<QString>&& sources, CppCodeParser& cpp_code_parser)
 {
-    CppCodeParser cpp_code_parser{};
-
     std::stable_sort(sources.begin(), sources.end());
 
     sources.erase (
@@ -576,7 +574,9 @@ static void processQdocconfFile(const QString &fileName)
             qCInfo(lcQdoc) << "Parse source files for" << project;
 
 
-        parseSourceFiles(std::move(sources));
+        auto headers = config.getHeaderFiles();
+        CppCodeParser cpp_code_parser(FnCommandParser(qdb, headers, clang_defines, pch));
+        parseSourceFiles(std::move(sources), cpp_code_parser);
 
         if (config.get(CONFIG_LOGPROGRESS).asBool())
             qCInfo(lcQdoc) << "Source files parsed for" << project;