Rewrite Qt Testlib selftest to not rely on Qt Testlib itself

We use the Catch2 testing framework to test Qt Testlib, which also opens
up the possibility of using it for other internal testing once it's made
available through the build system.

The test now has a --rebase mode which will write out the actual results
as new expected files. Once we add the required post-processing to the
results to remove timestamps and other testrun-specific data we can
remove the standalone python script generate_expected_output.py that
today has to be kept in sync with the test itself.

No attempt has been made to clean up the comparison-functions, but
these could all benefit from moving their logic from the comparison
to the sanitization step. This will both make the expected files
more generic, and will reduce the diff once a failure occurs, since
we're not seeing all the hunks that the comparison-functions ignored.

Change-Id: I1769d42e7958d56d1ad5da958db0e8fe3a2a3c23
Reviewed-by: Lars Knoll <lars.knoll@qt.io>

1 files changed, 66 insertions, 0 deletions

diff --git a/tests/auto/testlib/selftests/catch.cpp b/tests/auto/testlib/selftests/catch.cpp
new file mode 100644
index 0000000000..5d27d400e3
--- /dev/null
+++ b/tests/auto/testlib/selftests/catch.cpp
@@ -0,0 +1,66 @@
+/****************************************************************************
+**
+** Copyright (C) 2020 The Qt Company Ltd.
+** Contact: https://www.qt.io/licensing/
+**
+** This file is part of the QtTest module of the Qt Toolkit.
+**
+** $QT_BEGIN_LICENSE:LGPL$
+** Commercial License Usage
+** Licensees holding valid commercial Qt licenses may use this file in
+** accordance with the commercial license agreement provided with the
+** Software or, alternatively, in accordance with the terms contained in
+** a written agreement between you and The Qt Company. For licensing terms
+** and conditions see https://www.qt.io/terms-conditions. For further
+** information use the contact form at https://www.qt.io/contact-us.
+**
+** GNU Lesser General Public License Usage
+** Alternatively, this file may be used under the terms of the GNU Lesser
+** General Public License version 3 as published by the Free Software
+** Foundation and appearing in the file LICENSE.LGPL3 included in the
+** packaging of this file. Please review the following information to
+** ensure the GNU Lesser General Public License version 3 requirements
+** will be met: https://www.gnu.org/licenses/lgpl-3.0.html.
+**
+** GNU General Public License Usage
+** Alternatively, this file may be used under the terms of the GNU
+** General Public License version 2.0 or (at your option) the GNU General
+** Public license version 3 or any later version approved by the KDE Free
+** Qt Foundation. The licenses are as published by the Free Software
+** Foundation and appearing in the file LICENSE.GPL2 and LICENSE.GPL3
+** included in the packaging of this file. Please review the following
+** information to ensure the GNU General Public License requirements will
+** be met: https://www.gnu.org/licenses/gpl-2.0.html and
+** https://www.gnu.org/licenses/gpl-3.0.html.
+**
+** $QT_END_LICENSE$
+**
+****************************************************************************/
+
+#define CATCH_CONFIG_RUNNER
+#define CATCH_CLARA_CONFIG_CONSOLE_WIDTH 1000
+
+#if defined(QT_NO_EXCEPTIONS)
+#define CATCH_CONFIG_DISABLE_EXCEPTIONS
+#endif
+
+#include "catch_p.h"
+
+QT_BEGIN_NAMESPACE
+
+namespace QTestPrivate {
+
+int catchMain(int argc, char **argv)
+{
+    Catch::Session session;
+
+    if (int returnCode = session.applyCommandLine(argc, argv))
+        return returnCode; // Command line error
+
+    return session.run();
+}
+
+} // namespace QTestPrivate
+
+QT_END_NAMESPACE
+