diff options
Diffstat (limited to 'src/qdoc/qdoc/tests/validateqdocoutputfiles/README.md')
-rw-r--r-- | src/qdoc/qdoc/tests/validateqdocoutputfiles/README.md | 82 |
1 files changed, 82 insertions, 0 deletions
diff --git a/src/qdoc/qdoc/tests/validateqdocoutputfiles/README.md b/src/qdoc/qdoc/tests/validateqdocoutputfiles/README.md new file mode 100644 index 000000000..728cbee62 --- /dev/null +++ b/src/qdoc/qdoc/tests/validateqdocoutputfiles/README.md @@ -0,0 +1,82 @@ +<!-- + Copyright (C) 2024 The Qt Company Ltd. + SPDX-License-Identifier: LicenseRef-Qt-Commercial OR GFDL-1.3-no-invariants-only +--> + +# QDoc output test +This is a test that validates the files QDoc generates. It is not a test for +QDoc itself. The test calls the qdoc executable for each project in the +`testdata` directory (see [Test project structure](# Test project structure), +below), and then compares the output to that in the `expected` directory for +the test project. Each test is identified in the test output by the name of the +test project directory *and* the name of the .qdocconf-file. + +The test creates a new temporary directory for each test project. The +comparison is done by running `git diff` on the content in this temporary +directory (the output directory) and the contents in the test's `expected` +directory. A non-zero exit code fails the test for that project, and the test +includes the diff in its output if this happens. + +## How to add a new test +1. Create a new directory in the `testdata` directory. The name of the new + directory should be descriptive of the test project. +2. Create a `.qdocconf` file in the new directory. See + [The .qdocconf file](# The .qdocconf file) below for further details. +3. Add the necessary files (.qdoc, .h/.cpp, .qml, etc) so that the project + can be built in a meaningful manner. +4. Run QDoc on the project. +5. Verify that the output looks correct. +6. Copy the output into `[testdata/[new test directory]/expected`. +7. Run the test executable and verify that the test output includes a **PASS** + line for the new test-case. +8. Push your change upstream. + +## Update the expected content for all tests +If you make a change to QDoc that causes significant changes in output, you may +need to update the expected output for many, or even all, tests. If you set the +environment variable `QDOC_REGENERATE_TESTDATA=1` before running the test, all +current test data will be removed, and QDoc will be run on each project to +generate new output. Note, however, that the comparison per project is skipped +in this scenario. + +## Test project structure +The `testdata` directory is where the test looks for projects to test. Each +project has its own directory, a `qdocconf` file, and an `expected` +directory that contains the expected file output from QDoc. + +The project directory must contain a .qdocconf whose name matches that of the +project directory, otherwise the project won't be picked up by the test. + +### The .qdocconf file +The `.qdocconf` file contains the configuration necessary to build the +QDoc project in the directory. Observe the following: +- **Important!** Make sure to set `locationinfo = false` to avoid test failures + due to differences in the location information in the generated output. +- Test projects should be warning free, so set `warninglimit.enabled = true`. +- The name of the .qdocconf-file must be identical as that of the directory + that contains it. This means that for a project **foo**, it should go into + `testdata/foo` and the configuration file should be + `testdata/foo/foo.qdocconf`. +- Any other `.qdocconf` file will not be picked up explicitly by the test. + This means that for a test project **foo** that resides in `testdata/foo`, + `testdata/foo/foo.qdocconf` will be treated as a test-case, while + `testdata/foo/bar.qdocconf` will not. However, `foo.qdocconf` may contain + `include(bar.qdocconf)`, and this will work as expected. +- The `warninglimit` should be set to the number of warnings expected + from QDoc for that specific project, if any are to be expected at all. +- You probably want to configure all output formats, otherwise only the HTML + output will be generated. The test will compare all output formats that are + generated to the expected output, so remember to specify the output directory + for each format. By convention, the output directories are named after the + format, for example `html`, `docbook`, `webxml`, etc. +- Place the sources for your new test in a subdirectory of the test project + directory. By convention, the sources are placed in a directory named `src`. + +### The `expected` directory +The `expected` directory contains the expected output from QDoc +for the project. If QDoc generates an empty directory (for example, it +always creates the `images/` directory for a project, whether +the project contains any images or not), that directory isn't tracked +in the `expected` directory, as **git** doesn't track directories, only +files. + |