aboutsummaryrefslogtreecommitdiffstats
path: root/testing
Commit message (Collapse)AuthorAgeFilesLines
* testing: solve flake8 warningsCristian Maureira-Fredes2021-03-037-55/+51
| | | | | | | Change-Id: I75f1a367c8a86ec586820bd4a45339773c15a70a Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io> (cherry picked from commit 2de3a763fa18d5c30c2cff25057f1b81ceeed231) Reviewed-by: Qt Cherry-pick Bot <cherrypick_bot@qt-project.org>
* testing: Add a "ci" keyword equivalent to the one in QtFriedemann Kleint2020-12-031-0/+2
| | | | | | | COIN-only failures can then be tagged accordingly. Change-Id: Ib7f5abaea48eee5feee5b676ba80421c66ffb08e Reviewed-by: Christian Tismer <tismer@stackless.com>
* Blacklist failing tests for Qt 6Friedemann Kleint2020-11-191-1/+1
| | | | | | | | | Tested using Qt 6 beta 5. Task-number: PYSIDE-1339 Task-number: PYSIDE-904 Change-Id: I378770df929327aa8aa502751baac482702520d7 Reviewed-by: Christian Tismer <tismer@stackless.com>
* python3: remove all obsolete "from __future__" import statementsChristian Tismer2020-11-091-1/+0
| | | | | | | | | | | Note that not all future statements can be removed: PEP 563 for instance deals with from __future__ import annotations Task-number: PYSIDE-904 Change-Id: Ia387ec87f3c68ca64d13e8e80191eda900f58638 Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
* testrunner: Report the number of failed testsFriedemann Kleint2020-11-061-3/+7
| | | | | Change-Id: I0e5f73d7bf0bb404dc77d6d016a211e61c09da91 Reviewed-by: Christian Tismer <tismer@stackless.com>
* Rename PySide2 to PySide6Friedemann Kleint2020-11-022-4/+4
| | | | | | | | Adapt CMake files, build scripts, tests and examples. Task-number: PYSIDE-904 Change-Id: I845f7b006e9ad274fed5444ec4c1f9dbe176ff88 Reviewed-by: Christian Tismer <tismer@stackless.com>
* replace **locals by f-strings where possibleChristian Tismer2020-11-022-13/+12
| | | | | | | | | This change affects mostly only my own sources which were prepared for the migration to Python 3.6 . Task-number: PYSIDE-904 Change-Id: I0c2cd59f6f625f51f876099c33005ac70ca39db9 Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
* Remove old pyside2-tools directoryCristian Maureira-Fredes2020-10-311-1/+1
| | | | | | | | | | | | The tools directory was brought back as sources/pyside-tools and currently only holds the pyside_tool.py script to create the entry points for rcc and uic. Since the submodule is not present anymore, we remove also the reference from the testing step. Change-Id: I0bfa4509d44c9bd2a3e5fbf4ddb7fff7a2750f4d Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
* TestRunner: Fix syntax errorFriedemann Kleint2020-10-291-1/+1
| | | | | | | Amends b78e5bd771f0d5a81692bc95f76929b092edf2f6. Change-Id: Idecff944c27c9473cd565224547a206199af8a30 Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
* testing: remove __future__ and Python2 referencesCristian Maureira-Fredes2020-10-288-28/+5
| | | | | | Change-Id: I80079d8a0956e3cc9b27ceb5b5ea2cfc6c9c9449 Reviewed-by: Christian Tismer <tismer@stackless.com> Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
* Rename shiboken2 to shiboken6Friedemann Kleint2020-10-282-4/+4
| | | | | | | | Adapt CMake files, build scripts, tests and examples. Task-number: PYSIDE-904 Change-Id: I4cb5ee4c8df539546014b08202a7b1e98ed3ff07 Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
* wheel_tester.py: No longer use class OPTIONS from the build scriptsFriedemann Kleint2020-10-071-12/+9
| | | | | | | | | The semantics have changed, the OPTIONS dict is not fully populated at startup. Task-number: PYSIDE-807 Change-Id: Ic076f4ec8d624119d11e6ee266e2f6ccdb702b1e Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
* Testing: Fix a quirk introduced by pruningChristian Tismer2020-07-161-1/+4
| | | | | | | | | | | The newly introduced history pruning has a timing problem on RHel 7.6. We disable pruning now when COIN is active. This makes anyway only sense for multiple builds on a developer machine but not in COIN. Change-Id: Ic4b4f2e42303e343d222b98655300509f83c7288 Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io> Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
* testing: Automate pruning of old history entriesChristian Tismer2020-07-141-1/+26
| | | | | | | | | | The always growing list of invalid directory entries in the history will get pruned to only keep the latest days entries. Change-Id: Ie104ca9874f985f0040950f650390798f45fc6d5 Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io> Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
* Improve wheel_tester script a bitChristian Tismer2020-03-201-5/+9
| | | | | | | | | | | Wheel_tester forces installation of "shiboken2_generator". This is not needed for compile_using_pyinstaller, but building modules with cmake. This was not clear. In effect, just the log messages were made a bit more verbose and a clarifying comment was added. Change-Id: I7d7d9ce1e9911461d087d685ab5c1cb66c94aa27 Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
* testrunner: Fix disrupted lines in the error logChristian Tismer2020-02-206-58/+139
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Windows inserts extra newlines into the error log when certain errors occur like "Exit code 0xc0000409\n***Exception:" and that newline caused the parser match to fail. Note that this is the normal stdout. CMake does not use stderr. It makes no sense to fix the pipe structure of the script. Instead, the only fix needed was recognition of line breaks in the error log. You can see this also in the failure listing The following tests FAILED: 1 - pysidetest_constructor_properties_test (Exit code 0xc0000409 ) The following improvements were done: - add an extra pass that checks for broken lines in the error log - add ad extra plausibility check or consecutive test numbers - improve the output, program structure and add some documentation When there should still an unforeseen bug occurs, it will be recognized by the plausi-check and the test repetitions are immediately cancelled. We could also fix the output not to contain the line breaks, but that breaks the principle of keeping the original output and needs discussion by the developers. Fixes: PYSIDE-1229 Change-Id: Ib71f3361e78eb59f3469da172c74c719e9f08706 Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
* Replace OPTION_* by a dictionaryCristián Maureira-Fredes2019-12-031-4/+3
| | | | | | | | | | | | Currently we are importing * from the options.py file, which is a problem for the linters to discover if we are properly using the options of the file. Having a dictionary provides also a better way of access these options, and it is more clean than having one variable per each option. Change-Id: Ie70bd88665357b85e2842c5df269857504ecccdf Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
* Make the scrape script robust enough, againChristian Tismer2019-10-311-2/+5
| | | | | | | | | | | | | | | | | The script scrape_testresults.py analyzes log files in order to find embedded script files for the registry. This script does not try to be absolute water-proof but handled the existing cases well. By chance, the text "BEGIN_FILE", which is used as an indicator, was found in a traceback from the generating script. We modified the search to ignore this text in single quotes, but also changed the originating file testing/runner.py to no longer contain this text in the future. Change-Id: I1fef1cf5e9b85fc94b7ee52edbdf4afa4bf4e35a Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
* wheel_tester.py: Add a command line option to disable wheel installationFriedemann Kleint2019-07-121-4/+12
| | | | | | | This is useful when testing in a local developer build. Change-Id: Ib875dabd21d437951d3909030b47805b807fac9d Reviewed-by: Christian Tismer <tismer@stackless.com>
* Fix PyInstaller after removal of COIN glitchesChristian Tismer2019-04-291-13/+20
| | | | | | | | | | | | | | After the COIN problems have been removed, the only remaining problem is PyInstaller, which crashes on macOS with Python 2.7.16 . For a full problem description, see the amendment to "Current COIN Wheel Tester Problems" To simplify a later problem analysis, "pip list" is included. Fixes: PYSIDE-996 Change-Id: Ie607ac190ab703a0b8cbe20483b4b60f81634d9d Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
* Provide a spec file for hello.app used by wheel_testerChristian Tismer2019-04-231-3/+6
| | | | | | | | | | | | | | | | | | | | | | The PyInstaller test was done with minimalistic options to keep things as simple as possible. This had the side effect that on certain Python versions and platforms things were touched that created errors, although those things were nowhere in use. It turned out that the generated hello_app.spec file contains the result of an in-depth analysis which touches so many non-existing modules. When using just the generated spec file and no other option, the analysis is almost skipped and the generation of an executable avoids further problems and is faster. This patch removes all arguments and uses a spec file instead. The arguments are kept in a comment, just in case. Change-Id: I17c43a588bc6f95f62fc3104a047dc08e8d8e70d Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io> Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
* Automatically Test Small Example With PyInstallerChristian Tismer2019-04-081-1/+40
| | | | | | | | | | | | | | | | | | | A simple hello.py script was modified for running in PyInstaller and stopping to execute after 2 seconds. The reason is to test that PyInstaller works correctly together with the embedded mode of the signature extension on all platforms. The script did first not work on Windows. This is now solved, after an import in pyside2_config.py is fixed. Currently, there are several configuration errors in COIN. Errors are therefore skipped in the PyInstaller build. The test tests only if the generated script works. Change-Id: I7a1b1e738d523b83cc3fe5beafa7e2579f9c7f48 Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
* testrunner: Fix error message about forgotten --build-testsFriedemann Kleint2019-03-181-1/+1
| | | | | | | | | | Fix variable name changed by f30e4db5169800c25bf79573f650fc2b08d13046, fixing error: NameError global name 'make_path' is not defined Change-Id: Ieee3ba884e122cbc6e2cec267653481552a041d9 Reviewed-by: Christian Tismer <tismer@stackless.com>
* Move the .pyqtc files to .pyproject filesFriedemann Kleint2019-03-012-9/+4
| | | | | | | Use the new project file format. Change-Id: I69f488c285343500edd999b746ce244a56504030 Reviewed-by: Christian Tismer <tismer@stackless.com>
* Modernize cmake buildAlexandru Croitor2019-02-071-7/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add a CMake super project that includes the shiboken2, PySide2 and pyside2-tools subprojects, so that it's possible to build everything from Qt Creator (or any other IDE that supports CMake) with minimal set up effort, and thus inform the IDE CMake integration of all relevant files, for easier code editing, navigation and refactoring. This also lays the foundation for allowing 3rd parties to use the shiboken2 generator to generate custom modules. This is achieved by eliminating various hardcoded paths for libraries and include directories. Start using CMake targets throughout the build code to correctly propagate link flags and include dirs for libshiboken and shiboken2 executable targets. Same for the libpyside target. Generate two separate cmake config files (build-tree / install-tree) that can be used with find_package(Shiboken2), to make sure that the PySide2 project can be built as part of the super project build. This is currently the only way I've found to allow the super build to work. Note that for the build-tree find_package() to work, the CMAKE_MODULE_PATH has to be adjusted in the super project file. The generated config files contain variables and logic that allow usage of the installed shiboken package in downstream projects (PySide2). This involves things like getting the includes and libraries for the currently found python interpreter, the shiboken build type (release or debug), was shiboken built with limited api support, etc. Generate 2 separate (build-tree and install-tree) config files for PySide2, similar to how it's done for the shiboken case, for pyside2-tools to build correctly. Install shiboken2 target files using install(EXPORT) to allow building PySide2 with an installed Shiboken2 package (as opposed to one that is built as part of the super project). Same with PySide2 targets for pyside2-tools subproject. Make sure not to redefine uninstall targets if they are already defined. Add a --shorter-paths setup.py option, which would be used by the Windows CI, to circumvent creating paths that are too long, and thus avoiding build issues. Output the build characteristics / classifiers into the generated build_history/YYYY-MM-DD_AAAAAA/build_dir.txt file, so it can be used by the test runner to properly filter out blacklisted tests. This was necessary due to the shorter paths options. Fix various issues regarding target includes and library dependencies. Remove certain duplicated cmake code (like limited api check and build type checks) in PySide2, given that that information will now be present in the exported shiboken2 config file. Include a short README.cmake.md file that describes how to build the super project. References used https://rix0r.nl/blog/2015/08/13/cmake-guide/ https://pabloariasal.github.io/2018/02/19/its-time-to-do-cmake-right/ https://gist.github.com/mbinna/c61dbb39bca0e4fb7d1f73b0d66a4fd1 https://cliutils.gitlab.io/modern-cmake/chapters/basics/functions.html https://cmake.org/cmake/help/v3.0/manual/cmake-packages.7.html https://github.com/ComicSansMS/libstratcom/blob/master/CMakeLists.txt Abandoned approach using ExternalProject references: https://cmake.org/cmake/help/latest/module/ExternalProject.html https://stackoverflow.com/questions/44990964/how-to-perform-cmakefind-package-at-build-stage-only Fixes: PYSIDE-919 Change-Id: Iaa15d20b279a04c5e16ce2795d03f912bc44a389 Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
* Build scripts: Add support for ninjaFriedemann Kleint2018-11-271-11/+23
| | | | | | | | | ninja will be recommended build tool/CMake generator for Qt due to its speed. Streamline the option parsing code and add it. Adapt the test runner to find the ctest command in the ninja build file. Change-Id: I61dd6fd4fb26a50af21432e10e7da86123240e0f Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
* testrunner.py: Add a small Qt Creator Python project file for editingFriedemann Kleint2018-11-241-0/+9
| | | | | Change-Id: Icc009fde4ac6266f4beaf29b235a6af27d1a7d1d Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
* Add tests to install built wheels and build some examplesAlexandru Croitor2018-10-121-0/+295
| | | | | | | | | | | | | | | | | | | | Augment coin_test_instructions.py to run a new script testing/wheel_tester.py, which tries to pip install the built wheels, and then tries to build the samplebinding and scriptableapplication examples. This allows us to confirm that the generated wheels are actually installable, and also hopefully prevent us from breaking the embeddable examples, by making sure that they at least build (and execute in the case of samplebinding). The change also modifies the examples to be able to take the python executable as build argument, so that wheel_tester can specify explicitly which python interpeter to use. Change-Id: I0f141e40ab86e3311dd25915c4e53b0af36aaae9 Reviewed-by: Christian Tismer <tismer@stackless.com> Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
* Format python files to use PEP8Cristian Maureira-Fredes2018-05-231-9/+9
| | | | | | | Change-Id: I8dc16c10565da8f93ba5cb7cace004f8eaf52236 Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io> Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io> Reviewed-by: Christian Tismer <tismer@stackless.com>
* Change license from all the filesCristian Maureira-Fredes2018-05-036-6/+6
| | | | | | | | | | | | | | | | | Removing the word 'project' from all the headers, and changing the PySide reference from the examples to Qt for Python: The following line was used inside the source/ and build_scripts/ directory: for i in $(grep -r "the Qt for Python project" * |grep -v "pyside2-tools" | awk '{print $1}' | sed 's/:.*//g');do sed -i 's/the\ Qt\ for\ Python\ project/Qt\ for\ Python/g' $i;done and the following line was used inside the examples/ directory: for i in $(grep -r "of the PySide" * |grep -v "pyside2-tools" | awk '{print $1}' | sed 's/:.*//g');do sed -i 's/of\ the\ PySide/of\ the\ Qt\ for\ Python/g' $i;done Change-Id: Ic480714686ad62ac4d81c670f87f1c2033d4ffa1 Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io> Reviewed-by: Alex Blasche <alexander.blasche@qt.io>
* Split and move various setup.py parts into different filesAlexandru Croitor2018-04-301-3/+12
| | | | | | | | | | | | | | | | | This is an initial effort to clean up setup.py. A new directory called build_scripts contains most of the logic for building, leaving setup.py as an entry point. The build_scripts directory contains the usual qtinfo, utils, and the setup.py content has been split into main.py and platform specific files under platforms subfolder. The testrunner script has been modified to find the new location of the utils module. Task-number: PYSIDE-558 Change-Id: I3e041d5116ca5c3f96c789317303b65a7b1bbd70 Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
* Registry: Trigger re-generation of files for Qt3DChristian Tismer2018-02-021-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | This is phase 2. The files are generated. I only changed the date in the license section. After this check-in, the update is done. From Phase 1: The function registry does not contain the new Qt3D module. This produces no error because the test is configured to only break on missing functions but not on new unknown functions. We provoke a reaction of the system by removing of the 5.9 registry files. The system will generate an error once and produce the desired output. But because of the multiple testing, the test will succeed because the generated file exist on the second run and therefore the test will succeed as a flaky test. There is only one run necessary for all platforms at once. A second check-in will then do the update with the generated data. I also had to fix the testrunner to produce a good listing without labels. Task-number: PYSIDE-487 Change-Id: I06a73d244ce306977fd16223eec4dc491fff3429 Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
* Merge remote-tracking branch 'origin/5.6' into 5.9Friedemann Kleint2018-01-121-0/+8
|\ | | | | | | Change-Id: I18387b329e61646e8d68e678140b533281e359cd
| * Enhance testrunner to ignore old test logsChristian Tismer2018-01-051-0/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | Testrunner had a virtual problem: When running testrunner without building the project again, and the test skips some runs that were not skipped before, then it will fetch an old test log from the former run. Cure: We check all logfiles first. If a file exists and skip is not set, then we remove the file. Then normal testing starts. Change-Id: Ib91baca97d3b0a5c0b68a8022fca52202b20e862 Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
* | Merge remote-tracking branch 'origin/5.6' into 5.9Friedemann Kleint2018-01-051-0/+14
|/ | | | Change-Id: I254cc725505349e73c7abc290afc7abb85156997
* small enhancement for testrunnerChristian Tismer2017-12-223-20/+18
| | | | | | | | | | | | The new testrunner now has the multiple test feature. We refine and correct the summary view a little bit, remove more dead code, add some comments and improve the output slightly. Change-Id: I152bf5cbe2171fb07de7e88054a42fc767dc14c0 Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io> Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org> Reviewed-by: Simo Fält <simo.falt@qt.io>
* testrunner 5: Improve the algorithmChristian Tismer2017-12-214-64/+245
| | | | | | | | | | | | | | | | | | | | | | Testrunner checks if it is running in COIN. If so, it runs each tested project 5 times and reports an error if 3 errors were found in a test function and not blacklisted. The time is measured, so we know if we can run all the tests five times or if we can run only the failed tests. At the moment, the option COIN_RERUN_FAILED_ONLY is activated by default. We can configure it by setting to false. Since this change turns the registry existence test into a flaky test (te tests generates it's missing file in the first run), the registry existence test is no more blacklisted. We simulate our own tee command now with decorated output. Task-number: PYSIDE-578 Change-Id: I6390cd50398a97a168c85c6bb778984c6a5b90fc Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
* testrunner 4: Improve the error listingChristian Tismer2017-12-212-3/+3
| | | | | | | | A simple change that makes the "FAIL!" entries easy to find. Task-number: PYSIDE-578 Change-Id: I953bf94912f101208c1cddb7772aa0c34e9ca1a8 Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
* testrunner 3: Replace (n)make and tee by ctestChristian Tismer2017-12-212-60/+73
| | | | | | | | | | | | | Before working on the algorithm, we simplify the testrunner by using ctest directly. This has also the advantage that we can control much better how the output is generated. By interrogating the Makefile, we can now detect early if ctest was configured in setup.py . Task-number: PYSIDE-578 Change-Id: I1157576bf5806044465d7a1409c3d26810f33f62 Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
* testrunner 2: Remove unused code and clean upChristian Tismer2017-12-212-235/+14
| | | | | | | | | | | Testrunner contains old learning code for blacklist that was never used. We remove that code. The new display of environment was correctly placed and reformatted. Task-number: PYSIDE-578 Change-Id: Ia508314aa1a87d07522386342059c14cb1254400 Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
* testrunner 1: Turn into a packageChristian Tismer2017-12-217-0/+1096
Testrunner is a medium-large module that has a number of different purposes. Since we want to improve the testing logic, it is better to clean the module up before adding even more code. For compatibitity, the testrunner module will stay in place, but redirect everything into the new "testing" package. Task-number: PYSIDE-578 Change-Id: I3d14f3c4c421cdd2bb7777691b56677aa4786dd8 Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
generated by cgit v1.2.3 (git 2.25.1) at 2024-04-30 15:01:04 +0000