diff options
| author | Alexandru Croitor <alexandru.croitor@qt.io> | 2018-07-20 18:54:05 +0200 |
|---|---|---|
| committer | Alexandru Croitor <alexandru.croitor@qt.io> | 2018-10-12 14:45:35 +0000 |
| commit | 43fe3494a9d902034896e3afa7b5158c77163be0 (patch) | |
| tree | 07bbd20b8bd322fdf062b87dfade1e1e74d80433 | |
| parent | c6c9f057cddc0e0b885a648489264538eba0a158 (diff) | |
Allow building shiboken2 and PySide2 as separate wheels
Actually this creates 3 wheel packages:
- shiboken2 (the python module and libshiboken shared library)
- shiboken2-generator (contains the generator executable, libclang and
dependent Qt libraries)
- PySide2 (the PySide2 modules and Qt shared libraries, and tools like
rcc, uic)
Calling the setup.py script will not do the actual build now (in the
sense of calling CMake, make, etc.). Instead it will spawn new
processes (via subprocess.call) calling the same setup.py script,
but with different arguments. These "sub-invocations" will do the
actual building. Thus, the "top-level invocation" will decide which
packages to build and delegate that to the "sub-invocations" of
setup.py.
A new optional command line argument is introduced called
"--build-type" which defaults to "all", and can also be set to
"shiboken2", "shiboken2-generator" and "pyside2". A user can choose
which packages to build using this option. The "top-level invocation"
uses this option to decide how many "sub-invocations" to execute.
A new command line argument called "--internal-build-type"
takes the same values as the one above. It defines which package
will actually be built in the new spawned "sub-invocation" process.
The "top-level invocation" sets this automatically for each
"sub-invocation" depending on the value of "--build-type".
This option is also useful for developers that may want to debug the
python building code in the "sub-invocation".
Developers can set this manually via the command line, and
thus avoid the process spawning indirection.
A new class Config is introduced to facilitate storage of
the various state needed for building a single package.
A new class SetupRunner is introduced that takes care of the
"--build-type" and "--internal-build-type" argument handling
and delegation of "sub-invocations".
A new class Options is introduced to 'hopefully', in the future, streamline
the mess of option handling that we currently have.
setup.py now is now simplified to mostly just call
SetupRunner.run_setup().
Certain refactorings were done to facilitate further clean-up of the
build code, the current code is definitely not the end all be all.
Various other changes that were needed to implement the wheel
separation:
- a new cmake_helpers directory is added to share common cmake
code between packages.
- the custom popenasync.py file is removed in favor of using
subprocess.call in as many places as possible, and thus
avoid 10 different functions for process creation.
- Manifest.in is removed, because copying to the setuptools
build dir is now done directly by prepare_packages functions.
- because prepare_packages copies directly to the setuptools
build dir, avoiding the pyside_package dir, we do less copying
of big Qt files now.
- versioning of PySide2 and shiboken2 packages is now separate.
shiboken2 and shiboken2-generator share the same versions for
now though.
- shiboken2 is now listed as a required package for PySide2, to
facilitate pip requirements.txt dependencies.
- coin_build_instructions currently needs to install an unreleased
version of wheel, due to a bug that breaks installation of
generated wheel files.
- added separate command line options to pyside2_config.py for
shiboken2-module and shiboken2-generator.
- adapted samplebinding and scriptableapplication projects due to
shiboken being a separate package.
- adapted pyside2-tool and shiboken2-tool python scripts for setup
tools entry points.
- made some optimizations not to invoke cmake for shiboken2-generator
when doing a top-level "all" build.
- fixed unnecessary rpaths not to be included on Linux (mainly the
Qt rpaths).
Task-nubmer: PYSIDE-749
Change-Id: I0336043955624c1d12ed254802c442608cced5fb
Reviewed-by: Christian Tismer <tismer@stackless.com>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
37 files changed, 2108 insertions, 1466 deletions
diff --git a/.gitignore b/.gitignore index 6a8b69288..903fc81f3 100644 --- a/.gitignore +++ b/.gitignore @@ -1,7 +1,6 @@ /build /dist /pyside*_build -/pyside*_package /pyside*_install /PySide /PySide-*.*.* @@ -13,3 +12,5 @@ distribute-*.egg distribute-*.tar.gz explore2 build_history/2* +*.qdocconf +*.qdocconf.in diff --git a/MANIFEST.in b/MANIFEST.in deleted file mode 100644 index dd79b2380..000000000 --- a/MANIFEST.in +++ /dev/null @@ -1,40 +0,0 @@ -# -# MANIFEST.in -# -# Manifest template for creating the PySide source distribution. - -include MANIFEST.in -include CHANGES.rst -include README.rst -include ez_setup.py -include setup.py -include popenasync.py -include qtinfo.py -include utils.py - -# sources -recursive-include sources/patchelf ** -recursive-include sources/shiboken2 ** -recursive-include sources/pyside2 ** -recursive-include sources/pyside2-tools ** -# ignore .git -recursive-exclude sources/shiboken2/.git ** -recursive-exclude sources/pyside2/.git ** -recursive-exclude sources/pyside2-tools/.git ** - -# PySide package -recursive-include pyside_package/PySide2 ** -recursive-include pyside_package/PySide2/docs ** -recursive-include pyside_package/PySide2/plugins ** -recursive-include pyside_package/PySide2s ** -recursive-include pyside_package/PySide2/translations ** -recursive-include pyside_package/PySide2include ** -recursive-include pyside_package/PySide2/typesystems ** -recursive-include pyside_package/PySide2/examples ** - -# pysideuic package -recursive-include pyside_package/pysideuic ** -recursive-include pyside_package/pysideuic/Compiler ** -recursive-include pyside_package/pysideuic/port_v2 ** -recursive-include pyside_package/pysideuic/port_v3 ** -recursive-include pyside_package/pysideuic/widget-plugins ** @@ -1,53 +1,9 @@ -# PySide2 +# Qt For Python -### Introduction - -PySide is the [Python Qt bindings project](http://wiki.qt.io/PySide2), providing +Qt For Python is the [Python Qt bindings project](http://wiki.qt.io/PySide2), providing access to the complete Qt 5.x framework as well as to generator tools for rapidly generating bindings for any C++ libraries. -The PySide project is developed in the open, with all facilities you'd expect -from any modern OSS project such as all code in a git repository and an open -design process. We welcome any contribution conforming to the -[Qt Contribution Agreement](https://www.qt.io/contributionagreement/). - - -PySide 2 supports Qt5. For building, please read about -[getting started](https://wiki.qt.io/PySide2_GettingStarted). -Then download the sources by running - - git clone https://code.qt.io/pyside/pyside-setup - -### Building - -#### Dependencies - -PySide versions following 5.6 use a C++ parser based on -[Clang](http://clang.org/). The Clang library (C-bindings), version 3.9 or -higher is required for building. Prebuilt versions of it can be downloaded from -[download.qt.io](http://download.qt.io/development_releases/prebuilt/libclang/). - -After unpacking the archive, set the environment variable *LLVM_INSTALL_DIR* to -point to the folder containing the *include* and *lib* directories of Clang: - - 7z x .../libclang-release_39-linux-Rhel7.2-gcc5.3-x86_64.7z - export LLVM_INSTALL_DIR=$PWD/libclang - -On Windows: - - 7z x .../libclang-release_39-windows-vs2015_64.7z - SET LLVM_INSTALL_DIR=%CD%\libclang - -#### Build Instructions - -You might consider using a virtual environment as described at -[getting started](https://wiki.qt.io/PySide2_GettingStarted). -You should be able to build: - - cd pyside-setup - python setup.py install +shiboken2 is the generator used to build the bindings. -The setup script will try to find the location of the qmake tool of the Qt -version to be used and the cmake build tool in the path. Non-standard -locations can be specified by the *--qmake=path_to_qmake* or -*--cmake=path_to_cmake* command line options. +See README.pyside2.md and README.shiboken2.md for details. diff --git a/README.pyside2.md b/README.pyside2.md new file mode 100644 index 000000000..cde05940d --- /dev/null +++ b/README.pyside2.md @@ -0,0 +1,53 @@ +# PySide2 + +### Introduction + +PySide is the [Python Qt bindings project](http://wiki.qt.io/PySide2), providing +access to the complete Qt 5.x framework as well as to generator tools for rapidly +generating bindings for any C++ libraries. + +The PySide project is developed in the open, with all facilities you'd expect +from any modern OSS project such as all code in a git repository and an open +design process. We welcome any contribution conforming to the +[Qt Contribution Agreement](https://www.qt.io/contributionagreement/). + + +PySide 2 supports Qt5. For building, please read about +[getting started](https://wiki.qt.io/PySide2_GettingStarted). +Then download the sources by running + + git clone https://code.qt.io/pyside/pyside-setup + +### Building + +#### Dependencies + +PySide versions following 5.6 use a C++ parser based on +[Clang](http://clang.org/). The Clang library (C-bindings), version 3.9 or +higher is required for building. Prebuilt versions of it can be downloaded from +[download.qt.io](http://download.qt.io/development_releases/prebuilt/libclang/). + +After unpacking the archive, set the environment variable *LLVM_INSTALL_DIR* to +point to the folder containing the *include* and *lib* directories of Clang: + + 7z x .../libclang-release_39-linux-Rhel7.2-gcc5.3-x86_64.7z + export LLVM_INSTALL_DIR=$PWD/libclang + +On Windows: + + 7z x .../libclang-release_39-windows-vs2015_64.7z + SET LLVM_INSTALL_DIR=%CD%\libclang + +#### Build Instructions + +You might consider using a virtual environment as described at +[getting started](https://wiki.qt.io/PySide2_GettingStarted). +You should be able to build: + + cd pyside-setup + python setup.py install + +The setup script will try to find the location of the qmake tool of the Qt +version to be used and the cmake build tool in the path. Non-standard +locations can be specified by the *--qmake=path_to_qmake* or +*--cmake=path_to_cmake* command line options. diff --git a/README.shiboken2-generator.md b/README.shiboken2-generator.md new file mode 100644 index 000000000..c71c6a8de --- /dev/null +++ b/README.shiboken2-generator.md @@ -0,0 +1 @@ +# shiboken2-generator diff --git a/README.shiboken2.md b/README.shiboken2.md new file mode 100644 index 000000000..f98f63c57 --- /dev/null +++ b/README.shiboken2.md @@ -0,0 +1 @@ +# shiboken2 module diff --git a/build_scripts/build_scripts.pyqtc b/build_scripts/build_scripts.pyqtc index eb4b58a63..1fc1c9664 100644 --- a/build_scripts/build_scripts.pyqtc +++ b/build_scripts/build_scripts.pyqtc @@ -1,8 +1,10 @@ __init__.py +config.py main.py options.py platforms qtinfo.py +setup_runner.py utils.py wheel_override.py platforms/__init__.py @@ -11,3 +13,6 @@ platforms/macos.py platforms/unix.py platforms/windows_desktop.py ../setup.py +../coin_build_instructions.py +../coin_test_instructions.py + diff --git a/build_scripts/config.py b/build_scripts/config.py new file mode 100644 index 000000000..78d7d4040 --- /dev/null +++ b/build_scripts/config.py @@ -0,0 +1,393 @@ +############################################################################# +## +## Copyright (C) 2018 The Qt Company Ltd. +## Contact: https://www.qt.io/licensing/ +## +## This file is part of Qt for Python. +## +## $QT_BEGIN_LICENSE:LGPL$ +## Commercial License Usage +## Licensees holding valid commercial Qt licenses may use this file in +## accordance with the commercial license agreement provided with the +## Software or, alternatively, in accordance with the terms contained in +## a written agreement between you and The Qt Company. For licensing terms +## and conditions see https://www.qt.io/terms-conditions. For further +## information use the contact form at https://www.qt.io/contact-us. +## +## GNU Lesser General Public License Usage +## Alternatively, this file may be used under the terms of the GNU Lesser +## General Public License version 3 as published by the Free Software +## Foundation and appearing in the file LICENSE.LGPL3 included in the +## packaging of this file. Please review the following information to +## ensure the GNU Lesser General Public License version 3 requirements +## will be met: https://www.gnu.org/licenses/lgpl-3.0.html. +## +## GNU General Public License Usage +## Alternatively, this file may be used under the terms of the GNU +## General Public License version 2.0 or (at your option) the GNU General +## Public license version 3 or any later version approved by the KDE Free +## Qt Foundation. The licenses are as published by the Free Software +## Foundation and appearing in the file LICENSE.GPL2 and LICENSE.GPL3 +## included in the packaging of this file. Please review the following +## information to ensure the GNU General Public License requirements will +## be met: https://www.gnu.org/licenses/gpl-2.0.html and +## https://www.gnu.org/licenses/gpl-3.0.html. +## +## $QT_END_LICENSE$ +## +############################################################################# + +import sys, os +import distutils.log as log + + +class Config(object): + def __init__(self): + # Constants + self._build_type_all = "all" + self._invocation_type_top_level = "top-level" + self._invocation_type_internal = "internal" + + # The keyword arguments which will be given to setuptools.setup + self.setup_kwargs = {} + + # The setup.py invocation type. + # top-level + # internal + self.invocation_type = None + + # The type of the top-level build. + # all - build shiboken2 module, shiboken2-generator and PySide2 + # modules + # shiboken2 - build only shiboken2 module + # shiboken2-generator - build only the shiboken2-generator + # pyside2 - build only PySide2 modules + self.build_type = None + + # The internal build type, used for internal invocations of + # setup.py to build a specific module only. + self.internal_build_type = None + + # Options that can be given to --build-type and + # --internal-build-type + self.shiboken_module_option_name = "shiboken2" + self.shiboken_generator_option_name = "shiboken2-generator" + self.pyside_option_name = "pyside2" + + # Names to be passed to setuptools.setup() name key, + # so not package name, but rather project name as it appears + # in the wheel name and on PyPi. + self.shiboken_module_st_name = "shiboken2" + self.shiboken_generator_st_name = "shiboken2-generator" + self.pyside_st_name = "PySide2" + + # Used by check_allowed_python_version to validate the + # interpreter version. + self.python_version_classifiers = [ + 'Programming Language :: Python', + 'Programming Language :: Python :: 2', + 'Programming Language :: Python :: 2.7', + 'Programming Language :: Python :: 3', + 'Programming Language :: Python :: 3.3', + 'Programming Language :: Python :: 3.4', + 'Programming Language :: Python :: 3.5', + 'Programming Language :: Python :: 3.6', + 'Programming Language :: Python :: 3.7', + ] + + self.setup_script_dir = None + + def init_config(self, build_type=None, internal_build_type=None, + cmd_class_dict=None, package_version=None, + ext_modules=None, setup_script_dir=None): + """ + Sets up the global singleton config which is used in many parts + of the setup process. + """ + + # if --internal-build-type was passed, it means that this is a + # sub-invocation to build a specific package. + if internal_build_type: + self.set_is_internal_invocation() + self.set_internal_build_type(internal_build_type) + else: + self.set_is_top_level_invocation() + + # --build-type was specified explicitly, so set it. Otherwise + # default to all. + if build_type: + self.build_type = build_type + else: + self.build_type = self._build_type_all + + self.setup_script_dir = setup_script_dir + + setup_kwargs = {} + setup_kwargs['long_description'] = self.get_long_description() + setup_kwargs['long_description_content_type'] = 'text/markdown', + setup_kwargs['keywords'] = 'Qt' + setup_kwargs['author'] = 'Qt for Python Team' + setup_kwargs['author_email'] = 'pyside@qt-project.org' + setup_kwargs['url'] = 'https://www.pyside.org' + setup_kwargs['download_url'] = 'https://download.qt.io/official_releases/QtForPython' + setup_kwargs['license'] = 'LGPL' + setup_kwargs['zip_safe'] = False + setup_kwargs['cmdclass'] = cmd_class_dict + setup_kwargs['version'] = package_version + + # Setting these two keys is still a bit of a discussion point. + # In general not setting them will allow using "build" and + # "bdist_wheel" just fine. What they do, is they specify to the + # setuptools.command.build_py command that certain pure python + # modules (.py files) exist in the specified package location, + # and that they should be copied over to the setuptools build + # dir. + # But it doesn't really make sense for us, because we copy all + # the necessary files to the build dir via prepare_packages() + # function anyway. + # If we don't set them, the build_py sub-command will be + # skipped, but the build command will still be executed, which + # is where we run cmake / make. + # The only plausible usage of it, is if we will implement a + # correctly functioning setup.py develop command (or bdist_egg). + # But currently that doesn't seem to work. + setup_kwargs['packages'] = self.get_setup_tools_packages_for_current_build() + setup_kwargs['package_dir'] = self.get_package_name_to_dir_path_mapping() + + # Add a bogus extension module (will never be built here since + # we are overriding the build command to do it using cmake) so + # things like bdist_egg will know that there are extension + # modules and will name the dist with the full platform info. + setup_kwargs['ext_modules'] = ext_modules + + common_classifiers = [ + 'Development Status :: 5 - Production/Stable', + 'Environment :: Console', + 'Environment :: MacOS X', + 'Environment :: X11 Applications :: Qt', + 'Environment :: Win32 (MS Windows)', + 'Intended Audience :: Developers', + 'License :: OSI Approved :: GNU Library or Lesser General Public License (LGPL)', + 'Operating System :: MacOS :: MacOS X', + 'Operating System :: POSIX', + 'Operating System :: POSIX :: Linux', + 'Operating System :: Microsoft', + 'Operating System :: Microsoft :: Windows', + 'Programming Language :: C++'] + common_classifiers.extend(self.python_version_classifiers) + common_classifiers.extend([ + 'Topic :: Database', + 'Topic :: Software Development', + 'Topic :: Software Development :: Code Generators', + 'Topic :: Software Development :: Libraries :: Application Frameworks', + 'Topic :: Software Development :: User Interfaces', + 'Topic :: Software Development :: Widget Sets']) + setup_kwargs['classifiers'] = common_classifiers + + if self.internal_build_type == self.shiboken_module_option_name: + setup_kwargs['name'] = self.shiboken_module_st_name + setup_kwargs['description'] = "Python / C++ bindings helper module", + setup_kwargs['entry_points'] = {} + + elif self.internal_build_type == self.shiboken_generator_option_name: + setup_kwargs['name'] = self.shiboken_generator_st_name + setup_kwargs['description'] = "Python / C++ bindings generator", + setup_kwargs['install_requires'] = [self.shiboken_module_st_name] + setup_kwargs['entry_points'] = { + 'console_scripts': [ + 'shiboken2 = {}.scripts.shiboken_tool:main'.format(self.package_name()), + ] + } + + elif self.internal_build_type == self.pyside_option_name: + setup_kwargs['name'] = self.pyside_st_name + setup_kwargs['description'] = ("Python bindings for the Qt cross-platform application" + " and UI framework"), + setup_kwargs['install_requires'] = [self.shiboken_module_st_name] + setup_kwargs['entry_points'] = { + 'console_scripts': [ + 'pyside2-uic = {}.scripts.uic:main'.format(self.package_name()), + 'pyside2-rcc = {}.scripts.pyside_tool:main'.format(self.package_name()), + 'pyside2-lupdate = {}.scripts.pyside_tool:main'.format(self.package_name()), + ] + } + self.setup_kwargs = setup_kwargs + + def get_long_description(self): + readme_filename = 'README.md' + changes_filename = 'CHANGES.rst' + + if self.is_internal_shiboken_module_build(): + readme_filename = 'README.shiboken2.md' + elif self.is_internal_shiboken_generator_build(): + readme_filename = 'README.shiboken2-generator.md' + elif self.is_internal_pyside_build(): + readme_filename = 'README.pyside2.md' + + content = '' + changes = '' + try: + with open(os.path.join(self.setup_script_dir, readme_filename)) as f: + readme = f.read() + except Exception as e: + log.error("Couldn't read contents of {}.".format(readme_filename)) + raise + + # Don't include CHANGES.rst for now, because we have not decided + # how to handle change files yet. + include_changes = False + if include_changes: + try: + with open(os.path.join(self.setup_script_dir, changes_filename)) as f: + changes = f.read() + except Exception as e: + log.error("Couldn't read contents of {}".format(changes_filename)) + raise + content += readme + + if changes: + content += "\n\n" + changes + + return content + + def package_name(self): + """ + Returns package name as it appears in Python's site-packages + directory. + + Package names can only be delimited by underscores, and not by + dashes. + """ + if self.is_internal_shiboken_module_build(): + return "shiboken2" + elif self.is_internal_shiboken_generator_build(): + return "shiboken2_generator" + elif self.is_internal_pyside_build(): + return "PySide2" + else: + return None + + def get_setup_tools_packages_for_current_build(self): + """ + Returns a list of packages for setup tools to consider in the + build_py command, so that it can copy the pure python files. + Not really necessary because it's done in prepare_packages() + anyway. + + This is really just to satisfy some checks in setuptools + build_py command, and if we ever properly implement the develop + command. + """ + if self.internal_build_type == self.pyside_option_name: + return [ + config.package_name(), + 'pyside2uic', + 'pyside2uic.Compiler', + 'pyside2uic.port_v{}'.format(sys.version_info[0]) + ] + elif self.internal_build_type == self.shiboken_module_option_name: + return [self.package_name()] + else: + return [] + + def get_package_name_to_dir_path_mapping(self): + """ + Used in setuptools.setup 'package_dir' argument to specify where + the actual module packages are located. + + For example when building the shiboken module, setuptools will + expect to find the "shiboken2" module sources under + "sources/shiboken2/shibokenmodule". + + This is really just to satisfy some checks in setuptools + build_py command, and if we ever properly implement the develop + command. + """ + if self.is_internal_shiboken_module_build(): + return { + self.package_name(): "sources/shiboken2/shibokenmodule" + } + elif self.is_internal_shiboken_generator_build(): + # This is left empty on purpose, because the shiboken + # generator doesn't have a python module for now. + return {} + elif self.is_internal_pyside_build(): + return { + self.package_name(): "sources/pyside2/PySide2", + "pyside2uic": "sources/pyside2-tools/pyside2uic" + } + else: + return {} + + def get_buildable_extensions(self): + """ + Used by PysideBuild.run to build the CMake projects. + :return: A list of directory names under the sources directory. + """ + if self.is_internal_shiboken_module_build() or self.is_internal_shiboken_generator_build(): + return ['shiboken2'] + elif self.is_internal_pyside_build(): + return ['pyside2', 'pyside2-tools'] + return None + + def set_is_top_level_invocation(self): + self.invocation_type = self._invocation_type_top_level + + def set_is_internal_invocation(self): + self.invocation_type = self._invocation_type_internal + + def is_top_level_invocation(self): + return self.invocation_type == self._invocation_type_top_level + + def is_internal_invocation(self): + return self.invocation_type == self._invocation_type_internal + + def is_top_level_build_all(self): + return self.build_type == self._build_type_all + + def is_top_level_build_shiboken_module(self): + return self.build_type == self.shiboken_module_option_name + + def is_top_level_build_shiboken_generator(self): + return self.build_type == self.shiboken_generator_option_name + + def is_top_level_build_pyside(self): + return self.build_type == self.pyside_option_name + + def set_internal_build_type(self, internal_build_type): + self.internal_build_type = internal_build_type + + def is_internal_shiboken_module_build(self): + return self.internal_build_type == self.shiboken_module_option_name + + def is_internal_shiboken_generator_build(self): + return self.internal_build_type == self.shiboken_generator_option_name + + def is_internal_pyside_build(self): + return self.internal_build_type == self.pyside_option_name + + def is_internal_shiboken_generator_build_and_part_of_top_level_all(self): + """ + Used to skip certain build rules and output, when we know that + the CMake build of shiboken was already done as part of the + top-level "all" build when shiboken2-module was built. + """ + return self.is_internal_shiboken_generator_build() and self.is_top_level_build_all() + + def get_allowed_top_level_build_values(self): + return [ + self._build_type_all, + self.shiboken_module_option_name, + self.shiboken_generator_option_name, + self.pyside_option_name + ] + + def get_allowed_internal_build_values(self): + return [ + self.shiboken_module_option_name, + self.shiboken_generator_option_name, + self.pyside_option_name + ] + + +config = Config() diff --git a/build_scripts/main.py b/build_scripts/main.py index b64d6f1a9..50f751caa 100644 --- a/build_scripts/main.py +++ b/build_scripts/main.py @@ -42,6 +42,7 @@ from distutils.version import LooseVersion import os import time +from .config import config from .utils import memoize, get_python_dict from .options import * @@ -90,15 +91,11 @@ def get_setuptools_extension_modules(): extension_modules = [Extension(*extension_args, **extension_kwargs)] return extension_modules -# Buildable extensions. -contained_modules = ['shiboken2', 'pyside2', 'pyside2-tools'] # Git submodules: ["submodule_name", # "location_relative_to_sources_folder"] submodules = [["pyside2-tools"]] -pyside_package_dir_name = "pyside_package" - try: import setuptools except ImportError: @@ -108,13 +105,8 @@ except ImportError: import sys import platform import re -import fnmatch - -import difflib # for a close match of dirname and module -import functools -from distutils import log -from distutils.errors import DistutilsOptionError +import distutils.log as log from distutils.errors import DistutilsSetupError from distutils.sysconfig import get_config_var from distutils.sysconfig import get_python_lib @@ -123,7 +115,7 @@ from distutils.command.build import build as _build from distutils.command.build_ext import build_ext as _build_ext from distutils.util import get_platform -from setuptools import setup, Extension +from setuptools import Extension from setuptools.command.install import install as _install from setuptools.command.install_lib import install_lib as _install_lib from setuptools.command.bdist_egg import bdist_egg as _bdist_egg @@ -132,39 +124,40 @@ from setuptools.command.build_py import build_py as _build_py from .qtinfo import QtInfo from .utils import rmtree, detect_clang, copyfile, copydir, run_process_output, run_process -from .utils import update_env_path, init_msvc_env, filter_match, macos_fix_rpaths_for_library +from .utils import update_env_path, init_msvc_env, filter_match +from .utils import macos_fix_rpaths_for_library +from .utils import linux_fix_rpaths_for_library from .platforms.unix import prepare_packages_posix from .platforms.windows_desktop import prepare_packages_win32 from .wheel_override import wheel_module_exists, get_bdist_wheel_override from textwrap import dedent -# make sure that setup.py is run with an allowed python version + def check_allowed_python_version(): + """ + Make sure that setup.py is run with an allowed python version. + """ + import re - pattern = "'Programming Language :: Python :: (\d+)\.(\d+)'" + pattern = "Programming Language :: Python :: (\d+)\.(\d+)" supported = [] - with open(setup_py_path) as setup: - for line in setup.readlines(): - found = re.search(pattern, line) - if found: - major = int(found.group(1)) - minor = int(found.group(2)) - supported.append( (major, minor) ) + + for line in config.python_version_classifiers: + found = re.search(pattern, line) + if found: + major = int(found.group(1)) + minor = int(found.group(2)) + supported.append( (major, minor) ) this_py = sys.version_info[:2] if this_py not in supported: - print("only these python versions are supported:", supported) + print("Unsupported python version detected. Only these python versions are supported: {}" + .format(supported)) sys.exit(1) -check_allowed_python_version() qt_src_dir = '' -# This is used automatically by distutils.command.install object, to -# specify final installation location. -OPTION_FINAL_INSTALL_PREFIX = option_value("prefix") - - if OPTION_QT_VERSION is None: OPTION_QT_VERSION = "5" if OPTION_QMAKE is None: @@ -293,8 +286,7 @@ def get_qt_version(): qt_version = qtinfo.version if not qt_version: - log.error("Failed to query the Qt version with qmake {0}".format( - self.qtinfo.qmake_command)) + log.error("Failed to query the Qt version with qmake {0}".format(qtinfo.qmake_command)) sys.exit(1) if LooseVersion(qtinfo.version) < LooseVersion("5.7"): @@ -304,12 +296,13 @@ def get_qt_version(): return qt_version + def prepare_build(): if (os.path.isdir(".git") and not OPTION_IGNOREGIT and not OPTION_ONLYPACKAGE and not OPTION_REUSE_BUILD): prepare_sub_modules() - # Clean up temp and package folders - for n in [pyside_package_dir_name, "build"]: + # Clean up temp build folder. + for n in ["build"]: d = os.path.join(setup_script_dir, n) if os.path.isdir(d): print("Removing {}".format(d)) @@ -318,13 +311,7 @@ def prepare_build(): except Exception as e: print('***** problem removing "{}"'.format(d)) print('ignored error: {}'.format(e)) - # Prepare package folders - ppdn = pyside_package_dir_name - absolute_paths = [os.path.join(ppdn, "PySide2"), - os.path.join(ppdn, "pyside2uic")] - for pkg in absolute_paths: - pkg_dir = os.path.join(setup_script_dir, pkg) - os.makedirs(pkg_dir) + # locate Qt sources for the documentation if OPTION_QT_SRC is None: install_prefix = qtinfo.prefix_dir @@ -389,26 +376,15 @@ class PysideBuildExt(_build_ext): pass - -# pyside_build_py and pyside_install_lib are reimplemented to preserve -# symlinks when distutils / setuptools copy files to various -# directories through the different build stages. class PysideBuildPy(_build_py): def __init__(self, *args, **kwargs): _build_py.__init__(self, *args, **kwargs) - def build_package_data(self): - """Copies files from pyside_package into build/xxx directory""" - - for package, src_dir, build_dir, filenames in self.data_files: - for filename in filenames: - target = os.path.join(build_dir, filename) - self.mkpath(os.path.dirname(target)) - srcfile = os.path.abspath(os.path.join(src_dir, filename)) - # Using our own copyfile makes sure to preserve symlinks. - copyfile(srcfile, target) +# _install_lib is reimplemented to preserve +# symlinks when distutils / setuptools copy files to various +# directories from the setup tools build dir to the install dir. class PysideInstallLib(_install_lib): def __init__(self, *args, **kwargs): @@ -539,10 +515,12 @@ class PysideBuild(_build): py_prefix = get_config_var("prefix") if not py_prefix or not os.path.exists(py_prefix): py_prefix = sys.prefix + self.py_prefix = py_prefix if sys.platform == "win32": py_scripts_dir = os.path.join(py_prefix, "Scripts") else: py_scripts_dir = os.path.join(py_prefix, "bin") + self.py_scripts_dir = py_scripts_dir if py_libdir is None or not os.path.exists(py_libdir): if sys.platform == "win32": py_libdir = os.path.join(py_prefix, "libs") @@ -656,7 +634,7 @@ class PysideBuild(_build): qt_version = get_qt_version() # Update the PATH environment variable - additional_paths = [py_scripts_dir, qt_dir] + additional_paths = [self.py_scripts_dir, qt_dir] # Add Clang to path for Windows. # Revisit once Clang is bundled with Qt. @@ -685,19 +663,11 @@ class PysideBuild(_build): install_dir = os.path.join(script_dir, prefix() + "_install", "{}".format(build_name)) - # Try to ensure that tools built by this script (such as shiboken2) - # are found before any that may already be installed on the system. - update_env_path([os.path.join(install_dir, 'bin')]) - - # Tell cmake to look here for *.cmake files - os.environ['CMAKE_PREFIX_PATH'] = install_dir - self.make_path = make_path self.make_generator = make_generator self.debug = OPTION_DEBUG self.script_dir = script_dir - self.pyside_package_dir = os.path.join(self.script_dir, - pyside_package_dir_name) + self.st_build_dir = os.path.join(self.script_dir, self.build_lib) self.sources_dir = sources_dir self.build_dir = build_dir self.install_dir = install_dir @@ -709,14 +679,61 @@ class PysideBuild(_build): self.site_packages_dir = get_python_lib(1, 0, prefix=install_dir) self.build_tests = OPTION_BUILDTESTS - setuptools_install_prefix = get_python_lib(1) - if OPTION_FINAL_INSTALL_PREFIX: - setuptools_install_prefix = OPTION_FINAL_INSTALL_PREFIX - # Save the shiboken build dir path for clang deployment # purposes. self.shiboken_build_dir = os.path.join(self.build_dir, "shiboken2") + self.log_pre_build_info() + + # Prepare folders + if not os.path.exists(self.sources_dir): + log.info("Creating sources folder {}...".format(self.sources_dir)) + os.makedirs(self.sources_dir) + if not os.path.exists(self.build_dir): + log.info("Creating build folder {}...".format(self.build_dir)) + os.makedirs(self.build_dir) + if not os.path.exists(self.install_dir): + log.info("Creating install folder {}...".format(self.install_dir)) + os.makedirs(self.install_dir) + + if not (OPTION_ONLYPACKAGE + and not config.is_internal_shiboken_generator_build_and_part_of_top_level_all()): + # Build extensions + for ext in config.get_buildable_extensions(): + self.build_extension(ext) + + if OPTION_BUILDTESTS: + # we record the latest successful build and note the + # build directory for supporting the tests. + timestamp = time.strftime('%Y-%m-%d_%H%M%S') + build_history = os.path.join(setup_script_dir, 'build_history') + unique_dir = os.path.join(build_history, timestamp) + os.makedirs(unique_dir) + fpath = os.path.join(unique_dir, 'build_dir.txt') + with open(fpath, 'w') as f: + print(build_dir, file=f) + log.info("Created {}".format(build_history)) + + if not OPTION_SKIP_PACKAGING: + # Build patchelf if needed + self.build_patchelf() + + # Prepare packages + self.prepare_packages() + + # Build packages + _build.run(self) + else: + log.info("Skipped preparing and building packages.") + log.info('*** Build completed') + + def log_pre_build_info(self): + if config.is_internal_shiboken_generator_build_and_part_of_top_level_all(): + return + + setuptools_install_prefix = get_python_lib(1) + if OPTION_FINAL_INSTALL_PREFIX: + setuptools_install_prefix = OPTION_FINAL_INSTALL_PREFIX log.info("=" * 30) log.info("Package version: {}".format(get_package_version())) log.info("Build type: {}".format(self.build_type)) @@ -726,40 +743,34 @@ class PysideBuild(_build): log.info("Make generator: {}".format(self.make_generator)) log.info("Make jobs: {}".format(OPTION_JOBS)) log.info("-" * 3) - log.info("setup.py directory: {}".format(self.script_dir)) log.info("Build scripts directory: {}".format(build_scripts_dir)) log.info("Sources directory: {}".format(self.sources_dir)) - log.info(dedent(""" - Building PySide2 will create and touch directories + Building {st_package_name} will create and touch directories in the following order: make build directory (py*_build/*/*) -> make install directory (py*_install/*/*) -> - {} directory (pyside_package/*) -> setuptools build directory (build/*/*) -> setuptools install directory (usually path-installed-python/lib/python*/site-packages/*) - """).format(pyside_package_dir_name)) - + """).format(st_package_name=config.package_name())) log.info("make build directory: {}".format(self.build_dir)) log.info("make install directory: {}".format(self.install_dir)) - log.info("{} directory: {}".format(pyside_package_dir_name, - self.pyside_package_dir)) - log.info("setuptools build directory: {}".format( - os.path.join(self.script_dir, "build"))) - log.info("setuptools install directory: {}".format( - setuptools_install_prefix)) - log.info("make-installed site-packages directory: {} \n" - " (only relevant for copying files from " - "'make install directory' to '{} directory'".format( - self.site_packages_dir, pyside_package_dir_name)) + log.info("setuptools build directory: {}".format(self.st_build_dir)) + log.info("setuptools install directory: {}".format(setuptools_install_prefix)) + log.info(dedent(""" + make-installed site-packages directory: {} + (only relevant for copying files from 'make install directory' + to 'setuptools build directory' + """).format( + self.site_packages_dir)) log.info("-" * 3) log.info("Python executable: {}".format(self.py_executable)) log.info("Python includes: {}".format(self.py_include_dir)) log.info("Python library: {}".format(self.py_library)) - log.info("Python prefix: {}".format(py_prefix)) - log.info("Python scripts: {}".format(py_scripts_dir)) + log.info("Python prefix: {}".format(self.py_prefix)) + log.info("Python scripts: {}".format(self.py_scripts_dir)) log.info("-" * 3) log.info("Qt qmake: {}".format(self.qtinfo.qmake_command)) log.info("Qt version: {}".format(self.qtinfo.version)) @@ -772,52 +783,11 @@ class PysideBuild(_build): if sys.platform == 'darwin': pyside_macos_deployment_target = ( PysideBuild.macos_pyside_min_deployment_target() - ) + ) log.info("MACOSX_DEPLOYMENT_TARGET set to: {}".format( pyside_macos_deployment_target)) log.info("=" * 30) - # Prepare folders - if not os.path.exists(self.sources_dir): - log.info("Creating sources folder {}...".format(self.sources_dir)) - os.makedirs(self.sources_dir) - if not os.path.exists(self.build_dir): - log.info("Creating build folder {}...".format(self.build_dir)) - os.makedirs(self.build_dir) - if not os.path.exists(self.install_dir): - log.info("Creating install folder {}...".format(self.install_dir)) - os.makedirs(self.install_dir) - - if not OPTION_ONLYPACKAGE: - # Build extensions - for ext in contained_modules: - self.build_extension(ext) - - if OPTION_BUILDTESTS: - # we record the latest successful build and note the - # build directory for supporting the tests. - timestamp = time.strftime('%Y-%m-%d_%H%M%S') - build_history = os.path.join(setup_script_dir, 'build_history') - unique_dir = os.path.join(build_history, timestamp) - os.makedirs(unique_dir) - fpath = os.path.join(unique_dir, 'build_dir.txt') - with open(fpath, 'w') as f: - print(build_dir, file=f) - log.info("Created {}".format(build_history)) - - if not OPTION_SKIP_PACKAGING: - # Build patchelf if needed - self.build_patchelf() - - # Prepare packages - self.prepare_packages() - - # Build packages - _build.run(self) - else: - log.info("Skipped preparing and building packages.") - log.info('*** Build completed') - @staticmethod def macos_qt_min_deployment_target(): target = qtinfo.macos_min_deployment_target @@ -953,6 +923,17 @@ class PysideBuild(_build): cmake_cmd.append("-DPYTHON_EXECUTABLE={}".format(self.py_executable)) cmake_cmd.append("-DPYTHON_INCLUDE_DIR={}".format(self.py_include_dir)) cmake_cmd.append("-DPYTHON_LIBRARY={}".format(self.py_library)) + + # If a custom shiboken cmake config directory path was provided, pass it to CMake. + if OPTION_SHIBOKEN_CONFIG_DIR and config.is_internal_pyside_build(): + if os.path.exists(OPTION_SHIBOKEN_CONFIG_DIR): + log.info("Using custom provided shiboken2 installation: {}" + .format(OPTION_SHIBOKEN_CONFIG_DIR)) + cmake_cmd.append("-DShiboken2_DIR={}".format(OPTION_SHIBOKEN_CONFIG_DIR)) + else: + log.info("Custom provided shiboken2 installation not found. Path given: {}" + .format(OPTION_SHIBOKEN_CONFIG_DIR)) + if OPTION_MODULE_SUBSET: module_sub_set = '' for m in OPTION_MODULE_SUBSET.split(','): @@ -1014,20 +995,20 @@ class PysideBuild(_build): cmake_cmd.append("-DPYSIDE_QT_CONF_PREFIX={}".format( pyside_qt_conf_prefix)) - # Pass package version to CMake, so this string can be - # embedded into _config.py file. - package_version = get_package_version() - cmake_cmd.append("-DPYSIDE_SETUP_PY_PACKAGE_VERSION={}".format( - package_version)) - - # In case if this is a snapshot build, also pass the - # timestamp as a separate value, because it the only - # version component that is actually generated by setup.py. - timestamp = '' - if OPTION_SNAPSHOT_BUILD: - timestamp = get_package_timestamp() - cmake_cmd.append("-DPYSIDE_SETUP_PY_PACKAGE_TIMESTAMP={}".format( - timestamp)) + # Pass package version to CMake, so this string can be + # embedded into _config.py file. + package_version = get_package_version() + cmake_cmd.append("-DPACKAGE_SETUP_PY_PACKAGE_VERSION={}".format( + package_version)) + + # In case if this is a snapshot build, also pass the + # timestamp as a separate value, because it is the only + # version component that is actually generated by setup.py. + timestamp = '' + if OPTION_SNAPSHOT_BUILD: + timestamp = get_package_timestamp() + cmake_cmd.append("-DPACKAGE_SETUP_PY_PACKAGE_TIMESTAMP={}".format( + timestamp)) if extension.lower() in ["shiboken2", "pyside2-tools"]: cmake_cmd.append("-DCMAKE_INSTALL_RPATH_USE_LINK_PATH=yes") @@ -1129,15 +1110,24 @@ class PysideBuild(_build): os.chdir(self.script_dir) def prepare_packages(self): + """ + This will copy all relevant files from the various locations in the "cmake install dir", + to the setup tools build dir (which is read from self.build_lib provided by distutils). + + After that setuptools.command.build_py is smart enough to copy everything + from the build dir to the install dir (the virtualenv site-packages for example). + """ try: - log.info("Preparing packages...") + log.info("\nPreparing setup tools build directory.\n") vars = { "site_packages_dir": self.site_packages_dir, "sources_dir": self.sources_dir, "install_dir": self.install_dir, "build_dir": self.build_dir, "script_dir": self.script_dir, - "pyside_package_dir": self.pyside_package_dir, + "st_build_dir": self.st_build_dir, + "cmake_package_name": config.package_name(), + "st_package_name": config.package_name(), "ssl_libs_dir": OPTION_OPENSSL, "py_version": self.py_version, "qt_version": self.qtinfo.version, @@ -1151,6 +1141,12 @@ class PysideBuild(_build): "qt_qml_dir": self.qtinfo.qml_dir, "target_arch": self.py_arch, } + + # Needed for correct file installation in generator build + # case. + if config.is_internal_shiboken_generator_build(): + vars['cmake_package_name'] = config.shiboken_module_option_name + os.chdir(self.script_dir) if sys.platform == "win32": @@ -1170,19 +1166,21 @@ class PysideBuild(_build): def get_built_pyside_config(self, vars): # Get config that contains list of built modules, and # SOVERSIONs of the built libraries. - pyside_package_dir = vars['pyside_package_dir'] - config_path = os.path.join(pyside_package_dir, "PySide2", "_config.py") - config = get_python_dict(config_path) - return config + st_build_dir = vars['st_build_dir'] + config_path = os.path.join(st_build_dir, config.package_name(), "_config.py") + temp_config = get_python_dict(config_path) + if 'built_modules' not in temp_config: + temp_config['built_modules'] = [] + return temp_config def is_webengine_built(self, built_modules): return ('WebEngineWidgets' in built_modules or 'WebEngineCore' in built_modules or 'WebEngine' in built_modules) - def prepare_standalone_clang(self, is_win = False): + def prepare_standalone_clang(self, is_win=False): """ - Copies the libclang library to the pyside package so that - shiboken executable works. + Copies the libclang library to the shiboken2-generator + package so that the shiboken executable works. """ log.info('Finding path to the libclang shared library.') cmake_cmd = [ @@ -1205,47 +1203,54 @@ class PysideBuild(_build): if not clang_lib_path: raise RuntimeError("Could not find the location of the libclang " - "library inside the CMake cache file.") + "library inside the CMake cache file.") - target_name = None if is_win: # clang_lib_path points to the static import library # (lib/libclang.lib), whereas we want to copy the shared # library (bin/libclang.dll). - clang_lib_path = re.sub(r'lib/libclang.lib$', 'bin/libclang.dll', - clang_lib_path) + clang_lib_path = re.sub(r'lib/libclang.lib$', + 'bin/libclang.dll', + clang_lib_path) else: - if sys.platform != 'darwin' and os.path.islink(clang_lib_path): - # On Linux, we get "libclang.so" from CMake which is - # a symlink: - # libclang.so -> libclang.so.6 -> libclang.so.6.0. - # shiboken2 links against libclang.so.6. So, we - # determine the target name by resolving just - # one symlink (note: os.path.realpath() resolves all). - target_name = os.readlink(clang_lib_path) - # We want to resolve any symlink on Linux and macOS, and - # copy the actual file. - clang_lib_path = os.path.realpath(clang_lib_path) - - if not target_name: - target_name = os.path.basename(clang_lib_path) - - # Path to directory containing libclang. - clang_lib_dir = os.path.dirname(clang_lib_path) - - # The destination will be the package folder near the other - # extension modules. - destination_dir = "{}/PySide2".format(os.path.join(self.script_dir, - 'pyside_package')) + # shiboken2 links against libclang.so.6 or a similarly + # named library. + # If the linked against library is a symlink, resolve + # the symlink once (but not all the way to the real + # file) on Linux and macOS, + # so that we get the path to the "SO version" symlink + # (the one used as the install name in the shared library + # dependency section). + # E.g. On Linux libclang.so -> libclang.so.6 -> + # libclang.so.6.0. + # "libclang.so.6" is the name we want for the copied file. + if os.path.islink(clang_lib_path): + link_target = os.readlink(clang_lib_path) + if os.path.isabs(link_target): + clang_lib_path = link_target + else: + # link_target is relative, transform to absolute. + clang_lib_path = os.path.join(os.path.dirname(clang_lib_path), link_target) + clang_lib_path = os.path.abspath(clang_lib_path) + + # The destination will be the shiboken package folder. + vars = {} + vars['st_build_dir'] = self.st_build_dir + vars['st_package_name'] = config.package_name() + destination_dir = "{st_build_dir}/{st_package_name}".format(**vars) + if os.path.exists(clang_lib_path): - log.info('Copying libclang shared library {} to the package folder as {}.'.format( - clang_lib_path, target_name)) basename = os.path.basename(clang_lib_path) - destination_path = os.path.join(destination_dir, target_name) + log.info('Copying libclang shared library {} to the package folder as {}.'.format( + clang_lib_path, basename)) + destination_path = os.path.join(destination_dir, basename) # Need to modify permissions in case file is not writable # (a reinstall would cause a permission denied error). - copyfile(clang_lib_path, destination_path, make_writable_by_owner=True) + copyfile(clang_lib_path, + destination_path, + force_copy_symlink=True, + make_writable_by_owner=True) else: raise RuntimeError("Error copying libclang library " "from {} to {}. ".format( @@ -1265,18 +1270,17 @@ class PysideBuild(_build): else: # Add rpath values pointing to $ORIGIN and the # installed qt lib directory. - local_rpath = '$ORIGIN/' - qt_lib_dir = self.qtinfo.libs_dir + final_rpath = self.qtinfo.libs_dir if OPTION_STANDALONE: - qt_lib_dir = "$ORIGIN/Qt/lib" - final_rpath = local_rpath + ':' + qt_lib_dir - cmd = [self._patchelf_path, '--set-rpath', final_rpath, srcpath] - if run_process(cmd) != 0: - raise RuntimeError("Error patching rpath in " + srcpath) + final_rpath = "$ORIGIN/Qt/lib" + override = OPTION_STANDALONE + linux_fix_rpaths_for_library(self._patchelf_path, srcpath, final_rpath, + override=override) elif sys.platform == 'darwin': pyside_libs = [lib for lib in os.listdir( package_path) if filter_match(lib, ["*.so", "*.dylib"])] + def rpath_cmd(srcpath): final_rpath = '' # Command line rpath option takes precedence over @@ -1308,15 +1312,6 @@ class PysideBuild(_build): "updated rpath (OS/X) in {}.".format(srcpath)) -try: - with open(os.path.join(setup_script_dir, 'README.rst')) as f: - README = f.read() - with open(os.path.join(setup_script_dir, 'CHANGES.rst')) as f: - CHANGES = f.read() -except IOError: - README = CHANGES = '' - - cmd_class_dict = { 'build': PysideBuild, 'build_py': PysideBuildPy, diff --git a/build_scripts/options.py b/build_scripts/options.py index fd8b0718e..daf3bb00e 100644 --- a/build_scripts/options.py +++ b/build_scripts/options.py @@ -38,16 +38,88 @@ ############################################################################# from __future__ import print_function +import sys +import os + + +class Options(object): + def __init__(self): + + # Dictionary containing values of all the possible options. + self.dict = {} + + def has_option(self, name): + """ Returns True if argument '--name' was passed on the command + line. """ + try: + sys.argv.remove("--{}".format(name)) + self.dict[name] = True + return True + except ValueError: + pass + return False + + def option_value(self, name, remove=True): + """ + Returns the value of a command line option or environment + variable. + + :param name: The name of the command line option or environment + variable. + + :param remove: Whether the option and its value should be + removed from sys.argv. Useful when there's a need to query for + the value and also pass it along to setuptools for example. + + :return: Either the option value or None. + """ + for index, option in enumerate(sys.argv): + if option == '--' + name: + if index + 1 >= len(sys.argv): + raise RuntimeError("The option {} requires a value".format(option)) + value = sys.argv[index + 1] + + if remove: + sys.argv[index:index + 2] = [] + + self.dict[name] = value + return value + + if option.startswith('--' + name + '='): + value = option[len(name) + 3:] + + if remove: + sys.argv[index:index + 1] = [] + + self.dict[name] = value + return value + + env_val = os.getenv(name.upper().replace('-', '_')) + self.dict[name] = env_val + return env_val + + +options = Options() + + +def has_option(name): + return options.has_option(name) + + +def option_value(*args,**kwargs): + return options.option_value(*args,**kwargs) -from .utils import has_option, option_value # Declare options +OPTION_BUILD_TYPE = option_value("build-type") +OPTION_INTERNAL_BUILD_TYPE = option_value("internal-build-type") OPTION_DEBUG = has_option("debug") OPTION_RELWITHDEBINFO = has_option('relwithdebinfo') OPTION_QMAKE = option_value("qmake") OPTION_QT_VERSION = option_value("qt") OPTION_CMAKE = option_value("cmake") OPTION_OPENSSL = option_value("openssl") +OPTION_SHIBOKEN_CONFIG_DIR = option_value("shiboken-config-dir") OPTION_ONLYPACKAGE = has_option("only-package") OPTION_STANDALONE = has_option("standalone") OPTION_MAKESPEC = option_value("make-spec") @@ -82,3 +154,7 @@ OPTION_SANITIZE_ADDRESS = has_option("sanitize-address") OPTION_SNAPSHOT_BUILD = has_option("snapshot-build") OPTION_LIMITED_API = option_value("limited-api") OPTION_PACKAGE_TIMESTAMP = option_value("package-timestamp") + +# This is used automatically by distutils.command.install object, to +# specify the final installation location. +OPTION_FINAL_INSTALL_PREFIX = option_value("prefix", remove=False) diff --git a/build_scripts/platforms/linux.py b/build_scripts/platforms/linux.py index 4c38fef6c..067179cdc 100644 --- a/build_scripts/platforms/linux.py +++ b/build_scripts/platforms/linux.py @@ -37,76 +37,100 @@ ## ############################################################################# -from ..options import * from ..utils import copydir, copyfile, copy_icu_libs, find_files_using_glob +from ..config import config -def prepare_standalone_package_linux(self, executables, vars): + +def prepare_standalone_package_linux(self, vars): built_modules = vars['built_modules'] - # <qt>/lib/* -> <setup>/PySide2/Qt/lib - destination_lib_dir = "{pyside_package_dir}/PySide2/Qt/lib" + constrain_modules = None + copy_plugins = True + copy_qml = True + copy_translations = True + copy_qt_conf = True + should_copy_icu_libs = True + + if config.is_internal_shiboken_generator_build(): + constrain_modules = ["Core", "Network", "Xml", "XmlPatterns"] + copy_plugins = False + copy_qml = False + copy_translations = False + copy_qt_conf = False + should_copy_icu_libs = False + + # <qt>/lib/* -> <setup>/{st_package_name}/Qt/lib + destination_lib_dir = "{st_build_dir}/{st_package_name}/Qt/lib" + + accepted_modules = ['libQt5*.so.?'] + if constrain_modules: + accepted_modules = ["libQt5" + module + "*.so.?" for module in constrain_modules] + accepted_modules.append("libicu*.so.??") + copydir("{qt_lib_dir}", destination_lib_dir, - filter=[ - "libQt5*.so.?", - "libicu*.so.??", - ], - recursive=False, vars=vars, force_copy_symlinks=True) - - # Check if ICU libraries were copied over to the destination - # Qt libdir. - resolved_destination_lib_dir = destination_lib_dir.format(**vars) - maybe_icu_libs = find_files_using_glob(resolved_destination_lib_dir, - "libicu*") - - # If no ICU libraries are present in the Qt libdir (like when - # Qt is built against system ICU, or in the Coin CI where ICU - # libs are in a different directory) try to find out / resolve - # which ICU libs are used by QtCore (if used at all) using a - # custom written ldd, and copy the ICU libs to the Pyside Qt - # dir if necessary. We choose the QtCore lib to inspect, by - # checking which QtCore library the shiboken2 executable uses. - if not maybe_icu_libs: - copy_icu_libs(self._patchelf_path, resolved_destination_lib_dir) + filter=accepted_modules, + recursive=False, vars=vars, force_copy_symlinks=True) + + if should_copy_icu_libs: + # Check if ICU libraries were copied over to the destination + # Qt libdir. + resolved_destination_lib_dir = destination_lib_dir.format(**vars) + maybe_icu_libs = find_files_using_glob(resolved_destination_lib_dir, + "libicu*") + + # If no ICU libraries are present in the Qt libdir (like when + # Qt is built against system ICU, or in the Coin CI where ICU + # libs are in a different directory) try to find out / resolve + # which ICU libs are used by QtCore (if used at all) using a + # custom written ldd, and copy the ICU libs to the Pyside Qt + # dir if necessary. We choose the QtCore lib to inspect, by + # checking which QtCore library the shiboken2 executable uses. + if not maybe_icu_libs: + copy_icu_libs(self._patchelf_path, resolved_destination_lib_dir) if self.is_webengine_built(built_modules): copydir("{qt_lib_execs_dir}", - "{pyside_package_dir}/PySide2/Qt/libexec", + "{st_build_dir}/{st_package_name}/Qt/libexec", filter=None, recursive=False, vars=vars) copydir("{qt_prefix_dir}/resources", - "{pyside_package_dir}/PySide2/Qt/resources", + "{st_build_dir}/{st_package_name}/Qt/resources", filter=None, recursive=False, vars=vars) - # <qt>/plugins/* -> <setup>/PySide2/Qt/plugins - copydir("{qt_plugins_dir}", - "{pyside_package_dir}/PySide2/Qt/plugins", - filter=["*.so"], - recursive=True, - vars=vars) - - # <qt>/qml/* -> <setup>/PySide2/Qt/qml - copydir("{qt_qml_dir}", - "{pyside_package_dir}/PySide2/Qt/qml", - filter=None, - force=False, - recursive=True, - ignore=["*.so.debug"], - vars=vars) - - # <qt>/translations/* -> <setup>/PySide2/Qt/translations - - copydir("{qt_translations_dir}", - "{pyside_package_dir}/PySide2/Qt/translations", - filter=["*.qm", "*.pak"], - force=False, - vars=vars) - - # Copy the qt.conf file to libexec. - copyfile( - "{build_dir}/pyside2/PySide2/qt.conf", - "{pyside_package_dir}/PySide2/Qt/libexec", - vars=vars) + if copy_plugins: + # <qt>/plugins/* -> <setup>/{st_package_name}/Qt/plugins + copydir("{qt_plugins_dir}", + "{st_build_dir}/{st_package_name}/Qt/plugins", + filter=["*.so"], + recursive=True, + vars=vars) + + if copy_qml: + # <qt>/qml/* -> <setup>/{st_package_name}/Qt/qml + copydir("{qt_qml_dir}", + "{st_build_dir}/{st_package_name}/Qt/qml", + filter=None, + force=False, + recursive=True, + ignore=["*.so.debug"], + vars=vars) + + if copy_translations: + # <qt>/translations/* -> + # <setup>/{st_package_name}/Qt/translations + copydir("{qt_translations_dir}", + "{st_build_dir}/{st_package_name}/Qt/translations", + filter=["*.qm", "*.pak"], + force=False, + vars=vars) + + if copy_qt_conf: + # Copy the qt.conf file to libexec. + copyfile( + "{build_dir}/pyside2/{st_package_name}/qt.conf", + "{st_build_dir}/{st_package_name}/Qt/libexec", + vars=vars) diff --git a/build_scripts/platforms/macos.py b/build_scripts/platforms/macos.py index 936f4ca90..49f02754d 100644 --- a/build_scripts/platforms/macos.py +++ b/build_scripts/platforms/macos.py @@ -37,12 +37,29 @@ ## ############################################################################# -import fnmatch, os +import fnmatch +import os from ..utils import copydir, copyfile, macos_fix_rpaths_for_library +from ..config import config -def prepare_standalone_package_macos(self, executables, vars): + +def prepare_standalone_package_macos(self, vars): built_modules = vars['built_modules'] + constrain_modules = None + copy_plugins = True + copy_qml = True + copy_translations = True + copy_qt_conf = True + + if config.is_internal_shiboken_generator_build(): + constrain_modules = ["Core", "Network", "Xml", "XmlPatterns"] + constrain_frameworks = ['Qt' + name + '.framework' for name in constrain_modules] + copy_plugins = False + copy_qml = False + copy_translations = False + copy_qt_conf = False + # Directory filter for skipping unnecessary files. def general_dir_filter(dir_name, parent_full_path, dir_full_path): if fnmatch.fnmatch(dir_name, "*.dSYM"): @@ -52,6 +69,7 @@ def prepare_standalone_package_macos(self, executables, vars): # Filter out debug plugins and qml plugins in the # debug_and_release config. no_copy_debug = True + def file_variant_filter(file_name, file_full_path): if self.qtinfo.build_type != 'debug_and_release': return True @@ -59,17 +77,16 @@ def prepare_standalone_package_macos(self, executables, vars): return False return True - # <qt>/lib/* -> <setup>/PySide2/Qt/lib + # <qt>/lib/* -> <setup>/{st_package_name}/Qt/lib if self.qt_is_framework_build(): - framework_built_modules = [ - 'Qt' + name + '.framework' for name in built_modules] - - def framework_dir_filter(dir_name, parent_full_path, - dir_full_path): + def framework_dir_filter(dir_name, parent_full_path, dir_full_path): if '.framework' in dir_name: - if dir_name.startswith('QtWebEngine') and not \ - self.is_webengine_built(built_modules): + if (dir_name.startswith('QtWebEngine') and + not self.is_webengine_built(built_modules)): + return False + if constrain_modules and dir_name not in constrain_frameworks: return False + if dir_name in ['Headers', 'fonts']: return False if dir_full_path.endswith('Versions/Current'): @@ -84,6 +101,7 @@ def prepare_standalone_package_macos(self, executables, vars): # Filter out debug frameworks in the # debug_and_release config. no_copy_debug = True + def framework_variant_filter(file_name, file_full_path): if self.qtinfo.build_type != 'debug_and_release': return True @@ -93,7 +111,7 @@ def prepare_standalone_package_macos(self, executables, vars): return False return True - copydir("{qt_lib_dir}", "{pyside_package_dir}/PySide2/Qt/lib", + copydir("{qt_lib_dir}", "{st_build_dir}/{st_package_name}/Qt/lib", recursive=True, vars=vars, ignore=["*.la", "*.a", "*.cmake", "*.pc", "*.prl"], dir_filter_function=framework_dir_filter, @@ -104,7 +122,7 @@ def prepare_standalone_package_macos(self, executables, vars): # from Versions/5/Helpers, thus adding two more levels of # directory hierarchy. if self.is_webengine_built(built_modules): - qt_lib_path = "{pyside_package_dir}/PySide2/Qt/lib".format( + qt_lib_path = "{st_build_dir}/{st_package_name}/Qt/lib".format( **vars) bundle = "QtWebEngineCore.framework/Helpers/" bundle += "QtWebEngineProcess.app" @@ -120,9 +138,11 @@ def prepare_standalone_package_macos(self, executables, vars): if 'WebKit' not in built_modules: ignored_modules.extend(['libQt5WebKit*.dylib']) accepted_modules = ['libQt5*.5.dylib'] + if constrain_modules: + accepted_modules = ["libQt5" + module + "*.5.dylib" for module in constrain_modules] copydir("{qt_lib_dir}", - "{pyside_package_dir}/PySide2/Qt/lib", + "{st_build_dir}/{st_package_name}/Qt/lib", filter=accepted_modules, ignore=ignored_modules, file_filter_function=file_variant_filter, @@ -130,53 +150,58 @@ def prepare_standalone_package_macos(self, executables, vars): if self.is_webengine_built(built_modules): copydir("{qt_lib_execs_dir}", - "{pyside_package_dir}/PySide2/Qt/libexec", + "{st_build_dir}/{st_package_name}/Qt/libexec", filter=None, recursive=False, vars=vars) copydir("{qt_prefix_dir}/resources", - "{pyside_package_dir}/PySide2/Qt/resources", + "{st_build_dir}/{st_package_name}/Qt/resources", filter=None, recursive=False, vars=vars) # Fix rpath for WebEngine process executable. - pyside_package_dir = vars['pyside_package_dir'] - qt_libexec_path = "{}/PySide2/Qt/libexec".format(pyside_package_dir) + qt_libexec_path = "{st_build_dir}/{st_package_name}/Qt/libexec".format(**vars) binary = "QtWebEngineProcess" final_path = os.path.join(qt_libexec_path, binary) rpath = "@loader_path/../lib" macos_fix_rpaths_for_library(final_path, rpath) - # Copy the qt.conf file to libexec. - copyfile( - "{build_dir}/pyside2/PySide2/qt.conf", - "{pyside_package_dir}/PySide2/Qt/libexec", - vars=vars) + if copy_qt_conf: + # Copy the qt.conf file to libexec. + copyfile( + "{build_dir}/pyside2/{st_package_name}/qt.conf", + "{st_build_dir}/{st_package_name}/Qt/libexec", + vars=vars) + + if copy_plugins: + # <qt>/plugins/* -> <setup>/{st_package_name}/Qt/plugins + copydir("{qt_plugins_dir}", + "{st_build_dir}/{st_package_name}/Qt/plugins", + filter=["*.dylib"], + recursive=True, + dir_filter_function=general_dir_filter, + file_filter_function=file_variant_filter, + vars=vars) + - # <qt>/plugins/* -> <setup>/PySide2/Qt/plugins - copydir("{qt_plugins_dir}", - "{pyside_package_dir}/PySide2/Qt/plugins", - filter=["*.dylib"], - recursive=True, - dir_filter_function=general_dir_filter, - file_filter_function=file_variant_filter, - vars=vars) - - # <qt>/qml/* -> <setup>/PySide2/Qt/qml - copydir("{qt_qml_dir}", - "{pyside_package_dir}/PySide2/Qt/qml", - filter=None, - recursive=True, - force=False, - dir_filter_function=general_dir_filter, - file_filter_function=file_variant_filter, - vars=vars) - - # <qt>/translations/* -> <setup>/PySide2/Qt/translations - copydir("{qt_translations_dir}", - "{pyside_package_dir}/PySide2/Qt/translations", - filter=["*.qm", "*.pak"], - force=False, - vars=vars) + if copy_qml: + # <qt>/qml/* -> <setup>/{st_package_name}/Qt/qml + copydir("{qt_qml_dir}", + "{st_build_dir}/{st_package_name}/Qt/qml", + filter=None, + recursive=True, + force=False, + dir_filter_function=general_dir_filter, + file_filter_function=file_variant_filter, + vars=vars) + + if copy_translations: + # <qt>/translations/* -> + # <setup>/{st_package_name}/Qt/translations + copydir("{qt_translations_dir}", + "{st_build_dir}/{st_package_name}/Qt/translations", + filter=["*.qm", "*.pak"], + force=False, + vars=vars) diff --git a/build_scripts/platforms/unix.py b/build_scripts/platforms/unix.py index e7fa92390..7dce11612 100644 --- a/build_scripts/platforms/unix.py +++ b/build_scripts/platforms/unix.py @@ -37,73 +37,31 @@ ## ############################################################################# -import os, re, sys +import os +import sys from .linux import prepare_standalone_package_linux from .macos import prepare_standalone_package_macos + +from ..config import config from ..options import * from ..utils import copydir, copyfile, rmtree, makefile from ..utils import regenerate_qt_resources + def prepare_packages_posix(self, vars): executables = [] - # <build>/shiboken2/doc/html/* -> - # <setup>/PySide2/docs/shiboken2 - copydir( - "{build_dir}/shiboken2/doc/html", - "{pyside_package_dir}/PySide2/docs/shiboken2", - force=False, vars=vars) - # <install>/lib/site-packages/PySide2/* -> <setup>/PySide2 - copydir( - "{site_packages_dir}/PySide2", - "{pyside_package_dir}/PySide2", - vars=vars) - # <install>/lib/site-packages/shiboken2.so -> - # <setup>/PySide2/shiboken2.so - shiboken_module_name = 'shiboken2.so' - shiboken_src_path = "{site_packages_dir}".format(**vars) - maybe_shiboken_names = [f for f in os.listdir(shiboken_src_path) - if re.match(r'shiboken.*\.so', f)] - if maybe_shiboken_names: - shiboken_module_name = maybe_shiboken_names[0] - vars.update({'shiboken_module_name': shiboken_module_name}) - copyfile( - "{site_packages_dir}/{shiboken_module_name}", - "{pyside_package_dir}/PySide2/{shiboken_module_name}", - vars=vars) - # <install>/lib/site-packages/pyside2uic/* -> - # <setup>/pyside2uic + + # <install>/lib/site-packages/{st_package_name}/* -> + # <setup>/{st_package_name} + # This copies the module .so/.dylib files and various .py files + # (__init__, config, git version, etc.) copydir( - "{site_packages_dir}/pyside2uic", - "{pyside_package_dir}/pyside2uic", - force=False, vars=vars) - if sys.version_info[0] > 2: - rmtree("{pyside_package_dir}/pyside2uic/port_v2".format(**vars)) - else: - rmtree("{pyside_package_dir}/pyside2uic/port_v3".format(**vars)) - # <install>/bin/pyside2-uic -> PySide2/scripts/uic.py - makefile( - "{pyside_package_dir}/PySide2/scripts/__init__.py", + "{site_packages_dir}/{st_package_name}", + "{st_build_dir}/{st_package_name}", vars=vars) - copyfile( - "{install_dir}/bin/pyside2-uic", - "{pyside_package_dir}/PySide2/scripts/uic.py", - force=False, vars=vars) - copyfile( - "{install_dir}/bin/pyside_tool.py", - "{pyside_package_dir}/PySide2/scripts/pyside_tool.py", - force=False, vars=vars) - # <install>/bin/* -> PySide2/ - executables.extend(copydir( - "{install_dir}/bin/", - "{pyside_package_dir}/PySide2", - filter=[ - "pyside2-lupdate", - "pyside2-rcc", - "shiboken2", - ], - recursive=False, vars=vars)) - # <install>/lib/lib* -> PySide2/ - config = self.get_built_pyside_config(vars) + + generated_config = self.get_built_pyside_config(vars) + def adjusted_lib_name(name, version): postfix = '' if sys.platform.startswith('linux'): @@ -111,61 +69,144 @@ def prepare_packages_posix(self, vars): elif sys.platform == 'darwin': postfix = '.' + version + '.dylib' return name + postfix - copydir( - "{install_dir}/lib/", - "{pyside_package_dir}/PySide2", - filter=[ - adjusted_lib_name("libpyside*", - config['pyside_library_soversion']), - adjusted_lib_name("libshiboken*", - config['shiboken_library_soversion']), - ], - recursive=False, vars=vars, force_copy_symlinks=True) - # <install>/share/PySide2/typesystems/* -> - # <setup>/PySide2/typesystems - copydir( - "{install_dir}/share/PySide2/typesystems", - "{pyside_package_dir}/PySide2/typesystems", - vars=vars) - # <install>/include/* -> <setup>/PySide2/include - copydir( - "{install_dir}/include", - "{pyside_package_dir}/PySide2/include", - vars=vars) - # <source>/pyside2/PySide2/support/* -> - # <setup>/PySide2/support/* - copydir( - "{build_dir}/pyside2/PySide2/support", - "{pyside_package_dir}/PySide2/support", - vars=vars) - if not OPTION_NOEXAMPLES: - # examples/* -> <setup>/PySide2/examples - copydir(os.path.join(self.script_dir, "examples"), - "{pyside_package_dir}/PySide2/examples", - force=False, vars=vars) - # Re-generate examples Qt resource files for Python 3 - # compatibility - if sys.version_info[0] == 3: - examples_path = "{pyside_package_dir}/PySide2/examples".format( - **vars) - pyside_rcc_path = "{install_dir}/bin/pyside2-rcc".format( - **vars) - pyside_rcc_options = '-py3' - regenerate_qt_resources(examples_path, pyside_rcc_path, - pyside_rcc_options) + + if config.is_internal_shiboken_module_build(): + # <build>/shiboken2/doc/html/* -> + # <setup>/{st_package_name}/docs/shiboken2 + copydir( + "{build_dir}/shiboken2/doc/html", + "{st_build_dir}/{st_package_name}/docs/shiboken2", + force=False, vars=vars) + + # <install>/lib/lib* -> {st_package_name}/ + copydir( + "{install_dir}/lib/", + "{st_build_dir}/{st_package_name}", + filter=[ + adjusted_lib_name("libshiboken*", + generated_config['shiboken_library_soversion']), + ], + recursive=False, vars=vars, force_copy_symlinks=True) + + if config.is_internal_shiboken_generator_build(): + # <install>/bin/* -> {st_package_name}/ + executables.extend(copydir( + "{install_dir}/bin/", + "{st_build_dir}/{st_package_name}", + filter=[ + "shiboken2", + ], + recursive=False, vars=vars)) + + # Used to create scripts directory. + makefile( + "{st_build_dir}/{st_package_name}/scripts/shiboken_tool.py", + vars=vars) + + # For setting up setuptools entry points. + copyfile( + "{install_dir}/bin/shiboken_tool.py", + "{st_build_dir}/{st_package_name}/scripts/shiboken_tool.py", + force=False, vars=vars) + + if config.is_internal_shiboken_generator_build() or config.is_internal_pyside_build(): + # <install>/include/* -> <setup>/{st_package_name}/include + copydir( + "{install_dir}/include/{cmake_package_name}", + "{st_build_dir}/{st_package_name}/include", + vars=vars) + + if config.is_internal_pyside_build(): + # <install>/lib/site-packages/pyside2uic/* -> + # <setup>/pyside2uic + copydir( + "{site_packages_dir}/pyside2uic", + "{st_build_dir}/pyside2uic", + force=False, vars=vars) + if sys.version_info[0] > 2: + rmtree("{st_build_dir}/pyside2uic/port_v2".format(**vars)) + else: + rmtree("{st_build_dir}/pyside2uic/port_v3".format(**vars)) + + # <install>/bin/pyside2-uic -> {st_package_name}/scripts/uic.py + makefile( + "{st_build_dir}/{st_package_name}/scripts/__init__.py", + vars=vars) + copyfile( + "{install_dir}/bin/pyside2-uic", + "{st_build_dir}/{st_package_name}/scripts/uic.py", + force=False, vars=vars) + + # For setting up setuptools entry points + copyfile( + "{install_dir}/bin/pyside_tool.py", + "{st_build_dir}/{st_package_name}/scripts/pyside_tool.py", + force=False, vars=vars) + + # <install>/bin/* -> {st_package_name}/ + executables.extend(copydir( + "{install_dir}/bin/", + "{st_build_dir}/{st_package_name}", + filter=[ + "pyside2-lupdate", + "pyside2-rcc", + ], + recursive=False, vars=vars)) + + # <install>/lib/lib* -> {st_package_name}/ + copydir( + "{install_dir}/lib/", + "{st_build_dir}/{st_package_name}", + filter=[ + adjusted_lib_name("libpyside*", + generated_config['pyside_library_soversion']), + ], + recursive=False, vars=vars, force_copy_symlinks=True) + + # <install>/share/{st_package_name}/typesystems/* -> + # <setup>/{st_package_name}/typesystems + copydir( + "{install_dir}/share/{st_package_name}/typesystems", + "{st_build_dir}/{st_package_name}/typesystems", + vars=vars) + + # <source>/pyside2/{st_package_name}/support/* -> + # <setup>/{st_package_name}/support/* + copydir( + "{build_dir}/pyside2/{st_package_name}/support", + "{st_build_dir}/{st_package_name}/support", + vars=vars) + + if not OPTION_NOEXAMPLES: + # examples/* -> <setup>/{st_package_name}/examples + copydir(os.path.join(self.script_dir, "examples"), + "{st_build_dir}/{st_package_name}/examples", + force=False, vars=vars) + # Re-generate examples Qt resource files for Python 3 + # compatibility + if sys.version_info[0] == 3: + examples_path = "{st_build_dir}/{st_package_name}/examples".format( + **vars) + pyside_rcc_path = "{install_dir}/bin/pyside2-rcc".format( + **vars) + pyside_rcc_options = '-py3' + regenerate_qt_resources(examples_path, pyside_rcc_path, + pyside_rcc_options) + # Copy Qt libs to package if OPTION_STANDALONE: - vars['built_modules'] = config['built_modules'] - if sys.platform == 'darwin': - prepare_standalone_package_macos(self, executables, vars) - else: - prepare_standalone_package_linux(self, executables, vars) + if config.is_internal_pyside_build() or config.is_internal_shiboken_generator_build(): + vars['built_modules'] = generated_config['built_modules'] + if sys.platform == 'darwin': + prepare_standalone_package_macos(self, vars) + else: + prepare_standalone_package_linux(self, vars) - # Copy over clang before rpath patching. - self.prepare_standalone_clang(is_win=False) + if config.is_internal_shiboken_generator_build(): + # Copy over clang before rpath patching. + self.prepare_standalone_clang(is_win=False) # Update rpath to $ORIGIN - if (sys.platform.startswith('linux') or - sys.platform.startswith('darwin')): - self.update_rpath("{pyside_package_dir}/PySide2".format(**vars), - executables) + if sys.platform.startswith('linux') or sys.platform.startswith('darwin'): + rpath_path = "{st_build_dir}/{st_package_name}".format(**vars) + self.update_rpath(rpath_path, executables) diff --git a/build_scripts/platforms/windows_desktop.py b/build_scripts/platforms/windows_desktop.py index aa9487ec6..6307238a1 100644 --- a/build_scripts/platforms/windows_desktop.py +++ b/build_scripts/platforms/windows_desktop.py @@ -38,153 +38,225 @@ ############################################################################# import functools -import os, re, sys +import os +import sys + +from ..config import config from ..options import * from ..utils import copydir, copyfile, rmtree, makefile from ..utils import regenerate_qt_resources, filter_match from ..utils import download_and_extract_7z + def prepare_packages_win32(self, vars): # For now, debug symbols will not be shipped into the package. copy_pdbs = False pdbs = [] if (self.debug or self.build_type == 'RelWithDebInfo') and copy_pdbs: pdbs = ['*.pdb'] - # <install>/lib/site-packages/PySide2/* -> <setup>/PySide2 + + # <install>/lib/site-packages/{st_package_name}/* -> + # <setup>/{st_package_name} + # This copies the module .pyd files and various .py files + # (__init__, config, git version, etc.) copydir( - "{site_packages_dir}/PySide2", - "{pyside_package_dir}/PySide2", + "{site_packages_dir}/{st_package_name}", + "{st_build_dir}/{st_package_name}", vars=vars) - built_modules = self.get_built_pyside_config(vars)['built_modules'] - # <build>/pyside2/PySide2/*.pdb -> <setup>/PySide2 - copydir( - "{build_dir}/pyside2/PySide2", - "{pyside_package_dir}/PySide2", - filter=pdbs, - recursive=False, vars=vars) + if config.is_internal_shiboken_module_build(): + # <build>/shiboken2/doc/html/* -> + # <setup>/{st_package_name}/docs/shiboken2 + copydir( + "{build_dir}/shiboken2/doc/html", + "{st_build_dir}/{st_package_name}/docs/shiboken2", + force=False, vars=vars) + + # <install>/bin/*.dll -> {st_package_name}/ + copydir( + "{install_dir}/bin/", + "{st_build_dir}/{st_package_name}", + filter=["shiboken*.dll"], + recursive=False, vars=vars) - # <build>/shiboken2/doc/html/* -> - # <setup>/PySide2/docs/shiboken2 - copydir( - "{build_dir}/shiboken2/doc/html", - "{pyside_package_dir}/PySide2/docs/shiboken2", - force=False, vars=vars) - - # <install>/lib/site-packages/shiboken2.pyd -> - # <setup>/PySide2/shiboken2.pyd - shiboken_module_name = 'shiboken2.pyd' - shiboken_src_path = "{site_packages_dir}".format(**vars) - maybe_shiboken_names = [f for f in os.listdir(shiboken_src_path) - if re.match(r'shiboken.*\.pyd', f)] - if maybe_shiboken_names: - shiboken_module_name = maybe_shiboken_names[0] - vars.update({'shiboken_module_name': shiboken_module_name}) - copyfile( - "{site_packages_dir}/{shiboken_module_name}", - "{pyside_package_dir}/PySide2/{shiboken_module_name}", - vars=vars) - # @TODO: Fix this .pdb file not to overwrite release - # {shibokengenerator}.pdb file. - # Task-number: PYSIDE-615 - copydir( - "{build_dir}/shiboken2/shibokenmodule", - "{pyside_package_dir}/PySide2", - filter=pdbs, - recursive=False, vars=vars) + # <install>/lib/*.lib -> {st_package_name}/ + copydir( + "{install_dir}/lib/", + "{st_build_dir}/{st_package_name}", + filter=["shiboken*.lib"], + recursive=False, vars=vars) - # <install>/lib/site-packages/pyside2uic/* -> - # <setup>/pyside2uic - copydir( - "{site_packages_dir}/pyside2uic", - "{pyside_package_dir}/pyside2uic", - force=False, vars=vars) - if sys.version_info[0] > 2: - rmtree("{pyside_package_dir}/pyside2uic/port_v2".format(**vars)) - else: - rmtree("{pyside_package_dir}/pyside2uic/port_v3".format(**vars)) + # @TODO: Fix this .pdb file not to overwrite release + # {shibokengenerator}.pdb file. + # Task-number: PYSIDE-615 + copydir( + "{build_dir}/shiboken2/shibokenmodule", + "{st_build_dir}/{st_package_name}", + filter=pdbs, + recursive=False, vars=vars) - # <install>/bin/pyside2-uic -> PySide2/scripts/uic.py - makefile( - "{pyside_package_dir}/PySide2/scripts/__init__.py", - vars=vars) - copyfile( - "{install_dir}/bin/pyside2-uic", - "{pyside_package_dir}/PySide2/scripts/uic.py", - force=False, vars=vars) - - # For setting up entry points - copyfile( - "{install_dir}/bin/pyside_tool.py", - "{pyside_package_dir}/PySide2/scripts/pyside_tool.py", - force=False, vars=vars) - - # <install>/bin/*.exe,*.dll,*.pdb -> PySide2/ - copydir( - "{install_dir}/bin/", - "{pyside_package_dir}/PySide2", - filter=["*.exe", "*.dll"], - recursive=False, vars=vars) - # @TODO: Fix this .pdb file not to overwrite release - # {shibokenmodule}.pdb file. - # Task-number: PYSIDE-615 - copydir( - "{build_dir}/shiboken2/generator", - "{pyside_package_dir}/PySide2", - filter=pdbs, - recursive=False, vars=vars) + # pdb files for libshiboken and libpyside + copydir( + "{build_dir}/shiboken2/libshiboken", + "{st_build_dir}/{st_package_name}", + filter=pdbs, + recursive=False, vars=vars) - # <install>/lib/*.lib -> PySide2/ - copydir( - "{install_dir}/lib/", - "{pyside_package_dir}/PySide2", - filter=["*.lib"], - recursive=False, vars=vars) + if config.is_internal_shiboken_generator_build(): + # <install>/bin/*.dll -> {st_package_name}/ + copydir( + "{install_dir}/bin/", + "{st_build_dir}/{st_package_name}", + filter=["shiboken*.exe"], + recursive=False, vars=vars) - # <install>/share/PySide2/typesystems/* -> - # <setup>/PySide2/typesystems - copydir( - "{install_dir}/share/PySide2/typesystems", - "{pyside_package_dir}/PySide2/typesystems", - vars=vars) + # Used to create scripts directory. + makefile( + "{st_build_dir}/{st_package_name}/scripts/shiboken_tool.py", + vars=vars) - # <install>/include/* -> <setup>/PySide2/include - copydir( - "{install_dir}/include", - "{pyside_package_dir}/PySide2/include", - vars=vars) + # For setting up setuptools entry points. + copyfile( + "{install_dir}/bin/shiboken_tool.py", + "{st_build_dir}/{st_package_name}/scripts/shiboken_tool.py", + force=False, vars=vars) + + # @TODO: Fix this .pdb file not to overwrite release + # {shibokenmodule}.pdb file. + # Task-number: PYSIDE-615 + copydir( + "{build_dir}/shiboken2/generator", + "{st_build_dir}/{st_package_name}", + filter=pdbs, + recursive=False, vars=vars) - # <source>/pyside2/PySide2/support/* -> - # <setup>/PySide2/support/* - copydir( - "{build_dir}/pyside2/PySide2/support", - "{pyside_package_dir}/PySide2/support", - vars=vars) + if config.is_internal_shiboken_generator_build() or config.is_internal_pyside_build(): + # <install>/include/* -> <setup>/{st_package_name}/include + copydir( + "{install_dir}/include/{cmake_package_name}", + "{st_build_dir}/{st_package_name}/include", + vars=vars) + + if config.is_internal_pyside_build(): + # <build>/pyside2/{st_package_name}/*.pdb -> + # <setup>/{st_package_name} + copydir( + "{build_dir}/pyside2/{st_package_name}", + "{st_build_dir}/{st_package_name}", + filter=pdbs, + recursive=False, vars=vars) + + # <install>/lib/site-packages/pyside2uic/* -> + # <setup>/pyside2uic + copydir( + "{site_packages_dir}/pyside2uic", + "{st_build_dir}/pyside2uic", + force=False, vars=vars) + if sys.version_info[0] > 2: + rmtree("{st_build_dir}/pyside2uic/port_v2".format(**vars)) + else: + rmtree("{st_build_dir}/pyside2uic/port_v3".format(**vars)) + + # <install>/bin/pyside2-uic -> {st_package_name}/scripts/uic.py + makefile( + "{st_build_dir}/{st_package_name}/scripts/__init__.py", + vars=vars) + copyfile( + "{install_dir}/bin/pyside2-uic", + "{st_build_dir}/{st_package_name}/scripts/uic.py", + force=False, vars=vars) + + # For setting up setuptools entry points + copyfile( + "{install_dir}/bin/pyside_tool.py", + "{st_build_dir}/{st_package_name}/scripts/pyside_tool.py", + force=False, vars=vars) + + # <install>/bin/*.exe,*.dll -> {st_package_name}/ + copydir( + "{install_dir}/bin/", + "{st_build_dir}/{st_package_name}", + filter=["pyside*.exe", "pyside*.dll"], + recursive=False, vars=vars) + + # <install>/lib/*.lib -> {st_package_name}/ + copydir( + "{install_dir}/lib/", + "{st_build_dir}/{st_package_name}", + filter=["pyside*.lib"], + recursive=False, vars=vars) + + # <install>/share/{st_package_name}/typesystems/* -> + # <setup>/{st_package_name}/typesystems + copydir( + "{install_dir}/share/{st_package_name}/typesystems", + "{st_build_dir}/{st_package_name}/typesystems", + vars=vars) + + # <source>/pyside2/{st_package_name}/support/* -> + # <setup>/{st_package_name}/support/* + copydir( + "{build_dir}/pyside2/{st_package_name}/support", + "{st_build_dir}/{st_package_name}/support", + vars=vars) + + copydir( + "{build_dir}/pyside2/libpyside", + "{st_build_dir}/{st_package_name}", + filter=pdbs, + recursive=False, vars=vars) - if not OPTION_NOEXAMPLES: - # examples/* -> <setup>/PySide2/examples - copydir(os.path.join(self.script_dir, "examples"), - "{pyside_package_dir}/PySide2/examples", + if not OPTION_NOEXAMPLES: + # examples/* -> <setup>/{st_package_name}/examples + copydir(os.path.join(self.script_dir, "examples"), + "{st_build_dir}/{st_package_name}/examples", + force=False, vars=vars) + # Re-generate examples Qt resource files for Python 3 + # compatibility + if sys.version_info[0] == 3: + examples_path = "{st_build_dir}/{st_package_name}/examples".format( + **vars) + pyside_rcc_path = "{install_dir}/bin/pyside2-rcc".format( + **vars) + pyside_rcc_options = '-py3' + regenerate_qt_resources(examples_path, pyside_rcc_path, + pyside_rcc_options) + + if vars['ssl_libs_dir']: + # <ssl_libs>/* -> <setup>/{st_package_name}/openssl + copydir("{ssl_libs_dir}", "{st_build_dir}/{st_package_name}/openssl", + filter=[ + "libeay32.dll", + "ssleay32.dll"], force=False, vars=vars) - # Re-generate examples Qt resource files for Python 3 - # compatibility - if sys.version_info[0] == 3: - examples_path = "{pyside_package_dir}/PySide2/examples".format( - **vars) - pyside_rcc_path = "{install_dir}/bin/pyside2-rcc".format( - **vars) - pyside_rcc_options = '-py3' - regenerate_qt_resources(examples_path, pyside_rcc_path, - pyside_rcc_options) - - # <ssl_libs>/* -> <setup>/PySide2/openssl - copydir("{ssl_libs_dir}", "{pyside_package_dir}/PySide2/openssl", - filter=[ - "libeay32.dll", - "ssleay32.dll"], - force=False, vars=vars) - - # <qt>/bin/*.dll and Qt *.exe -> <setup>/PySide2 + + if config.is_internal_pyside_build() or config.is_internal_shiboken_generator_build(): + copy_qt_artifacts(self, copy_pdbs, vars) + + +def copy_qt_artifacts(self, copy_pdbs, vars): + built_modules = self.get_built_pyside_config(vars)['built_modules'] + + constrain_modules = None + copy_plugins = True + copy_qml = True + copy_translations = True + copy_qt_conf = True + copy_qt_permanent_artifacts = True + copy_msvc_redist = False + copy_clang = False + + if config.is_internal_shiboken_generator_build(): + constrain_modules = ["Core", "Network", "Xml", "XmlPatterns"] + copy_plugins = False + copy_qml = False + copy_translations = False + copy_qt_conf = False + copy_qt_permanent_artifacts = False + copy_msvc_redist = True + copy_clang = True + + # <qt>/bin/*.dll and Qt *.exe -> <setup>/{st_package_name} qt_artifacts_permanent = [ "opengl*.dll", "d3d*.dll", @@ -195,6 +267,7 @@ def prepare_packages_win32(self, vars): "lconvert.exe", "qtdiag.exe" ] + # MSVC redistributable msvc_redist = [ "concrt140.dll", @@ -218,8 +291,14 @@ def prepare_packages_win32(self, vars): else: egl_suffix = '' qt_artifacts_egl = [a.format(egl_suffix) for a in qt_artifacts_egl] - qt_artifacts_permanent += qt_artifacts_egl - qt_artifacts_permanent += msvc_redist + + artifacts = [] + if copy_qt_permanent_artifacts: + artifacts += qt_artifacts_permanent + artifacts += qt_artifacts_egl + + if copy_msvc_redist: + artifacts += msvc_redist # Extract Qt dependency dll's when building on Qt CI # There is no proper CI env variable, so using agent launch params @@ -231,15 +310,22 @@ def prepare_packages_win32(self, vars): zip_file = "pyside_qt_deps_32.7z" download_and_extract_7z(redist_url + zip_file, "{qt_bin_dir}".format(**vars)) - copydir("{qt_bin_dir}", "{pyside_package_dir}/PySide2", - filter=qt_artifacts_permanent, - recursive=False, vars=vars) + if artifacts: + copydir("{qt_bin_dir}", + "{st_build_dir}/{st_package_name}", + filter=artifacts, recursive=False, vars=vars) - # <qt>/bin/*.dll and Qt *.pdbs -> <setup>/PySide2 part two + # <qt>/bin/*.dll and Qt *.pdbs -> <setup>/{st_package_name} part two # File filter to copy only debug or only release files. - qt_dll_patterns = ["Qt5*{}.dll", "lib*{}.dll"] - if copy_pdbs: - qt_dll_patterns += ["Qt5*{}.pdb", "lib*{}.pdb"] + if constrain_modules: + qt_dll_patterns = ["Qt5" + x + "{}.dll" for x in constrain_modules] + if copy_pdbs: + qt_dll_patterns += ["Qt5" + x + "{}.pdb" for x in constrain_modules] + else: + qt_dll_patterns = ["Qt5*{}.dll", "lib*{}.dll"] + if copy_pdbs: + qt_dll_patterns += ["Qt5*{}.pdb", "lib*{}.pdb"] + def qt_build_config_filter(patterns, file_name, file_full_path): release = [a.format('') for a in patterns] debug = [a.format('d') for a in patterns] @@ -289,56 +375,60 @@ def prepare_packages_win32(self, vars): return False qt_dll_filter = functools.partial(qt_build_config_filter, - qt_dll_patterns) - copydir("{qt_bin_dir}", "{pyside_package_dir}/PySide2", - file_filter_function=qt_dll_filter, - recursive=False, vars=vars) - - # <qt>/plugins/* -> <setup>/PySide2/plugins - plugin_dll_patterns = ["*{}.dll"] - pdb_pattern = "*{}.pdb" - if copy_pdbs: - plugin_dll_patterns += [pdb_pattern] - plugin_dll_filter = functools.partial(qt_build_config_filter, - plugin_dll_patterns) - copydir("{qt_plugins_dir}", "{pyside_package_dir}/PySide2/plugins", - file_filter_function=plugin_dll_filter, - vars=vars) + qt_dll_patterns) + copydir("{qt_bin_dir}", + "{st_build_dir}/{st_package_name}", + file_filter_function=qt_dll_filter, + recursive=False, vars=vars) - # <qt>/translations/* -> <setup>/PySide2/translations - copydir("{qt_translations_dir}", - "{pyside_package_dir}/PySide2/translations", - filter=["*.qm", "*.pak"], - force=False, - vars=vars) + if copy_plugins: + # <qt>/plugins/* -> <setup>/{st_package_name}/plugins + plugin_dll_patterns = ["*{}.dll"] + pdb_pattern = "*{}.pdb" + if copy_pdbs: + plugin_dll_patterns += [pdb_pattern] + plugin_dll_filter = functools.partial(qt_build_config_filter, + plugin_dll_patterns) + copydir("{qt_plugins_dir}", "{st_build_dir}/{st_package_name}/plugins", + file_filter_function=plugin_dll_filter, + vars=vars) - # <qt>/qml/* -> <setup>/PySide2/qml - qml_dll_patterns = ["*{}.dll"] - qml_ignore_patterns = qml_dll_patterns + [pdb_pattern] - qml_ignore = [a.format('') for a in qml_ignore_patterns] + if copy_translations: + # <qt>/translations/* -> <setup>/{st_package_name}/translations + copydir("{qt_translations_dir}", + "{st_build_dir}/{st_package_name}/translations", + filter=["*.qm", "*.pak"], + force=False, + vars=vars) - # Copy all files that are not dlls and pdbs (.qml, qmldir). - copydir("{qt_qml_dir}", "{pyside_package_dir}/PySide2/qml", - ignore=qml_ignore, - force=False, - recursive=True, - vars=vars) + if copy_qml: + # <qt>/qml/* -> <setup>/{st_package_name}/qml + qml_dll_patterns = ["*{}.dll"] + qml_ignore_patterns = qml_dll_patterns + [pdb_pattern] + qml_ignore = [a.format('') for a in qml_ignore_patterns] + + # Copy all files that are not dlls and pdbs (.qml, qmldir). + copydir("{qt_qml_dir}", "{st_build_dir}/{st_package_name}/qml", + ignore=qml_ignore, + force=False, + recursive=True, + vars=vars) - if copy_pdbs: - qml_dll_patterns += [pdb_pattern] - qml_dll_filter = functools.partial(qt_build_config_filter, - qml_dll_patterns) + if copy_pdbs: + qml_dll_patterns += [pdb_pattern] + qml_dll_filter = functools.partial(qt_build_config_filter, + qml_dll_patterns) - # Copy all dlls (and possibly pdbs). - copydir("{qt_qml_dir}", "{pyside_package_dir}/PySide2/qml", - file_filter_function=qml_dll_filter, - force=False, - recursive=True, - vars=vars) + # Copy all dlls (and possibly pdbs). + copydir("{qt_qml_dir}", "{st_build_dir}/{st_package_name}/qml", + file_filter_function=qml_dll_filter, + force=False, + recursive=True, + vars=vars) if self.is_webengine_built(built_modules): copydir("{qt_prefix_dir}/resources", - "{pyside_package_dir}/PySide2/resources", + "{st_build_dir}/{st_package_name}/resources", filter=None, recursive=False, vars=vars) @@ -346,26 +436,16 @@ def prepare_packages_win32(self, vars): filter = 'QtWebEngineProcess{}.exe'.format( 'd' if self.debug else '') copydir("{qt_bin_dir}", - "{pyside_package_dir}/PySide2", + "{st_build_dir}/{st_package_name}", filter=[filter], recursive=False, vars=vars) - # Copy the qt.conf file to prefix dir. - copyfile( - "{build_dir}/pyside2/PySide2/qt.conf", - "{pyside_package_dir}/PySide2", - vars=vars) - - self.prepare_standalone_clang(is_win=True) + if copy_qt_conf: + # Copy the qt.conf file to prefix dir. + copyfile( + "{build_dir}/pyside2/{st_package_name}/qt.conf", + "{st_build_dir}/{st_package_name}", + vars=vars) - # pdb files for libshiboken and libpyside - copydir( - "{build_dir}/shiboken2/libshiboken", - "{pyside_package_dir}/PySide2", - filter=pdbs, - recursive=False, vars=vars) - copydir( - "{build_dir}/pyside2/libpyside", - "{pyside_package_dir}/PySide2", - filter=pdbs, - recursive=False, vars=vars) + if copy_clang: + self.prepare_standalone_clang(is_win=True) diff --git a/build_scripts/setup_runner.py b/build_scripts/setup_runner.py new file mode 100644 index 000000000..a1526793e --- /dev/null +++ b/build_scripts/setup_runner.py @@ -0,0 +1,165 @@ +############################################################################# +## +## Copyright (C) 2018 The Qt Company Ltd. +## Contact: https://www.qt.io/licensing/ +## +## This file is part of Qt for Python. +## +## $QT_BEGIN_LICENSE:LGPL$ +## Commercial License Usage +## Licensees holding valid commercial Qt licenses may use this file in +## accordance with the commercial license agreement provided with the +## Software or, alternatively, in accordance with the terms contained in +## a written agreement between you and The Qt Company. For licensing terms +## and conditions see https://www.qt.io/terms-conditions. For further +## information use the contact form at https://www.qt.io/contact-us. +## +## GNU Lesser General Public License Usage +## Alternatively, this file may be used under the terms of the GNU Lesser +## General Public License version 3 as published by the Free Software +## Foundation and appearing in the file LICENSE.LGPL3 included in the +## packaging of this file. Please review the following information to +## ensure the GNU Lesser General Public License version 3 requirements +## will be met: https://www.gnu.org/licenses/lgpl-3.0.html. +## +## GNU General Public License Usage +## Alternatively, this file may be used under the terms of the GNU +## General Public License version 2.0 or (at your option) the GNU General +## Public license version 3 or any later version approved by the KDE Free +## Qt Foundation. The licenses are as published by the Free Software +## Foundation and appearing in the file LICENSE.GPL2 and LICENSE.GPL3 +## included in the packaging of this file. Please review the following +## information to ensure the GNU General Public License requirements will +## be met: https://www.gnu.org/licenses/gpl-2.0.html and +## https://www.gnu.org/licenses/gpl-3.0.html. +## +## $QT_END_LICENSE$ +## +############################################################################# + +import sys, os, textwrap + +from build_scripts.config import config +from build_scripts.main import get_package_version, get_setuptools_extension_modules +from build_scripts.main import cmd_class_dict +from build_scripts.options import OPTION_BUILD_TYPE, OPTION_INTERNAL_BUILD_TYPE +from build_scripts.utils import run_process + +from setuptools import setup + + +class SetupRunner(object): + def __init__(self, orig_argv): + self.invocations_list = [] + + # Keep the original args around in case we ever need to pass + # modified arguments to the sub invocations. + self.orig_argv = orig_argv + self.sub_argv = list(orig_argv) + + self.setup_script_dir = os.getcwd() + + @staticmethod + def cmd_line_argument_is_in_args(argument, args): + """ Check if command line argument was passed in args. """ + return any(arg for arg in list(args) if "--" + argument in arg) + + @staticmethod + def remove_cmd_line_argument_in_args(argument, args): + """ Remove command line argument from args. """ + return [arg for arg in list(args) if "--" + argument not in arg] + + @staticmethod + def construct_cmd_line_argument(name, value=None): + """ Constructs a command line argument given name and value. """ + if not value: + return "--{}".format(name) + return "--{}={}".format(name, value) + + @staticmethod + def construct_internal_build_type_cmd_line_argument(internal_build_type): + return SetupRunner.construct_cmd_line_argument("internal-build-type", internal_build_type) + + def add_setup_internal_invocation(self, build_type, reuse_build=False): + """ Enqueues a script sub-invocation to be executed later. """ + internal_build_type_arg = self.construct_internal_build_type_cmd_line_argument(build_type) + setup_cmd = [sys.executable] + self.sub_argv + [internal_build_type_arg] + + # Add --reuse-build option if requested and not already present. + if reuse_build and not self.cmd_line_argument_is_in_args("reuse-build", self.sub_argv): + setup_cmd.append(self.construct_cmd_line_argument("reuse-build")) + self.invocations_list.append(setup_cmd) + + def run_setup(self): + """ + Decide what kind of build is requested and then execute it. + In the top-level invocation case, the script + will spawn setup.py again (possibly multiple times). + In the internal invocation case, the script + will run setuptools.setup(). + """ + + # Prepare initial config. + config.init_config(build_type=OPTION_BUILD_TYPE, + internal_build_type=OPTION_INTERNAL_BUILD_TYPE, + cmd_class_dict=cmd_class_dict, + package_version=get_package_version(), + ext_modules=get_setuptools_extension_modules(), + setup_script_dir=self.setup_script_dir) + + # This is an internal invocation of setup.py, so start actual + # build. + if config.is_internal_invocation(): + if config.internal_build_type not in config.get_allowed_internal_build_values(): + raise RuntimeError("Invalid '{}' option given to --internal-build-type. " + .format(config.internal_build_type)) + self.run_setuptools_setup() + return + + # This is a top-level invocation of setup.py, so figure out what + # modules we will build and depending on that, call setup.py + # multiple times with different arguments. + if config.build_type not in config.get_allowed_top_level_build_values(): + raise RuntimeError("Invalid '{}' option given to --build-type. " + .format(config.build_type)) + + # Build everything: shiboken2, shiboken2-generator and PySide2. + if config.is_top_level_build_all(): + self.add_setup_internal_invocation(config.shiboken_module_option_name) + + # Reuse the shiboken build for the generator package instead + # of rebuilding it again. + self.add_setup_internal_invocation(config.shiboken_generator_option_name, + reuse_build=True) + + self.add_setup_internal_invocation(config.pyside_option_name) + + elif config.is_top_level_build_shiboken_module(): + self.add_setup_internal_invocation(config.shiboken_module_option_name) + + elif config.is_top_level_build_shiboken_generator(): + self.add_setup_internal_invocation(config.shiboken_generator_option_name) + + elif config.is_top_level_build_pyside(): + self.add_setup_internal_invocation(config.pyside_option_name) + + for cmd in self.invocations_list: + cmd_as_string = " ".join(cmd) + print("\nRunning process: {}\n".format(cmd_as_string)) + exit_code = run_process(cmd) + if exit_code != 0: + msg = textwrap.dedent(""" + setup.py invocation failed with exit code: {}.\n\n + setup.py invocation was: {} + """).format(exit_code, cmd_as_string) + raise RuntimeError(msg) + + @staticmethod + def run_setuptools_setup(): + """ + Runs setuptools.setup() once in a single setup.py + sub-invocation. + """ + + kwargs = config.setup_kwargs + setup(**kwargs) diff --git a/build_scripts/utils.py b/build_scripts/utils.py index 7160630d1..165366e26 100644 --- a/build_scripts/utils.py +++ b/build_scripts/utils.py @@ -39,17 +39,13 @@ import sys import os -import stat import re import stat import errno -import time import shutil import subprocess import fnmatch -import glob import itertools -import popenasync import glob # There is no urllib.request in Python2 @@ -58,11 +54,9 @@ try: except ImportError: import urllib -from distutils import log +import distutils.log as log from distutils.errors import DistutilsOptionError from distutils.errors import DistutilsSetupError -from distutils.spawn import spawn -from distutils.spawn import DistutilsExecError try: WindowsError @@ -70,32 +64,6 @@ except NameError: WindowsError = None -def has_option(name): - try: - sys.argv.remove("--{}".format(name)) - return True - except ValueError: - pass - return False - - -def option_value(name): - for index, option in enumerate(sys.argv): - if option == '--' + name: - if index+1 >= len(sys.argv): - raise DistutilsOptionError("The option {} requires a " - "value".format(option)) - value = sys.argv[index+1] - sys.argv[index:index+2] = [] - return value - if option.startswith('--' + name + '='): - value = option[len(name)+3:] - sys.argv[index:index+1] = [] - return value - env_val = os.getenv(name.upper().replace('-', '_')) - return env_val - - def filter_match(name, patterns): for pattern in patterns: if pattern is None: @@ -182,7 +150,6 @@ def find_vcdir(version): """ from distutils.msvc9compiler import VS_BASE from distutils.msvc9compiler import Reg - from distutils import log vsbase = VS_BASE % version try: productdir = Reg.get_value(r"{}\Setup\VC".format(vsbase), "productdir") @@ -416,13 +383,15 @@ def rmtree(dirname, ignore=False): os.chmod(path, stat.S_IRWXU| stat.S_IRWXG| stat.S_IRWXO) # 0777 func(path) else: - raise + raise IOError shutil.rmtree(dirname, ignore_errors=ignore, onerror=handle_remove_readonly) def run_process_output(args, initial_env=None): if initial_env is None: initial_env = os.environ - std_out = subprocess.Popen(args, env = initial_env, universal_newlines = 1, + std_out = subprocess.Popen(args, + env = initial_env, + universal_newlines = 1, stdout=subprocess.PIPE).stdout result = [] for raw_line in std_out.readlines(): @@ -431,53 +400,21 @@ def run_process_output(args, initial_env=None): return result def run_process(args, initial_env=None): - def _log(buffer, check_new_line=False): - ends_with_new_line = False - if buffer.endswith('\n'): - ends_with_new_line = True - if check_new_line and buffer.find('\n') == -1: - return buffer - lines = buffer.splitlines() - buffer = '' - if check_new_line and not ends_with_new_line: - buffer = lines[-1] - lines = lines[:-1] - for line in lines: - log.info(line.rstrip('\r')) - return buffer - _log("Running process in {0}: {1}".format(os.getcwd(), - " ".join([(" " in x and '"{0}"'.format(x) or x) for x in args]))) - - if sys.platform != "win32": - try: - spawn(args) - return 0 - except DistutilsExecError: - return -1 - - shell = False - if sys.platform == "win32": - shell = True + """ + Run process until completion and return the process exit code. + Prints both stdout and stderr to the console. + No output is captured. + """ + log.info("Running process in directory {0}: command {1}".format( + os.getcwd(), + " ".join([(" " in x and '"{0}"'.format(x) or x) for x in args])) + ) if initial_env is None: initial_env = os.environ - proc = popenasync.Popen(args, - stdin = subprocess.PIPE, - stdout = subprocess.PIPE, - stderr = subprocess.STDOUT, - universal_newlines = 1, - shell = shell, - env = initial_env) - - log_buffer = None; - while proc.poll() is None: - log_buffer = _log(proc.read_async(wait=0.1, e=0)) - if log_buffer: - _log(log_buffer) - - proc.wait() - return proc.returncode + exit_code = subprocess.call(args, stderr=subprocess.STDOUT, env=initial_env) + return exit_code def get_environment_from_batch_command(env_cmd, initial=None): @@ -665,6 +602,9 @@ def macos_get_rpaths(libpath): ctr += 3 return rpaths +def macos_add_rpath(rpath, library_path): + back_tick('install_name_tool -add_rpath {rpath} {library_path}'.format( + rpath=rpath, library_path=library_path)) def macos_fix_rpaths_for_library(library_path, qt_lib_dir): """ Adds required rpath load commands to given library. @@ -703,8 +643,7 @@ def macos_fix_rpaths_for_library(library_path, qt_lib_dir): break if needs_loader_path and "@loader_path" not in existing_rpath_commands: - back_tick('install_name_tool -add_rpath {rpath} {library_path}'.format( - rpath="@loader_path", library_path=library_path)) + macos_add_rpath("@loader_path", library_path) # If the library depends on a Qt library, add an rpath load comment # pointing to the Qt lib directory. @@ -738,8 +677,7 @@ def macos_add_qt_rpath(library_path, qt_lib_dir, break if needs_qt_rpath: - back_tick('install_name_tool -add_rpath {rpath} {library_path}'.format( - rpath=qt_lib_dir, library_path=library_path)) + macos_add_rpath(qt_lib_dir, library_path) # Find an executable specified by a glob pattern ('foo*') in the OS path def find_glob_in_path(pattern): @@ -996,6 +934,17 @@ def copy_icu_libs(patchelf, destination_lib_dir): new_rpaths_string = ":".join(rpaths) linux_set_rpaths(patchelf, qt_core_library_path, new_rpaths_string) + +def linux_run_read_elf(executable_path): + cmd = "readelf -d {}".format(executable_path) + (out, err, code) = back_tick(cmd, True) + if code != 0: + raise RuntimeError("Running `readelf -d {}` failed with error " + "output:\n {}. ".format(executable_path, err)) + lines = split_and_strip(out) + return lines + + def linux_set_rpaths(patchelf, executable_path, rpath_string): """ Patches the `executable_path` with a new rpath string. """ @@ -1005,18 +954,32 @@ def linux_set_rpaths(patchelf, executable_path, rpath_string): raise RuntimeError("Error patching rpath in {}".format( executable_path)) + +def linux_get_dependent_libraries(executable_path): + """ + Returns a list of libraries that executable_path depends on. + """ + + lines = linux_run_read_elf(executable_path) + pattern = re.compile(r"^.+?\(NEEDED\).+?\[(.+?)\]$") + + library_lines = [] + for line in lines: + match = pattern.search(line) + if match: + library_line = match.group(1) + library_lines.append(library_line) + + return library_lines + + def linux_get_rpaths(executable_path): """ Returns a list of run path values embedded in the executable or just an empty list. """ - cmd = "readelf -d {}".format(executable_path) - (out, err, code) = back_tick(cmd, True) - if code != 0: - raise RuntimeError("Running `readelf -d {}` failed with error " - "output:\n {}. ".format(executable_path, err)) - lines = split_and_strip(out) + lines = linux_run_read_elf(executable_path) pattern = re.compile(r"^.+?\(RUNPATH\).+?\[(.+?)\]$") rpath_line = None @@ -1033,6 +996,7 @@ def linux_get_rpaths(executable_path): return rpaths + def rpaths_has_origin(rpaths): """ Return True if the specified list of rpaths has an "$ORIGIN" value @@ -1048,6 +1012,39 @@ def rpaths_has_origin(rpaths): return True return False + +def linux_needs_qt_rpath(executable_path): + """ + Returns true if library_path depends on Qt libraries. + """ + + dependencies = linux_get_dependent_libraries(executable_path) + + # Check if any library dependencies are Qt libraries (hacky). + needs_qt_rpath = False + for dep in dependencies: + if 'Qt' in dep: + needs_qt_rpath = True + break + return needs_qt_rpath + + +def linux_fix_rpaths_for_library(patchelf, executable_path, qt_rpath, override=False): + """ + Adds or overrides required rpaths in given executable / library. + """ + rpaths = ['$ORIGIN/'] + existing_rpaths = [] + if not override: + existing_rpaths = linux_get_rpaths(executable_path) + rpaths.extend(existing_rpaths) + + if linux_needs_qt_rpath(executable_path) and qt_rpath not in existing_rpaths: + rpaths.append(qt_rpath) + + rpaths_string = ':'.join(rpaths) + linux_set_rpaths(patchelf, executable_path, rpaths_string) + def memoize(function): """ Decorator to wrap a function with a memoizing callable. @@ -1076,9 +1073,27 @@ def get_python_dict(python_script_path): "file: {}.".format(python_script_path)) raise -def install_pip_dependencies(env_pip, packages): +def install_pip_wheel_package(env_pip): + # Need to install an unreleased wheel version, due to a bug that + # will generate a wheel which will not be installable. + # See https://github.com/pypa/wheel/issues/263 + wheel_url = "git+https://github.com/pypa/wheel.git@fbf3e3ada64d36ca7bb9c1422f5a1ccdba7e4dcf" + install_pip_package_from_url_specifier(env_pip, wheel_url) + +def install_pip_package_from_url_specifier(env_pip, url, upgrade=True): + args = [env_pip, "install", url] + if upgrade: + args.append("--upgrade") + args.append(url) + run_instruction(args, "Failed to install {}".format(url)) + +def install_pip_dependencies(env_pip, packages, upgrade=True): for p in packages: - run_instruction([env_pip, "install", p], "Failed to install " + p) + args = [env_pip, "install"] + if upgrade: + args.append("--upgrade") + args.append(p) + run_instruction(args, "Failed to install " + p) def get_qtci_virtualEnv(python_ver, host, hostArch, targetArch): _pExe = "python" @@ -1105,9 +1120,11 @@ def get_qtci_virtualEnv(python_ver, host, hostArch, targetArch): _pExe = "python3" return(_pExe, _env, env_pip, env_python) -def run_instruction(instruction, error): +def run_instruction(instruction, error, initial_env=None): + if initial_env is None: + initial_env = os.environ print("Running Coin instruction: " + ' '.join(str(e) for e in instruction)) - result = subprocess.call(instruction) + result = subprocess.call(instruction, env=initial_env) if result != 0: print("ERROR : " + error) exit(result) diff --git a/coin_build_instructions.py b/coin_build_instructions.py index 9f9a74bc9..6ef17246a 100644 --- a/coin_build_instructions.py +++ b/coin_build_instructions.py @@ -36,9 +36,10 @@ ## $QT_END_LICENSE$ ## ############################################################################# -from build_scripts.utils import has_option -from build_scripts.utils import option_value +from build_scripts.options import has_option +from build_scripts.options import option_value from build_scripts.utils import install_pip_dependencies +from build_scripts.utils import install_pip_wheel_package from build_scripts.utils import get_qtci_virtualEnv from build_scripts.utils import run_instruction from build_scripts.utils import rmtree @@ -98,8 +99,11 @@ def call_setup(python_ver): _pExe, _env, env_pip, env_python = get_qtci_virtualEnv(python_ver, CI_HOST_OS, CI_HOST_ARCH, CI_TARGET_ARCH) rmtree(_env, True) run_instruction(["virtualenv", "-p", _pExe, _env], "Failed to create virtualenv") - install_pip_dependencies(env_pip, ["six", "wheel"]) - cmd = [env_python, "setup.py"] + + install_pip_dependencies(env_pip, ["six", "setuptools"]) + install_pip_wheel_package(env_pip) + + cmd = [env_python, "-u", "setup.py"] if CI_RELEASE_CONF: cmd += ["bdist_wheel", "--standalone"] else: @@ -121,7 +125,23 @@ def call_setup(python_ver): cmd += ["--package-timestamp=" + CI_INTEGRATION_ID] - run_instruction(cmd, "Failed to run setup.py") + env = os.environ + if CI_HOST_OS == "MacOS": + # On Python 3, setuptools.dist.handle_display_options does some + # weird sys.stdout.detach-ing if the stdout encoding is + # different from utf-8. This causes issues when running + # subprocess.call() because that access the original stdout + # object stored in sys.__stdout__ which was detached, and + # results in an exception being thrown. + # The Coin macOS locale by default is US-ASCII, and that + # triggers the above issue. Set the encoding to UTF-8 which + # makes sure to skip over the detach-ing code. + # Relevant links to the issue: + # https://bugs.python.org/issue15216 + # https://bitbucket.org/tarek/distribute/issues/334/fix-for-311-breaks-packages-that-use + # https://github.com/pypa/virtualenv/issues/359 + env['LC_CTYPE'] = 'UTF-8' + run_instruction(cmd, "Failed to run setup.py", initial_env=env) def run_build_instructions(): if not acceptCITestConfiguration(CI_HOST_OS, CI_HOST_OS_VER, CI_TARGET_ARCH, CI_COMPILER): diff --git a/coin_test_instructions.py b/coin_test_instructions.py index c2c5c07ea..5ecb4c17a 100644 --- a/coin_test_instructions.py +++ b/coin_test_instructions.py @@ -36,9 +36,10 @@ ## $QT_END_LICENSE$ ## ############################################################################# -from build_scripts.utils import has_option -from build_scripts.utils import option_value +from build_scripts.options import has_option +from build_scripts.options import option_value from build_scripts.utils import install_pip_dependencies +from build_scripts.utils import install_pip_wheel_package from build_scripts.utils import get_qtci_virtualEnv from build_scripts.utils import run_instruction from build_scripts.utils import rmtree @@ -66,7 +67,8 @@ def call_testrunner(python_ver, buildnro): _pExe, _env, env_pip, env_python = get_qtci_virtualEnv(python_ver, CI_HOST_OS, CI_HOST_ARCH, CI_TARGET_ARCH) rmtree(_env, True) run_instruction(["virtualenv", "-p", _pExe, _env], "Failed to create virtualenv") - install_pip_dependencies(env_pip, ["six", "wheel"]) + install_pip_dependencies(env_pip, ["six", "setuptools"]) + install_pip_wheel_package(env_pip) cmd = [env_python, "testrunner.py", "test", "--blacklist", "build_history/blacklist.txt", "--buildno=" + buildnro] diff --git a/examples/samplebinding/CMakeLists.txt b/examples/samplebinding/CMakeLists.txt index 03ab85754..f5212c449 100644 --- a/examples/samplebinding/CMakeLists.txt +++ b/examples/samplebinding/CMakeLists.txt @@ -64,14 +64,15 @@ macro(pyside2_config option output_var) endif() endmacro() -# Query for the shiboken path, Python path, include paths and linker flags. -pyside2_config(--pyside2 pyside2_path) -pyside2_config(--python-include python_include_dir) -pyside2_config(--shiboken-include shiboken_include_dir 1) -pyside2_config(--shiboken-shared-libraries-cmake shiboken_shared_libraries 0) -pyside2_config(--python-link-cmake python_linking_data 0) - -set(shiboken_path "${pyside2_path}/shiboken2${CMAKE_EXECUTABLE_SUFFIX}") +# Query for the shiboken generator path, Python path, include paths and linker flags. +pyside2_config(--shiboken2-module-path shiboken2_module_path) +pyside2_config(--shiboken2-generator-path shiboken2_generator_path) +pyside2_config(--python-include-path python_include_dir) +pyside2_config(--shiboken2-generator-include-path shiboken_include_dir 1) +pyside2_config(--shiboken2-module-shared-libraries-cmake shiboken_shared_libraries 0) +pyside2_config(--python-link-flags-cmake python_linking_data 0) + +set(shiboken_path "${shiboken2_generator_path}/shiboken2${CMAKE_EXECUTABLE_SUFFIX}") if(NOT EXISTS ${shiboken_path}) message(FATAL_ERROR "Shiboken executable not found at path: ${shiboken_path}") endif() @@ -87,7 +88,7 @@ endif() # Enable rpaths so that the built shared libraries find their dependencies. set(CMAKE_SKIP_BUILD_RPATH FALSE) set(CMAKE_BUILD_WITH_INSTALL_RPATH TRUE) -set(CMAKE_INSTALL_RPATH ${pyside2_path} ${CMAKE_CURRENT_SOURCE_DIR}) +set(CMAKE_INSTALL_RPATH ${shiboken2_module_path} ${CMAKE_CURRENT_SOURCE_DIR}) set(CMAKE_INSTALL_RPATH_USE_LINK_PATH TRUE) # ============================================================================================= # !!! End of dubious section. diff --git a/examples/scriptableapplication/CMakeLists.txt b/examples/scriptableapplication/CMakeLists.txt index 4119b6756..71e7869ab 100644 --- a/examples/scriptableapplication/CMakeLists.txt +++ b/examples/scriptableapplication/CMakeLists.txt @@ -35,14 +35,20 @@ macro(pyside2_config option output_var) endif() endmacro() -# Query for the PySide2 path, Python path, include paths and linker flags. -pyside2_config(--pyside2 PYSIDE2_PATH) -pyside2_config(--python-include PYTHON_INCLUDE_DIR) -pyside2_config(--pyside2-include PYSIDE2_INCLUDE_DIR 1) +# Query for the shiboken2-generator path, PySide2 path, Python path, include paths and linker flags. +pyside2_config(--shiboken2-module-path SHIBOKEN2_MODULE_PATH) +pyside2_config(--shiboken2-generator-path SHIBOKEN2_GENERATOR_PATH) +pyside2_config(--pyside2-path PYSIDE2_PATH) + +pyside2_config(--python-include-path PYTHON_INCLUDE_DIR) +pyside2_config(--shiboken2-generator-include-path SHIBOKEN2_GENERATOR_INCLUDE_DIR 1) +pyside2_config(--pyside2-include-path PYSIDE2_INCLUDE_DIR 1) + +pyside2_config(--python-link-flags-cmake PYTHON_LINKING_DATA 0) +pyside2_config(--shiboken2-module-shared-libraries-cmake SHIBOKEN2_MODULE_SHARED_LIBRARIES 0) pyside2_config(--pyside2-shared-libraries-cmake PYSIDE2_SHARED_LIBRARIES 0) -pyside2_config(--python-link-cmake PYTHON_LINKING_DATA 0) -set(SHIBOKEN_PATH "${PYSIDE2_PATH}/shiboken2${CMAKE_EXECUTABLE_SUFFIX}") +set(SHIBOKEN_PATH "${SHIBOKEN2_GENERATOR_PATH}/shiboken2${CMAKE_EXECUTABLE_SUFFIX}") if(NOT EXISTS ${SHIBOKEN_PATH}) message(FATAL_ERROR "Shiboken executable not found at path: ${SHIBOKEN_PATH}") @@ -122,7 +128,7 @@ endforeach() # Enable rpaths so that the example can be executed from the build dir. set(CMAKE_SKIP_BUILD_RPATH FALSE) set(CMAKE_BUILD_WITH_INSTALL_RPATH TRUE) -set(CMAKE_INSTALL_RPATH ${PYSIDE2_PATH}) +set(CMAKE_INSTALL_RPATH ${PYSIDE2_PATH} ${SHIBOKEN2_MODULE_PATH}) set(CMAKE_INSTALL_RPATH_USE_LINK_PATH TRUE) # ============================================================================================= # !!! End of dubious section. @@ -139,11 +145,13 @@ target_sources(${PROJECT_NAME} PUBLIC ${SOURCES}) # Apply relevant include and link flags. target_include_directories(${PROJECT_NAME} PRIVATE ${PYTHON_INCLUDE_DIR}) +target_include_directories(${PROJECT_NAME} PRIVATE ${SHIBOKEN2_GENERATOR_INCLUDE_DIR}) target_include_directories(${PROJECT_NAME} PRIVATE ${PYSIDE2_INCLUDE_DIR}) target_include_directories(${PROJECT_NAME} PRIVATE ${PYSIDE2_ADDITIONAL_INCLUDES}) target_include_directories(${PROJECT_NAME} PRIVATE ${CMAKE_SOURCE_DIR}) target_link_libraries(${PROJECT_NAME} PRIVATE Qt5::Widgets) +target_link_libraries(${PROJECT_NAME} PRIVATE ${SHIBOKEN2_MODULE_SHARED_LIBRARIES}) target_link_libraries(${PROJECT_NAME} PRIVATE ${PYSIDE2_SHARED_LIBRARIES}) # Find and link to the python library. @@ -176,7 +184,8 @@ if(WIN32) # Add custom target to hard link PySide2 shared libraries (just like in qmake example), so you # don't have to set PATH manually to point to the PySide2 package. - foreach(LIBRARY_PATH ${PYSIDE2_SHARED_LIBRARIES}) + set(shared_libraries ${SHIBOKEN2_MODULE_SHARED_LIBRARIES} ${PYSIDE2_SHARED_LIBRARIES}) + foreach(LIBRARY_PATH ${shared_libraries}) string(REGEX REPLACE ".lib$" ".dll" LIBRARY_PATH ${LIBRARY_PATH}) get_filename_component(BASE_NAME ${LIBRARY_PATH} NAME) file(TO_NATIVE_PATH ${LIBRARY_PATH} SOURCE_PATH) diff --git a/examples/scriptableapplication/pyside2.pri b/examples/scriptableapplication/pyside2.pri index 17be4392f..a2dc516cf 100644 --- a/examples/scriptableapplication/pyside2.pri +++ b/examples/scriptableapplication/pyside2.pri @@ -1,30 +1,46 @@ PYSIDE_CONFIG = $$PWD/../utils/pyside2_config.py -PYSIDE2 = $$system(python $$PYSIDE_CONFIG --pyside2) +SHIBOKEN2_GENERATOR = $$system(python $$PYSIDE_CONFIG --shiboken2-generator-path) +isEmpty(SHIBOKEN2_GENERATOR): error(Unable to locate the shiboken2-generator package location) + +SHIBOKEN2_MODULE = $$system(python $$PYSIDE_CONFIG --shiboken2-module-path) +isEmpty(SHIBOKEN2_MODULE): error(Unable to locate the shiboken2 package location) + +PYSIDE2 = $$system(python $$PYSIDE_CONFIG --pyside2-path) isEmpty(PYSIDE2): error(Unable to locate the PySide2 package location) -PYTHON_INCLUDE = $$system(python $$PYSIDE_CONFIG --python-include) +PYTHON_INCLUDE = $$system(python $$PYSIDE_CONFIG --python-include-path) isEmpty(PYTHON_INCLUDE): error(Unable to locate the Python include headers directory) -PYTHON_LFLAGS = $$system(python $$PYSIDE_CONFIG --python-link) +PYTHON_LFLAGS = $$system(python $$PYSIDE_CONFIG --python-link-flags-qmake) isEmpty(PYTHON_LFLAGS): error(Unable to locate the Python library for linking) -PYSIDE2_INCLUDE = $$system(python $$PYSIDE_CONFIG --pyside2-include) +SHIBOKEN2_INCLUDE = $$system(python $$PYSIDE_CONFIG --shiboken2-generator-include-path) +isEmpty(SHIBOKEN2_INCLUDE): error(Unable to locate the shiboken include headers directory) + +PYSIDE2_INCLUDE = $$system(python $$PYSIDE_CONFIG --pyside2-include-path) isEmpty(PYSIDE2_INCLUDE): error(Unable to locate the PySide2 include headers directory) -PYSIDE2_LFLAGS = $$system(python $$PYSIDE_CONFIG --pyside2-link) +SHIBOKEN2_LFLAGS = $$system(python $$PYSIDE_CONFIG --shiboken2-module-qmake-lflags) +isEmpty(SHIBOKEN2_LFLAGS): error(Unable to locate the shiboken libraries for linking) + +PYSIDE2_LFLAGS = $$system(python $$PYSIDE_CONFIG --pyside2-qmake-lflags) isEmpty(PYSIDE2_LFLAGS): error(Unable to locate the PySide2 libraries for linking) -PYSIDE2_SHARED_LIBRARIES = $$system(python $$PYSIDE_CONFIG --pyside2-shared-libraries) +SHIBOKEN2_SHARED_LIBRARIES = $$system(python $$PYSIDE_CONFIG --shiboken2-module-shared-libraries-qmake) +isEmpty(SHIBOKEN2_SHARED_LIBRARIES): error(Unable to locate the used shiboken2 module shared libraries) + +PYSIDE2_SHARED_LIBRARIES = $$system(python $$PYSIDE_CONFIG --pyside2-shared-libraries-qmake) isEmpty(PYSIDE2_SHARED_LIBRARIES): error(Unable to locate the used PySide2 shared libraries) -INCLUDEPATH += "$$PYTHON_INCLUDE" $$PYSIDE2_INCLUDE -LIBS += $$PYTHON_LFLAGS $$PYSIDE2_LFLAGS +INCLUDEPATH += "$$PYTHON_INCLUDE" $$PYSIDE2_INCLUDE $$SHIBOKEN2_INCLUDE +LIBS += $$PYTHON_LFLAGS $$PYSIDE2_LFLAGS $$SHIBOKEN2_LFLAGS !build_pass:message(INCLUDEPATH is $$INCLUDEPATH) !build_pass:message(LIBS are $$LIBS) !build_pass:message(Using $$PYSIDE2) !win32 { - QMAKE_RPATHDIR += $$PYSIDE2 + !build_pass:message(RPATH will include $$PYSIDE2 and $$SHIBOKEN2_MODULE) + QMAKE_RPATHDIR += $$PYSIDE2 $$SHIBOKEN2_MODULE } diff --git a/examples/scriptableapplication/scriptableapplication.pro b/examples/scriptableapplication/scriptableapplication.pro index 8a09b0abf..8ebab9476 100644 --- a/examples/scriptableapplication/scriptableapplication.pro +++ b/examples/scriptableapplication/scriptableapplication.pro @@ -23,7 +23,7 @@ SHIBOKEN_OPTIONS = --generator-set=shiboken --enable-parent-ctor-heuristic \ win32:SHIBOKEN_OPTIONS += --avoid-protected-hack # Prepare the shiboken tool -QT_TOOL.shiboken.binary = $$system_path($$PYSIDE2/shiboken2) +QT_TOOL.shiboken.binary = $$system_path($$SHIBOKEN2_GENERATOR/shiboken2) qtPrepareTool(SHIBOKEN, shiboken) # Shiboken run that adds the module wrapper to GENERATED_SOURCES @@ -61,7 +61,7 @@ win32 { hard_link_libraries.CONFIG = no_link target_predeps explicit_dependencies hard_link_libraries.output = $$out_dir/${QMAKE_FILE_BASE}${QMAKE_FILE_EXT} hard_link_libraries.commands = mklink /H $$shell_path($$out_dir/${QMAKE_FILE_BASE}${QMAKE_FILE_EXT}) $$shell_path(${QMAKE_FILE_IN}) - hard_link_libraries.input = PYSIDE2_SHARED_LIBRARIES + hard_link_libraries.input = PYSIDE2_SHARED_LIBRARIES SHIBOKEN2_SHARED_LIBRARIES } QMAKE_EXTRA_COMPILERS += shiboken module_wrapper_dummy_command diff --git a/examples/utils/pyside2_config.py b/examples/utils/pyside2_config.py index 21a1238b1..c62b38cad 100644 --- a/examples/utils/pyside2_config.py +++ b/examples/utils/pyside2_config.py @@ -41,25 +41,103 @@ import os, glob, re, sys from distutils import sysconfig +generic_error = (' Did you forget to activate your virtualenv? Or perhaps' + ' you forgot to build / install PySide2 into your currently active Python' + ' environment?') +pyside2_error = 'Unable to locate PySide2.' + generic_error +shiboken2_module_error = 'Unable to locate shiboken2-module.' + generic_error +shiboken2_generator_error = 'Unable to locate shiboken2-generator.' + generic_error +pyside2_libs_error = 'Unable to locate the PySide2 shared libraries.' + generic_error +python_link_error = 'Unable to locate the Python library for linking.' +python_include_error = 'Unable to locate the Python include headers directory.' + +options = [] + +# option, function, error, description +options.append(("--shiboken2-module-path", + lambda: find_shiboken2_module(), + shiboken2_module_error, + "Print shiboken2 module location")) +options.append(("--shiboken2-generator-path", + lambda: find_shiboken2_generator(), + shiboken2_generator_error, + "Print shiboken2 generator location")) +options.append(("--pyside2-path", lambda: find_pyside2(), pyside2_error, + "Print PySide2 location")) + +options.append(("--python-include-path", + lambda: get_python_include_path(), + python_include_error, + "Print Python include path")) +options.append(("--shiboken2-generator-include-path", + lambda: get_package_include_path(Package.shiboken2_generator), + pyside2_error, + "Print shiboken2 generator include paths")) +options.append(("--pyside2-include-path", + lambda: get_package_include_path(Package.pyside2), + pyside2_error, + "Print PySide2 include paths")) + +options.append(("--python-link-flags-qmake", lambda: python_link_flags_qmake(), python_link_error, + "Print python link flags for qmake")) +options.append(("--python-link-flags-cmake", lambda: python_link_flags_cmake(), python_link_error, + "Print python link flags for cmake")) + +options.append(("--shiboken2-module-qmake-lflags", + lambda: get_package_qmake_lflags(Package.shiboken2_module), pyside2_error, + "Print shiboken2 shared library link flags for qmake")) +options.append(("--pyside2-qmake-lflags", + lambda: get_package_qmake_lflags(Package.pyside2), pyside2_error, + "Print PySide2 shared library link flags for qmake")) + +options.append(("--shiboken2-module-shared-libraries-qmake", + lambda: get_shared_libraries_qmake(Package.shiboken2_module), pyside2_libs_error, + "Print paths of shiboken2 shared libraries (.so's, .dylib's, .dll's) for qmake")) +options.append(("--shiboken2-module-shared-libraries-cmake", + lambda: get_shared_libraries_cmake(Package.shiboken2_module), pyside2_libs_error, + "Print paths of shiboken2 shared libraries (.so's, .dylib's, .dll's) for cmake")) + +options.append(("--pyside2-shared-libraries-qmake", + lambda: get_shared_libraries_qmake(Package.pyside2), pyside2_libs_error, + "Print paths of PySide2 shared libraries (.so's, .dylib's, .dll's) for qmake")) +options.append(("--pyside2-shared-libraries-cmake", + lambda: get_shared_libraries_cmake(Package.pyside2), pyside2_libs_error, + "Print paths of PySide2 shared libraries (.so's, .dylib's, .dll's) for cmake")) + +options_usage = '' +for i, (flag, _, _, description) in enumerate(options): + options_usage += ' {:<45} {}'.format(flag, description) + if i < len(options) - 1: + options_usage += '\n' + usage = """ -Utility to determine include/link options of PySide2 and Python for qmake +Utility to determine include/link options of shiboken2/PySide2 and Python for qmake/CMake projects +that would like to embed or build custom shiboken2/PySide2 bindings. Usage: pyside2_config.py [option] Options: - --python-include Print Python include path - --python-link Print Python link flags - --pyside2 Print PySide2 location - --pyside2-include Print PySide2 include paths - --pyside2-link Print PySide2 link flags - --pyside2-shared-libraries Print paths of PySide2 shared libraries (.so's, .dylib's, .dll's) - -a Print all - --help/-h Print this help -""" - -def cleanPath(path): +{} + -a Print all options and their values + --help/-h Print this help +""".format(options_usage) + +option = sys.argv[1] if len(sys.argv) == 2 else '-a' +if option == '-h' or option == '--help': + print(usage) + sys.exit(0) + + +class Package(object): + shiboken2_module = 1 + shiboken2_generator = 2 + pyside2 = 3 + + +def clean_path(path): return path if sys.platform != 'win32' else path.replace('\\', '/') -def sharedLibrarySuffix(): + +def shared_library_suffix(): if sys.platform == 'win32': return 'lib' elif sys.platform == 'darwin': @@ -68,7 +146,8 @@ def sharedLibrarySuffix(): else: return 'so.*' -def importSuffixes(): + +def import_suffixes(): if (sys.version_info >= (3, 4)): import importlib return importlib.machinery.EXTENSION_SUFFIXES @@ -79,25 +158,29 @@ def importSuffixes(): result.append(t[0]) return result -def isDebug(): - debugSuffix = '_d.pyd' if sys.platform == 'win32' else '_d.so' - return any([s.endswith(debugSuffix) for s in importSuffixes()]) -def sharedLibraryGlobPattern(): - glob = '*.' + sharedLibrarySuffix() +def is_debug(): + debug_suffix = '_d.pyd' if sys.platform == 'win32' else '_d.so' + return any([s.endswith(debug_suffix) for s in import_suffixes()]) + + +def shared_library_glob_pattern(): + glob = '*.' + shared_library_suffix() return glob if sys.platform == 'win32' else 'lib' + glob -def filterPySide2SharedLibraries(list, only_shiboken=False): - def predicate(item): - basename = os.path.basename(item) - if 'shiboken' in basename or ('pyside2' in basename and not only_shiboken): + +def filter_shared_libraries(libs_list): + def predicate(lib_name): + basename = os.path.basename(lib_name) + if 'shiboken' in basename or 'pyside2' in basename: return True return False - result = [item for item in list if predicate(item)] + result = [lib for lib in libs_list if predicate(lib)] return result + # Return qmake link option for a library file name -def linkOption(lib): +def link_option(lib): # On Linux: # Since we cannot include symlinks with wheel packages # we are using an absolute path for the libpyside and libshiboken @@ -112,24 +195,50 @@ def linkOption(lib): link += os.path.splitext(baseName)[0] return link -# Locate PySide2 via package path -def findPySide2(): + +# Locate PySide2 via sys.path package path. +def find_pyside2(): + return find_package_path("PySide2") + + +def find_shiboken2_module(): + return find_package_path("shiboken2") + + +def find_shiboken2_generator(): + return find_package_path("shiboken2_generator") + + +def find_package(which_package): + if which_package == Package.shiboken2_module: + return find_shiboken2_module() + if which_package == Package.shiboken2_generator: + return find_shiboken2_generator() + if which_package == Package.pyside2: + return find_pyside2() + return None + + +def find_package_path(dir_name): for p in sys.path: if 'site-' in p: - pyside2 = os.path.join(p, 'PySide2') - if os.path.exists(pyside2): - return cleanPath(os.path.realpath(pyside2)) + package = os.path.join(p, dir_name) + if os.path.exists(package): + return clean_path(os.path.realpath(package)) return None + # Return version as "3.5" -def pythonVersion(): +def python_version(): return str(sys.version_info[0]) + '.' + str(sys.version_info[1]) -def pythonInclude(): + +def get_python_include_path(): return sysconfig.get_python_inc() -def pythonLinkQmake(): - flags = pythonLinkData() + +def python_link_flags_qmake(): + flags = python_link_data() if sys.platform == 'win32': libdir = flags['libdir'] # This will add the "~1" shortcut for directories that @@ -146,25 +255,27 @@ def pythonLinkQmake(): # Linux and anything else return '-L{} -l{}'.format(flags['libdir'], flags['lib']) -def pythonLinkCmake(): - flags = pythonLinkData() + +def python_link_flags_cmake(): + flags = python_link_data() libdir = flags['libdir'] lib = re.sub(r'.dll$', '.lib', flags['lib']) return '{};{}'.format(libdir, lib) -def pythonLinkData(): + +def python_link_data(): # @TODO Fix to work with static builds of Python libdir = sysconfig.get_config_var('LIBDIR') if libdir is None: libdir = os.path.abspath(os.path.join( sysconfig.get_config_var('LIBDEST'), "..", "libs")) - version = pythonVersion() + version = python_version() version_no_dots = version.replace('.', '') flags = {} flags['libdir'] = libdir if sys.platform == 'win32': - suffix = '_d' if isDebug() else '' + suffix = '_d' if is_debug() else '' flags['lib'] = 'python{}{}'.format(version_no_dots, suffix) elif sys.platform == 'darwin': @@ -173,42 +284,44 @@ def pythonLinkData(): # Linux and anything else else: if sys.version_info[0] < 3: - suffix = '_d' if isDebug() else '' + suffix = '_d' if is_debug() else '' flags['lib'] = 'python{}{}'.format(version, suffix) else: flags['lib'] = 'python{}{}'.format(version, sys.abiflags) return flags -def pyside2Include(only_shiboken=False): - pySide2 = findPySide2() - if pySide2 is None: + +def get_package_include_path(which_package): + package_path = find_package(which_package) + if package_path is None: return None - includes = "{0}/include/shiboken2".format(pySide2) - if not only_shiboken: - includes = includes + " {0}/include/PySide2".format(pySide2) + includes = "{0}/include".format(package_path) return includes -def pyside2Link(): - pySide2 = findPySide2() - if pySide2 is None: + +def get_package_qmake_lflags(which_package): + package_path = find_package(which_package) + if package_path is None: return None - link = "-L{}".format(pySide2) - glob_result = glob.glob(os.path.join(pySide2, sharedLibraryGlobPattern())) - for lib in filterPySide2SharedLibraries(glob_result): + + link = "-L{}".format(package_path) + glob_result = glob.glob(os.path.join(package_path, shared_library_glob_pattern())) + for lib in filter_shared_libraries(glob_result): link += ' ' - link += linkOption(lib) + link += link_option(lib) return link -def pyside2SharedLibrariesData(only_shiboken=False): - pySide2 = findPySide2() - if pySide2 is None: + +def get_shared_libraries_data(which_package): + package_path = find_package(which_package) + if package_path is None: return None - glob_result = glob.glob(os.path.join(pySide2, sharedLibraryGlobPattern())) - filtered_libs = filterPySide2SharedLibraries(glob_result, only_shiboken) + glob_result = glob.glob(os.path.join(package_path, shared_library_glob_pattern())) + filtered_libs = filter_shared_libraries(glob_result) libs = [] if sys.platform == 'win32': for lib in filtered_libs: @@ -218,8 +331,9 @@ def pyside2SharedLibrariesData(only_shiboken=False): libs.append(lib) return libs -def pyside2SharedLibraries(): - libs = pyside2SharedLibrariesData() + +def get_shared_libraries_qmake(which_package): + libs = get_shared_libraries_data(which_package) if libs is None: return None @@ -238,80 +352,21 @@ def pyside2SharedLibraries(): libs_string += lib + ' ' return libs_string -def pyside2SharedLibrariesCmake(only_shiboken=False): - libs = pyside2SharedLibrariesData(only_shiboken) + +def get_shared_libraries_cmake(which_package): + libs = get_shared_libraries_data(which_package) result = ';'.join(libs) return result -option = sys.argv[1] if len(sys.argv) == 2 else '-a' -if option == '-h' or option == '--help': - print(usage) - sys.exit(0) -generic_error = (' Did you forget to activate your virtualenv? Or perhaps' - ' you forgot to build / install PySide2 into your currently active Python' - ' environment?') -pyside2_error = 'Unable to locate PySide2.' + generic_error -pyside2_libs_error = 'Unable to locate the PySide2 shared libraries.' + generic_error -python_link_error = 'Unable to locate the Python library for linking.' +print_all = option == "-a" +for argument, handler, error, _ in options: + if option == argument or print_all: + handler_result = handler() + if handler_result is None: + sys.exit(error) -if option == '--pyside2' or option == '-a': - pySide2 = findPySide2() - if pySide2 is None: - sys.exit(pyside2_error) - print(pySide2) - -if option == '--pyside2-link' or option == '-a': - l = pyside2Link() - if l is None: - sys.exit(pyside2_error) - - print(l) - -if option == '--shiboken-include' or option == '-a': - i = pyside2Include(only_shiboken=True) - if i is None: - sys.exit(pyside2_error) - print(i) - -if option == '--pyside2-include' or option == '-a': - i = pyside2Include() - if i is None: - sys.exit(pyside2_error) - print(i) - -if option == '--python-include' or option == '-a': - i = pythonInclude() - if i is None: - sys.exit('Unable to locate the Python include headers directory.') - print(i) - -if option == '--python-link' or option == '-a': - l = pythonLinkQmake() - if l is None: - sys.exit(python_link_error) - print(l) - -if option == '--python-link-cmake' or option == '-a': - l = pythonLinkCmake() - if l is None: - sys.exit(python_link_error) - print(l) - -if option == '--pyside2-shared-libraries' or option == '-a': - l = pyside2SharedLibraries() - if l is None: - sys.exit(pyside2_libs_error) - print(l) - -if option == '--pyside2-shared-libraries-cmake' or option == '-a': - l = pyside2SharedLibrariesCmake() - if l is None: - sys.exit(pyside2_libs_error) - print(l) - -if option == '--shiboken-shared-libraries-cmake' or option == '-a': - l = pyside2SharedLibrariesCmake(only_shiboken=True) - if l is None: - sys.exit(pyside2_libs_error) - print(l) + line = handler_result + if print_all: + line = "{:<40}: ".format(argument) + line + print(line) diff --git a/popenasync.py b/popenasync.py deleted file mode 100644 index 77faf9e0c..000000000 --- a/popenasync.py +++ /dev/null @@ -1,360 +0,0 @@ -############################################################################# -## -## Copyright (C) 2017 The Qt Company Ltd. -## Contact: https://www.qt.io/licensing/ -## -## This file is part of Qt for Python. -## -## $QT_BEGIN_LICENSE:LGPL$ -## Commercial License Usage -## Licensees holding valid commercial Qt licenses may use this file in -## accordance with the commercial license agreement provided with the -## Software or, alternatively, in accordance with the terms contained in -## a written agreement between you and The Qt Company. For licensing terms -## and conditions see https://www.qt.io/terms-conditions. For further -## information use the contact form at https://www.qt.io/contact-us. -## -## GNU Lesser General Public License Usage -## Alternatively, this file may be used under the terms of the GNU Lesser -## General Public License version 3 as published by the Free Software -## Foundation and appearing in the file LICENSE.LGPL3 included in the -## packaging of this file. Please review the following information to -## ensure the GNU Lesser General Public License version 3 requirements -## will be met: https://www.gnu.org/licenses/lgpl-3.0.html. -## -## GNU General Public License Usage -## Alternatively, this file may be used under the terms of the GNU -## General Public License version 2.0 or (at your option) the GNU General -## Public license version 3 or any later version approved by the KDE Free -## Qt Foundation. The licenses are as published by the Free Software -## Foundation and appearing in the file LICENSE.GPL2 and LICENSE.GPL3 -## included in the packaging of this file. Please review the following -## information to ensure the GNU General Public License requirements will -## be met: https://www.gnu.org/licenses/gpl-2.0.html and -## https://www.gnu.org/licenses/gpl-3.0.html. -## -## $QT_END_LICENSE$ -## -############################################################################# - -################################################################################ -""" - -Modification of http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/440554 - -""" - -#################################### IMPORTS ################################### - -import os -import subprocess -import errno -import time -import sys -import unittest -import tempfile - -def geterror (): - return sys.exc_info()[1] - -if sys.version_info >= (3,): - null_byte = '\x00'.encode('ascii') -else: - null_byte = '\x00' - -mswindows = (sys.platform == "win32") - -if mswindows: - if sys.version_info >= (3,): - # Test date should be in ascii. - def encode(s): - return s.encode('ascii', 'ignore') - - def decode(b): - return b.decode('ascii', 'ignore') - else: - # Strings only; do nothing - def encode(s): - return s - - def decode(b): - return b - - try: - import ctypes - from ctypes.wintypes import DWORD - kernel32 = ctypes.windll.kernel32 - TerminateProcess = ctypes.windll.kernel32.TerminateProcess - def WriteFile(handle, data, ol = None): - c_written = DWORD() - success = ctypes.windll.kernel32.WriteFile(handle, - ctypes.create_string_buffer(encode(data)), len(data), - ctypes.byref(c_written), ol) - return ctypes.windll.kernel32.GetLastError(), c_written.value - def ReadFile(handle, desired_bytes, ol = None): - c_read = DWORD() - buffer = ctypes.create_string_buffer(desired_bytes+1) - success = ctypes.windll.kernel32.ReadFile(handle, buffer, - desired_bytes, ctypes.byref(c_read), ol) - buffer[c_read.value] = null_byte - return ctypes.windll.kernel32.GetLastError(), decode(buffer.value) - def PeekNamedPipe(handle, desired_bytes): - c_avail = DWORD() - c_message = DWORD() - if desired_bytes > 0: - c_read = DWORD() - buffer = ctypes.create_string_buffer(desired_bytes+1) - success = ctypes.windll.kernel32.PeekNamedPipe(handle, buffer, - desired_bytes, ctypes.byref(c_read), ctypes.byref(c_avail), - ctypes.byref(c_message)) - buffer[c_read.value] = null_byte - return decode(buffer.value), c_avail.value, c_message.value - else: - success = ctypes.windll.kernel32.PeekNamedPipe(handle, None, - desired_bytes, None, ctypes.byref(c_avail), - ctypes.byref(c_message)) - return "", c_avail.value, c_message.value - - except ImportError: - from win32file import ReadFile, WriteFile - from win32pipe import PeekNamedPipe - from win32api import TerminateProcess - import msvcrt - -else: - from signal import SIGINT, SIGTERM, SIGKILL - import select - import fcntl - -################################### CONSTANTS ################################## - -PIPE = subprocess.PIPE - -################################################################################ - -class Popen(subprocess.Popen): - def __init__(self, *args, **kwargs): - subprocess.Popen.__init__(self, *args, **kwargs) - - def recv(self, maxsize=None): - return self._recv('stdout', maxsize) - - def recv_err(self, maxsize=None): - return self._recv('stderr', maxsize) - - def send_recv(self, input='', maxsize=None): - return self.send(input), self.recv(maxsize), self.recv_err(maxsize) - - def read_async(self, wait=.1, e=1, tr=5, stderr=0): - if tr < 1: - tr = 1 - x = time.time()+ wait - y = [] - r = '' - pr = self.recv - if stderr: - pr = self.recv_err - while time.time() < x or r: - r = pr() - if r is None: - if e: - raise Exception("Other end disconnected!") - else: - break - elif r: - y.append(r) - else: - time.sleep(max((x-time.time())/tr, 0)) - return ''.join(y) - - def send_all(self, data): - while len(data): - sent = self.send(data) - if sent is None: - raise Exception("Other end disconnected!") - data = buffer(data, sent) - - def get_conn_maxsize(self, which, maxsize): - if maxsize is None: - maxsize = 1024 - elif maxsize < 1: - maxsize = 1 - return getattr(self, which), maxsize - - def _close(self, which): - conn = getattr(self, which) - flags = fcntl.fcntl(conn, fcntl.F_GETFL) - if not conn.closed: - fcntl.fcntl(conn, fcntl.F_SETFL, flags & ~os.O_NONBLOCK) - assert conn.read() == '' - getattr(self, which).close() - setattr(self, which, None) - - if mswindows: - def kill(self): - # Recipes - #http://me.in-berlin.de/doc/python/faq/windows.html#how-do-i-emulate-os-kill-in-windows - #http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/347462 - - """kill function for Win32""" - TerminateProcess(int(self._handle), 0) # returns None - - def send(self, input): - if not self.stdin: - return None - - try: - x = msvcrt.get_osfhandle(self.stdin.fileno()) - (errCode, written) = WriteFile(x, input) - except ValueError: - return self._close('stdin') - except (subprocess.pywintypes.error, Exception): - if geterror()[0] in (109, errno.ESHUTDOWN): - return self._close('stdin') - raise - - return written - - def _recv(self, which, maxsize): - conn, maxsize = self.get_conn_maxsize(which, maxsize) - if conn is None: - return None - - try: - x = msvcrt.get_osfhandle(conn.fileno()) - (read, nAvail, nMessage) = PeekNamedPipe(x, 0) - if maxsize < nAvail: - nAvail = maxsize - if nAvail > 0: - (errCode, read) = ReadFile(x, nAvail, None) - except ValueError: - return self._close(which) - except (subprocess.pywintypes.error, Exception): - if geterror()[0] in (109, errno.ESHUTDOWN): - return self._close(which) - raise - - if self.universal_newlines: - # Translate newlines. For Python 3.x assume read is text. - # If bytes then another solution is needed. - read = read.replace("\r\n", "\n").replace("\r", "\n") - return read - - else: - def kill(self): - for i, sig in enumerate([SIGTERM, SIGKILL] * 2): - if i % 2 == 0: os.kill(self.pid, sig) - time.sleep((i * (i % 2) / 5.0) + 0.01) - - killed_pid, stat = os.waitpid(self.pid, os.WNOHANG) - if killed_pid != 0: return - - def send(self, input): - if not self.stdin: - return None - - if not select.select([], [self.stdin], [], 0)[1]: - return 0 - - try: - written = os.write(self.stdin.fileno(), input) - except OSError: - if geterror()[0] == errno.EPIPE: #broken pipe - return self._close('stdin') - raise - - return written - - def _recv(self, which, maxsize): - conn, maxsize = self.get_conn_maxsize(which, maxsize) - if conn is None: - return None - - flags = fcntl.fcntl(conn, fcntl.F_GETFL) - if not conn.closed: - fcntl.fcntl(conn, fcntl.F_SETFL, flags| os.O_NONBLOCK) - - try: - if not select.select([conn], [], [], 0)[0]: - return '' - - try: - r = conn.read(maxsize) - except IOError as e: - if e.errno == errno.EAGAIN: - return '' - raise - if not r: - return self._close(which) - - if self.universal_newlines: - r = r.replace("\r\n", "\n").replace("\r", "\n") - return r - finally: - if not conn.closed: - fcntl.fcntl(conn, fcntl.F_SETFL, flags) - -################################################################################ - -def proc_in_time_or_kill(cmd, time_out, wd = None, env = None): - proc = Popen ( - cmd, cwd = wd, env = env, - stdin = subprocess.PIPE, stdout = subprocess.PIPE, - stderr = subprocess.STDOUT, universal_newlines = 1 - ) - - ret_code = None - response = [] - - t = time.time() - while ret_code is None and ((time.time() -t) < time_out): - ret_code = proc.poll() - response += [proc.read_async(wait=0.1, e=0)] - - if ret_code is None: - ret_code = '"Process timed out (time_out = {} secs) '.format(time_out) - try: - proc.kill() - ret_code += 'and was successfully terminated"' - except Exception: - ret_code += ("and termination failed " - "(exception: {})".format(geterror(),)) - - return ret_code, ''.join(response) - -################################################################################ - -class AsyncTest(unittest.TestCase): - def test_proc_in_time_or_kill(self): - ret_code, response = proc_in_time_or_kill( - [sys.executable, '-c', 'while 1: pass'], time_out = 1 - ) - - self.assert_( 'rocess timed out' in ret_code ) - self.assert_( 'successfully terminated' in ret_code ) - -################################################################################ - -def _example(): - if sys.platform == 'win32': - shell, commands, tail = ('cmd', ('echo "hello"', 'echo "HELLO WORLD"'), - '\r\n') - else: - shell, commands, tail = ('sh', ('ls', 'echo HELLO WORLD'), '\n') - - a = Popen(shell, stdin=PIPE, stdout=PIPE) - sys.stdout.write(a.read_async()) - sys.stdout.write(" ") - for cmd in commands: - a.send_all(cmd + tail) - sys.stdout.write(a.read_async()) - sys.stdout.write(" ") - a.send_all('exit' + tail) - print (a.read_async(e=0)) - a.wait() - -################################################################################ - -if __name__ == '__main__': - if 1: unittest.main() - else: _example() @@ -42,12 +42,18 @@ from __future__ import print_function """ This is a distutils setup-script for the Qt for Python project -To build PySide2 simply execute: +To build both shiboken2 and PySide2 simply execute: python setup.py build or python setup.py install to build and install into your current Python installation. +The same setup.py script is used to build all the components of the +project: + - shiboken2 (the supporting Python module) + - shiboken2-generator (the bindings generation executable) + - PySide2 + - pyside2-tools Optionally, one can specify the location of qmake and cmake if it is not on the current PATH with: @@ -56,6 +62,23 @@ and --cmake=/path/to/bin/cmake respectively. +By default, all of the above is built when no special options are +passed to the script. You can use the --build-type parameter to specify +which things should be built: + --build-type=shiboken2 - build / package only the python module + --build-type=shiboken2-generator - build / package the generator + executable + --build-type=pyside2 - build / package the PySide2 bindings and + and pyside2-tools + --build-type=all - the implicit default to build all of the above + + +When building PySide2, optionally, one can specify the location of the +shiboken2 cmake config path if it is not on the current PATH with: + --shiboken-config-dir=/path/to/shiboken/cmake/config/dir +This is useful if you did a cmake installation of shiboken2 into +a custom location. + For Windows, if OpenSSL support is required, it's necessary to specify the directory path that contains the OpenSSL shared libraries "libeay32.dll" and "ssleay32.dll", for example: @@ -79,7 +102,7 @@ not specified. You can use the option `--only-package` if you want to create more binary packages (bdist_wheel, bdist_egg, ...) without rebuilding the -entire PySide2 every time: +entire project every time: e.g.: @@ -89,7 +112,7 @@ e.g.: --cmake=c:\tools\cmake\bin\cmake.exe --openssl=c:\libs\OpenSSL32bit\bin -* Then, we create a bdist_egg reusing PySide2 build with option +* Then, we create a bdist_egg reusing the PySide2 build with option `--only-package`: python setup.py bdist_egg --only-package @@ -110,7 +133,7 @@ new environment variable called PYSIDE_DISABLE_INTERNAL_QT_CONF is introduced. You should assign the integer "1" to disable the internal `qt.conf`, -or "0" (or leave empty) to keep usining the internal `qt.conf` file. +or "0" (or leave empty) to keep using the internal `qt.conf` file. DEVELOPMENT OPTIONS: @@ -230,78 +253,18 @@ this_file = os.path.abspath(this_file) if os.path.dirname(this_file): os.chdir(os.path.dirname(this_file)) -from build_scripts.main import get_package_version, get_setuptools_extension_modules -from build_scripts.main import pyside_package_dir_name -from build_scripts.main import cmd_class_dict -from build_scripts.main import README, CHANGES -from setuptools import setup, Extension +# Save the original command line arguments to pass them on to the setup +# mechanism. +original_argv = list(sys.argv) -# The __version__ variable is just for PEP compliancy, and shouldn't be -# used as a value source. +from build_scripts.main import get_package_version, check_allowed_python_version +from build_scripts.setup_runner import SetupRunner + +# The __version__ variable is just for PEP compliance, and shouldn't be +# used as a value source. Use get_package_version() instead. __version__ = get_package_version() -extension_modules = get_setuptools_extension_modules() - -setup( - name = "PySide2", - version = get_package_version(), - description = ("Python bindings for the Qt cross-platform application and " - "UI framework"), - long_description = README + "\n\n" + CHANGES, - long_description_content_type = 'text/markdown', - classifiers = [ - 'Development Status :: 5 - Production/Stable', - 'Environment :: Console', - 'Environment :: MacOS X', - 'Environment :: X11 Applications :: Qt', - 'Environment :: Win32 (MS Windows)', - 'Intended Audience :: Developers', - 'License :: OSI Approved :: GNU Library or Lesser General Public License (LGPL)', - 'Operating System :: MacOS :: MacOS X', - 'Operating System :: POSIX', - 'Operating System :: POSIX :: Linux', - 'Operating System :: Microsoft', - 'Operating System :: Microsoft :: Windows', - 'Programming Language :: C++', - 'Programming Language :: Python', - 'Programming Language :: Python :: 2', - 'Programming Language :: Python :: 2.7', - 'Programming Language :: Python :: 3', - 'Programming Language :: Python :: 3.5', - 'Programming Language :: Python :: 3.6', - 'Programming Language :: Python :: 3.7', - 'Topic :: Database', - 'Topic :: Software Development', - 'Topic :: Software Development :: Code Generators', - 'Topic :: Software Development :: Libraries :: Application Frameworks', - 'Topic :: Software Development :: User Interfaces', - 'Topic :: Software Development :: Widget Sets', - ], - keywords = 'Qt', - author = 'Qt for Python Team', - author_email = 'pyside@qt-project.org', - url = 'https://www.pyside.org', - download_url = 'https://download.qt.io/official_releases/QtForPython/', - license = 'LGPL', - packages = ['PySide2', 'pyside2uic', - 'pyside2uic.Compiler', - 'pyside2uic.port_v{}'.format(sys.version_info[0]) ], - package_dir = {'': pyside_package_dir_name}, - include_package_data = True, - zip_safe = False, - entry_points = { - 'console_scripts': [ - 'pyside2-uic = PySide2.scripts.uic:main', - 'pyside2-rcc = PySide2.scripts.pyside_tool:main', - 'pyside2-lupdate = PySide2.scripts.pyside_tool:main', - 'shiboken2 = PySide2.scripts.pyside_tool:main', - ] - }, - cmdclass = cmd_class_dict, - # Add a bogus extension module (will never be built here since we - # are overriding the build command to do it using cmake) so things - # like bdist_egg will know that there are extension modules and - # will name the dist with the full platform info. - ext_modules = extension_modules, - ext_package = 'PySide2', -) +check_allowed_python_version() + +setup_runner = SetupRunner(original_argv) +setup_runner.run_setup() diff --git a/sources/cmake_helpers/helpers.cmake b/sources/cmake_helpers/helpers.cmake new file mode 100644 index 000000000..dd2e98a3a --- /dev/null +++ b/sources/cmake_helpers/helpers.cmake @@ -0,0 +1,63 @@ +macro(compute_config_py_values + full_version_var_name + ) + string(TIMESTAMP PACKAGE_BUILD_DATE "%Y-%m-%dT%H:%M:%S+00:00" UTC) + if (PACKAGE_BUILD_DATE) + set(PACKAGE_BUILD_DATE "__build_date__ = '${PACKAGE_BUILD_DATE}'") + endif() + + if (PACKAGE_SETUP_PY_PACKAGE_VERSION) + set(PACKAGE_SETUP_PY_PACKAGE_VERSION_ASSIGNMENT "__setup_py_package_version__ = '${PACKAGE_SETUP_PY_PACKAGE_VERSION}'") + set(FINAL_PACKAGE_VERSION ${PACKAGE_SETUP_PY_PACKAGE_VERSION}) + else() + set(FINAL_PACKAGE_VERSION ${${full_version_var_name}}) + endif() + + if (PACKAGE_SETUP_PY_PACKAGE_TIMESTAMP) + set(PACKAGE_SETUP_PY_PACKAGE_TIMESTAMP_ASSIGNMENT "__setup_py_package_timestamp__ = '${PACKAGE_SETUP_PY_PACKAGE_TIMESTAMP}'") + else() + set(PACKAGE_SETUP_PY_PACKAGE_TIMESTAMP_ASSIGNMENT "") + endif() + + find_package(Git) + if(GIT_FOUND) + # Check if current source folder is inside a git repo, so that commit information can be + # queried. + execute_process( + COMMAND ${GIT_EXECUTABLE} rev-parse --git-dir + OUTPUT_VARIABLE PACKAGE_SOURCE_IS_INSIDE_REPO + ERROR_QUIET + OUTPUT_STRIP_TRAILING_WHITESPACE) + + if(PACKAGE_SOURCE_IS_INSIDE_REPO) + # Force git dates to be UTC-based. + set(ENV{TZ} UTC) + execute_process( + COMMAND ${GIT_EXECUTABLE} --no-pager show --date=format-local:%Y-%m-%dT%H:%M:%S+00:00 -s --format=%cd HEAD + OUTPUT_VARIABLE PACKAGE_BUILD_COMMIT_DATE + OUTPUT_STRIP_TRAILING_WHITESPACE) + if(PACKAGE_BUILD_COMMIT_DATE) + set(PACKAGE_BUILD_COMMIT_DATE "__build_commit_date__ = '${PACKAGE_BUILD_COMMIT_DATE}'") + endif() + unset(ENV{TZ}) + + execute_process( + COMMAND ${GIT_EXECUTABLE} rev-parse HEAD + OUTPUT_VARIABLE PACKAGE_BUILD_COMMIT_HASH + OUTPUT_STRIP_TRAILING_WHITESPACE) + if(PACKAGE_BUILD_COMMIT_HASH) + set(PACKAGE_BUILD_COMMIT_HASH "__build_commit_hash__ = '${PACKAGE_BUILD_COMMIT_HASH}'") + endif() + + execute_process( + COMMAND ${GIT_EXECUTABLE} describe HEAD + OUTPUT_VARIABLE PACKAGE_BUILD_COMMIT_HASH_DESCRIBED + OUTPUT_STRIP_TRAILING_WHITESPACE) + if(PACKAGE_BUILD_COMMIT_HASH_DESCRIBED) + set(PACKAGE_BUILD_COMMIT_HASH_DESCRIBED "__build_commit_hash_described__ = '${PACKAGE_BUILD_COMMIT_HASH_DESCRIBED}'") + endif() + + endif() + endif() + +endmacro() diff --git a/sources/pyside2/CMakeLists.txt b/sources/pyside2/CMakeLists.txt index 358c119ae..25598bb5e 100644 --- a/sources/pyside2/CMakeLists.txt +++ b/sources/pyside2/CMakeLists.txt @@ -8,8 +8,10 @@ cmake_policy(SET CMP0046 NEW) project(pysidebindings) -set(CMAKE_MODULE_PATH ${CMAKE_SOURCE_DIR}/cmake/Macros/ +set(CMAKE_MODULE_PATH ${CMAKE_SOURCE_DIR}/../cmake_helpers/ + ${CMAKE_SOURCE_DIR}/cmake/Macros/ ${CMAKE_MODULE_PATH}) +include(helpers) option(USE_PYTHON_VERSION "Use specific python version to build pyside2." "") @@ -244,64 +246,7 @@ else() CACHE STRING "PySide2 version [full]" FORCE) endif() -string(TIMESTAMP PYSIDE_BUILD_DATE "%Y-%m-%dT%H:%M:%S+00:00" UTC) -if (PYSIDE_BUILD_DATE) - set(PYSIDE_BUILD_DATE "__build_date__ = '${PYSIDE_BUILD_DATE}'") -endif() - -if (PYSIDE_SETUP_PY_PACKAGE_VERSION) - set(PYSIDE_SETUP_PY_PACKAGE_VERSION_ASSIGNMENT "__setup_py_package_version__ = '${PYSIDE_SETUP_PY_PACKAGE_VERSION}'") - set(FINAL_PACKAGE_VERSION ${PYSIDE_SETUP_PY_PACKAGE_VERSION}) -else() - set(FINAL_PACKAGE_VERSION ${BINDING_API_VERSION_FULL}) -endif() - -if (PYSIDE_SETUP_PY_PACKAGE_TIMESTAMP) - set(PYSIDE_SETUP_PY_PACKAGE_TIMESTAMP_ASSIGNMENT "__setup_py_package_timestamp__ = '${PYSIDE_SETUP_PY_PACKAGE_TIMESTAMP}'") -else() - set(PYSIDE_SETUP_PY_PACKAGE_TIMESTAMP_ASSIGNMENT "__setup_py_package_timestamp__ = ''") -endif() - -find_package(Git) -if(GIT_FOUND) - # Check if current source folder is inside a git repo, so that commit information can be - # queried. - execute_process( - COMMAND ${GIT_EXECUTABLE} rev-parse --git-dir - OUTPUT_VARIABLE PYSIDE_SOURCE_IS_INSIDE_REPO - ERROR_QUIET - OUTPUT_STRIP_TRAILING_WHITESPACE) - - if(PYSIDE_SOURCE_IS_INSIDE_REPO) - # Force git dates to be UTC-based. - set(ENV{TZ} UTC) - execute_process( - COMMAND ${GIT_EXECUTABLE} --no-pager show --date=format-local:%Y-%m-%dT%H:%M:%S+00:00 -s --format=%cd HEAD - OUTPUT_VARIABLE PYSIDE_BUILD_COMMIT_DATE - OUTPUT_STRIP_TRAILING_WHITESPACE) - if(PYSIDE_BUILD_COMMIT_DATE) - set(PYSIDE_BUILD_COMMIT_DATE "__build_commit_date__ = '${PYSIDE_BUILD_COMMIT_DATE}'") - endif() - unset(ENV{TZ}) - - execute_process( - COMMAND ${GIT_EXECUTABLE} rev-parse HEAD - OUTPUT_VARIABLE PYSIDE_BUILD_COMMIT_HASH - OUTPUT_STRIP_TRAILING_WHITESPACE) - if(PYSIDE_BUILD_COMMIT_HASH) - set(PYSIDE_BUILD_COMMIT_HASH "__build_commit_hash__ = '${PYSIDE_BUILD_COMMIT_HASH}'") - endif() - - execute_process( - COMMAND ${GIT_EXECUTABLE} describe HEAD - OUTPUT_VARIABLE PYSIDE_BUILD_COMMIT_HASH_DESCRIBED - OUTPUT_STRIP_TRAILING_WHITESPACE) - if(PYSIDE_BUILD_COMMIT_HASH_DESCRIBED) - set(PYSIDE_BUILD_COMMIT_HASH_DESCRIBED "__build_commit_hash_described__ = '${PYSIDE_BUILD_COMMIT_HASH_DESCRIBED}'") - endif() - - endif() -endif() +compute_config_py_values(BINDING_API_VERSION) include(PySideModules) diff --git a/sources/pyside2/PySide2/__init__.py.in b/sources/pyside2/PySide2/__init__.py.in index ab50ef776..631f5f13a 100644 --- a/sources/pyside2/PySide2/__init__.py.in +++ b/sources/pyside2/PySide2/__init__.py.in @@ -4,18 +4,22 @@ __all__ = list("Qt" + body for body in __version__ = "@FINAL_PACKAGE_VERSION@" __version_info__ = (@BINDING_API_MAJOR_VERSION@, @BINDING_API_MINOR_VERSION@, @BINDING_API_MICRO_VERSION@, "@BINDING_API_PRE_RELEASE_VERSION_TYPE@", "@BINDING_API_PRE_RELEASE_VERSION@") -@PYSIDE_BUILD_DATE@ -@PYSIDE_BUILD_COMMIT_DATE@ -@PYSIDE_BUILD_COMMIT_HASH@ -@PYSIDE_BUILD_COMMIT_HASH_DESCRIBED@ - -# Timestamp used for snapshot build, which is part of snapshot package version. -@PYSIDE_SETUP_PY_PACKAGE_TIMESTAMP_ASSIGNMENT@ - def _setupQtDirectories(): import sys import os + # On Windows we need to explicitly import the shiboken2 module so + # that the libshiboken.dll dependency is loaded by the time a + # Qt module is imported. Otherwise due to PATH not containing + # the shiboken2 module path, the Qt module import would fail + # due to the missing libshiboken dll. + # We need to do the same on Linux and macOS, because we do not + # embed rpaths into the PySide2 libraries that would point to + # the libshiboken library location. Importing the module + # loads the libraries into the process memory beforehand, and + # thus takes care of it for us. + import shiboken2 + pyside_package_dir = os.path.abspath(os.path.dirname(__file__)) # Used by signature module. os.environ["PYSIDE_PACKAGE_DIR"] = pyside_package_dir diff --git a/sources/pyside2/PySide2/_config.py.in b/sources/pyside2/PySide2/_config.py.in index 31a2f7a50..740e9a001 100644 --- a/sources/pyside2/PySide2/_config.py.in +++ b/sources/pyside2/PySide2/_config.py.in @@ -8,10 +8,9 @@ pyside_library_soversion = str(@PYSIDE_SO_VERSION@) version = "@FINAL_PACKAGE_VERSION@" version_info = (@BINDING_API_MAJOR_VERSION@, @BINDING_API_MINOR_VERSION@, @BINDING_API_MICRO_VERSION@, "@BINDING_API_PRE_RELEASE_VERSION_TYPE@", "@BINDING_API_PRE_RELEASE_VERSION@") -@PYSIDE_BUILD_DATE@ -@PYSIDE_BUILD_COMMIT_DATE@ -@PYSIDE_BUILD_COMMIT_HASH@ -@PYSIDE_BUILD_COMMIT_HASH_DESCRIBED@ - -# Timestamp used for snapshot build, which is part of snapshot package version. -@PYSIDE_SETUP_PY_PACKAGE_TIMESTAMP_ASSIGNMENT@ +@PACKAGE_BUILD_DATE@ +@PACKAGE_BUILD_COMMIT_DATE@ +@PACKAGE_BUILD_COMMIT_HASH@ +@PACKAGE_BUILD_COMMIT_HASH_DESCRIBED@ +@PACKAGE_SETUP_PY_PACKAGE_TIMESTAMP_ASSIGNMENT@ +@PACKAGE_SETUP_PY_PACKAGE_VERSION_ASSIGNMENT@ diff --git a/sources/shiboken2/CMakeLists.txt b/sources/shiboken2/CMakeLists.txt index 1af84fca1..12a9b8773 100644 --- a/sources/shiboken2/CMakeLists.txt +++ b/sources/shiboken2/CMakeLists.txt @@ -5,8 +5,9 @@ include(CheckIncludeFileCXX) cmake_minimum_required(VERSION 3.1) cmake_policy(VERSION 3.1) -set(CMAKE_MODULE_PATH ${CMAKE_SOURCE_DIR}/cmake/Modules/ +set(CMAKE_MODULE_PATH ${CMAKE_SOURCE_DIR}/../cmake_helpers/ ${CMAKE_MODULE_PATH}) +include(helpers) find_package(Qt5 5.7 REQUIRED COMPONENTS Core Xml XmlPatterns) @@ -158,6 +159,8 @@ list(GET SHIBOKEN_VERSION_OUTPUT 4 shiboken_PRE_RELEASE_VERSION) set(shiboken2_VERSION "${shiboken_MAJOR_VERSION}.${shiboken_MINOR_VERSION}.${shiboken_MICRO_VERSION}") set(shiboken2_library_so_version "${shiboken_MAJOR_VERSION}.${shiboken_MINOR_VERSION}") +compute_config_py_values(shiboken2_VERSION) + ## For debugging the PYTHON* variables message("PYTHONLIBS_FOUND: " ${PYTHONLIBS_FOUND}) message("PYTHON_LIBRARIES: " ${PYTHON_LIBRARIES}) diff --git a/sources/shiboken2/generator/CMakeLists.txt b/sources/shiboken2/generator/CMakeLists.txt index 032118666..fb8058b2d 100644 --- a/sources/shiboken2/generator/CMakeLists.txt +++ b/sources/shiboken2/generator/CMakeLists.txt @@ -38,3 +38,33 @@ target_link_libraries(shiboken2 configure_file(shibokenconfig.h.in "${CMAKE_CURRENT_BINARY_DIR}/shibokenconfig.h" @ONLY) install(TARGETS shiboken2 DESTINATION bin) + +set(shiboken_generator_package_name "shiboken2_generator") + +configure_file("${CMAKE_CURRENT_SOURCE_DIR}/_config.py.in" + "${CMAKE_CURRENT_BINARY_DIR}/_config.py" @ONLY) +install(FILES "${CMAKE_CURRENT_BINARY_DIR}/_config.py" + DESTINATION "${PYTHON_SITE_PACKAGES}/${shiboken_generator_package_name}") + +configure_file("${CMAKE_CURRENT_SOURCE_DIR}/__init__.py.in" + "${CMAKE_CURRENT_BINARY_DIR}/__init__.py" @ONLY) +install(FILES "${CMAKE_CURRENT_BINARY_DIR}/__init__.py" + DESTINATION "${PYTHON_SITE_PACKAGES}/${shiboken_generator_package_name}") + +# shiboken2 setuptools entry point +install(FILES ${CMAKE_CURRENT_SOURCE_DIR}/../shiboken_tool.py + DESTINATION bin + PERMISSIONS + OWNER_EXECUTE OWNER_WRITE OWNER_READ + GROUP_EXECUTE GROUP_READ + WORLD_EXECUTE WORLD_READ) + +# Use absolute path instead of relative path, to avoid ninja build errors due to +# duplicate file dependency inconsistency. +set(shiboken_version_relative_path "${CMAKE_CURRENT_SOURCE_DIR}/../shiboken_version.py") +get_filename_component(shiboken_version_path ${shiboken_version_relative_path} ABSOLUTE) +configure_file("${shiboken_version_path}" + "${CMAKE_CURRENT_BINARY_DIR}/_git_shiboken_generator_version.py" @ONLY) + +install(FILES "${CMAKE_CURRENT_BINARY_DIR}/_git_shiboken_generator_version.py" + DESTINATION "${PYTHON_SITE_PACKAGES}/${shiboken_generator_package_name}") diff --git a/sources/shiboken2/generator/__init__.py.in b/sources/shiboken2/generator/__init__.py.in new file mode 100644 index 000000000..4be6a833b --- /dev/null +++ b/sources/shiboken2/generator/__init__.py.in @@ -0,0 +1,2 @@ +__version__ = "@FINAL_PACKAGE_VERSION@" +__version_info__ = (@shiboken_MAJOR_VERSION@, @shiboken_MINOR_VERSION@, @shiboken_MICRO_VERSION@, "@shiboken_PRE_RELEASE_VERSION_TYPE@", "@shiboken_PRE_RELEASE_VERSION@") diff --git a/sources/shiboken2/generator/_config.py.in b/sources/shiboken2/generator/_config.py.in new file mode 100644 index 000000000..985735fa4 --- /dev/null +++ b/sources/shiboken2/generator/_config.py.in @@ -0,0 +1,9 @@ +version = "@FINAL_PACKAGE_VERSION@" +version_info = (@shiboken_MAJOR_VERSION@, @shiboken_MINOR_VERSION@, @shiboken_MICRO_VERSION@, "@shiboken_PRE_RELEASE_VERSION_TYPE@", "@shiboken_PRE_RELEASE_VERSION@") + +@PACKAGE_BUILD_DATE@ +@PACKAGE_BUILD_COMMIT_DATE@ +@PACKAGE_BUILD_COMMIT_HASH@ +@PACKAGE_BUILD_COMMIT_HASH_DESCRIBED@ +@PACKAGE_SETUP_PY_PACKAGE_TIMESTAMP_ASSIGNMENT@ +@PACKAGE_SETUP_PY_PACKAGE_VERSION_ASSIGNMENT@ diff --git a/sources/shiboken2/shiboken_tool.py b/sources/shiboken2/shiboken_tool.py new file mode 100755 index 000000000..8494c5d57 --- /dev/null +++ b/sources/shiboken2/shiboken_tool.py @@ -0,0 +1,53 @@ +#!/usr/bin/env python +# -*- coding: utf-8 -*- +############################################################################# +## +## Copyright (C) 2018 The Qt Company Ltd. +## Contact: https://www.qt.io/licensing/ +## +## This file is part of Qt for Python. +## +## $QT_BEGIN_LICENSE:LGPL$ +## Commercial License Usage +## Licensees holding valid commercial Qt licenses may use this file in +## accordance with the commercial license agreement provided with the +## Software or, alternatively, in accordance with the terms contained in +## a written agreement between you and The Qt Company. For licensing terms +## and conditions see https://www.qt.io/terms-conditions. For further +## information use the contact form at https://www.qt.io/contact-us. +## +## GNU Lesser General Public License Usage +## Alternatively, this file may be used under the terms of the GNU Lesser +## General Public License version 3 as published by the Free Software +## Foundation and appearing in the file LICENSE.LGPL3 included in the +## packaging of this file. Please review the following information to +## ensure the GNU Lesser General Public License version 3 requirements +## will be met: https://www.gnu.org/licenses/lgpl-3.0.html. +## +## GNU General Public License Usage +## Alternatively, this file may be used under the terms of the GNU +## General Public License version 2.0 or (at your option) the GNU General +## Public license version 3 or any later version approved by the KDE Free +## Qt Foundation. The licenses are as published by the Free Software +## Foundation and appearing in the file LICENSE.GPL2 and LICENSE.GPL3 +## included in the packaging of this file. Please review the following +## information to ensure the GNU General Public License requirements will +## be met: https://www.gnu.org/licenses/gpl-2.0.html and +## https://www.gnu.org/licenses/gpl-3.0.html. +## +## $QT_END_LICENSE$ +## +############################################################################# +import sys +import os +import subprocess + +def main(): + # The tools listed as entrypoints in setup.py are copied to 'scripts/..' + cmd = os.path.join("..", os.path.basename(sys.argv[0])) + command = [os.path.join(os.path.dirname(os.path.realpath(__file__)), cmd)] + command.extend(sys.argv[1:]) + sys.exit(subprocess.call(command)) + +if __name__ == "__main__": + main() diff --git a/sources/shiboken2/shibokenmodule/CMakeLists.txt b/sources/shiboken2/shibokenmodule/CMakeLists.txt index f2d7b30f2..517aecba8 100644 --- a/sources/shiboken2/shibokenmodule/CMakeLists.txt +++ b/sources/shiboken2/shibokenmodule/CMakeLists.txt @@ -40,4 +40,24 @@ target_link_libraries(shibokenmodule add_dependencies(shibokenmodule shiboken2) -install(TARGETS shibokenmodule DESTINATION ${PYTHON_SITE_PACKAGES}) +install(TARGETS shibokenmodule DESTINATION ${PYTHON_SITE_PACKAGES}/shiboken2) + +configure_file("${CMAKE_CURRENT_SOURCE_DIR}/_config.py.in" + "${CMAKE_CURRENT_BINARY_DIR}/_config.py" @ONLY) +install(FILES "${CMAKE_CURRENT_BINARY_DIR}/_config.py" + DESTINATION "${PYTHON_SITE_PACKAGES}/shiboken2") + +configure_file("${CMAKE_CURRENT_SOURCE_DIR}/__init__.py.in" + "${CMAKE_CURRENT_BINARY_DIR}/__init__.py" @ONLY) +install(FILES "${CMAKE_CURRENT_BINARY_DIR}/__init__.py" + DESTINATION "${PYTHON_SITE_PACKAGES}/shiboken2") + +# Use absolute path instead of relative path, to avoid ninja build errors due to +# duplicate file dependency inconsistency. +set(shiboken_version_relative_path "${CMAKE_CURRENT_SOURCE_DIR}/../shiboken_version.py") +get_filename_component(shiboken_version_path ${shiboken_version_relative_path} ABSOLUTE) +configure_file("${shiboken_version_path}" + "${CMAKE_CURRENT_BINARY_DIR}/_git_shiboken_module_version.py" @ONLY) + +install(FILES "${CMAKE_CURRENT_BINARY_DIR}/_git_shiboken_module_version.py" + DESTINATION "${PYTHON_SITE_PACKAGES}/shiboken2") diff --git a/sources/shiboken2/shibokenmodule/__init__.py.in b/sources/shiboken2/shibokenmodule/__init__.py.in new file mode 100644 index 000000000..81ab0063a --- /dev/null +++ b/sources/shiboken2/shibokenmodule/__init__.py.in @@ -0,0 +1,4 @@ +__version__ = "@FINAL_PACKAGE_VERSION@" +__version_info__ = (@shiboken_MAJOR_VERSION@, @shiboken_MINOR_VERSION@, @shiboken_MICRO_VERSION@, "@shiboken_PRE_RELEASE_VERSION_TYPE@", "@shiboken_PRE_RELEASE_VERSION@") + +from .shiboken2 import * diff --git a/sources/shiboken2/shibokenmodule/_config.py.in b/sources/shiboken2/shibokenmodule/_config.py.in new file mode 100644 index 000000000..9607e5ca7 --- /dev/null +++ b/sources/shiboken2/shibokenmodule/_config.py.in @@ -0,0 +1,11 @@ +shiboken_library_soversion = str(@shiboken2_library_so_version@) + +version = "@FINAL_PACKAGE_VERSION@" +version_info = (@shiboken_MAJOR_VERSION@, @shiboken_MINOR_VERSION@, @shiboken_MICRO_VERSION@, "@shiboken_PRE_RELEASE_VERSION_TYPE@", "@shiboken_PRE_RELEASE_VERSION@") + +@PACKAGE_BUILD_DATE@ +@PACKAGE_BUILD_COMMIT_DATE@ +@PACKAGE_BUILD_COMMIT_HASH@ +@PACKAGE_BUILD_COMMIT_HASH_DESCRIBED@ +@PACKAGE_SETUP_PY_PACKAGE_TIMESTAMP_ASSIGNMENT@ +@PACKAGE_SETUP_PY_PACKAGE_VERSION_ASSIGNMENT@ |