aboutsummaryrefslogtreecommitdiffstats
path: root/build_scripts
diff options
context:
space:
mode:
authorAlexandru Croitor <alexandru.croitor@qt.io>2018-07-20 18:54:05 +0200
committerAlexandru Croitor <alexandru.croitor@qt.io>2018-10-12 14:45:35 +0000
commit43fe3494a9d902034896e3afa7b5158c77163be0 (patch)
tree07bbd20b8bd322fdf062b87dfade1e1e74d80433 /build_scripts
parentc6c9f057cddc0e0b885a648489264538eba0a158 (diff)
Allow building shiboken2 and PySide2 as separate wheels
Actually this creates 3 wheel packages: - shiboken2 (the python module and libshiboken shared library) - shiboken2-generator (contains the generator executable, libclang and dependent Qt libraries) - PySide2 (the PySide2 modules and Qt shared libraries, and tools like rcc, uic) Calling the setup.py script will not do the actual build now (in the sense of calling CMake, make, etc.). Instead it will spawn new processes (via subprocess.call) calling the same setup.py script, but with different arguments. These "sub-invocations" will do the actual building. Thus, the "top-level invocation" will decide which packages to build and delegate that to the "sub-invocations" of setup.py. A new optional command line argument is introduced called "--build-type" which defaults to "all", and can also be set to "shiboken2", "shiboken2-generator" and "pyside2". A user can choose which packages to build using this option. The "top-level invocation" uses this option to decide how many "sub-invocations" to execute. A new command line argument called "--internal-build-type" takes the same values as the one above. It defines which package will actually be built in the new spawned "sub-invocation" process. The "top-level invocation" sets this automatically for each "sub-invocation" depending on the value of "--build-type". This option is also useful for developers that may want to debug the python building code in the "sub-invocation". Developers can set this manually via the command line, and thus avoid the process spawning indirection. A new class Config is introduced to facilitate storage of the various state needed for building a single package. A new class SetupRunner is introduced that takes care of the "--build-type" and "--internal-build-type" argument handling and delegation of "sub-invocations". A new class Options is introduced to 'hopefully', in the future, streamline the mess of option handling that we currently have. setup.py now is now simplified to mostly just call SetupRunner.run_setup(). Certain refactorings were done to facilitate further clean-up of the build code, the current code is definitely not the end all be all. Various other changes that were needed to implement the wheel separation: - a new cmake_helpers directory is added to share common cmake code between packages. - the custom popenasync.py file is removed in favor of using subprocess.call in as many places as possible, and thus avoid 10 different functions for process creation. - Manifest.in is removed, because copying to the setuptools build dir is now done directly by prepare_packages functions. - because prepare_packages copies directly to the setuptools build dir, avoiding the pyside_package dir, we do less copying of big Qt files now. - versioning of PySide2 and shiboken2 packages is now separate. shiboken2 and shiboken2-generator share the same versions for now though. - shiboken2 is now listed as a required package for PySide2, to facilitate pip requirements.txt dependencies. - coin_build_instructions currently needs to install an unreleased version of wheel, due to a bug that breaks installation of generated wheel files. - added separate command line options to pyside2_config.py for shiboken2-module and shiboken2-generator. - adapted samplebinding and scriptableapplication projects due to shiboken being a separate package. - adapted pyside2-tool and shiboken2-tool python scripts for setup tools entry points. - made some optimizations not to invoke cmake for shiboken2-generator when doing a top-level "all" build. - fixed unnecessary rpaths not to be included on Linux (mainly the Qt rpaths). Task-nubmer: PYSIDE-749 Change-Id: I0336043955624c1d12ed254802c442608cced5fb Reviewed-by: Christian Tismer <tismer@stackless.com> Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Diffstat (limited to 'build_scripts')
-rw-r--r--build_scripts/build_scripts.pyqtc5
-rw-r--r--build_scripts/config.py393
-rw-r--r--build_scripts/main.py387
-rw-r--r--build_scripts/options.py78
-rw-r--r--build_scripts/platforms/linux.py136
-rw-r--r--build_scripts/platforms/macos.py119
-rw-r--r--build_scripts/platforms/unix.py261
-rw-r--r--build_scripts/platforms/windows_desktop.py468
-rw-r--r--build_scripts/setup_runner.py165
-rw-r--r--build_scripts/utils.py203
10 files changed, 1518 insertions, 697 deletions
diff --git a/build_scripts/build_scripts.pyqtc b/build_scripts/build_scripts.pyqtc
index eb4b58a63..1fc1c9664 100644
--- a/build_scripts/build_scripts.pyqtc
+++ b/build_scripts/build_scripts.pyqtc
@@ -1,8 +1,10 @@
__init__.py
+config.py
main.py
options.py
platforms
qtinfo.py
+setup_runner.py
utils.py
wheel_override.py
platforms/__init__.py
@@ -11,3 +13,6 @@ platforms/macos.py
platforms/unix.py
platforms/windows_desktop.py
../setup.py
+../coin_build_instructions.py
+../coin_test_instructions.py
+
diff --git a/build_scripts/config.py b/build_scripts/config.py
new file mode 100644
index 000000000..78d7d4040
--- /dev/null
+++ b/build_scripts/config.py
@@ -0,0 +1,393 @@
+#############################################################################
+##
+## Copyright (C) 2018 The Qt Company Ltd.
+## Contact: https://www.qt.io/licensing/
+##
+## This file is part of Qt for Python.
+##
+## $QT_BEGIN_LICENSE:LGPL$
+## Commercial License Usage
+## Licensees holding valid commercial Qt licenses may use this file in
+## accordance with the commercial license agreement provided with the
+## Software or, alternatively, in accordance with the terms contained in
+## a written agreement between you and The Qt Company. For licensing terms
+## and conditions see https://www.qt.io/terms-conditions. For further
+## information use the contact form at https://www.qt.io/contact-us.
+##
+## GNU Lesser General Public License Usage
+## Alternatively, this file may be used under the terms of the GNU Lesser
+## General Public License version 3 as published by the Free Software
+## Foundation and appearing in the file LICENSE.LGPL3 included in the
+## packaging of this file. Please review the following information to
+## ensure the GNU Lesser General Public License version 3 requirements
+## will be met: https://www.gnu.org/licenses/lgpl-3.0.html.
+##
+## GNU General Public License Usage
+## Alternatively, this file may be used under the terms of the GNU
+## General Public License version 2.0 or (at your option) the GNU General
+## Public license version 3 or any later version approved by the KDE Free
+## Qt Foundation. The licenses are as published by the Free Software
+## Foundation and appearing in the file LICENSE.GPL2 and LICENSE.GPL3
+## included in the packaging of this file. Please review the following
+## information to ensure the GNU General Public License requirements will
+## be met: https://www.gnu.org/licenses/gpl-2.0.html and
+## https://www.gnu.org/licenses/gpl-3.0.html.
+##
+## $QT_END_LICENSE$
+##
+#############################################################################
+
+import sys, os
+import distutils.log as log
+
+
+class Config(object):
+ def __init__(self):
+ # Constants
+ self._build_type_all = "all"
+ self._invocation_type_top_level = "top-level"
+ self._invocation_type_internal = "internal"
+
+ # The keyword arguments which will be given to setuptools.setup
+ self.setup_kwargs = {}
+
+ # The setup.py invocation type.
+ # top-level
+ # internal
+ self.invocation_type = None
+
+ # The type of the top-level build.
+ # all - build shiboken2 module, shiboken2-generator and PySide2
+ # modules
+ # shiboken2 - build only shiboken2 module
+ # shiboken2-generator - build only the shiboken2-generator
+ # pyside2 - build only PySide2 modules
+ self.build_type = None
+
+ # The internal build type, used for internal invocations of
+ # setup.py to build a specific module only.
+ self.internal_build_type = None
+
+ # Options that can be given to --build-type and
+ # --internal-build-type
+ self.shiboken_module_option_name = "shiboken2"
+ self.shiboken_generator_option_name = "shiboken2-generator"
+ self.pyside_option_name = "pyside2"
+
+ # Names to be passed to setuptools.setup() name key,
+ # so not package name, but rather project name as it appears
+ # in the wheel name and on PyPi.
+ self.shiboken_module_st_name = "shiboken2"
+ self.shiboken_generator_st_name = "shiboken2-generator"
+ self.pyside_st_name = "PySide2"
+
+ # Used by check_allowed_python_version to validate the
+ # interpreter version.
+ self.python_version_classifiers = [
+ 'Programming Language :: Python',
+ 'Programming Language :: Python :: 2',
+ 'Programming Language :: Python :: 2.7',
+ 'Programming Language :: Python :: 3',
+ 'Programming Language :: Python :: 3.3',
+ 'Programming Language :: Python :: 3.4',
+ 'Programming Language :: Python :: 3.5',
+ 'Programming Language :: Python :: 3.6',
+ 'Programming Language :: Python :: 3.7',
+ ]
+
+ self.setup_script_dir = None
+
+ def init_config(self, build_type=None, internal_build_type=None,
+ cmd_class_dict=None, package_version=None,
+ ext_modules=None, setup_script_dir=None):
+ """
+ Sets up the global singleton config which is used in many parts
+ of the setup process.
+ """
+
+ # if --internal-build-type was passed, it means that this is a
+ # sub-invocation to build a specific package.
+ if internal_build_type:
+ self.set_is_internal_invocation()
+ self.set_internal_build_type(internal_build_type)
+ else:
+ self.set_is_top_level_invocation()
+
+ # --build-type was specified explicitly, so set it. Otherwise
+ # default to all.
+ if build_type:
+ self.build_type = build_type
+ else:
+ self.build_type = self._build_type_all
+
+ self.setup_script_dir = setup_script_dir
+
+ setup_kwargs = {}
+ setup_kwargs['long_description'] = self.get_long_description()
+ setup_kwargs['long_description_content_type'] = 'text/markdown',
+ setup_kwargs['keywords'] = 'Qt'
+ setup_kwargs['author'] = 'Qt for Python Team'
+ setup_kwargs['author_email'] = 'pyside@qt-project.org'
+ setup_kwargs['url'] = 'https://www.pyside.org'
+ setup_kwargs['download_url'] = 'https://download.qt.io/official_releases/QtForPython'
+ setup_kwargs['license'] = 'LGPL'
+ setup_kwargs['zip_safe'] = False
+ setup_kwargs['cmdclass'] = cmd_class_dict
+ setup_kwargs['version'] = package_version
+
+ # Setting these two keys is still a bit of a discussion point.
+ # In general not setting them will allow using "build" and
+ # "bdist_wheel" just fine. What they do, is they specify to the
+ # setuptools.command.build_py command that certain pure python
+ # modules (.py files) exist in the specified package location,
+ # and that they should be copied over to the setuptools build
+ # dir.
+ # But it doesn't really make sense for us, because we copy all
+ # the necessary files to the build dir via prepare_packages()
+ # function anyway.
+ # If we don't set them, the build_py sub-command will be
+ # skipped, but the build command will still be executed, which
+ # is where we run cmake / make.
+ # The only plausible usage of it, is if we will implement a
+ # correctly functioning setup.py develop command (or bdist_egg).
+ # But currently that doesn't seem to work.
+ setup_kwargs['packages'] = self.get_setup_tools_packages_for_current_build()
+ setup_kwargs['package_dir'] = self.get_package_name_to_dir_path_mapping()
+
+ # Add a bogus extension module (will never be built here since
+ # we are overriding the build command to do it using cmake) so
+ # things like bdist_egg will know that there are extension
+ # modules and will name the dist with the full platform info.
+ setup_kwargs['ext_modules'] = ext_modules
+
+ common_classifiers = [
+ 'Development Status :: 5 - Production/Stable',
+ 'Environment :: Console',
+ 'Environment :: MacOS X',
+ 'Environment :: X11 Applications :: Qt',
+ 'Environment :: Win32 (MS Windows)',
+ 'Intended Audience :: Developers',
+ 'License :: OSI Approved :: GNU Library or Lesser General Public License (LGPL)',
+ 'Operating System :: MacOS :: MacOS X',
+ 'Operating System :: POSIX',
+ 'Operating System :: POSIX :: Linux',
+ 'Operating System :: Microsoft',
+ 'Operating System :: Microsoft :: Windows',
+ 'Programming Language :: C++']
+ common_classifiers.extend(self.python_version_classifiers)
+ common_classifiers.extend([
+ 'Topic :: Database',
+ 'Topic :: Software Development',
+ 'Topic :: Software Development :: Code Generators',
+ 'Topic :: Software Development :: Libraries :: Application Frameworks',
+ 'Topic :: Software Development :: User Interfaces',
+ 'Topic :: Software Development :: Widget Sets'])
+ setup_kwargs['classifiers'] = common_classifiers
+
+ if self.internal_build_type == self.shiboken_module_option_name:
+ setup_kwargs['name'] = self.shiboken_module_st_name
+ setup_kwargs['description'] = "Python / C++ bindings helper module",
+ setup_kwargs['entry_points'] = {}
+
+ elif self.internal_build_type == self.shiboken_generator_option_name:
+ setup_kwargs['name'] = self.shiboken_generator_st_name
+ setup_kwargs['description'] = "Python / C++ bindings generator",
+ setup_kwargs['install_requires'] = [self.shiboken_module_st_name]
+ setup_kwargs['entry_points'] = {
+ 'console_scripts': [
+ 'shiboken2 = {}.scripts.shiboken_tool:main'.format(self.package_name()),
+ ]
+ }
+
+ elif self.internal_build_type == self.pyside_option_name:
+ setup_kwargs['name'] = self.pyside_st_name
+ setup_kwargs['description'] = ("Python bindings for the Qt cross-platform application"
+ " and UI framework"),
+ setup_kwargs['install_requires'] = [self.shiboken_module_st_name]
+ setup_kwargs['entry_points'] = {
+ 'console_scripts': [
+ 'pyside2-uic = {}.scripts.uic:main'.format(self.package_name()),
+ 'pyside2-rcc = {}.scripts.pyside_tool:main'.format(self.package_name()),
+ 'pyside2-lupdate = {}.scripts.pyside_tool:main'.format(self.package_name()),
+ ]
+ }
+ self.setup_kwargs = setup_kwargs
+
+ def get_long_description(self):
+ readme_filename = 'README.md'
+ changes_filename = 'CHANGES.rst'
+
+ if self.is_internal_shiboken_module_build():
+ readme_filename = 'README.shiboken2.md'
+ elif self.is_internal_shiboken_generator_build():
+ readme_filename = 'README.shiboken2-generator.md'
+ elif self.is_internal_pyside_build():
+ readme_filename = 'README.pyside2.md'
+
+ content = ''
+ changes = ''
+ try:
+ with open(os.path.join(self.setup_script_dir, readme_filename)) as f:
+ readme = f.read()
+ except Exception as e:
+ log.error("Couldn't read contents of {}.".format(readme_filename))
+ raise
+
+ # Don't include CHANGES.rst for now, because we have not decided
+ # how to handle change files yet.
+ include_changes = False
+ if include_changes:
+ try:
+ with open(os.path.join(self.setup_script_dir, changes_filename)) as f:
+ changes = f.read()
+ except Exception as e:
+ log.error("Couldn't read contents of {}".format(changes_filename))
+ raise
+ content += readme
+
+ if changes:
+ content += "\n\n" + changes
+
+ return content
+
+ def package_name(self):
+ """
+ Returns package name as it appears in Python's site-packages
+ directory.
+
+ Package names can only be delimited by underscores, and not by
+ dashes.
+ """
+ if self.is_internal_shiboken_module_build():
+ return "shiboken2"
+ elif self.is_internal_shiboken_generator_build():
+ return "shiboken2_generator"
+ elif self.is_internal_pyside_build():
+ return "PySide2"
+ else:
+ return None
+
+ def get_setup_tools_packages_for_current_build(self):
+ """
+ Returns a list of packages for setup tools to consider in the
+ build_py command, so that it can copy the pure python files.
+ Not really necessary because it's done in prepare_packages()
+ anyway.
+
+ This is really just to satisfy some checks in setuptools
+ build_py command, and if we ever properly implement the develop
+ command.
+ """
+ if self.internal_build_type == self.pyside_option_name:
+ return [
+ config.package_name(),
+ 'pyside2uic',
+ 'pyside2uic.Compiler',
+ 'pyside2uic.port_v{}'.format(sys.version_info[0])
+ ]
+ elif self.internal_build_type == self.shiboken_module_option_name:
+ return [self.package_name()]
+ else:
+ return []
+
+ def get_package_name_to_dir_path_mapping(self):
+ """
+ Used in setuptools.setup 'package_dir' argument to specify where
+ the actual module packages are located.
+
+ For example when building the shiboken module, setuptools will
+ expect to find the "shiboken2" module sources under
+ "sources/shiboken2/shibokenmodule".
+
+ This is really just to satisfy some checks in setuptools
+ build_py command, and if we ever properly implement the develop
+ command.
+ """
+ if self.is_internal_shiboken_module_build():
+ return {
+ self.package_name(): "sources/shiboken2/shibokenmodule"
+ }
+ elif self.is_internal_shiboken_generator_build():
+ # This is left empty on purpose, because the shiboken
+ # generator doesn't have a python module for now.
+ return {}
+ elif self.is_internal_pyside_build():
+ return {
+ self.package_name(): "sources/pyside2/PySide2",
+ "pyside2uic": "sources/pyside2-tools/pyside2uic"
+ }
+ else:
+ return {}
+
+ def get_buildable_extensions(self):
+ """
+ Used by PysideBuild.run to build the CMake projects.
+ :return: A list of directory names under the sources directory.
+ """
+ if self.is_internal_shiboken_module_build() or self.is_internal_shiboken_generator_build():
+ return ['shiboken2']
+ elif self.is_internal_pyside_build():
+ return ['pyside2', 'pyside2-tools']
+ return None
+
+ def set_is_top_level_invocation(self):
+ self.invocation_type = self._invocation_type_top_level
+
+ def set_is_internal_invocation(self):
+ self.invocation_type = self._invocation_type_internal
+
+ def is_top_level_invocation(self):
+ return self.invocation_type == self._invocation_type_top_level
+
+ def is_internal_invocation(self):
+ return self.invocation_type == self._invocation_type_internal
+
+ def is_top_level_build_all(self):
+ return self.build_type == self._build_type_all
+
+ def is_top_level_build_shiboken_module(self):
+ return self.build_type == self.shiboken_module_option_name
+
+ def is_top_level_build_shiboken_generator(self):
+ return self.build_type == self.shiboken_generator_option_name
+
+ def is_top_level_build_pyside(self):
+ return self.build_type == self.pyside_option_name
+
+ def set_internal_build_type(self, internal_build_type):
+ self.internal_build_type = internal_build_type
+
+ def is_internal_shiboken_module_build(self):
+ return self.internal_build_type == self.shiboken_module_option_name
+
+ def is_internal_shiboken_generator_build(self):
+ return self.internal_build_type == self.shiboken_generator_option_name
+
+ def is_internal_pyside_build(self):
+ return self.internal_build_type == self.pyside_option_name
+
+ def is_internal_shiboken_generator_build_and_part_of_top_level_all(self):
+ """
+ Used to skip certain build rules and output, when we know that
+ the CMake build of shiboken was already done as part of the
+ top-level "all" build when shiboken2-module was built.
+ """
+ return self.is_internal_shiboken_generator_build() and self.is_top_level_build_all()
+
+ def get_allowed_top_level_build_values(self):
+ return [
+ self._build_type_all,
+ self.shiboken_module_option_name,
+ self.shiboken_generator_option_name,
+ self.pyside_option_name
+ ]
+
+ def get_allowed_internal_build_values(self):
+ return [
+ self.shiboken_module_option_name,
+ self.shiboken_generator_option_name,
+ self.pyside_option_name
+ ]
+
+
+config = Config()
diff --git a/build_scripts/main.py b/build_scripts/main.py
index b64d6f1a9..50f751caa 100644
--- a/build_scripts/main.py
+++ b/build_scripts/main.py
@@ -42,6 +42,7 @@ from distutils.version import LooseVersion
import os
import time
+from .config import config
from .utils import memoize, get_python_dict
from .options import *
@@ -90,15 +91,11 @@ def get_setuptools_extension_modules():
extension_modules = [Extension(*extension_args, **extension_kwargs)]
return extension_modules
-# Buildable extensions.
-contained_modules = ['shiboken2', 'pyside2', 'pyside2-tools']
# Git submodules: ["submodule_name",
# "location_relative_to_sources_folder"]
submodules = [["pyside2-tools"]]
-pyside_package_dir_name = "pyside_package"
-
try:
import setuptools
except ImportError:
@@ -108,13 +105,8 @@ except ImportError:
import sys
import platform
import re
-import fnmatch
-
-import difflib # for a close match of dirname and module
-import functools
-from distutils import log
-from distutils.errors import DistutilsOptionError
+import distutils.log as log
from distutils.errors import DistutilsSetupError
from distutils.sysconfig import get_config_var
from distutils.sysconfig import get_python_lib
@@ -123,7 +115,7 @@ from distutils.command.build import build as _build
from distutils.command.build_ext import build_ext as _build_ext
from distutils.util import get_platform
-from setuptools import setup, Extension
+from setuptools import Extension
from setuptools.command.install import install as _install
from setuptools.command.install_lib import install_lib as _install_lib
from setuptools.command.bdist_egg import bdist_egg as _bdist_egg
@@ -132,39 +124,40 @@ from setuptools.command.build_py import build_py as _build_py
from .qtinfo import QtInfo
from .utils import rmtree, detect_clang, copyfile, copydir, run_process_output, run_process
-from .utils import update_env_path, init_msvc_env, filter_match, macos_fix_rpaths_for_library
+from .utils import update_env_path, init_msvc_env, filter_match
+from .utils import macos_fix_rpaths_for_library
+from .utils import linux_fix_rpaths_for_library
from .platforms.unix import prepare_packages_posix
from .platforms.windows_desktop import prepare_packages_win32
from .wheel_override import wheel_module_exists, get_bdist_wheel_override
from textwrap import dedent
-# make sure that setup.py is run with an allowed python version
+
def check_allowed_python_version():
+ """
+ Make sure that setup.py is run with an allowed python version.
+ """
+
import re
- pattern = "'Programming Language :: Python :: (\d+)\.(\d+)'"
+ pattern = "Programming Language :: Python :: (\d+)\.(\d+)"
supported = []
- with open(setup_py_path) as setup:
- for line in setup.readlines():
- found = re.search(pattern, line)
- if found:
- major = int(found.group(1))
- minor = int(found.group(2))
- supported.append( (major, minor) )
+
+ for line in config.python_version_classifiers:
+ found = re.search(pattern, line)
+ if found:
+ major = int(found.group(1))
+ minor = int(found.group(2))
+ supported.append( (major, minor) )
this_py = sys.version_info[:2]
if this_py not in supported:
- print("only these python versions are supported:", supported)
+ print("Unsupported python version detected. Only these python versions are supported: {}"
+ .format(supported))
sys.exit(1)
-check_allowed_python_version()
qt_src_dir = ''
-# This is used automatically by distutils.command.install object, to
-# specify final installation location.
-OPTION_FINAL_INSTALL_PREFIX = option_value("prefix")
-
-
if OPTION_QT_VERSION is None:
OPTION_QT_VERSION = "5"
if OPTION_QMAKE is None:
@@ -293,8 +286,7 @@ def get_qt_version():
qt_version = qtinfo.version
if not qt_version:
- log.error("Failed to query the Qt version with qmake {0}".format(
- self.qtinfo.qmake_command))
+ log.error("Failed to query the Qt version with qmake {0}".format(qtinfo.qmake_command))
sys.exit(1)
if LooseVersion(qtinfo.version) < LooseVersion("5.7"):
@@ -304,12 +296,13 @@ def get_qt_version():
return qt_version
+
def prepare_build():
if (os.path.isdir(".git") and not OPTION_IGNOREGIT and
not OPTION_ONLYPACKAGE and not OPTION_REUSE_BUILD):
prepare_sub_modules()
- # Clean up temp and package folders
- for n in [pyside_package_dir_name, "build"]:
+ # Clean up temp build folder.
+ for n in ["build"]:
d = os.path.join(setup_script_dir, n)
if os.path.isdir(d):
print("Removing {}".format(d))
@@ -318,13 +311,7 @@ def prepare_build():
except Exception as e:
print('***** problem removing "{}"'.format(d))
print('ignored error: {}'.format(e))
- # Prepare package folders
- ppdn = pyside_package_dir_name
- absolute_paths = [os.path.join(ppdn, "PySide2"),
- os.path.join(ppdn, "pyside2uic")]
- for pkg in absolute_paths:
- pkg_dir = os.path.join(setup_script_dir, pkg)
- os.makedirs(pkg_dir)
+
# locate Qt sources for the documentation
if OPTION_QT_SRC is None:
install_prefix = qtinfo.prefix_dir
@@ -389,26 +376,15 @@ class PysideBuildExt(_build_ext):
pass
-
-# pyside_build_py and pyside_install_lib are reimplemented to preserve
-# symlinks when distutils / setuptools copy files to various
-# directories through the different build stages.
class PysideBuildPy(_build_py):
def __init__(self, *args, **kwargs):
_build_py.__init__(self, *args, **kwargs)
- def build_package_data(self):
- """Copies files from pyside_package into build/xxx directory"""
-
- for package, src_dir, build_dir, filenames in self.data_files:
- for filename in filenames:
- target = os.path.join(build_dir, filename)
- self.mkpath(os.path.dirname(target))
- srcfile = os.path.abspath(os.path.join(src_dir, filename))
- # Using our own copyfile makes sure to preserve symlinks.
- copyfile(srcfile, target)
+# _install_lib is reimplemented to preserve
+# symlinks when distutils / setuptools copy files to various
+# directories from the setup tools build dir to the install dir.
class PysideInstallLib(_install_lib):
def __init__(self, *args, **kwargs):
@@ -539,10 +515,12 @@ class PysideBuild(_build):
py_prefix = get_config_var("prefix")
if not py_prefix or not os.path.exists(py_prefix):
py_prefix = sys.prefix
+ self.py_prefix = py_prefix
if sys.platform == "win32":
py_scripts_dir = os.path.join(py_prefix, "Scripts")
else:
py_scripts_dir = os.path.join(py_prefix, "bin")
+ self.py_scripts_dir = py_scripts_dir
if py_libdir is None or not os.path.exists(py_libdir):
if sys.platform == "win32":
py_libdir = os.path.join(py_prefix, "libs")
@@ -656,7 +634,7 @@ class PysideBuild(_build):
qt_version = get_qt_version()
# Update the PATH environment variable
- additional_paths = [py_scripts_dir, qt_dir]
+ additional_paths = [self.py_scripts_dir, qt_dir]
# Add Clang to path for Windows.
# Revisit once Clang is bundled with Qt.
@@ -685,19 +663,11 @@ class PysideBuild(_build):
install_dir = os.path.join(script_dir, prefix() + "_install",
"{}".format(build_name))
- # Try to ensure that tools built by this script (such as shiboken2)
- # are found before any that may already be installed on the system.
- update_env_path([os.path.join(install_dir, 'bin')])
-
- # Tell cmake to look here for *.cmake files
- os.environ['CMAKE_PREFIX_PATH'] = install_dir
-
self.make_path = make_path
self.make_generator = make_generator
self.debug = OPTION_DEBUG
self.script_dir = script_dir
- self.pyside_package_dir = os.path.join(self.script_dir,
- pyside_package_dir_name)
+ self.st_build_dir = os.path.join(self.script_dir, self.build_lib)
self.sources_dir = sources_dir
self.build_dir = build_dir
self.install_dir = install_dir
@@ -709,14 +679,61 @@ class PysideBuild(_build):
self.site_packages_dir = get_python_lib(1, 0, prefix=install_dir)
self.build_tests = OPTION_BUILDTESTS
- setuptools_install_prefix = get_python_lib(1)
- if OPTION_FINAL_INSTALL_PREFIX:
- setuptools_install_prefix = OPTION_FINAL_INSTALL_PREFIX
-
# Save the shiboken build dir path for clang deployment
# purposes.
self.shiboken_build_dir = os.path.join(self.build_dir, "shiboken2")
+ self.log_pre_build_info()
+
+ # Prepare folders
+ if not os.path.exists(self.sources_dir):
+ log.info("Creating sources folder {}...".format(self.sources_dir))
+ os.makedirs(self.sources_dir)
+ if not os.path.exists(self.build_dir):
+ log.info("Creating build folder {}...".format(self.build_dir))
+ os.makedirs(self.build_dir)
+ if not os.path.exists(self.install_dir):
+ log.info("Creating install folder {}...".format(self.install_dir))
+ os.makedirs(self.install_dir)
+
+ if not (OPTION_ONLYPACKAGE
+ and not config.is_internal_shiboken_generator_build_and_part_of_top_level_all()):
+ # Build extensions
+ for ext in config.get_buildable_extensions():
+ self.build_extension(ext)
+
+ if OPTION_BUILDTESTS:
+ # we record the latest successful build and note the
+ # build directory for supporting the tests.
+ timestamp = time.strftime('%Y-%m-%d_%H%M%S')
+ build_history = os.path.join(setup_script_dir, 'build_history')
+ unique_dir = os.path.join(build_history, timestamp)
+ os.makedirs(unique_dir)
+ fpath = os.path.join(unique_dir, 'build_dir.txt')
+ with open(fpath, 'w') as f:
+ print(build_dir, file=f)
+ log.info("Created {}".format(build_history))
+
+ if not OPTION_SKIP_PACKAGING:
+ # Build patchelf if needed
+ self.build_patchelf()
+
+ # Prepare packages
+ self.prepare_packages()
+
+ # Build packages
+ _build.run(self)
+ else:
+ log.info("Skipped preparing and building packages.")
+ log.info('*** Build completed')
+
+ def log_pre_build_info(self):
+ if config.is_internal_shiboken_generator_build_and_part_of_top_level_all():
+ return
+
+ setuptools_install_prefix = get_python_lib(1)
+ if OPTION_FINAL_INSTALL_PREFIX:
+ setuptools_install_prefix = OPTION_FINAL_INSTALL_PREFIX
log.info("=" * 30)
log.info("Package version: {}".format(get_package_version()))
log.info("Build type: {}".format(self.build_type))
@@ -726,40 +743,34 @@ class PysideBuild(_build):
log.info("Make generator: {}".format(self.make_generator))
log.info("Make jobs: {}".format(OPTION_JOBS))
log.info("-" * 3)
-
log.info("setup.py directory: {}".format(self.script_dir))
log.info("Build scripts directory: {}".format(build_scripts_dir))
log.info("Sources directory: {}".format(self.sources_dir))
-
log.info(dedent("""
- Building PySide2 will create and touch directories
+ Building {st_package_name} will create and touch directories
in the following order:
make build directory (py*_build/*/*) ->
make install directory (py*_install/*/*) ->
- {} directory (pyside_package/*) ->
setuptools build directory (build/*/*) ->
setuptools install directory
(usually path-installed-python/lib/python*/site-packages/*)
- """).format(pyside_package_dir_name))
-
+ """).format(st_package_name=config.package_name()))
log.info("make build directory: {}".format(self.build_dir))
log.info("make install directory: {}".format(self.install_dir))
- log.info("{} directory: {}".format(pyside_package_dir_name,
- self.pyside_package_dir))
- log.info("setuptools build directory: {}".format(
- os.path.join(self.script_dir, "build")))
- log.info("setuptools install directory: {}".format(
- setuptools_install_prefix))
- log.info("make-installed site-packages directory: {} \n"
- " (only relevant for copying files from "
- "'make install directory' to '{} directory'".format(
- self.site_packages_dir, pyside_package_dir_name))
+ log.info("setuptools build directory: {}".format(self.st_build_dir))
+ log.info("setuptools install directory: {}".format(setuptools_install_prefix))
+ log.info(dedent("""
+ make-installed site-packages directory: {}
+ (only relevant for copying files from 'make install directory'
+ to 'setuptools build directory'
+ """).format(
+ self.site_packages_dir))
log.info("-" * 3)
log.info("Python executable: {}".format(self.py_executable))
log.info("Python includes: {}".format(self.py_include_dir))
log.info("Python library: {}".format(self.py_library))
- log.info("Python prefix: {}".format(py_prefix))
- log.info("Python scripts: {}".format(py_scripts_dir))
+ log.info("Python prefix: {}".format(self.py_prefix))
+ log.info("Python scripts: {}".format(self.py_scripts_dir))
log.info("-" * 3)
log.info("Qt qmake: {}".format(self.qtinfo.qmake_command))
log.info("Qt version: {}".format(self.qtinfo.version))
@@ -772,52 +783,11 @@ class PysideBuild(_build):
if sys.platform == 'darwin':
pyside_macos_deployment_target = (
PysideBuild.macos_pyside_min_deployment_target()
- )
+ )
log.info("MACOSX_DEPLOYMENT_TARGET set to: {}".format(
pyside_macos_deployment_target))
log.info("=" * 30)
- # Prepare folders
- if not os.path.exists(self.sources_dir):
- log.info("Creating sources folder {}...".format(self.sources_dir))
- os.makedirs(self.sources_dir)
- if not os.path.exists(self.build_dir):
- log.info("Creating build folder {}...".format(self.build_dir))
- os.makedirs(self.build_dir)
- if not os.path.exists(self.install_dir):
- log.info("Creating install folder {}...".format(self.install_dir))
- os.makedirs(self.install_dir)
-
- if not OPTION_ONLYPACKAGE:
- # Build extensions
- for ext in contained_modules:
- self.build_extension(ext)
-
- if OPTION_BUILDTESTS:
- # we record the latest successful build and note the
- # build directory for supporting the tests.
- timestamp = time.strftime('%Y-%m-%d_%H%M%S')
- build_history = os.path.join(setup_script_dir, 'build_history')
- unique_dir = os.path.join(build_history, timestamp)
- os.makedirs(unique_dir)
- fpath = os.path.join(unique_dir, 'build_dir.txt')
- with open(fpath, 'w') as f:
- print(build_dir, file=f)
- log.info("Created {}".format(build_history))
-
- if not OPTION_SKIP_PACKAGING:
- # Build patchelf if needed
- self.build_patchelf()
-
- # Prepare packages
- self.prepare_packages()
-
- # Build packages
- _build.run(self)
- else:
- log.info("Skipped preparing and building packages.")
- log.info('*** Build completed')
-
@staticmethod
def macos_qt_min_deployment_target():
target = qtinfo.macos_min_deployment_target
@@ -953,6 +923,17 @@ class PysideBuild(_build):
cmake_cmd.append("-DPYTHON_EXECUTABLE={}".format(self.py_executable))
cmake_cmd.append("-DPYTHON_INCLUDE_DIR={}".format(self.py_include_dir))
cmake_cmd.append("-DPYTHON_LIBRARY={}".format(self.py_library))
+
+ # If a custom shiboken cmake config directory path was provided, pass it to CMake.
+ if OPTION_SHIBOKEN_CONFIG_DIR and config.is_internal_pyside_build():
+ if os.path.exists(OPTION_SHIBOKEN_CONFIG_DIR):
+ log.info("Using custom provided shiboken2 installation: {}"
+ .format(OPTION_SHIBOKEN_CONFIG_DIR))
+ cmake_cmd.append("-DShiboken2_DIR={}".format(OPTION_SHIBOKEN_CONFIG_DIR))
+ else:
+ log.info("Custom provided shiboken2 installation not found. Path given: {}"
+ .format(OPTION_SHIBOKEN_CONFIG_DIR))
+
if OPTION_MODULE_SUBSET:
module_sub_set = ''
for m in OPTION_MODULE_SUBSET.split(','):
@@ -1014,20 +995,20 @@ class PysideBuild(_build):
cmake_cmd.append("-DPYSIDE_QT_CONF_PREFIX={}".format(
pyside_qt_conf_prefix))
- # Pass package version to CMake, so this string can be
- # embedded into _config.py file.
- package_version = get_package_version()
- cmake_cmd.append("-DPYSIDE_SETUP_PY_PACKAGE_VERSION={}".format(
- package_version))
-
- # In case if this is a snapshot build, also pass the
- # timestamp as a separate value, because it the only
- # version component that is actually generated by setup.py.
- timestamp = ''
- if OPTION_SNAPSHOT_BUILD:
- timestamp = get_package_timestamp()
- cmake_cmd.append("-DPYSIDE_SETUP_PY_PACKAGE_TIMESTAMP={}".format(
- timestamp))
+ # Pass package version to CMake, so this string can be
+ # embedded into _config.py file.
+ package_version = get_package_version()
+ cmake_cmd.append("-DPACKAGE_SETUP_PY_PACKAGE_VERSION={}".format(
+ package_version))
+
+ # In case if this is a snapshot build, also pass the
+ # timestamp as a separate value, because it is the only
+ # version component that is actually generated by setup.py.
+ timestamp = ''
+ if OPTION_SNAPSHOT_BUILD:
+ timestamp = get_package_timestamp()
+ cmake_cmd.append("-DPACKAGE_SETUP_PY_PACKAGE_TIMESTAMP={}".format(
+ timestamp))
if extension.lower() in ["shiboken2", "pyside2-tools"]:
cmake_cmd.append("-DCMAKE_INSTALL_RPATH_USE_LINK_PATH=yes")
@@ -1129,15 +1110,24 @@ class PysideBuild(_build):
os.chdir(self.script_dir)
def prepare_packages(self):
+ """
+ This will copy all relevant files from the various locations in the "cmake install dir",
+ to the setup tools build dir (which is read from self.build_lib provided by distutils).
+
+ After that setuptools.command.build_py is smart enough to copy everything
+ from the build dir to the install dir (the virtualenv site-packages for example).
+ """
try:
- log.info("Preparing packages...")
+ log.info("\nPreparing setup tools build directory.\n")
vars = {
"site_packages_dir": self.site_packages_dir,
"sources_dir": self.sources_dir,
"install_dir": self.install_dir,
"build_dir": self.build_dir,
"script_dir": self.script_dir,
- "pyside_package_dir": self.pyside_package_dir,
+ "st_build_dir": self.st_build_dir,
+ "cmake_package_name": config.package_name(),
+ "st_package_name": config.package_name(),
"ssl_libs_dir": OPTION_OPENSSL,
"py_version": self.py_version,
"qt_version": self.qtinfo.version,
@@ -1151,6 +1141,12 @@ class PysideBuild(_build):
"qt_qml_dir": self.qtinfo.qml_dir,
"target_arch": self.py_arch,
}
+
+ # Needed for correct file installation in generator build
+ # case.
+ if config.is_internal_shiboken_generator_build():
+ vars['cmake_package_name'] = config.shiboken_module_option_name
+
os.chdir(self.script_dir)
if sys.platform == "win32":
@@ -1170,19 +1166,21 @@ class PysideBuild(_build):
def get_built_pyside_config(self, vars):
# Get config that contains list of built modules, and
# SOVERSIONs of the built libraries.
- pyside_package_dir = vars['pyside_package_dir']
- config_path = os.path.join(pyside_package_dir, "PySide2", "_config.py")
- config = get_python_dict(config_path)
- return config
+ st_build_dir = vars['st_build_dir']
+ config_path = os.path.join(st_build_dir, config.package_name(), "_config.py")
+ temp_config = get_python_dict(config_path)
+ if 'built_modules' not in temp_config:
+ temp_config['built_modules'] = []
+ return temp_config
def is_webengine_built(self, built_modules):
return ('WebEngineWidgets' in built_modules or 'WebEngineCore' in built_modules
or 'WebEngine' in built_modules)
- def prepare_standalone_clang(self, is_win = False):
+ def prepare_standalone_clang(self, is_win=False):
"""
- Copies the libclang library to the pyside package so that
- shiboken executable works.
+ Copies the libclang library to the shiboken2-generator
+ package so that the shiboken executable works.
"""
log.info('Finding path to the libclang shared library.')
cmake_cmd = [
@@ -1205,47 +1203,54 @@ class PysideBuild(_build):
if not clang_lib_path:
raise RuntimeError("Could not find the location of the libclang "
- "library inside the CMake cache file.")
+ "library inside the CMake cache file.")
- target_name = None
if is_win:
# clang_lib_path points to the static import library
# (lib/libclang.lib), whereas we want to copy the shared
# library (bin/libclang.dll).
- clang_lib_path = re.sub(r'lib/libclang.lib$', 'bin/libclang.dll',
- clang_lib_path)
+ clang_lib_path = re.sub(r'lib/libclang.lib$',
+ 'bin/libclang.dll',
+ clang_lib_path)
else:
- if sys.platform != 'darwin' and os.path.islink(clang_lib_path):
- # On Linux, we get "libclang.so" from CMake which is
- # a symlink:
- # libclang.so -> libclang.so.6 -> libclang.so.6.0.
- # shiboken2 links against libclang.so.6. So, we
- # determine the target name by resolving just
- # one symlink (note: os.path.realpath() resolves all).
- target_name = os.readlink(clang_lib_path)
- # We want to resolve any symlink on Linux and macOS, and
- # copy the actual file.
- clang_lib_path = os.path.realpath(clang_lib_path)
-
- if not target_name:
- target_name = os.path.basename(clang_lib_path)
-
- # Path to directory containing libclang.
- clang_lib_dir = os.path.dirname(clang_lib_path)
-
- # The destination will be the package folder near the other
- # extension modules.
- destination_dir = "{}/PySide2".format(os.path.join(self.script_dir,
- 'pyside_package'))
+ # shiboken2 links against libclang.so.6 or a similarly
+ # named library.
+ # If the linked against library is a symlink, resolve
+ # the symlink once (but not all the way to the real
+ # file) on Linux and macOS,
+ # so that we get the path to the "SO version" symlink
+ # (the one used as the install name in the shared library
+ # dependency section).
+ # E.g. On Linux libclang.so -> libclang.so.6 ->
+ # libclang.so.6.0.
+ # "libclang.so.6" is the name we want for the copied file.
+ if os.path.islink(clang_lib_path):
+ link_target = os.readlink(clang_lib_path)
+ if os.path.isabs(link_target):
+ clang_lib_path = link_target
+ else:
+ # link_target is relative, transform to absolute.
+ clang_lib_path = os.path.join(os.path.dirname(clang_lib_path), link_target)
+ clang_lib_path = os.path.abspath(clang_lib_path)
+
+ # The destination will be the shiboken package folder.
+ vars = {}
+ vars['st_build_dir'] = self.st_build_dir
+ vars['st_package_name'] = config.package_name()
+ destination_dir = "{st_build_dir}/{st_package_name}".format(**vars)
+
if os.path.exists(clang_lib_path):
- log.info('Copying libclang shared library {} to the package folder as {}.'.format(
- clang_lib_path, target_name))
basename = os.path.basename(clang_lib_path)
- destination_path = os.path.join(destination_dir, target_name)
+ log.info('Copying libclang shared library {} to the package folder as {}.'.format(
+ clang_lib_path, basename))
+ destination_path = os.path.join(destination_dir, basename)
# Need to modify permissions in case file is not writable
# (a reinstall would cause a permission denied error).
- copyfile(clang_lib_path, destination_path, make_writable_by_owner=True)
+ copyfile(clang_lib_path,
+ destination_path,
+ force_copy_symlink=True,
+ make_writable_by_owner=True)
else:
raise RuntimeError("Error copying libclang library "
"from {} to {}. ".format(
@@ -1265,18 +1270,17 @@ class PysideBuild(_build):
else:
# Add rpath values pointing to $ORIGIN and the
# installed qt lib directory.
- local_rpath = '$ORIGIN/'
- qt_lib_dir = self.qtinfo.libs_dir
+ final_rpath = self.qtinfo.libs_dir
if OPTION_STANDALONE:
- qt_lib_dir = "$ORIGIN/Qt/lib"
- final_rpath = local_rpath + ':' + qt_lib_dir
- cmd = [self._patchelf_path, '--set-rpath', final_rpath, srcpath]
- if run_process(cmd) != 0:
- raise RuntimeError("Error patching rpath in " + srcpath)
+ final_rpath = "$ORIGIN/Qt/lib"
+ override = OPTION_STANDALONE
+ linux_fix_rpaths_for_library(self._patchelf_path, srcpath, final_rpath,
+ override=override)
elif sys.platform == 'darwin':
pyside_libs = [lib for lib in os.listdir(
package_path) if filter_match(lib, ["*.so", "*.dylib"])]
+
def rpath_cmd(srcpath):
final_rpath = ''
# Command line rpath option takes precedence over
@@ -1308,15 +1312,6 @@ class PysideBuild(_build):
"updated rpath (OS/X) in {}.".format(srcpath))
-try:
- with open(os.path.join(setup_script_dir, 'README.rst')) as f:
- README = f.read()
- with open(os.path.join(setup_script_dir, 'CHANGES.rst')) as f:
- CHANGES = f.read()
-except IOError:
- README = CHANGES = ''
-
-
cmd_class_dict = {
'build': PysideBuild,
'build_py': PysideBuildPy,
diff --git a/build_scripts/options.py b/build_scripts/options.py
index fd8b0718e..daf3bb00e 100644
--- a/build_scripts/options.py
+++ b/build_scripts/options.py
@@ -38,16 +38,88 @@
#############################################################################
from __future__ import print_function
+import sys
+import os
+
+
+class Options(object):
+ def __init__(self):
+
+ # Dictionary containing values of all the possible options.
+ self.dict = {}
+
+ def has_option(self, name):
+ """ Returns True if argument '--name' was passed on the command
+ line. """
+ try:
+ sys.argv.remove("--{}".format(name))
+ self.dict[name] = True
+ return True
+ except ValueError:
+ pass
+ return False
+
+ def option_value(self, name, remove=True):
+ """
+ Returns the value of a command line option or environment
+ variable.
+
+ :param name: The name of the command line option or environment
+ variable.
+
+ :param remove: Whether the option and its value should be
+ removed from sys.argv. Useful when there's a need to query for
+ the value and also pass it along to setuptools for example.
+
+ :return: Either the option value or None.
+ """
+ for index, option in enumerate(sys.argv):
+ if option == '--' + name:
+ if index + 1 >= len(sys.argv):
+ raise RuntimeError("The option {} requires a value".format(option))
+ value = sys.argv[index + 1]
+
+ if remove:
+ sys.argv[index:index + 2] = []
+
+ self.dict[name] = value
+ return value
+
+ if option.startswith('--' + name + '='):
+ value = option[len(name) + 3:]
+
+ if remove:
+ sys.argv[index:index + 1] = []
+
+ self.dict[name] = value
+ return value
+
+ env_val = os.getenv(name.upper().replace('-', '_'))
+ self.dict[name] = env_val
+ return env_val
+
+
+options = Options()
+
+
+def has_option(name):
+ return options.has_option(name)
+
+
+def option_value(*args,**kwargs):
+ return options.option_value(*args,**kwargs)
-from .utils import has_option, option_value
# Declare options
+OPTION_BUILD_TYPE = option_value("build-type")
+OPTION_INTERNAL_BUILD_TYPE = option_value("internal-build-type")
OPTION_DEBUG = has_option("debug")
OPTION_RELWITHDEBINFO = has_option('relwithdebinfo')
OPTION_QMAKE = option_value("qmake")
OPTION_QT_VERSION = option_value("qt")
OPTION_CMAKE = option_value("cmake")
OPTION_OPENSSL = option_value("openssl")
+OPTION_SHIBOKEN_CONFIG_DIR = option_value("shiboken-config-dir")
OPTION_ONLYPACKAGE = has_option("only-package")
OPTION_STANDALONE = has_option("standalone")
OPTION_MAKESPEC = option_value("make-spec")
@@ -82,3 +154,7 @@ OPTION_SANITIZE_ADDRESS = has_option("sanitize-address")
OPTION_SNAPSHOT_BUILD = has_option("snapshot-build")
OPTION_LIMITED_API = option_value("limited-api")
OPTION_PACKAGE_TIMESTAMP = option_value("package-timestamp")
+
+# This is used automatically by distutils.command.install object, to
+# specify the final installation location.
+OPTION_FINAL_INSTALL_PREFIX = option_value("prefix", remove=False)
diff --git a/build_scripts/platforms/linux.py b/build_scripts/platforms/linux.py
index 4c38fef6c..067179cdc 100644
--- a/build_scripts/platforms/linux.py
+++ b/build_scripts/platforms/linux.py
@@ -37,76 +37,100 @@
##
#############################################################################
-from ..options import *
from ..utils import copydir, copyfile, copy_icu_libs, find_files_using_glob
+from ..config import config
-def prepare_standalone_package_linux(self, executables, vars):
+
+def prepare_standalone_package_linux(self, vars):
built_modules = vars['built_modules']
- # <qt>/lib/* -> <setup>/PySide2/Qt/lib
- destination_lib_dir = "{pyside_package_dir}/PySide2/Qt/lib"
+ constrain_modules = None
+ copy_plugins = True
+ copy_qml = True
+ copy_translations = True
+ copy_qt_conf = True
+ should_copy_icu_libs = True
+
+ if config.is_internal_shiboken_generator_build():
+ constrain_modules = ["Core", "Network", "Xml", "XmlPatterns"]
+ copy_plugins = False
+ copy_qml = False
+ copy_translations = False
+ copy_qt_conf = False
+ should_copy_icu_libs = False
+
+ # <qt>/lib/* -> <setup>/{st_package_name}/Qt/lib
+ destination_lib_dir = "{st_build_dir}/{st_package_name}/Qt/lib"
+
+ accepted_modules = ['libQt5*.so.?']
+ if constrain_modules:
+ accepted_modules = ["libQt5" + module + "*.so.?" for module in constrain_modules]
+ accepted_modules.append("libicu*.so.??")
+
copydir("{qt_lib_dir}", destination_lib_dir,
- filter=[
- "libQt5*.so.?",
- "libicu*.so.??",
- ],
- recursive=False, vars=vars, force_copy_symlinks=True)
-
- # Check if ICU libraries were copied over to the destination
- # Qt libdir.
- resolved_destination_lib_dir = destination_lib_dir.format(**vars)
- maybe_icu_libs = find_files_using_glob(resolved_destination_lib_dir,
- "libicu*")
-
- # If no ICU libraries are present in the Qt libdir (like when
- # Qt is built against system ICU, or in the Coin CI where ICU
- # libs are in a different directory) try to find out / resolve
- # which ICU libs are used by QtCore (if used at all) using a
- # custom written ldd, and copy the ICU libs to the Pyside Qt
- # dir if necessary. We choose the QtCore lib to inspect, by
- # checking which QtCore library the shiboken2 executable uses.
- if not maybe_icu_libs:
- copy_icu_libs(self._patchelf_path, resolved_destination_lib_dir)
+ filter=accepted_modules,
+ recursive=False, vars=vars, force_copy_symlinks=True)
+
+ if should_copy_icu_libs:
+ # Check if ICU libraries were copied over to the destination
+ # Qt libdir.
+ resolved_destination_lib_dir = destination_lib_dir.format(**vars)
+ maybe_icu_libs = find_files_using_glob(resolved_destination_lib_dir,
+ "libicu*")
+
+ # If no ICU libraries are present in the Qt libdir (like when
+ # Qt is built against system ICU, or in the Coin CI where ICU
+ # libs are in a different directory) try to find out / resolve
+ # which ICU libs are used by QtCore (if used at all) using a
+ # custom written ldd, and copy the ICU libs to the Pyside Qt
+ # dir if necessary. We choose the QtCore lib to inspect, by
+ # checking which QtCore library the shiboken2 executable uses.
+ if not maybe_icu_libs:
+ copy_icu_libs(self._patchelf_path, resolved_destination_lib_dir)
if self.is_webengine_built(built_modules):
copydir("{qt_lib_execs_dir}",
- "{pyside_package_dir}/PySide2/Qt/libexec",
+ "{st_build_dir}/{st_package_name}/Qt/libexec",
filter=None,
recursive=False,
vars=vars)
copydir("{qt_prefix_dir}/resources",
- "{pyside_package_dir}/PySide2/Qt/resources",
+ "{st_build_dir}/{st_package_name}/Qt/resources",
filter=None,
recursive=False,
vars=vars)
- # <qt>/plugins/* -> <setup>/PySide2/Qt/plugins
- copydir("{qt_plugins_dir}",
- "{pyside_package_dir}/PySide2/Qt/plugins",
- filter=["*.so"],
- recursive=True,
- vars=vars)
-
- # <qt>/qml/* -> <setup>/PySide2/Qt/qml
- copydir("{qt_qml_dir}",
- "{pyside_package_dir}/PySide2/Qt/qml",
- filter=None,
- force=False,
- recursive=True,
- ignore=["*.so.debug"],
- vars=vars)
-
- # <qt>/translations/* -> <setup>/PySide2/Qt/translations
-
- copydir("{qt_translations_dir}",
- "{pyside_package_dir}/PySide2/Qt/translations",
- filter=["*.qm", "*.pak"],
- force=False,
- vars=vars)
-
- # Copy the qt.conf file to libexec.
- copyfile(
- "{build_dir}/pyside2/PySide2/qt.conf",
- "{pyside_package_dir}/PySide2/Qt/libexec",
- vars=vars)
+ if copy_plugins:
+ # <qt>/plugins/* -> <setup>/{st_package_name}/Qt/plugins
+ copydir("{qt_plugins_dir}",
+ "{st_build_dir}/{st_package_name}/Qt/plugins",
+ filter=["*.so"],
+ recursive=True,
+ vars=vars)
+
+ if copy_qml:
+ # <qt>/qml/* -> <setup>/{st_package_name}/Qt/qml
+ copydir("{qt_qml_dir}",
+ "{st_build_dir}/{st_package_name}/Qt/qml",
+ filter=None,
+ force=False,
+ recursive=True,
+ ignore=["*.so.debug"],
+ vars=vars)
+
+ if copy_translations:
+ # <qt>/translations/* ->
+ # <setup>/{st_package_name}/Qt/translations
+ copydir("{qt_translations_dir}",
+ "{st_build_dir}/{st_package_name}/Qt/translations",
+ filter=["*.qm", "*.pak"],
+ force=False,
+ vars=vars)
+
+ if copy_qt_conf:
+ # Copy the qt.conf file to libexec.
+ copyfile(
+ "{build_dir}/pyside2/{st_package_name}/qt.conf",
+ "{st_build_dir}/{st_package_name}/Qt/libexec",
+ vars=vars)
diff --git a/build_scripts/platforms/macos.py b/build_scripts/platforms/macos.py
index 936f4ca90..49f02754d 100644
--- a/build_scripts/platforms/macos.py
+++ b/build_scripts/platforms/macos.py
@@ -37,12 +37,29 @@
##
#############################################################################
-import fnmatch, os
+import fnmatch
+import os
from ..utils import copydir, copyfile, macos_fix_rpaths_for_library
+from ..config import config
-def prepare_standalone_package_macos(self, executables, vars):
+
+def prepare_standalone_package_macos(self, vars):
built_modules = vars['built_modules']
+ constrain_modules = None
+ copy_plugins = True
+ copy_qml = True
+ copy_translations = True
+ copy_qt_conf = True
+
+ if config.is_internal_shiboken_generator_build():
+ constrain_modules = ["Core", "Network", "Xml", "XmlPatterns"]
+ constrain_frameworks = ['Qt' + name + '.framework' for name in constrain_modules]
+ copy_plugins = False
+ copy_qml = False
+ copy_translations = False
+ copy_qt_conf = False
+
# Directory filter for skipping unnecessary files.
def general_dir_filter(dir_name, parent_full_path, dir_full_path):
if fnmatch.fnmatch(dir_name, "*.dSYM"):
@@ -52,6 +69,7 @@ def prepare_standalone_package_macos(self, executables, vars):
# Filter out debug plugins and qml plugins in the
# debug_and_release config.
no_copy_debug = True
+
def file_variant_filter(file_name, file_full_path):
if self.qtinfo.build_type != 'debug_and_release':
return True
@@ -59,17 +77,16 @@ def prepare_standalone_package_macos(self, executables, vars):
return False
return True
- # <qt>/lib/* -> <setup>/PySide2/Qt/lib
+ # <qt>/lib/* -> <setup>/{st_package_name}/Qt/lib
if self.qt_is_framework_build():
- framework_built_modules = [
- 'Qt' + name + '.framework' for name in built_modules]
-
- def framework_dir_filter(dir_name, parent_full_path,
- dir_full_path):
+ def framework_dir_filter(dir_name, parent_full_path, dir_full_path):
if '.framework' in dir_name:
- if dir_name.startswith('QtWebEngine') and not \
- self.is_webengine_built(built_modules):
+ if (dir_name.startswith('QtWebEngine') and
+ not self.is_webengine_built(built_modules)):
+ return False
+ if constrain_modules and dir_name not in constrain_frameworks:
return False
+
if dir_name in ['Headers', 'fonts']:
return False
if dir_full_path.endswith('Versions/Current'):
@@ -84,6 +101,7 @@ def prepare_standalone_package_macos(self, executables, vars):
# Filter out debug frameworks in the
# debug_and_release config.
no_copy_debug = True
+
def framework_variant_filter(file_name, file_full_path):
if self.qtinfo.build_type != 'debug_and_release':
return True
@@ -93,7 +111,7 @@ def prepare_standalone_package_macos(self, executables, vars):
return False
return True
- copydir("{qt_lib_dir}", "{pyside_package_dir}/PySide2/Qt/lib",
+ copydir("{qt_lib_dir}", "{st_build_dir}/{st_package_name}/Qt/lib",
recursive=True, vars=vars,
ignore=["*.la", "*.a", "*.cmake", "*.pc", "*.prl"],
dir_filter_function=framework_dir_filter,
@@ -104,7 +122,7 @@ def prepare_standalone_package_macos(self, executables, vars):
# from Versions/5/Helpers, thus adding two more levels of
# directory hierarchy.
if self.is_webengine_built(built_modules):
- qt_lib_path = "{pyside_package_dir}/PySide2/Qt/lib".format(
+ qt_lib_path = "{st_build_dir}/{st_package_name}/Qt/lib".format(
**vars)
bundle = "QtWebEngineCore.framework/Helpers/"
bundle += "QtWebEngineProcess.app"
@@ -120,9 +138,11 @@ def prepare_standalone_package_macos(self, executables, vars):
if 'WebKit' not in built_modules:
ignored_modules.extend(['libQt5WebKit*.dylib'])
accepted_modules = ['libQt5*.5.dylib']
+ if constrain_modules:
+ accepted_modules = ["libQt5" + module + "*.5.dylib" for module in constrain_modules]
copydir("{qt_lib_dir}",
- "{pyside_package_dir}/PySide2/Qt/lib",
+ "{st_build_dir}/{st_package_name}/Qt/lib",
filter=accepted_modules,
ignore=ignored_modules,
file_filter_function=file_variant_filter,
@@ -130,53 +150,58 @@ def prepare_standalone_package_macos(self, executables, vars):
if self.is_webengine_built(built_modules):
copydir("{qt_lib_execs_dir}",
- "{pyside_package_dir}/PySide2/Qt/libexec",
+ "{st_build_dir}/{st_package_name}/Qt/libexec",
filter=None,
recursive=False,
vars=vars)
copydir("{qt_prefix_dir}/resources",
- "{pyside_package_dir}/PySide2/Qt/resources",
+ "{st_build_dir}/{st_package_name}/Qt/resources",
filter=None,
recursive=False,
vars=vars)
# Fix rpath for WebEngine process executable.
- pyside_package_dir = vars['pyside_package_dir']
- qt_libexec_path = "{}/PySide2/Qt/libexec".format(pyside_package_dir)
+ qt_libexec_path = "{st_build_dir}/{st_package_name}/Qt/libexec".format(**vars)
binary = "QtWebEngineProcess"
final_path = os.path.join(qt_libexec_path, binary)
rpath = "@loader_path/../lib"
macos_fix_rpaths_for_library(final_path, rpath)
- # Copy the qt.conf file to libexec.
- copyfile(
- "{build_dir}/pyside2/PySide2/qt.conf",
- "{pyside_package_dir}/PySide2/Qt/libexec",
- vars=vars)
+ if copy_qt_conf:
+ # Copy the qt.conf file to libexec.
+ copyfile(
+ "{build_dir}/pyside2/{st_package_name}/qt.conf",
+ "{st_build_dir}/{st_package_name}/Qt/libexec",
+ vars=vars)
+
+ if copy_plugins:
+ # <qt>/plugins/* -> <setup>/{st_package_name}/Qt/plugins
+ copydir("{qt_plugins_dir}",
+ "{st_build_dir}/{st_package_name}/Qt/plugins",
+ filter=["*.dylib"],
+ recursive=True,
+ dir_filter_function=general_dir_filter,
+ file_filter_function=file_variant_filter,
+ vars=vars)
+
- # <qt>/plugins/* -> <setup>/PySide2/Qt/plugins
- copydir("{qt_plugins_dir}",
- "{pyside_package_dir}/PySide2/Qt/plugins",
- filter=["*.dylib"],
- recursive=True,
- dir_filter_function=general_dir_filter,
- file_filter_function=file_variant_filter,
- vars=vars)
-
- # <qt>/qml/* -> <setup>/PySide2/Qt/qml
- copydir("{qt_qml_dir}",
- "{pyside_package_dir}/PySide2/Qt/qml",
- filter=None,
- recursive=True,
- force=False,
- dir_filter_function=general_dir_filter,
- file_filter_function=file_variant_filter,
- vars=vars)
-
- # <qt>/translations/* -> <setup>/PySide2/Qt/translations
- copydir("{qt_translations_dir}",
- "{pyside_package_dir}/PySide2/Qt/translations",
- filter=["*.qm", "*.pak"],
- force=False,
- vars=vars)
+ if copy_qml:
+ # <qt>/qml/* -> <setup>/{st_package_name}/Qt/qml
+ copydir("{qt_qml_dir}",
+ "{st_build_dir}/{st_package_name}/Qt/qml",
+ filter=None,
+ recursive=True,
+ force=False,
+ dir_filter_function=general_dir_filter,
+ file_filter_function=file_variant_filter,
+ vars=vars)
+
+ if copy_translations:
+ # <qt>/translations/* ->
+ # <setup>/{st_package_name}/Qt/translations
+ copydir("{qt_translations_dir}",
+ "{st_build_dir}/{st_package_name}/Qt/translations",
+ filter=["*.qm", "*.pak"],
+ force=False,
+ vars=vars)
diff --git a/build_scripts/platforms/unix.py b/build_scripts/platforms/unix.py
index e7fa92390..7dce11612 100644
--- a/build_scripts/platforms/unix.py
+++ b/build_scripts/platforms/unix.py
@@ -37,73 +37,31 @@
##
#############################################################################
-import os, re, sys
+import os
+import sys
from .linux import prepare_standalone_package_linux
from .macos import prepare_standalone_package_macos
+
+from ..config import config
from ..options import *
from ..utils import copydir, copyfile, rmtree, makefile
from ..utils import regenerate_qt_resources
+
def prepare_packages_posix(self, vars):
executables = []
- # <build>/shiboken2/doc/html/* ->
- # <setup>/PySide2/docs/shiboken2
- copydir(
- "{build_dir}/shiboken2/doc/html",
- "{pyside_package_dir}/PySide2/docs/shiboken2",
- force=False, vars=vars)
- # <install>/lib/site-packages/PySide2/* -> <setup>/PySide2
- copydir(
- "{site_packages_dir}/PySide2",
- "{pyside_package_dir}/PySide2",
- vars=vars)
- # <install>/lib/site-packages/shiboken2.so ->
- # <setup>/PySide2/shiboken2.so
- shiboken_module_name = 'shiboken2.so'
- shiboken_src_path = "{site_packages_dir}".format(**vars)
- maybe_shiboken_names = [f for f in os.listdir(shiboken_src_path)
- if re.match(r'shiboken.*\.so', f)]
- if maybe_shiboken_names:
- shiboken_module_name = maybe_shiboken_names[0]
- vars.update({'shiboken_module_name': shiboken_module_name})
- copyfile(
- "{site_packages_dir}/{shiboken_module_name}",
- "{pyside_package_dir}/PySide2/{shiboken_module_name}",
- vars=vars)
- # <install>/lib/site-packages/pyside2uic/* ->
- # <setup>/pyside2uic
+
+ # <install>/lib/site-packages/{st_package_name}/* ->
+ # <setup>/{st_package_name}
+ # This copies the module .so/.dylib files and various .py files
+ # (__init__, config, git version, etc.)
copydir(
- "{site_packages_dir}/pyside2uic",
- "{pyside_package_dir}/pyside2uic",
- force=False, vars=vars)
- if sys.version_info[0] > 2:
- rmtree("{pyside_package_dir}/pyside2uic/port_v2".format(**vars))
- else:
- rmtree("{pyside_package_dir}/pyside2uic/port_v3".format(**vars))
- # <install>/bin/pyside2-uic -> PySide2/scripts/uic.py
- makefile(
- "{pyside_package_dir}/PySide2/scripts/__init__.py",
+ "{site_packages_dir}/{st_package_name}",
+ "{st_build_dir}/{st_package_name}",
vars=vars)
- copyfile(
- "{install_dir}/bin/pyside2-uic",
- "{pyside_package_dir}/PySide2/scripts/uic.py",
- force=False, vars=vars)
- copyfile(
- "{install_dir}/bin/pyside_tool.py",
- "{pyside_package_dir}/PySide2/scripts/pyside_tool.py",
- force=False, vars=vars)
- # <install>/bin/* -> PySide2/
- executables.extend(copydir(
- "{install_dir}/bin/",
- "{pyside_package_dir}/PySide2",
- filter=[
- "pyside2-lupdate",
- "pyside2-rcc",
- "shiboken2",
- ],
- recursive=False, vars=vars))
- # <install>/lib/lib* -> PySide2/
- config = self.get_built_pyside_config(vars)
+
+ generated_config = self.get_built_pyside_config(vars)
+
def adjusted_lib_name(name, version):
postfix = ''
if sys.platform.startswith('linux'):
@@ -111,61 +69,144 @@ def prepare_packages_posix(self, vars):
elif sys.platform == 'darwin':
postfix = '.' + version + '.dylib'
return name + postfix
- copydir(
- "{install_dir}/lib/",
- "{pyside_package_dir}/PySide2",
- filter=[
- adjusted_lib_name("libpyside*",
- config['pyside_library_soversion']),
- adjusted_lib_name("libshiboken*",
- config['shiboken_library_soversion']),
- ],
- recursive=False, vars=vars, force_copy_symlinks=True)
- # <install>/share/PySide2/typesystems/* ->
- # <setup>/PySide2/typesystems
- copydir(
- "{install_dir}/share/PySide2/typesystems",
- "{pyside_package_dir}/PySide2/typesystems",
- vars=vars)
- # <install>/include/* -> <setup>/PySide2/include
- copydir(
- "{install_dir}/include",
- "{pyside_package_dir}/PySide2/include",
- vars=vars)
- # <source>/pyside2/PySide2/support/* ->
- # <setup>/PySide2/support/*
- copydir(
- "{build_dir}/pyside2/PySide2/support",
- "{pyside_package_dir}/PySide2/support",
- vars=vars)
- if not OPTION_NOEXAMPLES:
- # examples/* -> <setup>/PySide2/examples
- copydir(os.path.join(self.script_dir, "examples"),
- "{pyside_package_dir}/PySide2/examples",
- force=False, vars=vars)
- # Re-generate examples Qt resource files for Python 3
- # compatibility
- if sys.version_info[0] == 3:
- examples_path = "{pyside_package_dir}/PySide2/examples".format(
- **vars)
- pyside_rcc_path = "{install_dir}/bin/pyside2-rcc".format(
- **vars)
- pyside_rcc_options = '-py3'
- regenerate_qt_resources(examples_path, pyside_rcc_path,
- pyside_rcc_options)
+
+ if config.is_internal_shiboken_module_build():
+ # <build>/shiboken2/doc/html/* ->
+ # <setup>/{st_package_name}/docs/shiboken2
+ copydir(
+ "{build_dir}/shiboken2/doc/html",
+ "{st_build_dir}/{st_package_name}/docs/shiboken2",
+ force=False, vars=vars)
+
+ # <install>/lib/lib* -> {st_package_name}/
+ copydir(
+ "{install_dir}/lib/",
+ "{st_build_dir}/{st_package_name}",
+ filter=[
+ adjusted_lib_name("libshiboken*",
+ generated_config['shiboken_library_soversion']),
+ ],
+ recursive=False, vars=vars, force_copy_symlinks=True)
+
+ if config.is_internal_shiboken_generator_build():
+ # <install>/bin/* -> {st_package_name}/
+ executables.extend(copydir(
+ "{install_dir}/bin/",
+ "{st_build_dir}/{st_package_name}",
+ filter=[
+ "shiboken2",
+ ],
+ recursive=False, vars=vars))
+
+ # Used to create scripts directory.
+ makefile(
+ "{st_build_dir}/{st_package_name}/scripts/shiboken_tool.py",
+ vars=vars)
+
+ # For setting up setuptools entry points.
+ copyfile(
+ "{install_dir}/bin/shiboken_tool.py",
+ "{st_build_dir}/{st_package_name}/scripts/shiboken_tool.py",
+ force=False, vars=vars)
+
+ if config.is_internal_shiboken_generator_build() or config.is_internal_pyside_build():
+ # <install>/include/* -> <setup>/{st_package_name}/include
+ copydir(
+ "{install_dir}/include/{cmake_package_name}",
+ "{st_build_dir}/{st_package_name}/include",
+ vars=vars)
+
+ if config.is_internal_pyside_build():
+ # <install>/lib/site-packages/pyside2uic/* ->
+ # <setup>/pyside2uic
+ copydir(
+ "{site_packages_dir}/pyside2uic",
+ "{st_build_dir}/pyside2uic",
+ force=False, vars=vars)
+ if sys.version_info[0] > 2:
+ rmtree("{st_build_dir}/pyside2uic/port_v2".format(**vars))
+ else:
+ rmtree("{st_build_dir}/pyside2uic/port_v3".format(**vars))
+
+ # <install>/bin/pyside2-uic -> {st_package_name}/scripts/uic.py
+ makefile(
+ "{st_build_dir}/{st_package_name}/scripts/__init__.py",
+ vars=vars)
+ copyfile(
+ "{install_dir}/bin/pyside2-uic",
+ "{st_build_dir}/{st_package_name}/scripts/uic.py",
+ force=False, vars=vars)
+
+ # For setting up setuptools entry points
+ copyfile(
+ "{install_dir}/bin/pyside_tool.py",
+ "{st_build_dir}/{st_package_name}/scripts/pyside_tool.py",
+ force=False, vars=vars)
+
+ # <install>/bin/* -> {st_package_name}/
+ executables.extend(copydir(
+ "{install_dir}/bin/",
+ "{st_build_dir}/{st_package_name}",
+ filter=[
+ "pyside2-lupdate",
+ "pyside2-rcc",
+ ],
+ recursive=False, vars=vars))
+
+ # <install>/lib/lib* -> {st_package_name}/
+ copydir(
+ "{install_dir}/lib/",
+ "{st_build_dir}/{st_package_name}",
+ filter=[
+ adjusted_lib_name("libpyside*",
+ generated_config['pyside_library_soversion']),
+ ],
+ recursive=False, vars=vars, force_copy_symlinks=True)
+
+ # <install>/share/{st_package_name}/typesystems/* ->
+ # <setup>/{st_package_name}/typesystems
+ copydir(
+ "{install_dir}/share/{st_package_name}/typesystems",
+ "{st_build_dir}/{st_package_name}/typesystems",
+ vars=vars)
+
+ # <source>/pyside2/{st_package_name}/support/* ->
+ # <setup>/{st_package_name}/support/*
+ copydir(
+ "{build_dir}/pyside2/{st_package_name}/support",
+ "{st_build_dir}/{st_package_name}/support",
+ vars=vars)
+
+ if not OPTION_NOEXAMPLES:
+ # examples/* -> <setup>/{st_package_name}/examples
+ copydir(os.path.join(self.script_dir, "examples"),
+ "{st_build_dir}/{st_package_name}/examples",
+ force=False, vars=vars)
+ # Re-generate examples Qt resource files for Python 3
+ # compatibility
+ if sys.version_info[0] == 3:
+ examples_path = "{st_build_dir}/{st_package_name}/examples".format(
+ **vars)
+ pyside_rcc_path = "{install_dir}/bin/pyside2-rcc".format(
+ **vars)
+ pyside_rcc_options = '-py3'
+ regenerate_qt_resources(examples_path, pyside_rcc_path,
+ pyside_rcc_options)
+
# Copy Qt libs to package
if OPTION_STANDALONE:
- vars['built_modules'] = config['built_modules']
- if sys.platform == 'darwin':
- prepare_standalone_package_macos(self, executables, vars)
- else:
- prepare_standalone_package_linux(self, executables, vars)
+ if config.is_internal_pyside_build() or config.is_internal_shiboken_generator_build():
+ vars['built_modules'] = generated_config['built_modules']
+ if sys.platform == 'darwin':
+ prepare_standalone_package_macos(self, vars)
+ else:
+ prepare_standalone_package_linux(self, vars)
- # Copy over clang before rpath patching.
- self.prepare_standalone_clang(is_win=False)
+ if config.is_internal_shiboken_generator_build():
+ # Copy over clang before rpath patching.
+ self.prepare_standalone_clang(is_win=False)
# Update rpath to $ORIGIN
- if (sys.platform.startswith('linux') or
- sys.platform.startswith('darwin')):
- self.update_rpath("{pyside_package_dir}/PySide2".format(**vars),
- executables)
+ if sys.platform.startswith('linux') or sys.platform.startswith('darwin'):
+ rpath_path = "{st_build_dir}/{st_package_name}".format(**vars)
+ self.update_rpath(rpath_path, executables)
diff --git a/build_scripts/platforms/windows_desktop.py b/build_scripts/platforms/windows_desktop.py
index aa9487ec6..6307238a1 100644
--- a/build_scripts/platforms/windows_desktop.py
+++ b/build_scripts/platforms/windows_desktop.py
@@ -38,153 +38,225 @@
#############################################################################
import functools
-import os, re, sys
+import os
+import sys
+
+from ..config import config
from ..options import *
from ..utils import copydir, copyfile, rmtree, makefile
from ..utils import regenerate_qt_resources, filter_match
from ..utils import download_and_extract_7z
+
def prepare_packages_win32(self, vars):
# For now, debug symbols will not be shipped into the package.
copy_pdbs = False
pdbs = []
if (self.debug or self.build_type == 'RelWithDebInfo') and copy_pdbs:
pdbs = ['*.pdb']
- # <install>/lib/site-packages/PySide2/* -> <setup>/PySide2
+
+ # <install>/lib/site-packages/{st_package_name}/* ->
+ # <setup>/{st_package_name}
+ # This copies the module .pyd files and various .py files
+ # (__init__, config, git version, etc.)
copydir(
- "{site_packages_dir}/PySide2",
- "{pyside_package_dir}/PySide2",
+ "{site_packages_dir}/{st_package_name}",
+ "{st_build_dir}/{st_package_name}",
vars=vars)
- built_modules = self.get_built_pyside_config(vars)['built_modules']
- # <build>/pyside2/PySide2/*.pdb -> <setup>/PySide2
- copydir(
- "{build_dir}/pyside2/PySide2",
- "{pyside_package_dir}/PySide2",
- filter=pdbs,
- recursive=False, vars=vars)
+ if config.is_internal_shiboken_module_build():
+ # <build>/shiboken2/doc/html/* ->
+ # <setup>/{st_package_name}/docs/shiboken2
+ copydir(
+ "{build_dir}/shiboken2/doc/html",
+ "{st_build_dir}/{st_package_name}/docs/shiboken2",
+ force=False, vars=vars)
+
+ # <install>/bin/*.dll -> {st_package_name}/
+ copydir(
+ "{install_dir}/bin/",
+ "{st_build_dir}/{st_package_name}",
+ filter=["shiboken*.dll"],
+ recursive=False, vars=vars)
- # <build>/shiboken2/doc/html/* ->
- # <setup>/PySide2/docs/shiboken2
- copydir(
- "{build_dir}/shiboken2/doc/html",
- "{pyside_package_dir}/PySide2/docs/shiboken2",
- force=False, vars=vars)
-
- # <install>/lib/site-packages/shiboken2.pyd ->
- # <setup>/PySide2/shiboken2.pyd
- shiboken_module_name = 'shiboken2.pyd'
- shiboken_src_path = "{site_packages_dir}".format(**vars)
- maybe_shiboken_names = [f for f in os.listdir(shiboken_src_path)
- if re.match(r'shiboken.*\.pyd', f)]
- if maybe_shiboken_names:
- shiboken_module_name = maybe_shiboken_names[0]
- vars.update({'shiboken_module_name': shiboken_module_name})
- copyfile(
- "{site_packages_dir}/{shiboken_module_name}",
- "{pyside_package_dir}/PySide2/{shiboken_module_name}",
- vars=vars)
- # @TODO: Fix this .pdb file not to overwrite release
- # {shibokengenerator}.pdb file.
- # Task-number: PYSIDE-615
- copydir(
- "{build_dir}/shiboken2/shibokenmodule",
- "{pyside_package_dir}/PySide2",
- filter=pdbs,
- recursive=False, vars=vars)
+ # <install>/lib/*.lib -> {st_package_name}/
+ copydir(
+ "{install_dir}/lib/",
+ "{st_build_dir}/{st_package_name}",
+ filter=["shiboken*.lib"],
+ recursive=False, vars=vars)
- # <install>/lib/site-packages/pyside2uic/* ->
- # <setup>/pyside2uic
- copydir(
- "{site_packages_dir}/pyside2uic",
- "{pyside_package_dir}/pyside2uic",
- force=False, vars=vars)
- if sys.version_info[0] > 2:
- rmtree("{pyside_package_dir}/pyside2uic/port_v2".format(**vars))
- else:
- rmtree("{pyside_package_dir}/pyside2uic/port_v3".format(**vars))
+ # @TODO: Fix this .pdb file not to overwrite release
+ # {shibokengenerator}.pdb file.
+ # Task-number: PYSIDE-615
+ copydir(
+ "{build_dir}/shiboken2/shibokenmodule",
+ "{st_build_dir}/{st_package_name}",
+ filter=pdbs,
+ recursive=False, vars=vars)
- # <install>/bin/pyside2-uic -> PySide2/scripts/uic.py
- makefile(
- "{pyside_package_dir}/PySide2/scripts/__init__.py",
- vars=vars)
- copyfile(
- "{install_dir}/bin/pyside2-uic",
- "{pyside_package_dir}/PySide2/scripts/uic.py",
- force=False, vars=vars)
-
- # For setting up entry points
- copyfile(
- "{install_dir}/bin/pyside_tool.py",
- "{pyside_package_dir}/PySide2/scripts/pyside_tool.py",
- force=False, vars=vars)
-
- # <install>/bin/*.exe,*.dll,*.pdb -> PySide2/
- copydir(
- "{install_dir}/bin/",
- "{pyside_package_dir}/PySide2",
- filter=["*.exe", "*.dll"],
- recursive=False, vars=vars)
- # @TODO: Fix this .pdb file not to overwrite release
- # {shibokenmodule}.pdb file.
- # Task-number: PYSIDE-615
- copydir(
- "{build_dir}/shiboken2/generator",
- "{pyside_package_dir}/PySide2",
- filter=pdbs,
- recursive=False, vars=vars)
+ # pdb files for libshiboken and libpyside
+ copydir(
+ "{build_dir}/shiboken2/libshiboken",
+ "{st_build_dir}/{st_package_name}",
+ filter=pdbs,
+ recursive=False, vars=vars)
- # <install>/lib/*.lib -> PySide2/
- copydir(
- "{install_dir}/lib/",
- "{pyside_package_dir}/PySide2",
- filter=["*.lib"],
- recursive=False, vars=vars)
+ if config.is_internal_shiboken_generator_build():
+ # <install>/bin/*.dll -> {st_package_name}/
+ copydir(
+ "{install_dir}/bin/",
+ "{st_build_dir}/{st_package_name}",
+ filter=["shiboken*.exe"],
+ recursive=False, vars=vars)
- # <install>/share/PySide2/typesystems/* ->
- # <setup>/PySide2/typesystems
- copydir(
- "{install_dir}/share/PySide2/typesystems",
- "{pyside_package_dir}/PySide2/typesystems",
- vars=vars)
+ # Used to create scripts directory.
+ makefile(
+ "{st_build_dir}/{st_package_name}/scripts/shiboken_tool.py",
+ vars=vars)
- # <install>/include/* -> <setup>/PySide2/include
- copydir(
- "{install_dir}/include",
- "{pyside_package_dir}/PySide2/include",
- vars=vars)
+ # For setting up setuptools entry points.
+ copyfile(
+ "{install_dir}/bin/shiboken_tool.py",
+ "{st_build_dir}/{st_package_name}/scripts/shiboken_tool.py",
+ force=False, vars=vars)
+
+ # @TODO: Fix this .pdb file not to overwrite release
+ # {shibokenmodule}.pdb file.
+ # Task-number: PYSIDE-615
+ copydir(
+ "{build_dir}/shiboken2/generator",
+ "{st_build_dir}/{st_package_name}",
+ filter=pdbs,
+ recursive=False, vars=vars)
- # <source>/pyside2/PySide2/support/* ->
- # <setup>/PySide2/support/*
- copydir(
- "{build_dir}/pyside2/PySide2/support",
- "{pyside_package_dir}/PySide2/support",
- vars=vars)
+ if config.is_internal_shiboken_generator_build() or config.is_internal_pyside_build():
+ # <install>/include/* -> <setup>/{st_package_name}/include
+ copydir(
+ "{install_dir}/include/{cmake_package_name}",
+ "{st_build_dir}/{st_package_name}/include",
+ vars=vars)
+
+ if config.is_internal_pyside_build():
+ # <build>/pyside2/{st_package_name}/*.pdb ->
+ # <setup>/{st_package_name}
+ copydir(
+ "{build_dir}/pyside2/{st_package_name}",
+ "{st_build_dir}/{st_package_name}",
+ filter=pdbs,
+ recursive=False, vars=vars)
+
+ # <install>/lib/site-packages/pyside2uic/* ->
+ # <setup>/pyside2uic
+ copydir(
+ "{site_packages_dir}/pyside2uic",
+ "{st_build_dir}/pyside2uic",
+ force=False, vars=vars)
+ if sys.version_info[0] > 2:
+ rmtree("{st_build_dir}/pyside2uic/port_v2".format(**vars))
+ else:
+ rmtree("{st_build_dir}/pyside2uic/port_v3".format(**vars))
+
+ # <install>/bin/pyside2-uic -> {st_package_name}/scripts/uic.py
+ makefile(
+ "{st_build_dir}/{st_package_name}/scripts/__init__.py",
+ vars=vars)
+ copyfile(
+ "{install_dir}/bin/pyside2-uic",
+ "{st_build_dir}/{st_package_name}/scripts/uic.py",
+ force=False, vars=vars)
+
+ # For setting up setuptools entry points
+ copyfile(
+ "{install_dir}/bin/pyside_tool.py",
+ "{st_build_dir}/{st_package_name}/scripts/pyside_tool.py",
+ force=False, vars=vars)
+
+ # <install>/bin/*.exe,*.dll -> {st_package_name}/
+ copydir(
+ "{install_dir}/bin/",
+ "{st_build_dir}/{st_package_name}",
+ filter=["pyside*.exe", "pyside*.dll"],
+ recursive=False, vars=vars)
+
+ # <install>/lib/*.lib -> {st_package_name}/
+ copydir(
+ "{install_dir}/lib/",
+ "{st_build_dir}/{st_package_name}",
+ filter=["pyside*.lib"],
+ recursive=False, vars=vars)
+
+ # <install>/share/{st_package_name}/typesystems/* ->
+ # <setup>/{st_package_name}/typesystems
+ copydir(
+ "{install_dir}/share/{st_package_name}/typesystems",
+ "{st_build_dir}/{st_package_name}/typesystems",
+ vars=vars)
+
+ # <source>/pyside2/{st_package_name}/support/* ->
+ # <setup>/{st_package_name}/support/*
+ copydir(
+ "{build_dir}/pyside2/{st_package_name}/support",
+ "{st_build_dir}/{st_package_name}/support",
+ vars=vars)
+
+ copydir(
+ "{build_dir}/pyside2/libpyside",
+ "{st_build_dir}/{st_package_name}",
+ filter=pdbs,
+ recursive=False, vars=vars)
- if not OPTION_NOEXAMPLES:
- # examples/* -> <setup>/PySide2/examples
- copydir(os.path.join(self.script_dir, "examples"),
- "{pyside_package_dir}/PySide2/examples",
+ if not OPTION_NOEXAMPLES:
+ # examples/* -> <setup>/{st_package_name}/examples
+ copydir(os.path.join(self.script_dir, "examples"),
+ "{st_build_dir}/{st_package_name}/examples",
+ force=False, vars=vars)
+ # Re-generate examples Qt resource files for Python 3
+ # compatibility
+ if sys.version_info[0] == 3:
+ examples_path = "{st_build_dir}/{st_package_name}/examples".format(
+ **vars)
+ pyside_rcc_path = "{install_dir}/bin/pyside2-rcc".format(
+ **vars)
+ pyside_rcc_options = '-py3'
+ regenerate_qt_resources(examples_path, pyside_rcc_path,
+ pyside_rcc_options)
+
+ if vars['ssl_libs_dir']:
+ # <ssl_libs>/* -> <setup>/{st_package_name}/openssl
+ copydir("{ssl_libs_dir}", "{st_build_dir}/{st_package_name}/openssl",
+ filter=[
+ "libeay32.dll",
+ "ssleay32.dll"],
force=False, vars=vars)
- # Re-generate examples Qt resource files for Python 3
- # compatibility
- if sys.version_info[0] == 3:
- examples_path = "{pyside_package_dir}/PySide2/examples".format(
- **vars)
- pyside_rcc_path = "{install_dir}/bin/pyside2-rcc".format(
- **vars)
- pyside_rcc_options = '-py3'
- regenerate_qt_resources(examples_path, pyside_rcc_path,
- pyside_rcc_options)
-
- # <ssl_libs>/* -> <setup>/PySide2/openssl
- copydir("{ssl_libs_dir}", "{pyside_package_dir}/PySide2/openssl",
- filter=[
- "libeay32.dll",
- "ssleay32.dll"],
- force=False, vars=vars)
-
- # <qt>/bin/*.dll and Qt *.exe -> <setup>/PySide2
+
+ if config.is_internal_pyside_build() or config.is_internal_shiboken_generator_build():
+ copy_qt_artifacts(self, copy_pdbs, vars)
+
+
+def copy_qt_artifacts(self, copy_pdbs, vars):
+ built_modules = self.get_built_pyside_config(vars)['built_modules']
+
+ constrain_modules = None
+ copy_plugins = True
+ copy_qml = True
+ copy_translations = True
+ copy_qt_conf = True
+ copy_qt_permanent_artifacts = True
+ copy_msvc_redist = False
+ copy_clang = False
+
+ if config.is_internal_shiboken_generator_build():
+ constrain_modules = ["Core", "Network", "Xml", "XmlPatterns"]
+ copy_plugins = False
+ copy_qml = False
+ copy_translations = False
+ copy_qt_conf = False
+ copy_qt_permanent_artifacts = False
+ copy_msvc_redist = True
+ copy_clang = True
+
+ # <qt>/bin/*.dll and Qt *.exe -> <setup>/{st_package_name}
qt_artifacts_permanent = [
"opengl*.dll",
"d3d*.dll",
@@ -195,6 +267,7 @@ def prepare_packages_win32(self, vars):
"lconvert.exe",
"qtdiag.exe"
]
+
# MSVC redistributable
msvc_redist = [
"concrt140.dll",
@@ -218,8 +291,14 @@ def prepare_packages_win32(self, vars):
else:
egl_suffix = ''
qt_artifacts_egl = [a.format(egl_suffix) for a in qt_artifacts_egl]
- qt_artifacts_permanent += qt_artifacts_egl
- qt_artifacts_permanent += msvc_redist
+
+ artifacts = []
+ if copy_qt_permanent_artifacts:
+ artifacts += qt_artifacts_permanent
+ artifacts += qt_artifacts_egl
+
+ if copy_msvc_redist:
+ artifacts += msvc_redist
# Extract Qt dependency dll's when building on Qt CI
# There is no proper CI env variable, so using agent launch params
@@ -231,15 +310,22 @@ def prepare_packages_win32(self, vars):
zip_file = "pyside_qt_deps_32.7z"
download_and_extract_7z(redist_url + zip_file, "{qt_bin_dir}".format(**vars))
- copydir("{qt_bin_dir}", "{pyside_package_dir}/PySide2",
- filter=qt_artifacts_permanent,
- recursive=False, vars=vars)
+ if artifacts:
+ copydir("{qt_bin_dir}",
+ "{st_build_dir}/{st_package_name}",
+ filter=artifacts, recursive=False, vars=vars)
- # <qt>/bin/*.dll and Qt *.pdbs -> <setup>/PySide2 part two
+ # <qt>/bin/*.dll and Qt *.pdbs -> <setup>/{st_package_name} part two
# File filter to copy only debug or only release files.
- qt_dll_patterns = ["Qt5*{}.dll", "lib*{}.dll"]
- if copy_pdbs:
- qt_dll_patterns += ["Qt5*{}.pdb", "lib*{}.pdb"]
+ if constrain_modules:
+ qt_dll_patterns = ["Qt5" + x + "{}.dll" for x in constrain_modules]
+ if copy_pdbs:
+ qt_dll_patterns += ["Qt5" + x + "{}.pdb" for x in constrain_modules]
+ else:
+ qt_dll_patterns = ["Qt5*{}.dll", "lib*{}.dll"]
+ if copy_pdbs:
+ qt_dll_patterns += ["Qt5*{}.pdb", "lib*{}.pdb"]
+
def qt_build_config_filter(patterns, file_name, file_full_path):
release = [a.format('') for a in patterns]
debug = [a.format('d') for a in patterns]
@@ -289,56 +375,60 @@ def prepare_packages_win32(self, vars):
return False
qt_dll_filter = functools.partial(qt_build_config_filter,
- qt_dll_patterns)
- copydir("{qt_bin_dir}", "{pyside_package_dir}/PySide2",
- file_filter_function=qt_dll_filter,
- recursive=False, vars=vars)
-
- # <qt>/plugins/* -> <setup>/PySide2/plugins
- plugin_dll_patterns = ["*{}.dll"]
- pdb_pattern = "*{}.pdb"
- if copy_pdbs:
- plugin_dll_patterns += [pdb_pattern]
- plugin_dll_filter = functools.partial(qt_build_config_filter,
- plugin_dll_patterns)
- copydir("{qt_plugins_dir}", "{pyside_package_dir}/PySide2/plugins",
- file_filter_function=plugin_dll_filter,
- vars=vars)
+ qt_dll_patterns)
+ copydir("{qt_bin_dir}",
+ "{st_build_dir}/{st_package_name}",
+ file_filter_function=qt_dll_filter,
+ recursive=False, vars=vars)
- # <qt>/translations/* -> <setup>/PySide2/translations
- copydir("{qt_translations_dir}",
- "{pyside_package_dir}/PySide2/translations",
- filter=["*.qm", "*.pak"],
- force=False,
- vars=vars)
+ if copy_plugins:
+ # <qt>/plugins/* -> <setup>/{st_package_name}/plugins
+ plugin_dll_patterns = ["*{}.dll"]
+ pdb_pattern = "*{}.pdb"
+ if copy_pdbs:
+ plugin_dll_patterns += [pdb_pattern]
+ plugin_dll_filter = functools.partial(qt_build_config_filter,
+ plugin_dll_patterns)
+ copydir("{qt_plugins_dir}", "{st_build_dir}/{st_package_name}/plugins",
+ file_filter_function=plugin_dll_filter,
+ vars=vars)
- # <qt>/qml/* -> <setup>/PySide2/qml
- qml_dll_patterns = ["*{}.dll"]
- qml_ignore_patterns = qml_dll_patterns + [pdb_pattern]
- qml_ignore = [a.format('') for a in qml_ignore_patterns]
+ if copy_translations:
+ # <qt>/translations/* -> <setup>/{st_package_name}/translations
+ copydir("{qt_translations_dir}",
+ "{st_build_dir}/{st_package_name}/translations",
+ filter=["*.qm", "*.pak"],
+ force=False,
+ vars=vars)
- # Copy all files that are not dlls and pdbs (.qml, qmldir).
- copydir("{qt_qml_dir}", "{pyside_package_dir}/PySide2/qml",
- ignore=qml_ignore,
- force=False,
- recursive=True,
- vars=vars)
+ if copy_qml:
+ # <qt>/qml/* -> <setup>/{st_package_name}/qml
+ qml_dll_patterns = ["*{}.dll"]
+ qml_ignore_patterns = qml_dll_patterns + [pdb_pattern]
+ qml_ignore = [a.format('') for a in qml_ignore_patterns]
+
+ # Copy all files that are not dlls and pdbs (.qml, qmldir).
+ copydir("{qt_qml_dir}", "{st_build_dir}/{st_package_name}/qml",
+ ignore=qml_ignore,
+ force=False,
+ recursive=True,
+ vars=vars)
- if copy_pdbs:
- qml_dll_patterns += [pdb_pattern]
- qml_dll_filter = functools.partial(qt_build_config_filter,
- qml_dll_patterns)
+ if copy_pdbs:
+ qml_dll_patterns += [pdb_pattern]
+ qml_dll_filter = functools.partial(qt_build_config_filter,
+ qml_dll_patterns)
- # Copy all dlls (and possibly pdbs).
- copydir("{qt_qml_dir}", "{pyside_package_dir}/PySide2/qml",
- file_filter_function=qml_dll_filter,
- force=False,
- recursive=True,
- vars=vars)
+ # Copy all dlls (and possibly pdbs).
+ copydir("{qt_qml_dir}", "{st_build_dir}/{st_package_name}/qml",
+ file_filter_function=qml_dll_filter,
+ force=False,
+ recursive=True,
+ vars=vars)
if self.is_webengine_built(built_modules):
copydir("{qt_prefix_dir}/resources",
- "{pyside_package_dir}/PySide2/resources",
+ "{st_build_dir}/{st_package_name}/resources",
filter=None,
recursive=False,
vars=vars)
@@ -346,26 +436,16 @@ def prepare_packages_win32(self, vars):
filter = 'QtWebEngineProcess{}.exe'.format(
'd' if self.debug else '')
copydir("{qt_bin_dir}",
- "{pyside_package_dir}/PySide2",
+ "{st_build_dir}/{st_package_name}",
filter=[filter],
recursive=False, vars=vars)
- # Copy the qt.conf file to prefix dir.
- copyfile(
- "{build_dir}/pyside2/PySide2/qt.conf",
- "{pyside_package_dir}/PySide2",
- vars=vars)
-
- self.prepare_standalone_clang(is_win=True)
+ if copy_qt_conf:
+ # Copy the qt.conf file to prefix dir.
+ copyfile(
+ "{build_dir}/pyside2/{st_package_name}/qt.conf",
+ "{st_build_dir}/{st_package_name}",
+ vars=vars)
- # pdb files for libshiboken and libpyside
- copydir(
- "{build_dir}/shiboken2/libshiboken",
- "{pyside_package_dir}/PySide2",
- filter=pdbs,
- recursive=False, vars=vars)
- copydir(
- "{build_dir}/pyside2/libpyside",
- "{pyside_package_dir}/PySide2",
- filter=pdbs,
- recursive=False, vars=vars)
+ if copy_clang:
+ self.prepare_standalone_clang(is_win=True)
diff --git a/build_scripts/setup_runner.py b/build_scripts/setup_runner.py
new file mode 100644
index 000000000..a1526793e
--- /dev/null
+++ b/build_scripts/setup_runner.py
@@ -0,0 +1,165 @@
+#############################################################################
+##
+## Copyright (C) 2018 The Qt Company Ltd.
+## Contact: https://www.qt.io/licensing/
+##
+## This file is part of Qt for Python.
+##
+## $QT_BEGIN_LICENSE:LGPL$
+## Commercial License Usage
+## Licensees holding valid commercial Qt licenses may use this file in
+## accordance with the commercial license agreement provided with the
+## Software or, alternatively, in accordance with the terms contained in
+## a written agreement between you and The Qt Company. For licensing terms
+## and conditions see https://www.qt.io/terms-conditions. For further
+## information use the contact form at https://www.qt.io/contact-us.
+##
+## GNU Lesser General Public License Usage
+## Alternatively, this file may be used under the terms of the GNU Lesser
+## General Public License version 3 as published by the Free Software
+## Foundation and appearing in the file LICENSE.LGPL3 included in the
+## packaging of this file. Please review the following information to
+## ensure the GNU Lesser General Public License version 3 requirements
+## will be met: https://www.gnu.org/licenses/lgpl-3.0.html.
+##
+## GNU General Public License Usage
+## Alternatively, this file may be used under the terms of the GNU
+## General Public License version 2.0 or (at your option) the GNU General
+## Public license version 3 or any later version approved by the KDE Free
+## Qt Foundation. The licenses are as published by the Free Software
+## Foundation and appearing in the file LICENSE.GPL2 and LICENSE.GPL3
+## included in the packaging of this file. Please review the following
+## information to ensure the GNU General Public License requirements will
+## be met: https://www.gnu.org/licenses/gpl-2.0.html and
+## https://www.gnu.org/licenses/gpl-3.0.html.
+##
+## $QT_END_LICENSE$
+##
+#############################################################################
+
+import sys, os, textwrap
+
+from build_scripts.config import config
+from build_scripts.main import get_package_version, get_setuptools_extension_modules
+from build_scripts.main import cmd_class_dict
+from build_scripts.options import OPTION_BUILD_TYPE, OPTION_INTERNAL_BUILD_TYPE
+from build_scripts.utils import run_process
+
+from setuptools import setup
+
+
+class SetupRunner(object):
+ def __init__(self, orig_argv):
+ self.invocations_list = []
+
+ # Keep the original args around in case we ever need to pass
+ # modified arguments to the sub invocations.
+ self.orig_argv = orig_argv
+ self.sub_argv = list(orig_argv)
+
+ self.setup_script_dir = os.getcwd()
+
+ @staticmethod
+ def cmd_line_argument_is_in_args(argument, args):
+ """ Check if command line argument was passed in args. """
+ return any(arg for arg in list(args) if "--" + argument in arg)
+
+ @staticmethod
+ def remove_cmd_line_argument_in_args(argument, args):
+ """ Remove command line argument from args. """
+ return [arg for arg in list(args) if "--" + argument not in arg]
+
+ @staticmethod
+ def construct_cmd_line_argument(name, value=None):
+ """ Constructs a command line argument given name and value. """
+ if not value:
+ return "--{}".format(name)
+ return "--{}={}".format(name, value)
+
+ @staticmethod
+ def construct_internal_build_type_cmd_line_argument(internal_build_type):
+ return SetupRunner.construct_cmd_line_argument("internal-build-type", internal_build_type)
+
+ def add_setup_internal_invocation(self, build_type, reuse_build=False):
+ """ Enqueues a script sub-invocation to be executed later. """
+ internal_build_type_arg = self.construct_internal_build_type_cmd_line_argument(build_type)
+ setup_cmd = [sys.executable] + self.sub_argv + [internal_build_type_arg]
+
+ # Add --reuse-build option if requested and not already present.
+ if reuse_build and not self.cmd_line_argument_is_in_args("reuse-build", self.sub_argv):
+ setup_cmd.append(self.construct_cmd_line_argument("reuse-build"))
+ self.invocations_list.append(setup_cmd)
+
+ def run_setup(self):
+ """
+ Decide what kind of build is requested and then execute it.
+ In the top-level invocation case, the script
+ will spawn setup.py again (possibly multiple times).
+ In the internal invocation case, the script
+ will run setuptools.setup().
+ """
+
+ # Prepare initial config.
+ config.init_config(build_type=OPTION_BUILD_TYPE,
+ internal_build_type=OPTION_INTERNAL_BUILD_TYPE,
+ cmd_class_dict=cmd_class_dict,
+ package_version=get_package_version(),
+ ext_modules=get_setuptools_extension_modules(),
+ setup_script_dir=self.setup_script_dir)
+
+ # This is an internal invocation of setup.py, so start actual
+ # build.
+ if config.is_internal_invocation():
+ if config.internal_build_type not in config.get_allowed_internal_build_values():
+ raise RuntimeError("Invalid '{}' option given to --internal-build-type. "
+ .format(config.internal_build_type))
+ self.run_setuptools_setup()
+ return
+
+ # This is a top-level invocation of setup.py, so figure out what
+ # modules we will build and depending on that, call setup.py
+ # multiple times with different arguments.
+ if config.build_type not in config.get_allowed_top_level_build_values():
+ raise RuntimeError("Invalid '{}' option given to --build-type. "
+ .format(config.build_type))
+
+ # Build everything: shiboken2, shiboken2-generator and PySide2.
+ if config.is_top_level_build_all():
+ self.add_setup_internal_invocation(config.shiboken_module_option_name)
+
+ # Reuse the shiboken build for the generator package instead
+ # of rebuilding it again.
+ self.add_setup_internal_invocation(config.shiboken_generator_option_name,
+ reuse_build=True)
+
+ self.add_setup_internal_invocation(config.pyside_option_name)
+
+ elif config.is_top_level_build_shiboken_module():
+ self.add_setup_internal_invocation(config.shiboken_module_option_name)
+
+ elif config.is_top_level_build_shiboken_generator():
+ self.add_setup_internal_invocation(config.shiboken_generator_option_name)
+
+ elif config.is_top_level_build_pyside():
+ self.add_setup_internal_invocation(config.pyside_option_name)
+
+ for cmd in self.invocations_list:
+ cmd_as_string = " ".join(cmd)
+ print("\nRunning process: {}\n".format(cmd_as_string))
+ exit_code = run_process(cmd)
+ if exit_code != 0:
+ msg = textwrap.dedent("""
+ setup.py invocation failed with exit code: {}.\n\n
+ setup.py invocation was: {}
+ """).format(exit_code, cmd_as_string)
+ raise RuntimeError(msg)
+
+ @staticmethod
+ def run_setuptools_setup():
+ """
+ Runs setuptools.setup() once in a single setup.py
+ sub-invocation.
+ """
+
+ kwargs = config.setup_kwargs
+ setup(**kwargs)
diff --git a/build_scripts/utils.py b/build_scripts/utils.py
index 7160630d1..165366e26 100644
--- a/build_scripts/utils.py
+++ b/build_scripts/utils.py
@@ -39,17 +39,13 @@
import sys
import os
-import stat
import re
import stat
import errno
-import time
import shutil
import subprocess
import fnmatch
-import glob
import itertools
-import popenasync
import glob
# There is no urllib.request in Python2
@@ -58,11 +54,9 @@ try:
except ImportError:
import urllib
-from distutils import log
+import distutils.log as log
from distutils.errors import DistutilsOptionError
from distutils.errors import DistutilsSetupError
-from distutils.spawn import spawn
-from distutils.spawn import DistutilsExecError
try:
WindowsError
@@ -70,32 +64,6 @@ except NameError:
WindowsError = None
-def has_option(name):
- try:
- sys.argv.remove("--{}".format(name))
- return True
- except ValueError:
- pass
- return False
-
-
-def option_value(name):
- for index, option in enumerate(sys.argv):
- if option == '--' + name:
- if index+1 >= len(sys.argv):
- raise DistutilsOptionError("The option {} requires a "
- "value".format(option))
- value = sys.argv[index+1]
- sys.argv[index:index+2] = []
- return value
- if option.startswith('--' + name + '='):
- value = option[len(name)+3:]
- sys.argv[index:index+1] = []
- return value
- env_val = os.getenv(name.upper().replace('-', '_'))
- return env_val
-
-
def filter_match(name, patterns):
for pattern in patterns:
if pattern is None:
@@ -182,7 +150,6 @@ def find_vcdir(version):
"""
from distutils.msvc9compiler import VS_BASE
from distutils.msvc9compiler import Reg
- from distutils import log
vsbase = VS_BASE % version
try:
productdir = Reg.get_value(r"{}\Setup\VC".format(vsbase), "productdir")
@@ -416,13 +383,15 @@ def rmtree(dirname, ignore=False):
os.chmod(path, stat.S_IRWXU| stat.S_IRWXG| stat.S_IRWXO) # 0777
func(path)
else:
- raise
+ raise IOError
shutil.rmtree(dirname, ignore_errors=ignore, onerror=handle_remove_readonly)
def run_process_output(args, initial_env=None):
if initial_env is None:
initial_env = os.environ
- std_out = subprocess.Popen(args, env = initial_env, universal_newlines = 1,
+ std_out = subprocess.Popen(args,
+ env = initial_env,
+ universal_newlines = 1,
stdout=subprocess.PIPE).stdout
result = []
for raw_line in std_out.readlines():
@@ -431,53 +400,21 @@ def run_process_output(args, initial_env=None):
return result
def run_process(args, initial_env=None):
- def _log(buffer, check_new_line=False):
- ends_with_new_line = False
- if buffer.endswith('\n'):
- ends_with_new_line = True
- if check_new_line and buffer.find('\n') == -1:
- return buffer
- lines = buffer.splitlines()
- buffer = ''
- if check_new_line and not ends_with_new_line:
- buffer = lines[-1]
- lines = lines[:-1]
- for line in lines:
- log.info(line.rstrip('\r'))
- return buffer
- _log("Running process in {0}: {1}".format(os.getcwd(),
- " ".join([(" " in x and '"{0}"'.format(x) or x) for x in args])))
-
- if sys.platform != "win32":
- try:
- spawn(args)
- return 0
- except DistutilsExecError:
- return -1
-
- shell = False
- if sys.platform == "win32":
- shell = True
+ """
+ Run process until completion and return the process exit code.
+ Prints both stdout and stderr to the console.
+ No output is captured.
+ """
+ log.info("Running process in directory {0}: command {1}".format(
+ os.getcwd(),
+ " ".join([(" " in x and '"{0}"'.format(x) or x) for x in args]))
+ )
if initial_env is None:
initial_env = os.environ
- proc = popenasync.Popen(args,
- stdin = subprocess.PIPE,
- stdout = subprocess.PIPE,
- stderr = subprocess.STDOUT,
- universal_newlines = 1,
- shell = shell,
- env = initial_env)
-
- log_buffer = None;
- while proc.poll() is None:
- log_buffer = _log(proc.read_async(wait=0.1, e=0))
- if log_buffer:
- _log(log_buffer)
-
- proc.wait()
- return proc.returncode
+ exit_code = subprocess.call(args, stderr=subprocess.STDOUT, env=initial_env)
+ return exit_code
def get_environment_from_batch_command(env_cmd, initial=None):
@@ -665,6 +602,9 @@ def macos_get_rpaths(libpath):
ctr += 3
return rpaths
+def macos_add_rpath(rpath, library_path):
+ back_tick('install_name_tool -add_rpath {rpath} {library_path}'.format(
+ rpath=rpath, library_path=library_path))
def macos_fix_rpaths_for_library(library_path, qt_lib_dir):
""" Adds required rpath load commands to given library.
@@ -703,8 +643,7 @@ def macos_fix_rpaths_for_library(library_path, qt_lib_dir):
break
if needs_loader_path and "@loader_path" not in existing_rpath_commands:
- back_tick('install_name_tool -add_rpath {rpath} {library_path}'.format(
- rpath="@loader_path", library_path=library_path))
+ macos_add_rpath("@loader_path", library_path)
# If the library depends on a Qt library, add an rpath load comment
# pointing to the Qt lib directory.
@@ -738,8 +677,7 @@ def macos_add_qt_rpath(library_path, qt_lib_dir,
break
if needs_qt_rpath:
- back_tick('install_name_tool -add_rpath {rpath} {library_path}'.format(
- rpath=qt_lib_dir, library_path=library_path))
+ macos_add_rpath(qt_lib_dir, library_path)
# Find an executable specified by a glob pattern ('foo*') in the OS path
def find_glob_in_path(pattern):
@@ -996,6 +934,17 @@ def copy_icu_libs(patchelf, destination_lib_dir):
new_rpaths_string = ":".join(rpaths)
linux_set_rpaths(patchelf, qt_core_library_path, new_rpaths_string)
+
+def linux_run_read_elf(executable_path):
+ cmd = "readelf -d {}".format(executable_path)
+ (out, err, code) = back_tick(cmd, True)
+ if code != 0:
+ raise RuntimeError("Running `readelf -d {}` failed with error "
+ "output:\n {}. ".format(executable_path, err))
+ lines = split_and_strip(out)
+ return lines
+
+
def linux_set_rpaths(patchelf, executable_path, rpath_string):
""" Patches the `executable_path` with a new rpath string. """
@@ -1005,18 +954,32 @@ def linux_set_rpaths(patchelf, executable_path, rpath_string):
raise RuntimeError("Error patching rpath in {}".format(
executable_path))
+
+def linux_get_dependent_libraries(executable_path):
+ """
+ Returns a list of libraries that executable_path depends on.
+ """
+
+ lines = linux_run_read_elf(executable_path)
+ pattern = re.compile(r"^.+?\(NEEDED\).+?\[(.+?)\]$")
+
+ library_lines = []
+ for line in lines:
+ match = pattern.search(line)
+ if match:
+ library_line = match.group(1)
+ library_lines.append(library_line)
+
+ return library_lines
+
+
def linux_get_rpaths(executable_path):
"""
Returns a list of run path values embedded in the executable or just
an empty list.
"""
- cmd = "readelf -d {}".format(executable_path)
- (out, err, code) = back_tick(cmd, True)
- if code != 0:
- raise RuntimeError("Running `readelf -d {}` failed with error "
- "output:\n {}. ".format(executable_path, err))
- lines = split_and_strip(out)
+ lines = linux_run_read_elf(executable_path)
pattern = re.compile(r"^.+?\(RUNPATH\).+?\[(.+?)\]$")
rpath_line = None
@@ -1033,6 +996,7 @@ def linux_get_rpaths(executable_path):
return rpaths
+
def rpaths_has_origin(rpaths):
"""
Return True if the specified list of rpaths has an "$ORIGIN" value
@@ -1048,6 +1012,39 @@ def rpaths_has_origin(rpaths):
return True
return False
+
+def linux_needs_qt_rpath(executable_path):
+ """
+ Returns true if library_path depends on Qt libraries.
+ """
+
+ dependencies = linux_get_dependent_libraries(executable_path)
+
+ # Check if any library dependencies are Qt libraries (hacky).
+ needs_qt_rpath = False
+ for dep in dependencies:
+ if 'Qt' in dep:
+ needs_qt_rpath = True
+ break
+ return needs_qt_rpath
+
+
+def linux_fix_rpaths_for_library(patchelf, executable_path, qt_rpath, override=False):
+ """
+ Adds or overrides required rpaths in given executable / library.
+ """
+ rpaths = ['$ORIGIN/']
+ existing_rpaths = []
+ if not override:
+ existing_rpaths = linux_get_rpaths(executable_path)
+ rpaths.extend(existing_rpaths)
+
+ if linux_needs_qt_rpath(executable_path) and qt_rpath not in existing_rpaths:
+ rpaths.append(qt_rpath)
+
+ rpaths_string = ':'.join(rpaths)
+ linux_set_rpaths(patchelf, executable_path, rpaths_string)
+
def memoize(function):
"""
Decorator to wrap a function with a memoizing callable.
@@ -1076,9 +1073,27 @@ def get_python_dict(python_script_path):
"file: {}.".format(python_script_path))
raise
-def install_pip_dependencies(env_pip, packages):
+def install_pip_wheel_package(env_pip):
+ # Need to install an unreleased wheel version, due to a bug that
+ # will generate a wheel which will not be installable.
+ # See https://github.com/pypa/wheel/issues/263
+ wheel_url = "git+https://github.com/pypa/wheel.git@fbf3e3ada64d36ca7bb9c1422f5a1ccdba7e4dcf"
+ install_pip_package_from_url_specifier(env_pip, wheel_url)
+
+def install_pip_package_from_url_specifier(env_pip, url, upgrade=True):
+ args = [env_pip, "install", url]
+ if upgrade:
+ args.append("--upgrade")
+ args.append(url)
+ run_instruction(args, "Failed to install {}".format(url))
+
+def install_pip_dependencies(env_pip, packages, upgrade=True):
for p in packages:
- run_instruction([env_pip, "install", p], "Failed to install " + p)
+ args = [env_pip, "install"]
+ if upgrade:
+ args.append("--upgrade")
+ args.append(p)
+ run_instruction(args, "Failed to install " + p)
def get_qtci_virtualEnv(python_ver, host, hostArch, targetArch):
_pExe = "python"
@@ -1105,9 +1120,11 @@ def get_qtci_virtualEnv(python_ver, host, hostArch, targetArch):
_pExe = "python3"
return(_pExe, _env, env_pip, env_python)
-def run_instruction(instruction, error):
+def run_instruction(instruction, error, initial_env=None):
+ if initial_env is None:
+ initial_env = os.environ
print("Running Coin instruction: " + ' '.join(str(e) for e in instruction))
- result = subprocess.call(instruction)
+ result = subprocess.call(instruction, env=initial_env)
if result != 0:
print("ERROR : " + error)
exit(result)