Allow building shiboken2 and PySide2 as separate wheels

Actually this creates 3 wheel packages:
- shiboken2 (the python module and libshiboken shared library)
- shiboken2-generator (contains the generator executable, libclang and
  dependent Qt libraries)
- PySide2 (the PySide2 modules and Qt shared libraries, and tools like
  rcc, uic)

Calling the setup.py script will not do the actual build now (in the
sense of calling CMake, make, etc.). Instead it will spawn new
processes (via subprocess.call) calling the same setup.py script,
but with different arguments. These "sub-invocations" will do the
actual building. Thus, the "top-level invocation" will decide which
packages to build and delegate that to the "sub-invocations" of
setup.py.

A new optional command line argument is introduced called
"--build-type" which defaults to "all", and can also be set to
"shiboken2", "shiboken2-generator" and "pyside2". A user can choose
which packages to build using this option. The "top-level invocation"
uses this option to decide how many "sub-invocations" to execute.

A new command line argument called "--internal-build-type"
takes the same values as the one above. It defines which package
will actually be built in the new spawned "sub-invocation" process.
The "top-level invocation" sets this automatically for each
"sub-invocation" depending on the value of "--build-type".

This option is also useful for developers that may want to debug the
python building code in the "sub-invocation".
Developers can set this manually via the command line, and
thus avoid the process spawning indirection.

A new class Config is introduced to facilitate storage of
the various state needed for building a single package.

A new class SetupRunner is introduced that takes care of the
"--build-type" and "--internal-build-type" argument handling
and delegation of "sub-invocations".

A new class Options is introduced to 'hopefully', in the future, streamline
the mess of option handling that we currently have.

setup.py now is now simplified to mostly just call
SetupRunner.run_setup().

Certain refactorings were done to facilitate further clean-up of the
build code, the current code is definitely not the end all be all.

Various other changes that were needed to implement the wheel
separation:
- a new cmake_helpers directory is added to share common cmake
  code between packages.
- the custom popenasync.py file is removed in favor of using
  subprocess.call in as many places as possible, and thus
  avoid 10 different functions for process creation.
- Manifest.in is removed, because copying to the setuptools
  build dir is now done directly by prepare_packages functions.
- because prepare_packages copies directly to the setuptools
  build dir, avoiding the pyside_package dir, we do less copying
  of big Qt files now.
- versioning of PySide2 and shiboken2 packages is now separate.
  shiboken2 and shiboken2-generator share the same versions for
  now though.
- shiboken2 is now listed as a required package for PySide2, to
  facilitate pip requirements.txt dependencies.
- coin_build_instructions currently needs to install an unreleased
  version of wheel, due to a bug that breaks installation of
  generated wheel files.
- added separate command line options to pyside2_config.py for
  shiboken2-module and shiboken2-generator.
- adapted samplebinding and scriptableapplication projects due to
  shiboken being a separate package.
- adapted pyside2-tool and shiboken2-tool python scripts for setup
  tools entry points.
- made some optimizations not to invoke cmake for shiboken2-generator
  when doing a top-level "all" build.
- fixed unnecessary rpaths not to be included on Linux (mainly the
  Qt rpaths).

Task-nubmer: PYSIDE-749
Change-Id: I0336043955624c1d12ed254802c442608cced5fb
Reviewed-by: Christian Tismer <tismer@stackless.com>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>

1 files changed, 0 insertions, 360 deletions

diff --git a/popenasync.py b/popenasync.py
deleted file mode 100644
index 77faf9e0c..000000000
--- a/popenasync.py
+++ /dev/null
@@ -1,360 +0,0 @@
-#############################################################################
-##
-## Copyright (C) 2017 The Qt Company Ltd.
-## Contact: https://www.qt.io/licensing/
-##
-## This file is part of Qt for Python.
-##
-## $QT_BEGIN_LICENSE:LGPL$
-## Commercial License Usage
-## Licensees holding valid commercial Qt licenses may use this file in
-## accordance with the commercial license agreement provided with the
-## Software or, alternatively, in accordance with the terms contained in
-## a written agreement between you and The Qt Company. For licensing terms
-## and conditions see https://www.qt.io/terms-conditions. For further
-## information use the contact form at https://www.qt.io/contact-us.
-##
-## GNU Lesser General Public License Usage
-## Alternatively, this file may be used under the terms of the GNU Lesser
-## General Public License version 3 as published by the Free Software
-## Foundation and appearing in the file LICENSE.LGPL3 included in the
-## packaging of this file. Please review the following information to
-## ensure the GNU Lesser General Public License version 3 requirements
-## will be met: https://www.gnu.org/licenses/lgpl-3.0.html.
-##
-## GNU General Public License Usage
-## Alternatively, this file may be used under the terms of the GNU
-## General Public License version 2.0 or (at your option) the GNU General
-## Public license version 3 or any later version approved by the KDE Free
-## Qt Foundation. The licenses are as published by the Free Software
-## Foundation and appearing in the file LICENSE.GPL2 and LICENSE.GPL3
-## included in the packaging of this file. Please review the following
-## information to ensure the GNU General Public License requirements will
-## be met: https://www.gnu.org/licenses/gpl-2.0.html and
-## https://www.gnu.org/licenses/gpl-3.0.html.
-##
-## $QT_END_LICENSE$
-##
-#############################################################################
-
-################################################################################
-"""
-
-Modification of http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/440554
-
-"""
-
-#################################### IMPORTS ###################################
-
-import os
-import subprocess
-import errno
-import time
-import sys
-import unittest
-import tempfile
-
-def geterror ():
-    return sys.exc_info()[1]
-
-if sys.version_info >= (3,):
-    null_byte = '\x00'.encode('ascii')
-else:
-    null_byte = '\x00'
-
-mswindows = (sys.platform == "win32")
-
-if mswindows:
-    if sys.version_info >= (3,):
-        # Test date should be in ascii.
-        def encode(s):
-            return s.encode('ascii', 'ignore')
-
-        def decode(b):
-            return b.decode('ascii', 'ignore')
-    else:
-        # Strings only; do nothing
-        def encode(s):
-            return s
-
-        def decode(b):
-            return b
-
-    try:
-        import ctypes
-        from ctypes.wintypes import DWORD
-        kernel32 = ctypes.windll.kernel32
-        TerminateProcess = ctypes.windll.kernel32.TerminateProcess
-        def WriteFile(handle, data, ol = None):
-            c_written = DWORD()
-            success = ctypes.windll.kernel32.WriteFile(handle,
-                ctypes.create_string_buffer(encode(data)), len(data),
-                ctypes.byref(c_written), ol)
-            return ctypes.windll.kernel32.GetLastError(), c_written.value
-        def ReadFile(handle, desired_bytes, ol = None):
-            c_read = DWORD()
-            buffer = ctypes.create_string_buffer(desired_bytes+1)
-            success = ctypes.windll.kernel32.ReadFile(handle, buffer,
-                desired_bytes, ctypes.byref(c_read), ol)
-            buffer[c_read.value] = null_byte
-            return ctypes.windll.kernel32.GetLastError(), decode(buffer.value)
-        def PeekNamedPipe(handle, desired_bytes):
-            c_avail = DWORD()
-            c_message = DWORD()
-            if desired_bytes > 0:
-                c_read = DWORD()
-                buffer = ctypes.create_string_buffer(desired_bytes+1)
-                success = ctypes.windll.kernel32.PeekNamedPipe(handle, buffer,
-                    desired_bytes, ctypes.byref(c_read), ctypes.byref(c_avail),
-                    ctypes.byref(c_message))
-                buffer[c_read.value] = null_byte
-                return decode(buffer.value), c_avail.value, c_message.value
-            else:
-                success = ctypes.windll.kernel32.PeekNamedPipe(handle, None,
-                    desired_bytes, None, ctypes.byref(c_avail),
-                    ctypes.byref(c_message))
-                return "", c_avail.value, c_message.value
-
-    except ImportError:
-        from win32file import ReadFile, WriteFile
-        from win32pipe import PeekNamedPipe
-        from win32api import TerminateProcess
-    import msvcrt
-
-else:
-    from signal import SIGINT, SIGTERM, SIGKILL
-    import select
-    import fcntl
-
-################################### CONSTANTS ##################################
-
-PIPE = subprocess.PIPE
-
-################################################################################
-
-class Popen(subprocess.Popen):
-    def __init__(self, *args, **kwargs):
-        subprocess.Popen.__init__(self, *args, **kwargs)
-
-    def recv(self, maxsize=None):
-        return self._recv('stdout', maxsize)
-
-    def recv_err(self, maxsize=None):
-        return self._recv('stderr', maxsize)
-
-    def send_recv(self, input='', maxsize=None):
-        return self.send(input), self.recv(maxsize), self.recv_err(maxsize)
-
-    def read_async(self,  wait=.1, e=1, tr=5, stderr=0):
-        if tr < 1:
-            tr = 1
-        x = time.time()+ wait
-        y = []
-        r = ''
-        pr = self.recv
-        if stderr:
-            pr = self.recv_err
-        while time.time() < x or r:
-            r = pr()
-            if r is None:
-                if e:
-                    raise Exception("Other end disconnected!")
-                else:
-                    break
-            elif r:
-                y.append(r)
-            else:
-                time.sleep(max((x-time.time())/tr, 0))
-        return ''.join(y)
-
-    def send_all(self, data):
-        while len(data):
-            sent = self.send(data)
-            if sent is None:
-                raise Exception("Other end disconnected!")
-            data = buffer(data, sent)
-
-    def get_conn_maxsize(self, which, maxsize):
-        if maxsize is None:
-            maxsize = 1024
-        elif maxsize < 1:
-            maxsize = 1
-        return getattr(self, which), maxsize
-
-    def _close(self, which):
-        conn = getattr(self, which)
-        flags = fcntl.fcntl(conn, fcntl.F_GETFL)
-        if not conn.closed:
-            fcntl.fcntl(conn, fcntl.F_SETFL, flags & ~os.O_NONBLOCK)
-        assert conn.read() == ''
-        getattr(self, which).close()
-        setattr(self, which, None)
-
-    if mswindows:
-        def kill(self):
-            # Recipes
-            #http://me.in-berlin.de/doc/python/faq/windows.html#how-do-i-emulate-os-kill-in-windows
-            #http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/347462
-
-            """kill function for Win32"""
-            TerminateProcess(int(self._handle), 0) # returns None
-
-        def send(self, input):
-            if not self.stdin:
-                return None
-
-            try:
-                x = msvcrt.get_osfhandle(self.stdin.fileno())
-                (errCode, written) = WriteFile(x, input)
-            except ValueError:
-                return self._close('stdin')
-            except (subprocess.pywintypes.error, Exception):
-                if geterror()[0] in (109, errno.ESHUTDOWN):
-                    return self._close('stdin')
-                raise
-
-            return written
-
-        def _recv(self, which, maxsize):
-            conn, maxsize = self.get_conn_maxsize(which, maxsize)
-            if conn is None:
-                return None
-
-            try:
-                x = msvcrt.get_osfhandle(conn.fileno())
-                (read, nAvail, nMessage) = PeekNamedPipe(x, 0)
-                if maxsize < nAvail:
-                    nAvail = maxsize
-                if nAvail > 0:
-                    (errCode, read) = ReadFile(x, nAvail, None)
-            except ValueError:
-                return self._close(which)
-            except (subprocess.pywintypes.error, Exception):
-                if geterror()[0] in (109, errno.ESHUTDOWN):
-                    return self._close(which)
-                raise
-
-            if self.universal_newlines:
-                # Translate newlines. For Python 3.x assume read is text.
-                # If bytes then another solution is needed.
-                read = read.replace("\r\n", "\n").replace("\r", "\n")
-            return read
-
-    else:
-        def kill(self):
-            for i, sig in enumerate([SIGTERM, SIGKILL] * 2):
-                if i % 2 == 0: os.kill(self.pid, sig)
-                time.sleep((i * (i % 2) / 5.0)  + 0.01)
-
-                killed_pid, stat = os.waitpid(self.pid, os.WNOHANG)
-                if killed_pid != 0: return
-
-        def send(self, input):
-            if not self.stdin:
-                return None
-
-            if not select.select([], [self.stdin], [], 0)[1]:
-                return 0
-
-            try:
-                written = os.write(self.stdin.fileno(), input)
-            except OSError:
-                if geterror()[0] == errno.EPIPE: #broken pipe
-                    return self._close('stdin')
-                raise
-
-            return written
-
-        def _recv(self, which, maxsize):
-            conn, maxsize = self.get_conn_maxsize(which, maxsize)
-            if conn is None:
-                return None
-
-            flags = fcntl.fcntl(conn, fcntl.F_GETFL)
-            if not conn.closed:
-                fcntl.fcntl(conn, fcntl.F_SETFL, flags| os.O_NONBLOCK)
-
-            try:
-                if not select.select([conn], [], [], 0)[0]:
-                    return ''
-
-                try:
-                    r = conn.read(maxsize)
-                except IOError as e:
-                    if e.errno == errno.EAGAIN:
-                        return ''
-                    raise
-                if not r:
-                    return self._close(which)
-
-                if self.universal_newlines:
-                    r = r.replace("\r\n", "\n").replace("\r", "\n")
-                return r
-            finally:
-                if not conn.closed:
-                    fcntl.fcntl(conn, fcntl.F_SETFL, flags)
-
-################################################################################
-
-def proc_in_time_or_kill(cmd, time_out, wd = None, env = None):
-    proc = Popen (
-        cmd, cwd = wd, env = env,
-        stdin = subprocess.PIPE, stdout = subprocess.PIPE,
-        stderr = subprocess.STDOUT, universal_newlines = 1
-    )
-
-    ret_code = None
-    response = []
-
-    t = time.time()
-    while ret_code is None and ((time.time() -t) < time_out):
-        ret_code = proc.poll()
-        response += [proc.read_async(wait=0.1, e=0)]
-
-    if ret_code is None:
-        ret_code = '"Process timed out (time_out = {} secs) '.format(time_out)
-        try:
-            proc.kill()
-            ret_code += 'and was successfully terminated"'
-        except Exception:
-            ret_code += ("and termination failed "
-                "(exception: {})".format(geterror(),))
-
-    return ret_code, ''.join(response)
-
-################################################################################
-
-class AsyncTest(unittest.TestCase):
-    def test_proc_in_time_or_kill(self):
-        ret_code, response = proc_in_time_or_kill(
-            [sys.executable, '-c', 'while 1: pass'], time_out = 1
-        )
-
-        self.assert_( 'rocess timed out' in ret_code )
-        self.assert_( 'successfully terminated' in ret_code )
-
-################################################################################
-
-def _example():
-    if sys.platform == 'win32':
-        shell, commands, tail = ('cmd', ('echo "hello"', 'echo "HELLO WORLD"'),
-            '\r\n')
-    else:
-        shell, commands, tail = ('sh', ('ls', 'echo HELLO WORLD'), '\n')
-
-    a = Popen(shell, stdin=PIPE, stdout=PIPE)
-    sys.stdout.write(a.read_async())
-    sys.stdout.write(" ")
-    for cmd in commands:
-        a.send_all(cmd + tail)
-        sys.stdout.write(a.read_async())
-        sys.stdout.write(" ")
-    a.send_all('exit' + tail)
-    print (a.read_async(e=0))
-    a.wait()
-
-################################################################################
-
-if __name__ == '__main__':
-    if 1: unittest.main()
-    else: _example()