path: root/src/doc/qtautomotivesuite/src/qtautomotivesuite-sdk-repositories.qdoc

/****************************************************************************
**
** Copyright (C) 2018 The Qt Company Ltd.
** Contact: https://www.qt.io/licensing/
**
** This file is part of the documentation of the Qt Automotive Suite.
**
** $QT_BEGIN_LICENSE:FDL-QTAS$
** Commercial License Usage
** Licensees holding valid commercial Qt Automotive Suite licenses may use
** this file in accordance with the commercial license agreement provided
** with the Software or, alternatively, in accordance with the terms
** contained in a written agreement between you and The Qt Company.  For
** licensing terms and conditions see https://www.qt.io/terms-conditions.
** For further information use the contact form at https://www.qt.io/contact-us.
**
** GNU Free Documentation License Usage
** Alternatively, this file may be used under the terms of the GNU Free
** Documentation License version 1.3 as published by the Free Software
** Foundation and appearing in the file included in the packaging of
** this file. Please review the following information to ensure
** the GNU Free Documentation License version 1.3 requirements
** will be met: https://www.gnu.org/licenses/fdl-1.3.html.
** $QT_END_LICENSE$
**
****************************************************************************/

/*!
    \page qtas-sdk-repository-structure.html
    \title Planning Repository Structure
    \previouspage qtas-sdk-workflow.html
    \nextpage qtas-sdk-build-repositories.html

    You should carefully plan an online repository structure before building
    the repositories. Consider the following items when you implement your SDK:

    \section1 Amount of Updated Content

    A full update of a very large repository might not be optimal, because
    uploading the repository content would take a long time.
    See \l{Partially Updating Repositories} for practical tips
    how to update repositories partially.

    \section1 Flexible Repository Structure

    You can keep the online repository structure flexible if you do not hard
    code the repository addresses into the online installer. Instead of hard
    coding the addresses, you can make the online installer point to a
    single repository.

    For example, define a remote repository in your \e{config.xml} as follows:

    \badcode
    <RemoteRepositories>
    <Repository>
        <Url>https://mycompany.com/myonlinerepository/linux-x86_64/root</Url>
        <Enabled>1</Enabled>
        <DisplayName>MyCompany Linux-x64 root online repository</DisplayName>
    </Repository>
    </RemoteRepositories>
    \endcode

    In the configuration file structure demonstrated in \l{Building Online Installer},
    the remote repository is defined in
    \l{http://code.qt.io/cgit/qtsdk/qtsdk.git/tree/packaging-tools/configurations/linux/config.xml.template.linux-x64.qt5-sdk}.

    For more information about the configuration file elements, see \l{Configuration File}.

    In \e{https://mycompany.com/myonlinerepository/linux-x86_64/root/Updates.xml}
    under your root directory, define a list of updated repositories as follows:

    \badcode
    <Updates>
    <ApplicationName>{AnyApplication}</ApplicationName>
    <ApplicationVersion>IFW_REPOSITORY_FORMAT_VERSION</ApplicationVersion>
    <Checksum>true</Checksum>
    <RepositoryUpdate>
        <Repository action="add" url="https://mycompany.com/myonlinerepository/linux-x86_64/desktop/qtcreator" displayname="description..."/>
        <Repository action="add" url="https://mycompany.com/myonlinerepository/linux-x86_64/desktop/another_cool_tool" displayname="description..."/>
        <Repository action="add" url="https://mycompany.com/myonlinerepository/linux-x86_64/desktop/qt58" displayname="description..."/>
        ...
    </RepositoryUpdate>
    </Updates>
    \endcode

    Now you can make new repositories available to end users simply by adding a
    new line into the root repository \e{Updates.xml}. You can add repositories
    to any \e{Updates.xml} using the same syntax.

    For general information about online repositories, see
    \l{Creating Online Installers}.

    \section1 File Structure in Qt Releases

    A Qt release provides a concrete example of a tree structure of an online
    repository. The tree structure corresponds to the structure that an end user
    sees in the Qt installer wizard's \uicontrol{Select component} page.

    For example, the online installer contains the following tree structure:

    \badcode
    qt
      57
        msvc2013 32-bit
        msvc2013 64-bit
        ...
        Android x86
      58
        msvc2015 32-bit
        msvc2015 64-bit
        ...
      ...
      Tools
      Qt Creator 4.1.0
      MinGW 5.3.0
      ...
    \endcode

    You find a similar structure under the Qt release configuration directories
    in the \l{Qt SDK Git Repository}{Qt SDK Git repository}. For example,
    see \e{pkg_<Qt version>} directories under
    \l{http://code.qt.io/cgit/qtsdk/qtsdk.git/tree/packaging-tools/configurations/pkg_templates}.

    A particular package is included into an online repository via configuration
    files. For example, see
    \l{http://code.qt.io/cgit/qtsdk/qtsdk.git/tree/packaging-tools/configurations/linux/x64/58/x86_64-qt58-gcc-conf}.

    \section1 Qt Configuration Files

    The following syntax is used in the Qt configuration files:

    \table
    \header
        \li Property name
        \li Description
        \li Example
    \row
        \li archives
        \li A comma-separated list of values that will be specified in a detailed
        level later in the file.
        \li
        archives:   58.gcc_64.qtbase,
                    58.gcc_64.qtconnectivity
    \row
        \li target_install_base
        \li A common base directory for all archives in a component
        \li
        target_install_base:    /%QT_VERSION_MINOR%/gcc_64
    \row
        \li archive_url
        \li The location from where an archived content is fetched. The location can
        be an absolute file system path or a URL. By default, the path or the URL
        is appended to a base directory defined in \e{target_install_base}.
        \li
        archive_uri:    /qt/%QT_VERSION_MINOR%/latest/qtbase/qtbase-Linux-RHEL_6_6-GCC-Linux-RHEL_6_6-X86_64.7z
    \row
        \li package_strip_dirs
        \li Can get numerical values, for example 0, 1 or 3. The value defines
        the number of unnecessary directories that are stripped away from the path.
        \li \e{package_strip_dirs} is set as \e{3} and an archive path is
        \e{/home/qtbuilder/build/<actual content>}. After you have run \e{build_wrapper.py},
        the archive is repackaged so that it has only <actual content>
        in the repository root directory. If the path is not stripped,
        an unnecessary directory structure is created in the end user host
        when they install the component.
    \row
        \li target_install_dir
        \li Defines a subdirectory for archive installation.
        \li target_install_dir:    /lib
    \endtable

    A configuration file can refer to other configuration files. For example,
    see the \e{[PackageConfigurationFiles]} section in
    \l{http://code.qt.io/cgit/qtsdk/qtsdk.git/tree/packaging-tools/configurations/offline_installer_jobs/5.8/linux_x64}.
    It lists the included configuration files:

    \badcode
    [PackageConfigurationFiles]
    file_list:  qt-conf,
                qt-license-conf,
                qt-installer-changelog,
                qt58-src-conf,
                qt58-doc-examples-conf,
                x86_64-qt58-gcc-conf,
                x86_64-tools-qtcreator-qt58x-conf
    \endcode

    See \l{Package Directory} for general information about the package
    directory structure.
*/

/*!
    \page qtas-sdk-build-repositories.html
    \title Working with Online Repositories
    \previouspage qtas-sdk-repository-structure.html
    \nextpage qtas-sdk-troubleshooting-repositories.html

    You need to build online repositories when you add a new repository to
    a server or when you update an existing repository.

    \section1 Creating New Repository

    When you create an online repository that does not yet exist in the server,
    you can upload the repository content and add it to the root repository as
    instructed in \l{Flexible Repository Structure}.

    \section1 Updating Repository

    \l{Promoting Updates} describes the steps required for updating online repositories.

    Updating an online repository can vary greatly depending on the used online
    server system. Typically, updating contains the following steps:

    \list 1
        \li Uploading content files to the server. See \l{Uploading Content Files}.
        \li Uploading \e{Updates.xml} to the server. See \l{Uploading Updates.xml}.
    \endlist

    \section2 Uploading Content Files

    For example, a maintenance tool update could contain the following content files:

    \badcode
    mycompany.tools.maintenancetool/2.0.4-0meta.7z
    mycompany.tools.maintenancetool/2.0.4-0maintenancetool.7z
    mycompany.tools.maintenancetool/2.0.4-0maintenancetool.7z.sha1
    \endcode

    Each content file is prepended with a version number set in the \c{<Version>} element
    in the component's \e{package.xml} file. Thus you can copy the updated
    files to a same directory that contains current file versions. It is just
    important to remember to update the version numbers. For example, the
    maintenance tool directory could contain the following files:

    \badcode
    mycompany.tools.maintenancetool/2.0.2-0meta.7z
    mycompany.tools.maintenancetool/2.0.2-0maintenancetool.7z
    mycompany.tools.maintenancetool/2.0.2-0maintenancetool.7z.sha1
    mycompany.tools.maintenancetool/2.0.3-0meta.7z
    mycompany.tools.maintenancetool/2.0.3-0maintenancetool.7z
    mycompany.tools.maintenancetool/2.0.3-0maintenancetool.7z.sha1
    mycompany.tools.maintenancetool/2.0.4-0meta.7z
    mycompany.tools.maintenancetool/2.0.4-0maintenancetool.7z
    mycompany.tools.maintenancetool/2.0.4-0maintenancetool.7z.sha1
    \endcode

    \section2 Uploading Updates.xml

    If your online server uses mirroring, it is important to ensure that all
    content is synchronized to the server before you update \e{Updates.xml}.

    You can set a cache expiration time as zero for \e{Updates.xml} in order to
    reduce delay in synchronization.

    \section1 Maintenance Tool

    A maintenance tool is built when you build the
    \l{Building Qt Installer Framework} {Qt Installer Framework}. Usually,
    the maintenance tool is included in online repositories. It is installed to
    the end user host from the online repository with other installed content.

    Online repositories should always provide the latest version of the
    maintenance tool. We recommend that you build both an online installer and
    the maintenance tool with the Qt Installer Framework version with the same
    SHA-1. Otherwise, end users can install an old version of the maintenance
    tool via the online installer. Then, if some updates on the server side are
    available only via the latest maintenance tool version, the end users with
    the old maintenance tool cannot access them.

    For example, the maintenance tool online repository could be as follows:

    \badcode
    linux_x64/desktop/tools_maintenancetool/Updates.xml
        mycompany.tools.maintenancetool/2.0.4-0meta.7z
        mycompany.tools.maintenancetool/2.0.4-0maintenancetool.7z
        mycompany.tools.maintenancetool/2.0.4-0maintenancetool.7z.sha1
    \endcode
*/

/*!
    \page qtas-sdk-troubleshooting-repositories.html
    \title Troubleshooting and Best Practices
    \previouspage qtas-sdk-build-repositories.html
    \nextpage qtas-sdk-creation.html

    When you work with online repositories, it is important to understand
    how different actions affect the repositories. \l{Troubleshooting}
    lists use cases that should be handled carefully so they do not lead to
    errors in end user host. \l{Best Practices} lists some useful
    practices that might be helpful in your daily work with online repositories.

    \section1 Troubleshooting

    \section2 Dependencies to Online Repository

    Immediately after an end user installs something from an online repository
    via an online installer or a maintenance tool, a local dependency from the
    end user host to the online repository is established. Thus you should
    follow the following quidelines:

    \list
        \li Do not remove repositories from the online server simply by deleting
        the repositories. Instead, you should add the following setting in the
        root repository:
        \badcode
            <Repository action="remove" url="...">
        \endcode
        \li Do not manually change content in online repositories as it causes
        a SHA-1 checksum mismatch.
        \li If you push an update for a component in an online repository, you
        must always remember to increase the version number of the updated component.
        See \l{Updating Repository}.
    \endlist

    \section2 Unsuccessful Updates

    If something goes wrong while you are uploading either
    \l{Uploading Content Files}{content} or \l{Uploading Updates.xml}{Updates.xml},
    do not upload the previous versions from your own backup-files.
    The Qt Installer Framework does not support roll-back functionality. Thus the
    maintenance tool does not allow end users to install again the previous
    version that is known to work.

    You should create a new version of the broken component and upload it
    to the online repository. End users need to uninstall the broken component
    and install the fixed one instead.

    \section2 Mirroring and File Caching

    If the web server uses mirroring or file caching, it is important to ensure
    that all content has been uploaded to the server before you upload a new version
    of \e{Updates.xml}.

    Some web servers support setting cache expiration time to zero for some
    files. In this case, you could set the cache expiration time to zero for
    \e{Updates.xml}.

    \section1 Best Practices

    \section2 Uploading to Test Server

    You can use a test server where you upload the repository builds for testing
    purposes. After you have verified the content, you can upload it publicly
    available to the web server that is used by online installers.

*/