summaryrefslogtreecommitdiffstats
path: root/src/qtmoduleupdater
diff options
context:
space:
mode:
authorDaniel Smith <daniel.smith@qt.io>2022-02-21 09:13:54 +0100
committerDaniel Smith <daniel.smith@qt.io>2022-02-23 11:58:37 +0100
commitba3522460e863dab432025c193865e1add27c825 (patch)
treeee7df0bf885deb819123ece28489927fe835747c /src/qtmoduleupdater
parentce354bc02ccaa872230096ce5200360401078eb4 (diff)
Remove the old Qt Dependency Updater and refer users to the new location
This dependency update script/bot has been superseded and should be removed as it is no longer functional with modern qt6 and LTS branches. Users are directed to the new utility via a README left in this repo. Change-Id: I992f64383ac1b96084d46d8c4830ac0317259a01 Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
Diffstat (limited to 'src/qtmoduleupdater')
-rw-r--r--src/qtmoduleupdater/.gitignore5
-rw-r--r--src/qtmoduleupdater/Makefile11
-rw-r--r--src/qtmoduleupdater/README.md37
-rw-r--r--src/qtmoduleupdater/TODO.md4
-rw-r--r--src/qtmoduleupdater/autorun.go78
-rw-r--r--src/qtmoduleupdater/autorun.json5
-rw-r--r--src/qtmoduleupdater/autorun_schema.json18
-rw-r--r--src/qtmoduleupdater/dependenciesupdateresultenum_string.go25
-rw-r--r--src/qtmoduleupdater/gerrit.go271
-rw-r--r--src/qtmoduleupdater/go.mod10
-rw-r--r--src/qtmoduleupdater/go.sum15
-rw-r--r--src/qtmoduleupdater/main.go136
-rw-r--r--src/qtmoduleupdater/module.go504
-rw-r--r--src/qtmoduleupdater/module_test.go129
-rw-r--r--src/qtmoduleupdater/moduleupdatebatch.go423
-rw-r--r--src/qtmoduleupdater/qt5.go262
-rw-r--r--src/qtmoduleupdater/qt5_test.go46
-rw-r--r--src/qtmoduleupdater/repo.go445
-rw-r--r--src/qtmoduleupdater/repo_test.go182
-rw-r--r--src/qtmoduleupdater/slack.go74
20 files changed, 8 insertions, 2672 deletions
diff --git a/src/qtmoduleupdater/.gitignore b/src/qtmoduleupdater/.gitignore
deleted file mode 100644
index 36c014eb..00000000
--- a/src/qtmoduleupdater/.gitignore
+++ /dev/null
@@ -1,5 +0,0 @@
-git-repos
-.vscode
-qtmoduleupdater
-state_*.json
-coin-secrets
diff --git a/src/qtmoduleupdater/Makefile b/src/qtmoduleupdater/Makefile
deleted file mode 100644
index c6182712..00000000
--- a/src/qtmoduleupdater/Makefile
+++ /dev/null
@@ -1,11 +0,0 @@
-
-service_start:
- systemd-run --user --on-calendar="*-*-* *:00:00" --same-dir --unit qtmoduleupdater $(MAKE) -d service_iterate
-
-service_stop:
- systemctl --user stop qtmoduleupdater.timer
-
-service_iterate:
- git pull --rebase
- go build
- ./qtmoduleupdater -autorun
diff --git a/src/qtmoduleupdater/README.md b/src/qtmoduleupdater/README.md
index 04d7051d..967cebf5 100644
--- a/src/qtmoduleupdater/README.md
+++ b/src/qtmoduleupdater/README.md
@@ -1,32 +1,11 @@
-# Qt Module Updater
+# Qt Submodule Updater
-This is a tool that serves the purpose of automating the process of keeping pinned dependencies between Qt git repositories up-to-date.
+**The Qt Submodule Updater has been superseded by
+qt/qtrepotools/util/dependency_updater**
-Qt modules in git repositories depend on modules from other repositories and therefore each repository encodes its dependencies to other repositories using a configuration file called ```dependencies.yaml```. It lists the required and optional dependencies as well as the commit sha1s that are known to work.
-
-All repositories with their dependencies form a graph, with qtbase typically at the root. A newer version of qtbase shall result in a change to ```dependencies.yaml``` in qtsvg. Once approved by the CI system, a change to qtdeclarative is needed to pull in the newer version of qtsvg and implicitly qtbase.
-
-This tool automates the pushing of updates through the graph of dependencies and once all modules of qt5.git are complete, an update of submodule sha1s to qt5.git will be posted.
-
-## Algorithm
-
-The process of updating dependencies starts by collecting a list of all repositories and determining the root of the graph. That's typically qtbase. From there on, updates to all repositories are posted that only depend on the root. All other repositories remain in a "todo" list. The root is remembered in a "done" list and all repositories that we are currently trying to bring up-to-date are in a "pending" list. Once this process is started, the program saves its state in personal branch under refs/personal/qt_submodule_updater_bot/state/<branch> and terminates.
-
-The next time the Qt Module Updater is started, it resumes the state and begins checking the state of all pending updates. If an update succeeded, then the corresponding repository is added to the "done" list and we can prepare updates for repositories that have now their dependencies satisfied by picking them from the "todo" list. If the update failed, the repository is dropped from the batch of updates and all other repositories that directly or indirectly depend on the failed one are also removed. After every such iteration of processing pending updates and pushing new ones to Gerrit, the process terminates and saves its state.
-
-When the todo list is empty and there are no more pending updates, the batch update is complete. If during that update there were no failures, the Qt Module Updater will also push a change to qt5.git with an update to all submodule sha1s of the new consistent set of modules.
-
-## Usage
-
-The Qt Module Updater is written in Golang and requires at least version 1.13. To build the program, simply clone the repository and run
-
- go build
-
-When running the program, git repositories will be cloned from Qt's Gerrit instance and stored as bare clones in the ```git-repos``` sub-directory.
-
-Every invocation requires passing a ```-branch=``` that specifies the Qt version branch to use as reference. By default, repositories from ```qt/qt5``` are picked up, but it is possible to override this with the ```-product=``` parameter.
-
-When run in a production environment, it is desirable to pass the ```-stage-as-bot``` parameter, to ensure that changes are pushed as the special Qt Submodule Update bot.
-
-For manual testing, it is also possible to use the ```-manual-stage``` parameter to merely push changes to Gerrit but not automatically stage them.
+This tool has been deprecated in favor of a more extensible utility
+which can be operated on-demand or as a bot, and provides support
+for non-qt/ namespaced repos among other improvements.
+Obtain the new utility at:
+https://codereview.qt-project.org/gitweb?p=qt/qtrepotools.git;a=tree;f=util/dependency_updater;hb=refs/heads/master
diff --git a/src/qtmoduleupdater/TODO.md b/src/qtmoduleupdater/TODO.md
deleted file mode 100644
index 40f07e8f..00000000
--- a/src/qtmoduleupdater/TODO.md
+++ /dev/null
@@ -1,4 +0,0 @@
-# A brief list of pending things that need to be implemented
-
-* Be more lenient towards failures during the attempt to update dependencies.yml in a module:
- * We could keep "failed" updates in the pending list, but we have to be careful about terminating the overall algorithm. Might be as simple as considering remaining pending changes as all failed if the todo list is empty otherwise.
diff --git a/src/qtmoduleupdater/autorun.go b/src/qtmoduleupdater/autorun.go
deleted file mode 100644
index 03d190f5..00000000
--- a/src/qtmoduleupdater/autorun.go
+++ /dev/null
@@ -1,78 +0,0 @@
-/****************************************************************************
-**
-** Copyright (C) 2019 The Qt Company Ltd.
-** Contact: https://www.qt.io/licensing/
-**
-** This file is part of the repo tools module of the Qt Toolkit.
-**
-** $QT_BEGIN_LICENSE:GPL-EXCEPT$
-** Commercial License Usage
-** Licensees holding valid commercial Qt licenses may use this file in
-** accordance with the commercial license agreement provided with the
-** Software or, alternatively, in accordance with the terms contained in
-** a written agreement between you and The Qt Company. For licensing terms
-** and conditions see https://www.qt.io/terms-conditions. For further
-** information use the contact form at https://www.qt.io/contact-us.
-**
-** GNU General Public License Usage
-** Alternatively, this file may be used under the terms of the GNU
-** General Public License version 3 as published by the Free Software
-** Foundation with exceptions as appearing in the file LICENSE.GPL3-EXCEPT
-** included in the packaging of this file. Please review the following
-** information to ensure the GNU General Public License requirements will
-** be met: https://www.gnu.org/licenses/gpl-3.0.html.
-**
-** $QT_END_LICENSE$
-**
-****************************************************************************/
-package main
-
-import (
- "encoding/json"
- "fmt"
- "io/ioutil"
- "os"
-)
-
-// AutoRunSettings is used to read autorun.json.
-type AutoRunSettings struct {
- Version int `json:"version"`
- Branches []string `json:"branches"`
- ProductRefs map[string]string `json:"productRefs"`
-}
-
-func (settings *AutoRunSettings) load() error {
- file, err := os.Open("autorun.json")
- if err != nil {
- return fmt.Errorf("Error opening autorun.json for reading: %s", err)
- }
- defer file.Close()
- content, err := ioutil.ReadAll(file)
- if err != nil {
- return fmt.Errorf("Error reading from autorun.json: %s", err)
- }
- return json.Unmarshal(content, settings)
-}
-
-func (settings *AutoRunSettings) runUpdates(gerrit *gerritInstance) {
- // ### might make sense to turn this into a loop over known products or read from settings. On the other hand we want
- // to avoid coding products into the file for now but rather query a list of products from Gerrit (in the future). Therefore
- // I'm keeping the product out of the file
- product := "qt/qt5"
- for _, branch := range settings.Branches {
- productRef := ""
- if specificProductRef, ok := settings.ProductRefs[branch]; ok {
- productRef = specificProductRef
- }
- batch, err := newModuleUpdateBatch(product, branch, productRef)
- if err != nil {
- fmt.Printf("Error loading update batch state for %s/%s: %s\n", product, branch, err)
- continue
- }
-
- if err := batch.runOneIteration(gerrit); err != nil {
- fmt.Printf("Error iterating on update batch for %s/%s: %s\n", product, branch, err)
- continue
- }
- }
-}
diff --git a/src/qtmoduleupdater/autorun.json b/src/qtmoduleupdater/autorun.json
deleted file mode 100644
index 046f9c03..00000000
--- a/src/qtmoduleupdater/autorun.json
+++ /dev/null
@@ -1,5 +0,0 @@
-{
- "$schema": "./autorun_schema.json",
- "version": 1,
- "branches": ["dev"]
-}
diff --git a/src/qtmoduleupdater/autorun_schema.json b/src/qtmoduleupdater/autorun_schema.json
deleted file mode 100644
index 3d8ff75b..00000000
--- a/src/qtmoduleupdater/autorun_schema.json
+++ /dev/null
@@ -1,18 +0,0 @@
-{
- "title": "qtmoduleupdater auto-run configuration",
- "type": "object",
- "properties": {
- "version": {
- "type": "integer",
- "description": "The file format version expected, currently 1."
- },
- "branches": {
- "type": "array",
- "description": "List of qt5 branches to submit updates to.",
- "items": {
- "type": "string"
- }
- }
- },
- "required": ["version"]
-}
diff --git a/src/qtmoduleupdater/dependenciesupdateresultenum_string.go b/src/qtmoduleupdater/dependenciesupdateresultenum_string.go
deleted file mode 100644
index db80b97c..00000000
--- a/src/qtmoduleupdater/dependenciesupdateresultenum_string.go
+++ /dev/null
@@ -1,25 +0,0 @@
-// Code generated by "stringer -type=DependenciesUpdateResultEnum"; DO NOT EDIT.
-
-package main
-
-import "strconv"
-
-func _() {
- // An "invalid array index" compiler error signifies that the constant values have changed.
- // Re-run the stringer command to generate them again.
- var x [1]struct{}
- _ = x[DependenciesUpdateDependencyMissing-0]
- _ = x[DependenciesUpdateContentUpToDate-1]
- _ = x[DependenciesUpdateUpdateScheduled-2]
-}
-
-const _DependenciesUpdateResultEnum_name = "DependenciesUpdateDependencyMissingDependenciesUpdateContentUpToDateDependenciesUpdateUpdateScheduled"
-
-var _DependenciesUpdateResultEnum_index = [...]uint8{0, 35, 68, 101}
-
-func (i DependenciesUpdateResultEnum) String() string {
- if i < 0 || i >= DependenciesUpdateResultEnum(len(_DependenciesUpdateResultEnum_index)-1) {
- return "DependenciesUpdateResultEnum(" + strconv.FormatInt(int64(i), 10) + ")"
- }
- return _DependenciesUpdateResultEnum_name[_DependenciesUpdateResultEnum_index[i]:_DependenciesUpdateResultEnum_index[i+1]]
-}
diff --git a/src/qtmoduleupdater/gerrit.go b/src/qtmoduleupdater/gerrit.go
deleted file mode 100644
index d37deeb5..00000000
--- a/src/qtmoduleupdater/gerrit.go
+++ /dev/null
@@ -1,271 +0,0 @@
-/****************************************************************************
-**
-** Copyright (C) 2019 The Qt Company Ltd.
-** Contact: https://www.qt.io/licensing/
-**
-** This file is part of the repo tools module of the Qt Toolkit.
-**
-** $QT_BEGIN_LICENSE:GPL-EXCEPT$
-** Commercial License Usage
-** Licensees holding valid commercial Qt licenses may use this file in
-** accordance with the commercial license agreement provided with the
-** Software or, alternatively, in accordance with the terms contained in
-** a written agreement between you and The Qt Company. For licensing terms
-** and conditions see https://www.qt.io/terms-conditions. For further
-** information use the contact form at https://www.qt.io/contact-us.
-**
-** GNU General Public License Usage
-** Alternatively, this file may be used under the terms of the GNU
-** General Public License version 3 as published by the Free Software
-** Foundation with exceptions as appearing in the file LICENSE.GPL3-EXCEPT
-** included in the packaging of this file. Please review the following
-** information to ensure the GNU General Public License requirements will
-** be met: https://www.gnu.org/licenses/gpl-3.0.html.
-**
-** $QT_END_LICENSE$
-**
-****************************************************************************/
-package main
-
-import (
- "encoding/json"
- "fmt"
- "log"
- "net"
- "net/url"
- "os"
- "os/exec"
- "strings"
-)
-
-// GerritPatchSet corresponds to the patch set JSON object returned by Gerrit's JSON API.
-type GerritPatchSet struct {
- Number int `json:"number"`
- Revision string `json:"revision"`
- Parents []string `json:"parents"`
- Uploader json.RawMessage `json:"uploader"`
- CreatedOn uint `json:"createdOn"`
- Author json.RawMessage `json:"author"`
- SizeInsertions int `json:"sizeInsertions"`
- SizeDeletions int `json:"sizeDeletions"`
-}
-
-// GerritChangeOrStats corresponds to the JSON data returned by Gerrit's Query JSON API.
-type GerritChangeOrStats struct {
- Type string `json:"type"`
- RowCount int `json:"rowCount"`
- RunTimeMilliseconds int `json:"runTimeMilliseconds"`
-
- Project string `json:"project"`
- Branch string `json:"branch"`
- ID string `json:"id"`
- Number int `json:"number"`
- Subject string `json:"subject"`
- Owner json.RawMessage `json:"owner"`
- URL string `json:"url"`
- CreatedOn uint `json:"createdOn"`
- LastUpdated uint `json:"lastUpdated"`
- SortKey string `json:"sortKey"`
- Open bool `json:"open"`
- Status string `json:"status"`
- PatchSets []GerritPatchSet `json:"patchSets"`
-}
-
-func gerritSSHCommand(gerritURL url.URL, arguments ...string) (*exec.Cmd, error) {
- host, port, err := net.SplitHostPort(gerritURL.Host)
- if err != nil {
- return nil, fmt.Errorf("Error splitting host and port from gerrit URL: %s", err)
- }
-
- userAtHost := host
- if gerritURL.User != nil {
- userAtHost = gerritURL.User.Username() + "@" + host
- }
-
- newArgs := []string{"-oBatchMode=yes", userAtHost, "-p", port}
- newArgs = append(newArgs, arguments...)
- ssh := os.Getenv("GIT_SSH_COMMAND")
- if ssh != "" {
- commandLine := strings.Split(ssh, " ")
- ssh = commandLine[0]
- newArgs = append(commandLine[1:], newArgs...)
- } else {
- ssh = "ssh"
- }
- log.Printf("Running gerrit ssh command: 'ssh %v'\n", newArgs)
- return exec.Command(ssh, newArgs...), nil
-}
-
-func getGerritChangeStatus(project string, branch string, changeID string) (status string, err error) {
- gerritURL, err := RepoPushURL(project)
- if err != nil {
- return "", fmt.Errorf("Error parsing gerrit URL: %s", err)
- }
- queryString := fmt.Sprintf(`project:%s branch:%s %s`, project, branch, changeID)
- cmd, err := gerritSSHCommand(*gerritURL, "gerrit", "query", "--patch-sets", "--format JSON", queryString)
- if err != nil {
- return "", err
- }
- output, err := cmd.Output()
- if err != nil {
- return "", fmt.Errorf("Error running gerrit query command: %s", err)
- }
-
- var id string
-
- for _, line := range strings.Split(string(output), "\n") {
- line = strings.TrimSpace(line)
- if line == "" {
- continue
- }
-
- var field GerritChangeOrStats
- err = json.Unmarshal([]byte(line), &field)
- if err != nil {
- return "", fmt.Errorf("Error reading gerrit json response: %s:%s", err, string(output))
- }
- if field.Type == "stats" {
- if field.RowCount != 1 {
- return "", fmt.Errorf("unexpected row count %v when querying for existing gerrit change", field.RowCount)
- }
- continue
- }
-
- if field.Project != project {
- return "", fmt.Errorf("unexpectedly found change for a different project. Received %s, expected %s for %s", field.Project, project, changeID)
- }
- if id != "" {
- return "", fmt.Errorf("unexpectedly found multiple changes for change ID %s", changeID)
- }
- id = field.ID
- status = field.Status
- }
- return status, nil
-}
-
-func getExistingChange(project string, branch string) (gerritChangeID string, changeNumber int, patchSetNr int, status string, err error) {
- gerritURL, err := RepoPushURL(project)
- if err != nil {
- return "", 0, 0, "", fmt.Errorf("Error parsing gerrit URL: %s", err)
- }
- queryString := fmt.Sprintf(`project:%s branch:%s NOT(status:merged OR status:abandoned OR status:deferred) owner:self message:{Update dependencies on \'%s\' in %s}`, project, branch, branch, project)
- cmd, err := gerritSSHCommand(*gerritURL, "gerrit", "query", "--patch-sets", "--format JSON", queryString)
- if err != nil {
- return "", 0, 0, "", err
- }
- output, err := cmd.Output()
- if err != nil {
- return "", 0, 0, "", fmt.Errorf("Error running gerrit query command: %s", err)
- }
-
- var id string
-
- for _, line := range strings.Split(string(output), "\n") {
- line = strings.TrimSpace(line)
- if line == "" {
- continue
- }
-
- var field GerritChangeOrStats
- err = json.Unmarshal([]byte(line), &field)
- if err != nil {
- return "", 0, 0, "", fmt.Errorf("Error reading gerrit json response: %s:%s", err, string(output))
- }
- if field.Type == "stats" {
- if field.RowCount == 0 {
- return "", 0, 0, "", nil
- }
- if field.RowCount != 1 {
- return "", 0, 0, "", fmt.Errorf("unexpected row count %v when querying for existing gerrit changes", field.RowCount)
- }
- continue
- }
-
- if field.Project == project {
- if id != "" {
- return "", 0, 0, "", fmt.Errorf("unexpectedly found multiple changes for submodule updates: Id %s and %s", id, field.ID)
- }
- id = field.ID
- changeNumber = field.Number
- status = field.Status
- patchSetNr = 0
- for _, patchSet := range field.PatchSets {
- if patchSet.Number > patchSetNr {
- patchSetNr = patchSet.Number
- }
- }
- continue
- }
- }
- return id, changeNumber, patchSetNr, status, nil
-}
-
-func escapeGerritMessage(message string) string {
- replacer := strings.NewReplacer(`\`, `\\`, `"`, `\"`, `'`, `\'`)
- return `"` + replacer.Replace(message) + `"`
-}
-
-type gerritInstance struct {
- pushUserName string
- disableStaging bool
-}
-
-func (instance *gerritInstance) pushChange(repoPath string, branch string, commitID OID, summary string) error {
- repo, err := OpenRepository(repoPath)
- if err != nil {
- return err
- }
-
- pushURL, err := RepoURL(repoPath)
- if err != nil {
- return err
- }
- if instance.pushUserName != "" {
- pushURL.User = url.User(instance.pushUserName)
- }
-
- return repo.Push(pushURL, nil, commitID, "refs/for/"+branch)
-}
-
-func (instance *gerritInstance) reviewAndStageChange(repoPath string, branch string, commitID OID, summary string) error {
- if instance.disableStaging {
- return nil
- }
-
- pushURL, err := RepoPushURL(repoPath)
- if err != nil {
- return err
- }
- // Always review/approve as current user as the bot does not have approval rights.
- pushURL.User = nil
-
- reviewArgs := []string{"gerrit", "review", string(commitID)}
-
- if summary != "" {
- reviewArgs = append(reviewArgs, "-m", escapeGerritMessage(summary))
- }
- // Pass in sanity review, since the sanity bot runs only after a delay and thus the commit will get refused.
- reviewArgs = append(reviewArgs, "--code-review", "2", "--sanity-review", "1")
-
- updateCommand, err := gerritSSHCommand(*pushURL, reviewArgs...)
- if err != nil {
- return err
- }
- updateCommand.Stdout = os.Stdout
- updateCommand.Stderr = os.Stderr
- if err = updateCommand.Run(); err != nil {
- return err
- }
-
- stageArgs := []string{"gerrit-plugin-qt-workflow", "stage", string(commitID)}
- updateCommand, err = gerritSSHCommand(*pushURL, stageArgs...)
- if err != nil {
- return err
- }
- updateCommand.Stdout = os.Stdout
- updateCommand.Stderr = os.Stderr
- if err = updateCommand.Run(); err != nil {
- return err
- }
- return nil
-}
diff --git a/src/qtmoduleupdater/go.mod b/src/qtmoduleupdater/go.mod
deleted file mode 100644
index 075eced5..00000000
--- a/src/qtmoduleupdater/go.mod
+++ /dev/null
@@ -1,10 +0,0 @@
-module qtmoduleupdater
-
-go 1.13
-
-require (
- github.com/kardianos/osext v0.0.0-20190222173326-2bc1f35cddc0
- github.com/stretchr/testify v1.4.0
- github.com/vaughan0/go-ini v0.0.0-20130923145212-a98ad7ee00ec
- gopkg.in/yaml.v2 v2.2.2
-)
diff --git a/src/qtmoduleupdater/go.sum b/src/qtmoduleupdater/go.sum
deleted file mode 100644
index dd88ecff..00000000
--- a/src/qtmoduleupdater/go.sum
+++ /dev/null
@@ -1,15 +0,0 @@
-github.com/davecgh/go-spew v1.1.0 h1:ZDRjVQ15GmhC3fiQ8ni8+OwkZQO4DARzQgrnXU1Liz8=
-github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
-github.com/kardianos/osext v0.0.0-20190222173326-2bc1f35cddc0 h1:iQTw/8FWTuc7uiaSepXwyf3o52HaUYcV+Tu66S3F5GA=
-github.com/kardianos/osext v0.0.0-20190222173326-2bc1f35cddc0/go.mod h1:1NbS8ALrpOvjt0rHPNLyCIeMtbizbir8U//inJ+zuB8=
-github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
-github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
-github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
-github.com/stretchr/testify v1.4.0 h1:2E4SXV/wtOkTonXsotYi4li6zVWxYlZuYNCXe9XRJyk=
-github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
-github.com/vaughan0/go-ini v0.0.0-20130923145212-a98ad7ee00ec h1:DGmKwyZwEB8dI7tbLt/I/gQuP559o/0FrAkHKlQM/Ks=
-github.com/vaughan0/go-ini v0.0.0-20130923145212-a98ad7ee00ec/go.mod h1:owBmyHYMLkxyrugmfwE/DLJyW8Ro9mkphwuVErQ0iUw=
-gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
-gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
-gopkg.in/yaml.v2 v2.2.2 h1:ZCJp+EgiOT7lHqUV2J862kp8Qj64Jo6az82+3Td9dZw=
-gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
diff --git a/src/qtmoduleupdater/main.go b/src/qtmoduleupdater/main.go
deleted file mode 100644
index 3d2e2809..00000000
--- a/src/qtmoduleupdater/main.go
+++ /dev/null
@@ -1,136 +0,0 @@
-/****************************************************************************
-**
-** Copyright (C) 2019 The Qt Company Ltd.
-** Contact: https://www.qt.io/licensing/
-**
-** This file is part of the repo tools module of the Qt Toolkit.
-**
-** $QT_BEGIN_LICENSE:GPL-EXCEPT$
-** Commercial License Usage
-** Licensees holding valid commercial Qt licenses may use this file in
-** accordance with the commercial license agreement provided with the
-** Software or, alternatively, in accordance with the terms contained in
-** a written agreement between you and The Qt Company. For licensing terms
-** and conditions see https://www.qt.io/terms-conditions. For further
-** information use the contact form at https://www.qt.io/contact-us.
-**
-** GNU General Public License Usage
-** Alternatively, this file may be used under the terms of the GNU
-** General Public License version 3 as published by the Free Software
-** Foundation with exceptions as appearing in the file LICENSE.GPL3-EXCEPT
-** included in the packaging of this file. Please review the following
-** information to ensure the GNU General Public License requirements will
-** be met: https://www.gnu.org/licenses/gpl-3.0.html.
-**
-** $QT_END_LICENSE$
-**
-****************************************************************************/
-package main
-
-import (
- "flag"
- "fmt"
- "io/ioutil"
- "log"
- "os"
-)
-
-func setupEnvironmentForSubmoduleUpdateBot() (username string, err error) {
- submoduleUpdateBotKeyPath := "coin-secrets/submodule_update_bot_key_rsa"
- if _, err = os.Stat(submoduleUpdateBotKeyPath); os.IsNotExist(err) {
- err = fmt.Errorf("cannot locate submodule update bot SSH key file. Please copy it from the coin secrets repo into the current directory")
- return
- }
-
- os.Setenv("GIT_SSH_COMMAND", fmt.Sprintf("ssh -i %s", submoduleUpdateBotKeyPath))
- os.Setenv("GIT_SSH_USER", "qt_submodule_update_bot")
-
- os.Setenv("GIT_AUTHOR_NAME", "Qt Submodule Update Bot")
- os.Setenv("GIT_COMMITTER_NAME", "Qt Submodule Update Bot")
- os.Setenv("GIT_AUTHOR_EMAIL", "qt_submodule_update_bot@qt-project.org")
- os.Setenv("GIT_COMMITTER_EMAIL", "qt_submodule_update_bot@qt-project.org")
-
- username = "qt_submodule_update_bot"
- return
-}
-
-func appMain() error {
- var product string
- flag.StringVar(&product, "product", "qt/qt5" /*default*/, "Product repository to use as reference and push completed updates to")
- stageAsBot := false
- flag.BoolVar(&stageAsBot, "stage-as-bot", false /*default*/, "Push changes to Gerrit using the submodule update bot account")
- var branch string
- flag.StringVar(&branch, "branch", "", "Branch to update")
- var productRef string
- flag.StringVar(&productRef, "product-ref", "", "Git ref in qt5 to use as basis for a new round of updates")
- manualStage := false
- flag.BoolVar(&manualStage, "manual-stage", false /*default*/, "Do not stage changes automatically")
- summaryOnly := false
- flag.BoolVar(&summaryOnly, "summarize", false /*default*/, "")
- verbose := false
- flag.BoolVar(&verbose, "verbose", false /*default*/, "Enable verbose logging output")
- reset := false
- flag.BoolVar(&reset, "reset", false, "Reset the batch update state")
- autorun := false
- flag.BoolVar(&autorun, "autorun", false, "Run automatically by reading settings from autorun.json")
- flag.Parse()
-
- if !verbose {
- oldWriter := log.Writer()
- defer log.SetOutput(oldWriter)
- log.SetOutput(ioutil.Discard)
- }
-
- if autorun {
- stageAsBot = true
- }
-
- gerrit := &gerritInstance{}
- gerrit.disableStaging = manualStage
-
- if stageAsBot {
- var err error
- gerrit.pushUserName, err = setupEnvironmentForSubmoduleUpdateBot()
- if err != nil {
- return fmt.Errorf("error preparing environment to work as submodule-update user: %s", err)
- }
- initSlackIntegration()
- }
-
- if autorun {
- autorun := &AutoRunSettings{}
- if err := autorun.load(); err != nil {
- return err
- }
- autorun.runUpdates(gerrit)
- return nil
- }
-
- if branch == "" {
- return fmt.Errorf("missing branch. Please specify -branch=<name of branch>")
- }
-
- batch, err := newModuleUpdateBatch(product, branch, productRef)
- if err != nil {
- return err
- }
-
- if summaryOnly {
- batch.printSummary()
- return nil
- }
-
- if reset {
- batch.clearStateCommit(gerrit)
- return nil
- }
-
- return batch.runOneIteration(gerrit)
-}
-
-func main() {
- err := appMain()
- if err != nil {
- log.Fatalf("Error: %s\n", err)
- }
-}
diff --git a/src/qtmoduleupdater/module.go b/src/qtmoduleupdater/module.go
deleted file mode 100644
index 99244d0f..00000000
--- a/src/qtmoduleupdater/module.go
+++ /dev/null
@@ -1,504 +0,0 @@
-/****************************************************************************
-**
-** Copyright (C) 2019 The Qt Company Ltd.
-** Contact: https://www.qt.io/licensing/
-**
-** This file is part of the repo tools module of the Qt Toolkit.
-**
-** $QT_BEGIN_LICENSE:GPL-EXCEPT$
-** Commercial License Usage
-** Licensees holding valid commercial Qt licenses may use this file in
-** accordance with the commercial license agreement provided with the
-** Software or, alternatively, in accordance with the terms contained in
-** a written agreement between you and The Qt Company. For licensing terms
-** and conditions see https://www.qt.io/terms-conditions. For further
-** information use the contact form at https://www.qt.io/contact-us.
-**
-** GNU General Public License Usage
-** Alternatively, this file may be used under the terms of the GNU
-** General Public License version 3 as published by the Free Software
-** Foundation with exceptions as appearing in the file LICENSE.GPL3-EXCEPT
-** included in the packaging of this file. Please review the following
-** information to ensure the GNU General Public License requirements will
-** be met: https://www.gnu.org/licenses/gpl-3.0.html.
-**
-** $QT_END_LICENSE$
-**
-****************************************************************************/
-package main
-
-import (
- "bytes"
- "fmt"
- "log"
- "os"
- "path/filepath"
- "regexp"
- "sort"
- "strings"
-
- yaml "gopkg.in/yaml.v2"
-)
-
-// Module represents a git repository with a dependencies.yaml file
-// that needs updating.
-type Module struct {
- RepoPath string // relative path in Gerrit, such as qt/qtsvg
- RequiredDependencies []string // Dependencies as per dependencies.yaml
- OptionalDependencies []string // Dependencies as per dependencies.yaml
- Branch string
- Tip OID
-}
-
-// YAMLModule for unmarshaling module information from dependencies file.
-type YAMLModule struct {
- Ref string `yaml:"ref"`
- Required bool `yaml:"required"`
-}
-
-// YAMLDependenciesMap is a map from string to YAMLModule but always
-// produces a sorted YAML map when serializing
-type YAMLDependenciesMap map[string]*YAMLModule
-
-// YAMLDependencies for unmarshaling module information from dependencies file.
-type YAMLDependencies struct {
- Dependencies YAMLDependenciesMap `yaml:"dependencies"`
-}
-
-var shaValidator = regexp.MustCompile(`\b[0-9a-f]{40}\b`)
-
-// MarshalYAML implements the marshalling of the dependencies while
-// making sure the entries are sorted.
-func (depMap *YAMLDependenciesMap) MarshalYAML() (interface{}, error) {
- var sortedKeys []string
- for key := range *depMap {
- sortedKeys = append(sortedKeys, key)
- }
-
- sort.Strings(sortedKeys)
-
- var result yaml.MapSlice
-
- for _, key := range sortedKeys {
- entry := (*depMap)[key]
- result = append(result, yaml.MapItem{
- Key: key,
- Value: entry,
- })
- }
-
- return result, nil
-}
-
-// ToString Converts the yaml dependencies map into its string representation, for storage
-// in the dependencies.yaml file, for example.
-func (depMap *YAMLDependencies) ToString() (string, error) {
- output := &bytes.Buffer{}
- encoder := yaml.NewEncoder(output)
- if err := encoder.Encode(depMap); err != nil {
- return "", fmt.Errorf("Error encoding YAML dependencies: %s", err)
- }
- encoder.Close()
-
- return output.String(), nil
-}
-
-//go:generate stringer -type=DependenciesUpdateResultEnum
-
-// DependenciesUpdateResultEnum describes the different states after attempting to update the dependencies.yaml for a module.
-type DependenciesUpdateResultEnum int
-
-const (
- // DependenciesUpdateDependencyMissing indicates that a dependency is not available yet.
- DependenciesUpdateDependencyMissing DependenciesUpdateResultEnum = iota
- // DependenciesUpdateContentUpToDate indicates that no further updates to dependencies.yaml are required.
- DependenciesUpdateContentUpToDate
- // DependenciesUpdateUpdateScheduled indicates that an update to dependencies.yaml was necessary and has been pushed to Gerrit.
- DependenciesUpdateUpdateScheduled
-)
-
-type pathNotExistError struct {
- path string
-}
-
-func (p *pathNotExistError) Error() string {
- return fmt.Sprintf("Could not locate %s in git tree", p.path)
-}
-
-func readDependenciesYAML(repoPath string, repo Repository, commit OID) (dependencies *YAMLDependencies, err error) {
- path := "dependencies.yaml"
-
- tree, err := repo.ListTree(commit)
- if err != nil {
- return nil, fmt.Errorf("could not list tree for commit %s: %s", commit, err)
- }
-
- entry, ok := tree.Entries[path]
- if !ok {
- return nil, &pathNotExistError{path}
- }
-
- if entry.Type != ObjectBlob {
- return nil, fmt.Errorf("%s is not a file/blob", path)
- }
-
- blob, err := repo.LookupBlob(entry.ID)
- if err != nil {
- return nil, fmt.Errorf("Error looking up %s blob: %s", path, err)
- }
-
- yamlData := &YAMLDependencies{}
- err = yaml.Unmarshal(blob, yamlData)
- if err != nil {
- return nil, fmt.Errorf("Error unmarshaling dependencies.yaml: %s", err)
- }
-
- if yamlData.Dependencies != nil {
- for name, dependency := range yamlData.Dependencies {
- if strings.HasPrefix(name, "/") {
- continue
- }
- absoluteName := filepath.Clean(filepath.Join(repoPath, name))
- delete(yamlData.Dependencies, name)
- yamlData.Dependencies[absoluteName] = dependency
- }
- }
-
- return yamlData, nil
-}
-
-// NewModule constructs a new module type for a dependencies.yaml update
-// from a given qt5 submodule structure.
-func NewModule(moduleName string, branch string, qt5Modules map[string]*submodule) (*Module, error) {
- var repoPath string
- if !strings.Contains(moduleName, "/") {
- repoPath = "qt/" + moduleName
- } else {
- repoPath = moduleName
- }
-
- repo, err := OpenRepository(repoPath)
- if err != nil {
- return nil, fmt.Errorf("Error opening submodule %s: %s", moduleName, err)
- }
-
- if subModule, ok := qt5Modules[moduleName]; ok {
- branch = subModule.branch
- }
- headRef := "refs/heads/" + branch
-
- repoURL, err := RepoURL(repoPath)
- if err != nil {
- return nil, fmt.Errorf("could not find fetch url for %s: %s", moduleName, err)
- }
-
- moduleTipCommit, err := repo.Fetch(repoURL, headRef)
- if err != nil {
- return nil, fmt.Errorf("could not fetch repo tip %s of %s: %s", headRef, moduleName, err)
- }
-
- var sha string
-
- if repoPath == "qt/qtbase" {
- fmt.Printf("\nProposing to update %s to %s. You may now override this sha if desired.\n", repoPath, moduleTipCommit)
- fmt.Print("WARN: Overriding shas may cause an inconsistent set. Only do this if you know what you're doing.\n")
- for {
- if os.Getenv("CUSTOM_QTBASE") != "" {
- sha = os.Getenv("CUSTOM_QTBASE")
- fmt.Printf("Using custom Qtbase SHA from environment variable CUSTOM_QTBASE: %s\n", sha)
- fmt.Println("NOTICE: No sha validation occurs on input from environment variables.")
- moduleTipCommit = OID(strings.Trim(sha, " "))
- break
- } else if os.Getenv("AUTORUN") == "" {
- fmt.Print("\nPress Return to accept default proposal or enter a custom sha:\n->")
- fmt.Scanln(&sha)
- }
- if sha != "" {
- sha = strings.Trim(sha, " ")
- if shaValidator.MatchString(sha) {
- moduleTipCommit = OID(sha)
- fmt.Printf("Using custom sha for %s: %s\n", repoPath, moduleTipCommit)
- break
- } else {
- fmt.Printf("Custom sha \"%s\" is not a valid SHA1. Try again or accept the default.\n", sha)
- sha = ""
- }
- } else {
- fmt.Printf("Using default HEAD sha for %s: %s", repoPath, moduleTipCommit)
- break
- }
- }
- }
-
- yamlDependencies := &YAMLDependencies{}
- yamlDependencies.Dependencies = make(map[string]*YAMLModule)
-
- subModule, ok := qt5Modules[moduleName]
- if !ok {
- return nil, fmt.Errorf("could not find %s in .gitmodules in qt5.git", moduleName)
- }
-
- populateDependencies := func(required bool, dependencies []string) {
- for _, dependency := range dependencies {
- _, knownModule := qt5Modules[dependency]
- if !required && !knownModule {
- continue
- }
-
- var yamlModule YAMLModule
- yamlModule.Required = required
- yamlModule.Ref = string(subModule.headCommit)
-
- yamlDependencies.Dependencies[dependency] = &yamlModule
- }
- }
-
- populateDependencies(true, subModule.requiredDependencies)
- populateDependencies(false, subModule.optionalDependencies)
-
- result := &Module{}
- result.RepoPath = repoPath
- result.Branch = branch
- result.Tip = moduleTipCommit
-
- for dependency, yamlModule := range yamlDependencies.Dependencies {
- if yamlModule.Required {
- result.RequiredDependencies = append(result.RequiredDependencies, dependency)
- } else {
- result.OptionalDependencies = append(result.OptionalDependencies, dependency)
- }
- }
-
- return result, nil
-}
-
-func (module *Module) hasDependency(dependency string) bool {
- for _, dep := range module.RequiredDependencies {
- if dep == dependency {
- return true
- }
- }
- for _, dep := range module.OptionalDependencies {
- if dep == dependency {
- return true
- }
- }
- return false
-}
-
-func (module *Module) refreshTip() error {
- repo, err := OpenRepository(module.RepoPath)
- if err != nil {
- return fmt.Errorf("Error opening submodule %s: %s", module.RepoPath, err)
- }
-
- headRef := "refs/heads/" + module.Branch
-
- repoURL, err := RepoURL(module.RepoPath)
- if err != nil {
- return fmt.Errorf("could not find fetch url for %s: %s", module.RepoPath, err)
- }
-
- moduleTipCommit, err := repo.Fetch(repoURL, headRef)
- if err != nil {
- return fmt.Errorf("could not fetch repo tip %s of %s: %s", headRef, module.RepoPath, err)
- }
-
- module.Tip = moduleTipCommit
- return nil
-}
-
-func (module *Module) maybePrepareUpdatedDependenciesYaml(availableModules map[string]*Module) (yaml *YAMLDependencies, err error) {
- var proposedUpdate YAMLDependencies
- proposedUpdate.Dependencies = make(map[string]*YAMLModule)
-
- updateDeps := func(required bool, deps []string) (allDependenciesAvailable bool, err error) {
- for _, dep := range deps {
- depModule, ok := availableModules[dep]
- if !ok {
- return false, nil
- }
-
- yamlModule := &YAMLModule{}
- yamlModule.Required = required
- yamlModule.Ref = string(depModule.Tip)
-
- path, err := filepath.Rel(module.RepoPath, depModule.RepoPath)
- if err != nil {
- path = module.RepoPath
- }
- proposedUpdate.Dependencies[path] = yamlModule
- }
- return true, nil
- }
-
- if allDepsOk, err := updateDeps( /*required*/ true, module.RequiredDependencies); err != nil || !allDepsOk {
- return nil, err
- }
- if allDepsOk, err := updateDeps( /*required*/ false, module.OptionalDependencies); err != nil || !allDepsOk {
- return nil, err
- }
-
- return &proposedUpdate, nil
-}
-
-func lookupPathIndexEntry(index *Index, path string) (*IndexEntry, error) {
- for i := 0; i < index.EntryCount(); i++ {
- entry, err := index.EntryByIndex(i)
- if err != nil {
- return nil, err
- }
- if entry.Path == path {
- return entry, nil
- }
- }
- return nil, fmt.Errorf("could not locate path %s in index", path)
-}
-
-func (module *Module) generateChangeLogOfDependencies(oldDependencies *YAMLDependencies, newDependencies *YAMLDependencies) string {
- if oldDependencies == nil || newDependencies == nil || oldDependencies.Dependencies == nil || newDependencies.Dependencies == nil {
- log.Printf("Empty set of dependencies for change log update for %s\n", module.RepoPath)
- return ""
- }
-
- var changeLog []string
-
- for dependencyName, dependency := range newDependencies.Dependencies {
- dependencyRepoPath := filepath.Clean(filepath.Join(module.RepoPath, dependencyName))
- oldDependency, ok := oldDependencies.Dependencies[dependencyRepoPath]
- if !ok {
- log.Printf("Could not find module %s in the old dependencies table %v\n", dependencyRepoPath, oldDependencies.Dependencies)
- continue
- }
- oldSha1 := oldDependency.Ref
- newSha1 := dependency.Ref
-
- depRepo, err := OpenRepository(dependencyRepoPath)
- if err != nil {
- log.Printf("Could not open dependency repo %s for changelog analysis: %s", dependencyRepoPath, err)
- continue
- }
-
- changes, err := depRepo.LogOutput(`--pretty=format: %m %s`, "--first-parent", string(oldSha1)+".."+string(newSha1))
- if err != nil {
- log.Printf("Oddly git log failed: %s\n", err)
- continue
- }
- changeLog = append(changeLog, fmt.Sprintf("%s %s..%s:", dependencyRepoPath, oldSha1, newSha1))
- changeLog = append(changeLog, changes...)
- changeLog = append(changeLog, "")
- }
-
- summary := strings.Join(changeLog, "\n ")
- // Limit due to maximum command line size when talking to gerrit via ssh :(
- if len(summary) > 65000 {
- summary = summary[:65000]
- }
- return summary
-}
-
-type dependenciesUpdateResult struct {
- result DependenciesUpdateResultEnum
- changeID string
- commitID OID
- summary string
-}
-
-func (module *Module) updateDependenciesForModule(availableModules map[string]*Module) (result dependenciesUpdateResult, err error) {
- yamlObject, err := module.maybePrepareUpdatedDependenciesYaml(availableModules)
- if err != nil {
- return dependenciesUpdateResult{}, err
- }
- if yamlObject == nil {
- return dependenciesUpdateResult{result: DependenciesUpdateDependencyMissing}, nil
- }
-
- repo, err := OpenRepository(module.RepoPath)
- if err != nil {
- return dependenciesUpdateResult{}, fmt.Errorf("Error opening repo to retrieve tip: %s", err)
- }
-
- index, err := repo.NewIndex()
- if err != nil {
- return dependenciesUpdateResult{}, fmt.Errorf("Error creating temporary git index: %s", err)
- }
- defer index.Free()
-
- module.refreshTip()
-
- err = index.ReadTree(module.Tip)
- if err != nil {
- return dependenciesUpdateResult{}, fmt.Errorf("Error populating temporary index from tree: %s", err)
- }
-
- existingEntry, _ := lookupPathIndexEntry(index, "dependencies.yaml")
-
- updatedIndexEntryForFile := &IndexEntry{
- Permissions: "100644",
- Path: "dependencies.yaml",
- }
-
- yamlStr, err := yamlObject.ToString()
- if err != nil {
- return dependenciesUpdateResult{}, fmt.Errorf("Internal error encoding yaml to string: %s", err)
- }
-
- if err := index.HashObject(updatedIndexEntryForFile, []byte(yamlStr)); err != nil {
- return dependenciesUpdateResult{}, err
- }
-
- var summary string
-
- if existingEntry != nil {
- if updatedIndexEntryForFile.ID == existingEntry.ID {
- return dependenciesUpdateResult{result: DependenciesUpdateContentUpToDate}, nil
- }
-
- log.Printf("Found existing dependencies file in %s, trying to read it to compare\n", module.RepoPath)
- oldDependenciesFile, err := readDependenciesYAML(module.RepoPath, repo, module.Tip)
- if err != nil {
- log.Printf("Could not decode existing yaml dependencies file for change log generation purposes: %s\n", err)
- } else {
- summary = module.generateChangeLogOfDependencies(oldDependenciesFile, yamlObject)
- }
- }
-
- if err := index.Add(updatedIndexEntryForFile); err != nil {
- return dependenciesUpdateResult{}, err
- }
-
- newTree, err := index.WriteTree()
- if err != nil {
- return dependenciesUpdateResult{}, err
- }
-
- changeID, _, _, status, err := getExistingChange(module.RepoPath, module.Branch)
- if err != nil {
- return dependenciesUpdateResult{}, fmt.Errorf("failure to check for existing change id for module %s: %s", module.RepoPath, err)
- }
-
- if changeID == "" {
- changeID = fmt.Sprintf("I%s", newTree)
- } else if status == "STAGED" || status == "INTEGRATING" || status == "STAGING" {
- // Assume that this is still work in progress, so try again later.
- return dependenciesUpdateResult{result: DependenciesUpdateDependencyMissing}, nil
- }
-
- message := fmt.Sprintf("Update dependencies on '%s' in %s\n\nChange-Id: %s\n", module.Branch, module.RepoPath, changeID)
-
- parentCommit := module.Tip
-
- commitOid, err := repo.CommitTree(newTree, message, parentCommit)
- if err != nil {
- return dependenciesUpdateResult{}, fmt.Errorf("Error creating git commit for dependencies update in module %s: %s", module.RepoPath, err)
- }
-
- log.Printf("New update commit created for %s: %s", module.RepoPath, commitOid)
-
- return dependenciesUpdateResult{
- result: DependenciesUpdateUpdateScheduled,
- changeID: changeID,
- commitID: commitOid,
- summary: summary,
- }, nil
-}
diff --git a/src/qtmoduleupdater/module_test.go b/src/qtmoduleupdater/module_test.go
deleted file mode 100644
index 0e665642..00000000
--- a/src/qtmoduleupdater/module_test.go
+++ /dev/null
@@ -1,129 +0,0 @@
-/****************************************************************************
-**
-** Copyright (C) 2019 The Qt Company Ltd.
-** Contact: https://www.qt.io/licensing/
-**
-** This file is part of the repo tools module of the Qt Toolkit.
-**
-** $QT_BEGIN_LICENSE:GPL-EXCEPT$
-** Commercial License Usage
-** Licensees holding valid commercial Qt licenses may use this file in
-** accordance with the commercial license agreement provided with the
-** Software or, alternatively, in accordance with the terms contained in
-** a written agreement between you and The Qt Company. For licensing terms
-** and conditions see https://www.qt.io/terms-conditions. For further
-** information use the contact form at https://www.qt.io/contact-us.
-**
-** GNU General Public License Usage
-** Alternatively, this file may be used under the terms of the GNU
-** General Public License version 3 as published by the Free Software
-** Foundation with exceptions as appearing in the file LICENSE.GPL3-EXCEPT
-** included in the packaging of this file. Please review the following
-** information to ensure the GNU General Public License requirements will
-** be met: https://www.gnu.org/licenses/gpl-3.0.html.
-**
-** $QT_END_LICENSE$
-**
-****************************************************************************/
-package main
-
-import (
- "testing"
-
- "github.com/stretchr/testify/assert"
-)
-
-func TestModuleYamlMarshalling(t *testing.T) {
- var module YAMLDependencies
- module.Dependencies = make(map[string]*YAMLModule)
-
- module.Dependencies["a"] = &YAMLModule{
- Ref: "refs/heads/foo",
- Required: false,
- }
- module.Dependencies["b"] = &YAMLModule{
- Ref: "refs/heads/bar",
- Required: true,
- }
-
- var yamlStr string
- var err error
- yamlStr, err = module.ToString()
- assert.Nil(t, err, "Conversion to yaml string must succeed")
-
- assert.Equal(t, `dependencies:
- a:
- ref: refs/heads/foo
- required: false
- b:
- ref: refs/heads/bar
- required: true
-`, yamlStr, "Yaml output should be as expected")
-}
-
-func TestProposedUpdateFailsForModulesThatDependOnMoreThanQtBase(t *testing.T) {
- // Make sure that this always points to the latest LTS branch. If it fails, update it.
- ref := "refs/heads/5.12"
- qt5Modules, err := getQt5ProductModules("qt/qt5", ref, "")
-
- assert.Nil(t, err, "Retrieving qt5 modules expected to work")
-
- todoMap, availableModules, err := loadTodoAndDoneModuleMapFromSubModules(ref, qt5Modules)
- assert.Nil(t, err, "No error expected creating module map")
-
- _, ok := availableModules["qt/qtbase"]
- assert.True(t, ok, "qt/qtbase must be present in the module map")
-
- qtSvg, ok := todoMap["qt/qtsvg"]
- assert.True(t, ok, "qtsvg must be present in the module map")
- yamlObject, err := qtSvg.maybePrepareUpdatedDependenciesYaml(availableModules)
- assert.Nil(t, err, "No error expected preparing dependencies.yaml update")
- assert.NotNil(t, yamlObject, "Yaml object must be defined for qtsvg")
-
- yamlStr, err := yamlObject.ToString()
- assert.Nil(t, err, "Conversion to yaml string must succeed")
-
- assert.Nil(t, err, "It should be possible to create a new dependencies.yaml file for qtsvg")
- assert.NotEqual(t, "", yamlStr, "Yaml string must not be empty for qtsvg")
-
- qtDeclarative, ok := todoMap["qt/qtdeclarative"]
- assert.True(t, ok, "qtdeclarative must be present in the module map")
- yamlObject, err = qtDeclarative.maybePrepareUpdatedDependenciesYaml(availableModules)
-
- assert.Nil(t, err, "It should be possible to create a new dependencies.yaml file for qtdeclarative")
- assert.Nil(t, yamlObject, "Yaml string be empty for qtdeclarative because dependencies are not available yet")
-}
-
-func TestRemovalOfNonExistentOptionalDependencies(t *testing.T) {
- // Make sure that this always points to the latest LTS branch. If it fails, update it.
- ref := "refs/heads/5.12"
- qt5Modules, err := getQt5ProductModules("qt/qt5", ref, "")
- assert.Nil(t, err, "Retrieving qt5 modules expected to work")
-
- _, haveSvg := qt5Modules["qt/qtsvg"]
- assert.True(t, haveSvg, "qtsvg needs to be in qt5.git")
- delete(qt5Modules, "qt/qtsvg")
-
- qtDeclarative := qt5Modules["qt/qtdeclarative"]
- assert.NotNil(t, qtDeclarative, "need qtdeclarative")
-
- assert.Contains(t, qtDeclarative.optionalDependencies, "qt/qtsvg")
- qtDeclarativeModule, err := NewModule("qt/qtdeclarative", ref, qt5Modules)
- assert.Nil(t, err, "There shall not be any error creating the module")
-
- assert.NotNil(t, qtDeclarativeModule, "qtdeclarative module shall exist")
-
- assert.Contains(t, qtDeclarativeModule.RequiredDependencies, "qt/qtbase")
- assert.NotContains(t, qtDeclarativeModule.OptionalDependencies, "qt/qtsvg")
-}
-
-func TestQueryChangeStatus(t *testing.T) {
- status, err := getGerritChangeStatus("qt/qtbase", "dev", "Ie6f0e2e3bb198a95dd40e7416adc8ffb29f3b2ba")
- assert.Nil(t, err, "Querying should not produce an error")
- assert.Equal(t, "MERGED", status)
-
- status, err = getGerritChangeStatus("qt/qtbase", "dev", "I6e4349f4d72de307a579f59bb689fd0638690403")
- assert.Nil(t, err, "Querying should not produce an error")
- assert.Equal(t, "ABANDONED", status)
-
-}
diff --git a/src/qtmoduleupdater/moduleupdatebatch.go b/src/qtmoduleupdater/moduleupdatebatch.go
deleted file mode 100644
index 3c2d1ccb..00000000
--- a/src/qtmoduleupdater/moduleupdatebatch.go
+++ /dev/null
@@ -1,423 +0,0 @@
-/****************************************************************************
-**
-** Copyright (C) 2019 The Qt Company Ltd.
-** Contact: https://www.qt.io/licensing/
-**
-** This file is part of the repo tools module of the Qt Toolkit.
-**
-** $QT_BEGIN_LICENSE:GPL-EXCEPT$
-** Commercial License Usage
-** Licensees holding valid commercial Qt licenses may use this file in
-** accordance with the commercial license agreement provided with the
-** Software or, alternatively, in accordance with the terms contained in
-** a written agreement between you and The Qt Company. For licensing terms
-** and conditions see https://www.qt.io/terms-conditions. For further
-** information use the contact form at https://www.qt.io/contact-us.
-**
-** GNU General Public License Usage
-** Alternatively, this file may be used under the terms of the GNU
-** General Public License version 3 as published by the Free Software
-** Foundation with exceptions as appearing in the file LICENSE.GPL3-EXCEPT
-** included in the packaging of this file. Please review the following
-** information to ensure the GNU General Public License requirements will
-** be met: https://www.gnu.org/licenses/gpl-3.0.html.
-**
-** $QT_END_LICENSE$
-**
-****************************************************************************/
-package main
-
-import (
- "bytes"
- "encoding/json"
- "fmt"
- "log"
- "net/url"
- "os"
- "strings"
-)
-
-// PendingUpdate describes that a module needs an updated dependencies.yaml and we are waiting for the change
-// to succeed/fail
-type PendingUpdate struct {
- Module *Module
- ChangeID string
- CommitID OID
- IntegrationAttempts int
-}
-
-// ModuleUpdateBatch is used to serialize and de-serialize the module updating state, used for debugging.
-type ModuleUpdateBatch struct {
- Product string
- ProductRef string
- Branch string
- Todo map[string]*Module
- Done map[string]*Module
- Pending []*PendingUpdate
- FailedModuleCount int
-}
-
-func newModuleUpdateBatch(product string, branch string, productRef string) (*ModuleUpdateBatch, error) {
- batch := &ModuleUpdateBatch{
- Product: product,
- ProductRef: productRef,
- Branch: branch,
- }
- var err error
-
- err = batch.loadStateFromCommit()
- if os.IsNotExist(err) {
- err = batch.loadTodoList()
- if err != nil {
- return nil, err
- }
- }
- return batch, nil
-}
-
-func (batch *ModuleUpdateBatch) scheduleUpdates(gerrit *gerritInstance) error {
- for _, moduleToUpdate := range batch.Todo {
- update, err := moduleToUpdate.updateDependenciesForModule(batch.Done)
- if err != nil {
- return fmt.Errorf("fatal error proposing module update: %s", err)
- }
- log.Printf("Attempting update for module %s resulted in %v\n", moduleToUpdate.RepoPath, update.result)
- if update.result == DependenciesUpdateContentUpToDate {
- batch.Done[moduleToUpdate.RepoPath] = moduleToUpdate
- delete(batch.Todo, moduleToUpdate.RepoPath)
- } else if update.result == DependenciesUpdateDependencyMissing {
- // Nothing to be done, we are waiting for indirect dependencies
- } else if update.result == DependenciesUpdateUpdateScheduled {
- // push and stage
- if err = gerrit.pushChange(moduleToUpdate.RepoPath, moduleToUpdate.Branch, update.commitID, update.summary); err != nil {
- return fmt.Errorf("error pushing change upate: %s", err)
- }
-
- if err = gerrit.reviewAndStageChange(moduleToUpdate.RepoPath, moduleToUpdate.Branch, update.commitID, update.summary); err != nil {
- return fmt.Errorf("error pushing change upate: %s", err)
- }
-
- batch.Pending = append(batch.Pending, &PendingUpdate{moduleToUpdate, update.changeID, update.commitID, 0})
- delete(batch.Todo, moduleToUpdate.RepoPath)
- } else {
- return fmt.Errorf("invalid state returned by updateDependenciesForModule for %s", moduleToUpdate.RepoPath)
- }
- }
-
- return nil
-}
-
-func removeAllDirectAndIndirectDependencies(allModules *map[string]*Module, moduleToRemove string) {
- for moduleName, module := range *allModules {
- if module.hasDependency(moduleToRemove) {
- delete(*allModules, moduleName)
- removeAllDirectAndIndirectDependencies(allModules, module.RepoPath)
- }
- }
-}
-
-func (batch *ModuleUpdateBatch) checkPendingModules(gerrit *gerritInstance) {
- log.Println("Checking status of pending modules")
- var newPending []*PendingUpdate
- for _, pendingUpdate := range batch.Pending {
- module := pendingUpdate.Module
- status, err := getGerritChangeStatus(module.RepoPath, module.Branch, pendingUpdate.ChangeID)
- if err != nil {
- log.Printf(" status check of %s gave error: %s\n", module.RepoPath, err)
- } else {
- log.Printf(" status of %s: %s\n", module.RepoPath, status)
- }
- if err != nil || status == "STAGED" || status == "INTEGRATING" || status == "STAGING" {
- // no change yet
- newPending = append(newPending, pendingUpdate)
- continue
- } else if status == "MERGED" {
- module.refreshTip()
- batch.Done[module.RepoPath] = module
- } else if (status == "NEW" || status == "OPEN") && len(pendingUpdate.CommitID) > 0 && pendingUpdate.IntegrationAttempts < 3 {
- log.Printf(" %v integration attempts for %s - trying again\n", pendingUpdate.IntegrationAttempts, module.RepoPath)
- pendingUpdate.IntegrationAttempts++
- if err = gerrit.reviewAndStageChange(module.RepoPath, module.Branch, pendingUpdate.CommitID, ""); err != nil {
- log.Printf("error staging change update: %s -- ignoring though", err)
- }
- newPending = append(newPending, pendingUpdate)
- } else {
- // Abandoned or tried too many times possibly -- either way an error integrating the update
- removeAllDirectAndIndirectDependencies(&batch.Todo, module.RepoPath)
- batch.FailedModuleCount++
- url := fmt.Sprintf("https://codereview.qt-project.org/#/q/%s,n,z", pendingUpdate.CommitID)
- postMessageToSlack(fmt.Sprintf("Dependency update to %s in %s failed -- <%s>", module.RepoPath, batch.Branch, url))
- }
- }
- batch.Pending = newPending
-}
-
-func loadTodoAndDoneModuleMapFromSubModules(branch string, submodules map[string]*submodule) (todo map[string]*Module, done map[string]*Module, err error) {
- todoModules := make(map[string]*Module)
- doneModules := make(map[string]*Module)
-
- for name, submodule := range submodules {
- module, err := NewModule(name, branch, submodules)
- if err != nil {
- return nil, nil, fmt.Errorf("could not create internal module structure: %s", err)
- }
-
- if submodule.repoType == "inherited" || name == "qt/qtbase" || submodule.branch != branch {
- doneModules[module.RepoPath] = module
- } else {
- todoModules[module.RepoPath] = module
- }
- }
-
- return todoModules, doneModules, nil
-}
-
-func (batch *ModuleUpdateBatch) loadTodoList() error {
- log.Printf("Fetching %s modules from %s %s\n", batch.Product, batch.Branch, batch.ProductRef)
- qt5Modules, err := getQt5ProductModules(batch.Product, batch.Branch, batch.ProductRef)
- if err != nil {
- return fmt.Errorf("Error listing qt5 product modules: %s", err)
- }
-
- batch.Todo, batch.Done, err = loadTodoAndDoneModuleMapFromSubModules(batch.Branch, qt5Modules)
- return err
-}
-
-func sanitizeBranchOrRepo(s string) string {
- s = strings.ToLower(s)
- s = strings.ReplaceAll(s, "/", "_")
- s = strings.ReplaceAll(s, "-", "_")
- return s
-}
-
-func (batch *ModuleUpdateBatch) stateFileName() string {
- return fmt.Sprintf("state_%s_%s.json", sanitizeBranchOrRepo(batch.Product), sanitizeBranchOrRepo(batch.Branch))
-}
-
-func (batch *ModuleUpdateBatch) saveStateAsCommit(gerrit *gerritInstance) error {
- productRepo, err := OpenRepository(batch.Product)
- if err != nil {
- return fmt.Errorf("Error opening product repo: %s", err)
- }
-
- index, err := productRepo.NewIndex()
- if err != nil {
- return fmt.Errorf("Error creating temporary index for saving batch state: %s", err)
- }
- defer index.Free()
-
- jsonBuffer := &bytes.Buffer{}
- encoder := json.NewEncoder(jsonBuffer)
- encoder.SetIndent("", " ")
- if err = encoder.Encode(batch); err != nil {
- return fmt.Errorf("Error serializing module update batch state to json: %s", err)
- }
-
- indexEntry := &IndexEntry{
- Permissions: "100644",
- Path: "state.json",
- }
-
- if err = index.HashObject(indexEntry, jsonBuffer.Bytes()); err != nil {
- return fmt.Errorf("Error adding json serialized module update batch state to git database: %s", err)
- }
-
- if err = index.Add(indexEntry); err != nil {
- return fmt.Errorf("Error adding json serialized module update batch state to git index: %s", err)
- }
-
- tree, err := index.WriteTree()
- if err != nil {
- return fmt.Errorf("Error writing tree object with module update batch state: %s", err)
- }
-
- commit, err := productRepo.CommitTree(tree, "Module update batch state")
- if err != nil {
- return fmt.Errorf("Error writing commit for module update batch state: %s", err)
- }
-
- log.Println("Module state saved as commit", commit)
-
- pushURL, err := RepoURL(batch.Product)
- if err != nil {
- return fmt.Errorf("Error determining %s repo URL: %s", batch.Product, err)
- }
-
- if gerrit.pushUserName != "" {
- pushURL.User = url.User(gerrit.pushUserName)
- }
-
- targetRef := "refs/personal/" + pushURL.User.Username() + "/state/" + batch.Branch
-
- log.Printf("Saving batch state to %s\n", targetRef)
-
- return productRepo.Push(pushURL, []string{"-f"}, commit, targetRef)
-}
-
-func (batch *ModuleUpdateBatch) loadStateFromCommit() error {
- productRepo, err := OpenRepository(batch.Product)
- if err != nil {
- return fmt.Errorf("Error opening product repo: %s", err)
- }
-
- repoURL, err := RepoURL(batch.Product)
- if err != nil {
- return fmt.Errorf("Error determining %s repo URL: %s", batch.Product, err)
- }
-
- log.Printf("Fetching state.json from personal branch")
-
- // ### url
- stateCommit, err := productRepo.Fetch(repoURL, "refs/personal/qt_submodule_update_bot/state/"+batch.Branch)
- if err != nil {
- return os.ErrNotExist
- }
-
- index, err := productRepo.NewIndex()
- if err != nil {
- return fmt.Errorf("Error creating git index when trying to load batch state: %s", err)
- }
- defer index.Free()
-
- if err = index.ReadTree(stateCommit); err != nil {
- return fmt.Errorf("Error reading tree of state commit %s: %s", stateCommit, err)
- }
-
- indexEntry, err := lookupPathIndexEntry(index, "state.json")
- if err != nil {
- return fmt.Errorf("Error looking up state.json from index in state commit %s: %s", stateCommit, err)
- }
-
- stateJSON, err := productRepo.LookupBlob(indexEntry.ID)
- if err != nil {
- return fmt.Errorf("Error reading state json from git db: %s", err)
- }
-
- decoder := json.NewDecoder(bytes.NewBuffer(stateJSON))
- err = decoder.Decode(batch)
- if err != nil {
- return fmt.Errorf("Error decoding JSON state file: %s", err)
- }
-
- return nil
-}
-
-func (batch *ModuleUpdateBatch) clearStateCommit(gerrit *gerritInstance) error {
- productRepo, err := OpenRepository(batch.Product)
- if err != nil {
- return fmt.Errorf("Error opening product repo: %s", err)
- }
-
- pushURL, err := RepoURL(batch.Product)
- if err != nil {
- return fmt.Errorf("Error determining %s repo URL: %s", batch.Product, err)
- }
-
- targetRef := "refs/personal/qt_submodule_update_bot/state/" + batch.Branch
-
- log.Printf("Clearing batch state at %s\n", targetRef)
-
- return productRepo.Push(pushURL, nil, "", targetRef)
-}
-
-func (batch *ModuleUpdateBatch) saveState() error {
- fileName := batch.stateFileName()
- outputFile, err := os.Create(fileName)
- if err != nil {
- return fmt.Errorf("failed to create state file %s: %s", fileName, err)
- }
- defer outputFile.Close()
-
- encoder := json.NewEncoder(outputFile)
- encoder.SetIndent("", " ")
- return encoder.Encode(batch)
-}
-
-func (batch *ModuleUpdateBatch) loadState() error {
- fileName := batch.stateFileName()
- inputFile, err := os.Open(fileName)
- if err != nil {
- return err
- }
- defer inputFile.Close()
-
- decoder := json.NewDecoder(inputFile)
- err = decoder.Decode(batch)
- if err != nil {
- return fmt.Errorf("Error decoding JSON state file: %s", err)
- }
- return nil
-}
-
-func (batch *ModuleUpdateBatch) clearState() {
- os.Remove(batch.stateFileName())
-}
-
-func (batch *ModuleUpdateBatch) isDone() bool {
- return len(batch.Todo) == 0 && len(batch.Pending) == 0
-}
-
-func (batch *ModuleUpdateBatch) printSummary() {
- fmt.Fprintf(os.Stdout, "Summary of git repository dependency update for target branch %s based off of %s\n", batch.Branch, batch.Product)
-
- if batch.isDone() {
- if batch.FailedModuleCount > 0 {
- fmt.Fprintf(os.Stdout, " %v modules failed to be updated. Check Gerrit for the %s branch\n", batch.FailedModuleCount, batch.Branch)
- } else {
- fmt.Fprintf(os.Stdout, " No updates are necessary for any modules - everything is up-to-date\n")
- }
- return
- }
-
- if len(batch.Done) > 0 {
- fmt.Fprintf(os.Stdout, "The following modules have been brought up-to-date:\n")
-
- for name := range batch.Done {
- fmt.Println(" " + name)
- }
- }
-
- if len(batch.Pending) > 0 {
- fmt.Fprintf(os.Stdout, "The following modules are current in-progress:\n")
-
- for _, pending := range batch.Pending {
- fmt.Println(" " + pending.Module.RepoPath)
- }
- }
-
- fmt.Fprintf(os.Stdout, "The following modules are outdated and are either waiting for one of their dependencies or are ready for an update:\n")
- for name := range batch.Todo {
- fmt.Println(" " + name)
- }
-
- fmt.Println()
- fmt.Println()
-}
-
-func (batch *ModuleUpdateBatch) runOneIteration(gerrit *gerritInstance) error {
- batch.checkPendingModules(gerrit)
-
- if err := batch.scheduleUpdates(gerrit); err != nil {
- return err
- }
-
- batch.printSummary()
-
- if !batch.isDone() {
- err := batch.saveStateAsCommit(gerrit)
- if err != nil {
- return err
- }
- } else {
- if batch.FailedModuleCount == 0 {
- fmt.Println("Preparing qt5 update")
- if err := prepareQt5Update(batch.Product, batch.Branch, batch.ProductRef, batch.Done, gerrit); err != nil {
- return fmt.Errorf("error preparing qt5 update: %s", err)
- }
- }
-
- batch.clearStateCommit(gerrit)
- }
-
- return nil
-}
diff --git a/src/qtmoduleupdater/qt5.go b/src/qtmoduleupdater/qt5.go
deleted file mode 100644
index 5d7fba2f..00000000
--- a/src/qtmoduleupdater/qt5.go
+++ /dev/null
@@ -1,262 +0,0 @@
-/****************************************************************************
-**
-** Copyright (C) 2019 The Qt Company Ltd.
-** Contact: https://www.qt.io/licensing/
-**
-** This file is part of the repo tools module of the Qt Toolkit.
-**
-** $QT_BEGIN_LICENSE:GPL-EXCEPT$
-** Commercial License Usage
-** Licensees holding valid commercial Qt licenses may use this file in
-** accordance with the commercial license agreement provided with the
-** Software or, alternatively, in accordance with the terms contained in
-** a written agreement between you and The Qt Company. For licensing terms
-** and conditions see https://www.qt.io/terms-conditions. For further
-** information use the contact form at https://www.qt.io/contact-us.
-**
-** GNU General Public License Usage
-** Alternatively, this file may be used under the terms of the GNU
-** General Public License version 3 as published by the Free Software
-** Foundation with exceptions as appearing in the file LICENSE.GPL3-EXCEPT
-** included in the packaging of this file. Please review the following
-** information to ensure the GNU General Public License requirements will
-** be met: https://www.gnu.org/licenses/gpl-3.0.html.
-**
-** $QT_END_LICENSE$
-**
-****************************************************************************/
-package main
-
-import (
- "bytes"
- "fmt"
- "net"
- "net/url"
- "strings"
-
- "github.com/vaughan0/go-ini"
-)
-
-type submodule struct {
- url string
- branch string
- repoType string
- requiredDependencies []string
- optionalDependencies []string
- headCommit OID
-}
-
-func listSubmodules(repo Repository, repoURL *url.URL, commit OID) (modules map[string]*submodule, err error) {
- tree, err := repo.ListTree(commit)
- if err != nil {
- return
- }
-
- gitModulesEntry, ok := tree.Entries[".gitmodules"]
- if !ok {
- err = fmt.Errorf("could not locate .gitmodules in git tree")
- return
- }
- if gitModulesEntry.Type != ObjectBlob {
- err = fmt.Errorf(".gitmodules is not a file/blob")
- return
- }
- blob, err := repo.LookupBlob(gitModulesEntry.ID)
- if err != nil {
- err = fmt.Errorf("Error looking up .gitmodules blob: %s", err)
- return
- }
-
- baseURL := *repoURL
- baseURL.Path = baseURL.Path + "/repo.git"
- baseURL.Scheme = "ssh"
- host, _, err := net.SplitHostPort(baseURL.Host)
- if err != nil {
- err = fmt.Errorf("Error splitting host and port from base url %v", baseURL)
- return
- }
- baseURL.Host = host
-
- gitModules, err := ini.Load(bytes.NewBuffer(blob))
- for key, values := range gitModules {
- subModule := strings.TrimPrefix(key, `submodule "`)
- if subModule == key {
- continue
- }
- subModule = strings.TrimSuffix(subModule, `"`)
-
- if status, ok := values["status"]; ok {
- if status == "ignore" {
- continue
- } else if status == "additionalLibrary" {
- continue
- }
- } else if initRepo, ok := values["initrepo"]; ok {
- if initRepo != "true" {
- continue
- }
- }
-
- module := &submodule{}
-
- urlString, ok := values["url"]
- if !ok {
- err = fmt.Errorf("could not find submodule URL for submodule %s", subModule)
- return
- }
-
- var subModuleURL *url.URL
- subModuleURL, err = url.Parse(urlString)
- if err != nil {
- err = fmt.Errorf("Error parsing submodule url %s: %s", values["url"], err)
- return
- }
-
- if modules == nil {
- modules = make(map[string]*submodule)
- }
-
- module.url = baseURL.ResolveReference(subModuleURL).String()
- module.branch = values["branch"]
-
- if repoType, ok := values["repoType"]; ok {
- module.repoType = repoType
- }
-
- if requiredDependenciesAsString, ok := values["depends"]; ok {
-
- for _, dep := range strings.Split(requiredDependenciesAsString, " ") {
- module.requiredDependencies = append(module.requiredDependencies, "qt/"+dep)
- }
- }
-
- if optionalDependenciesAsString, ok := values["recommends"]; ok {
- for _, dep := range strings.Split(optionalDependenciesAsString, " ") {
- module.optionalDependencies = append(module.optionalDependencies, "qt/"+dep)
- }
- }
-
- if tree.Entries[subModule].Type != ObjectCommit {
- return nil, fmt.Errorf("submodule entry for %s does not point to a commit", subModule)
- }
-
- module.headCommit = tree.Entries[subModule].ID
-
- modules["qt/"+subModule] = module
- }
- return
-}
-
-func getQt5ProductModules(productProject string, branchOrRef string, productFetchRef string) (modules map[string]*submodule, err error) {
- if productFetchRef == "" {
- productFetchRef = branchOrRef
- }
- if !strings.HasPrefix(productFetchRef, "refs/") {
- productFetchRef = "refs/heads/" + productFetchRef
- }
-
- productRepoURL, err := RepoURL(productProject)
- if err != nil {
- return nil, fmt.Errorf("Error determining %s repo URL: %s", productProject, err)
- }
-
- productRepo, err := OpenRepository(productProject)
- if err != nil {
- return nil, fmt.Errorf("Error opening product repo: %s", err)
- }
-
- productHead, err := productRepo.Fetch(productRepoURL, productFetchRef)
- if err != nil {
- return nil, fmt.Errorf("Error fetching product repo: %s", err)
- }
-
- return listSubmodules(productRepo, productRepoURL, productHead)
-}
-
-func prepareQt5Update(product string, branch string, productFetchRef string, updatedModules map[string]*Module, gerrit *gerritInstance) error {
- productRepoURL, err := RepoURL(product)
- if err != nil {
- return fmt.Errorf("Error determining %s repo URL: %s", product, err)
- }
-
- productRepo, err := OpenRepository(product)
- if err != nil {
- return fmt.Errorf("Error opening product repo: %s", err)
- }
-
- productHead, err := productRepo.Fetch(productRepoURL, productFetchRef)
- if err != nil {
- return err
- }
-
- index, err := productRepo.NewIndex()
- if err != nil {
- return err
- }
- defer index.Free()
-
- if err = index.ReadTree(productHead); err != nil {
- return err
- }
-
- qt5Modules, err := listSubmodules(productRepo, productRepoURL, productHead)
- if err != nil {
- return fmt.Errorf("error retrieving list of submodules: %s", err)
- }
-
- for name, qt5Module := range qt5Modules {
- updatedModule, ok := updatedModules[name]
- if !ok {
- if qt5Module.branch != branch {
- continue
- }
- return fmt.Errorf("could not locate qt5 module %s in map of updated modules", name)
- }
-
- unprefixedPath := strings.TrimPrefix(name, "qt/")
-
- updatedEntry := &IndexEntry{
- Permissions: "160000",
- Path: unprefixedPath,
- ID: OID(updatedModule.Tip),
- }
-
- if err = index.Add(updatedEntry); err != nil {
- return fmt.Errorf("could not update submodule index entry for %s: %s", unprefixedPath, err)
- }
- }
-
- newTree, err := index.WriteTree()
- if err != nil {
- return fmt.Errorf("could not write index with updated submodule sha1s: %s", err)
- }
-
- changeID, _, _, _, err := getExistingChange(product, branch)
- if err != nil {
- return fmt.Errorf("error looking for an existing change while updating submodules: %s", err)
- }
-
- if changeID == "" {
- changeID = fmt.Sprintf("I%s", newTree)
- }
-
- message := fmt.Sprintf("Update submodules on '%s' in %s\n\nChange-Id: %s\n", branch, product, changeID)
-
- commitOid, err := productRepo.CommitTree(newTree, message, productHead)
- if err != nil {
- return fmt.Errorf("could not create new commit for submodule update: %s", err)
- }
-
- fmt.Printf("Created new commit for submodule update: %s\n", commitOid)
-
- if err = gerrit.pushChange(product, branch, commitOid, "Updating all submodules with a new consistent set"); err != nil {
- return fmt.Errorf("Error pushing qt5 change: %s", err)
- }
-
- if err := gerrit.reviewAndStageChange(product, branch, commitOid, "Updating all submodules with a new consistent set"); err != nil {
- return err
- }
- url := fmt.Sprintf("https://codereview.qt-project.org/#/q/%s,n,z", commitOid)
- postMessageToSlack(fmt.Sprintf("Updating all submodules in qt5 %s with a new consistent set: <%s>", branch, url))
- return nil
-}
diff --git a/src/qtmoduleupdater/qt5_test.go b/src/qtmoduleupdater/qt5_test.go
deleted file mode 100644
index de489e9a..00000000
--- a/src/qtmoduleupdater/qt5_test.go
+++ /dev/null
@@ -1,46 +0,0 @@
-/****************************************************************************
-**
-** Copyright (C) 2019 The Qt Company Ltd.
-** Contact: https://www.qt.io/licensing/
-**
-** This file is part of the repo tools module of the Qt Toolkit.
-**
-** $QT_BEGIN_LICENSE:GPL-EXCEPT$
-** Commercial License Usage
-** Licensees holding valid commercial Qt licenses may use this file in
-** accordance with the commercial license agreement provided with the
-** Software or, alternatively, in accordance with the terms contained in
-** a written agreement between you and The Qt Company. For licensing terms
-** and conditions see https://www.qt.io/terms-conditions. For further
-** information use the contact form at https://www.qt.io/contact-us.
-**
-** GNU General Public License Usage
-** Alternatively, this file may be used under the terms of the GNU
-** General Public License version 3 as published by the Free Software
-** Foundation with exceptions as appearing in the file LICENSE.GPL3-EXCEPT
-** included in the packaging of this file. Please review the following
-** information to ensure the GNU General Public License requirements will
-** be met: https://www.gnu.org/licenses/gpl-3.0.html.
-**
-** $QT_END_LICENSE$
-**
-****************************************************************************/
-package main
-
-import (
- "testing"
-
- "github.com/stretchr/testify/assert"
-)
-
-func TestGitModules(t *testing.T) {
- ref := "refs/heads/5.12"
- subModules, err := getQt5ProductModules("qt/qt5", ref, "")
- assert.Nil(t, err, "No errors expected retrieving the submodules")
-
- qqc, ok := subModules["qt/qtquickcontrols"]
- assert.True(t, ok, "Could not find qtquickcontrols in submodules")
-
- assert.Equal(t, []string{"qt/qtdeclarative"}, qqc.requiredDependencies)
- assert.Equal(t, []string{"qt/qtgraphicaleffects"}, qqc.optionalDependencies)
-}
diff --git a/src/qtmoduleupdater/repo.go b/src/qtmoduleupdater/repo.go
deleted file mode 100644
index a347c83c..00000000
--- a/src/qtmoduleupdater/repo.go
+++ /dev/null
@@ -1,445 +0,0 @@
-/****************************************************************************
-**
-** Copyright (C) 2019 The Qt Company Ltd.
-** Contact: https://www.qt.io/licensing/
-**
-** This file is part of the repo tools module of the Qt Toolkit.
-**
-** $QT_BEGIN_LICENSE:GPL-EXCEPT$
-** Commercial License Usage
-** Licensees holding valid commercial Qt licenses may use this file in
-** accordance with the commercial license agreement provided with the
-** Software or, alternatively, in accordance with the terms contained in
-** a written agreement between you and The Qt Company. For licensing terms
-** and conditions see https://www.qt.io/terms-conditions. For further
-** information use the contact form at https://www.qt.io/contact-us.
-**
-** GNU General Public License Usage
-** Alternatively, this file may be used under the terms of the GNU
-** General Public License version 3 as published by the Free Software
-** Foundation with exceptions as appearing in the file LICENSE.GPL3-EXCEPT
-** included in the packaging of this file. Please review the following
-** information to ensure the GNU General Public License requirements will
-** be met: https://www.gnu.org/licenses/gpl-3.0.html.
-**
-** $QT_END_LICENSE$
-**
-****************************************************************************/
-package main
-
-import (
- "bufio"
- "bytes"
- "fmt"
- "io/ioutil"
- "log"
- "net/url"
- "os"
- "os/exec"
- "path/filepath"
- "strings"
-)
-
-type commandWithCapturedOutput struct {
- cmd *exec.Cmd
- stdout bytes.Buffer
- stderr bytes.Buffer
-}
-
-func newCommandWithCapturedOutput(cmd *exec.Cmd) *commandWithCapturedOutput {
- result := &commandWithCapturedOutput{}
- result.cmd = cmd
- result.cmd.Stdout = &result.stdout
- result.cmd.Stderr = &result.stderr
- return result
-}
-
-func (c *commandWithCapturedOutput) Run() (string, error) {
- err := c.cmd.Run()
- if err != nil {
- return "", fmt.Errorf("Error running %s: %s\nStdout: %s\nStderr: %s", strings.Join(c.cmd.Args, " "), err, c.stdout.String(), c.stderr.String())
- }
- return c.stdout.String(), err
-}
-
-func (c *commandWithCapturedOutput) RunWithSpaceTrimmed() (string, error) {
- output, err := c.Run()
- return strings.TrimSpace(output), err
-}
-
-// Repository is a type that wraps various git operations on the given repository on the local disk.
-type Repository string
-
-// OID is a git object identifier, in the form of a SHA1 check-sum.
-type OID string
-
-// RepoURL returns a clone/fetch URL for the given project.
-func RepoURL(project string) (*url.URL, error) {
- gerritConfig := struct {
- URL string
- Port string
- }{
- "codereview.qt-project.org",
- "29418",
- }
- repo := &url.URL{}
- repo.Host = gerritConfig.URL + ":" + gerritConfig.Port
- repo.Path = "/" + project
- repo.Scheme = "ssh"
- return repo, nil
-}
-
-// RepoPushURL returns a URL to use for pusing changes to the project.
-func RepoPushURL(project string) (*url.URL, error) {
- pushURL, err := RepoURL(project)
- if err != nil {
- return nil, err
- }
- user := os.Getenv("GIT_SSH_USER")
- if user != "" {
- pushURL.User = url.User(user)
- }
- return pushURL, nil
-}
-
-// OpenRepository is used to create a new repository wrapper for the specified project.
-// If the repository doesn't exist yet, it will be cloned.
-func OpenRepository(project string) (Repository, error) {
- reposLocation := "git-repos"
- repoPath := filepath.Join(reposLocation, project)
- if _, err := os.Stat(repoPath); os.IsNotExist(err) {
- url, err := RepoURL(project)
- if err != nil {
- return "", err
- }
- log.Printf("Cloning missing repository %s from %s to %s\n", project, url, repoPath)
- cloneCmd := []string{"clone", "--bare"}
- for _, ref := range strings.Split(os.Getenv("QT_CI_REPO_REFERENCES"), ":") {
- path := ref + "/.git/modules/" + strings.Split(project, "/")[1]
- _, err := os.Stat(path)
- if err == nil {
- cloneCmd = append(cloneCmd, []string{"--reference", path}...)
- break
- }
- }
- cloneCmd = append(cloneCmd, []string{url.String(), repoPath}...)
- cmd := exec.Command("git", cloneCmd...)
- if err = cmd.Run(); err != nil {
- return "", err
- }
- }
- return Repository(repoPath), nil
-}
-
-func (repo Repository) gitCommand(command string, parameters ...string) *commandWithCapturedOutput {
- parameters = append([]string{"--git-dir=" + string(repo), command}, parameters...)
- return newCommandWithCapturedOutput(exec.Command("git", parameters...))
-}
-
-// LookupReference resolves the provided git reference by means of calling rev-parse.
-func (repo Repository) LookupReference(ref string) (OID, error) {
- rev, err := repo.gitCommand("rev-parse", ref).RunWithSpaceTrimmed()
- return OID(rev), err
-}
-
-// ObjectType denotes the different types of objects stored in a git repository.
-type ObjectType int
-
-const (
- // ObjectBlob refers to a pure data object
- ObjectBlob = iota
- // ObjectCommit refers to a git commit
- ObjectCommit
- // ObjectTree refers to a tree if blobs or trees
- ObjectTree
-)
-
-// TreeEntry describes the entry of a directory listing in git
-type TreeEntry struct {
- Permissions string
- Type ObjectType
- ID OID
-}
-
-// Tree is data structure representing the output of the git ls-tree command.
-type Tree struct {
- Repo Repository
- ID OID
- Entries map[string]TreeEntry
-}
-
-func (repo Repository) decodeLsTreeOutput(commit OID, output string) (*Tree, error) {
- result := &Tree{
- Repo: repo,
- ID: commit,
- Entries: make(map[string]TreeEntry),
- }
- for _, line := range bytes.Split([]byte(output), []byte{0}) {
- if len(line) == 0 {
- continue
- }
- var entry TreeEntry
- modeIndex := bytes.IndexByte(line, ' ')
- if modeIndex == -1 {
- return nil, fmt.Errorf("missing space after permission field while parsing git ls-tree output")
- }
- entry.Permissions = string(line[:modeIndex])
-
- line = line[modeIndex+1:]
-
- typeIndex := bytes.IndexByte(line, ' ')
- if typeIndex == -1 {
- return nil, fmt.Errorf("missing space after type field while parsing git ls-tree output")
- }
- typeName := line[:typeIndex]
- switch string(typeName) {
- case "tree":
- entry.Type = ObjectTree
- case "blob":
- entry.Type = ObjectBlob
- case "commit":
- entry.Type = ObjectCommit
- default:
- return nil, fmt.Errorf("unexpected entry type %s while parsing git ls-tree output", typeName)
- }
-
- line = line[typeIndex+1:]
-
- objectIndex := bytes.IndexByte(line, '\t')
- if objectIndex == -1 {
- return nil, fmt.Errorf("missing space after entry field while parsing git ls-tree output")
- }
- entry.ID = OID(string(line[:objectIndex]))
-
- name := string(line[objectIndex+1:])
-
- result.Entries[name] = entry
- }
- return result, nil
-}
-
-// ListTree retrieves a (non-recursive) directory listing of the specified commit.
-func (repo Repository) ListTree(commit OID) (*Tree, error) {
- output, err := repo.gitCommand("ls-tree", "-z", string(commit)).Run()
- if err != nil {
- return nil, err
- }
- return repo.decodeLsTreeOutput(commit, output)
-}
-
-// ListTreeWithPath retrieves a (non-recursive) directory listing of the specified commit with the specified path.
-func (repo Repository) ListTreeWithPath(commit OID, subPath string) (*Tree, error) {
- output, err := repo.gitCommand("ls-tree", "-z", string(commit), subPath).Run()
- if err != nil {
- return nil, err
- }
- return repo.decodeLsTreeOutput(commit, output)
-}
-
-// Fetch retrieves the specified refSpec from the given url. The result is fetched into FETCH_HEAD
-// and the value of FETCH_HEAD is returned.
-func (repo Repository) Fetch(url *url.URL, refSpec string) (sha1 OID, err error) {
- if fetch := os.Getenv("NO_FETCH"); len(fetch) == 0 {
- cmd := repo.gitCommand("fetch", url.String(), refSpec)
- if _, err = cmd.Run(); err != nil {
- return "", fmt.Errorf("Error running fetch command: %s", err)
- }
- }
- ref, err := repo.LookupReference("FETCH_HEAD")
- if err != nil {
- return "", fmt.Errorf("Error looking up FETCH_HEAD after fetch: %s", err)
- }
- return ref, nil
-}
-
-// Push is a wrapper around the git push commit, similar to the Push() function
-// but allowing additional options to be passed to the git push invocation.
-func (repo Repository) Push(url *url.URL, options []string, commit OID, targetRef string) error {
- refSpec := fmt.Sprintf("%s:%s", commit, targetRef)
- options = append(options, url.String(), refSpec)
- _, err := repo.gitCommand("push", options...).Run()
- return err
-}
-
-// LookupBlob returns the byte content of the specified blob object.
-func (repo Repository) LookupBlob(object OID) ([]byte, error) {
- output, err := repo.gitCommand("cat-file", "blob", string(object)).Run()
- if err != nil {
- return nil, err
- }
- return []byte(output), nil
-}
-
-// IndexEntry represents an entry in the virtual git index directory structure.
-type IndexEntry struct {
- Permissions string
- Path string
- ID OID
-}
-
-// Index is a wrapper around git operations that allow operating on a temporary index.
-type Index struct {
- file *os.File
- repo Repository
- cachedEntries []IndexEntry
- populated bool
-}
-
-// NewIndex creates a new git index based on a temporary file. Unless you'd like to
-// start with an empty tree, you may want to populate the index with ReadTree.
-// A newly created index should be freed with Free() at the end of the usage, in order
-// to remove the temporary file.
-func (repo Repository) NewIndex() (result *Index, err error) {
- result = &Index{}
- result.repo = repo
- result.file, err = ioutil.TempFile("", "")
- if err != nil {
- return nil, err
- }
- result.populated = false
- return result, nil
-}
-
-// CommitTree creates a new commit object from the specified tree, along with the specified message and parent commits.
-// The new commit id is returned.
-func (repo Repository) CommitTree(tree OID, message string, parents ...OID) (OID, error) {
- allParams := make([]string, 0, 1+2*len(parents))
- allParams = append(allParams, string(tree))
- for _, parent := range parents {
- allParams = append(allParams, "-p")
- allParams = append(allParams, string(parent))
- }
- cmd := repo.gitCommand("commit-tree", allParams...)
- cmd.cmd.Stdin = bytes.NewBufferString(message)
- commit, err := cmd.RunWithSpaceTrimmed()
- return OID(commit), err
-}
-
-// LogOutput is a wrapper around the git log command.
-func (repo Repository) LogOutput(options ...string) ([]string, error) {
- logOutput, err := repo.gitCommand("log", options...).Run()
- if err != nil {
- return nil, err
- }
- var log []string
- scanner := bufio.NewScanner(bytes.NewBufferString(logOutput))
- for scanner.Scan() {
- log = append(log, scanner.Text())
- }
- return log, nil
-}
-
-// Free is responsible for deleting the temporary index file.
-func (idx *Index) Free() {
- os.Remove(idx.file.Name())
- idx.file = nil
-}
-
-func (idx *Index) gitCommandWithIndex(command string, parameters ...string) *commandWithCapturedOutput {
- cmd := idx.repo.gitCommand(command, parameters...)
- cmd.cmd.Env = os.Environ()
- cmd.cmd.Env = append(cmd.cmd.Env, "GIT_INDEX_FILE="+idx.file.Name())
- return cmd
-}
-
-func (idx *Index) updateCachedEntries() error {
- idx.cachedEntries = make([]IndexEntry, 0)
-
- output, err := idx.gitCommandWithIndex("ls-files", "-z", "--stage").Run()
- if err != nil {
- return err
- }
- for _, line := range bytes.Split([]byte(output), []byte{0}) {
- if len(line) == 0 {
- continue
- }
- var entry IndexEntry
- modeIndex := bytes.IndexByte(line, ' ')
- if modeIndex == -1 {
- return fmt.Errorf("missing space after permission field while parsing git ls-files output")
- }
- entry.Permissions = string(line[:modeIndex])
- line = line[modeIndex+1:]
-
- objectIndex := bytes.IndexByte(line, ' ')
- if objectIndex == -1 {
- return fmt.Errorf("missing space after entry field while parsing git ls-files output")
- }
- entry.ID = OID(string(line[:objectIndex]))
- line = line[objectIndex+1:]
-
- stageIndex := bytes.IndexByte(line, '\t')
- if stageIndex == -1 {
- return fmt.Errorf("missing space after stage field while parsing git ls-files output")
- }
-
- entry.Path = string(line[stageIndex+1:])
- idx.cachedEntries = append(idx.cachedEntries, entry)
- }
- return nil
-}
-
-// EntryCount returns the number of directory/file entries in the index.
-func (idx *Index) EntryCount() int {
- return len(idx.cachedEntries)
-}
-
-// EntryByIndex returns the i-th entry in the directory index.
-func (idx *Index) EntryByIndex(i int) (*IndexEntry, error) {
- if i < 0 || i >= len(idx.cachedEntries) {
- return nil, fmt.Errorf("Index %v out of range in index (0 - %v)", i, len(idx.cachedEntries))
- }
-
- return &idx.cachedEntries[i], nil
-}
-
-// ReadTree populates the index from the specified tree object. This is implemented by calling git read-tree.
-func (idx *Index) ReadTree(tree OID) error {
- _, err := idx.gitCommandWithIndex("read-tree", "--index-output="+idx.file.Name(), string(tree)).Run()
- if err != nil {
- return err
- }
- idx.populated = true
- return idx.updateCachedEntries()
-}
-
-// Add adds a new entry to the index or updates an existing one if already present.
-func (idx *Index) Add(entry *IndexEntry) error {
- if !idx.populated {
- os.Remove(idx.file.Name())
- }
- _, err := idx.gitCommandWithIndex("update-index", "--add", "--cacheinfo", fmt.Sprintf("%s,%s,%s", entry.Permissions, entry.ID, entry.Path)).Run()
- if err != nil {
- return err
- }
- idx.populated = true
- return idx.updateCachedEntries()
-}
-
-// HashObject writes content b as git object to the database and updates the entry.
-func (idx *Index) HashObject(entry *IndexEntry, b []byte) error {
- tempfile, err := ioutil.TempFile("", "")
- if err != nil {
- return err
- }
- defer os.Remove(tempfile.Name())
-
- if _, err := tempfile.Write(b); err != nil {
- return err
- }
-
- newSha1, err := idx.gitCommandWithIndex("hash-object", "-w", tempfile.Name()).RunWithSpaceTrimmed()
- if err != nil {
- return err
- }
-
- entry.ID = OID(newSha1)
-
- return nil
-}
-
-// WriteTree writes the index to the git database as a tree object and returns the tree object id.
-func (idx *Index) WriteTree() (OID, error) {
- output, err := idx.gitCommandWithIndex("write-tree").RunWithSpaceTrimmed()
- return OID(output), err
-}
diff --git a/src/qtmoduleupdater/repo_test.go b/src/qtmoduleupdater/repo_test.go
deleted file mode 100644
index eac6e267..00000000
--- a/src/qtmoduleupdater/repo_test.go
+++ /dev/null
@@ -1,182 +0,0 @@
-/****************************************************************************
-**
-** Copyright (C) 2019 The Qt Company Ltd.
-** Contact: https://www.qt.io/licensing/
-**
-** This file is part of the repo tools module of the Qt Toolkit.
-**
-** $QT_BEGIN_LICENSE:GPL-EXCEPT$
-** Commercial License Usage
-** Licensees holding valid commercial Qt licenses may use this file in
-** accordance with the commercial license agreement provided with the
-** Software or, alternatively, in accordance with the terms contained in
-** a written agreement between you and The Qt Company. For licensing terms
-** and conditions see https://www.qt.io/terms-conditions. For further
-** information use the contact form at https://www.qt.io/contact-us.
-**
-** GNU General Public License Usage
-** Alternatively, this file may be used under the terms of the GNU
-** General Public License version 3 as published by the Free Software
-** Foundation with exceptions as appearing in the file LICENSE.GPL3-EXCEPT
-** included in the packaging of this file. Please review the following
-** information to ensure the GNU General Public License requirements will
-** be met: https://www.gnu.org/licenses/gpl-3.0.html.
-**
-** $QT_END_LICENSE$
-**
-****************************************************************************/
-package main
-
-import (
- "strings"
- "testing"
-
- "github.com/stretchr/testify/assert"
-)
-
-func TestRepo(t *testing.T) {
- repo, err := OpenRepository("qt/qtbase")
- if err != nil {
- t.Fatalf("Unexpected error opening qtbase repo: %s", err)
- }
- if !strings.HasSuffix(string(repo), "qt/qtbase") {
- t.Fatalf("Unexpected repo path %s", repo)
- }
-
- ref, err := repo.LookupReference("v5.5.0")
- if err != nil {
- t.Fatalf("Unexpected error looking up reference: %s", err)
- }
- if string(ref) != "2fde9f59eeab68ede92324e7613daf8be3eaf498" {
- t.Fatalf("Incorrect sha1 for v5.5.0 tag")
- }
-
- tree, err := repo.ListTree(ref)
- if err != nil {
- t.Fatalf("Unexpected error listing tree: %s", err)
- }
-
- if tree.ID != ref {
- t.Fatalf("Incorrect tree entry for %s", ref)
- }
-
- qtbaseProEntry, ok := tree.Entries["qtbase.pro"]
- if !ok {
- t.Fatalf("Missing qtbase.pro entry in tree listing")
- }
-
- if qtbaseProEntry.ID != "24d0f5287ba26ee0e53e34c8860c6c7baf7b0268" {
- t.Fatalf("Unexpected sha1 for qtbase.pro: %s", qtbaseProEntry.ID)
- }
-
- if qtbaseProEntry.Type != ObjectBlob {
- t.Fatalf("Unexpected entry type for qtbase.pro")
- }
-
- if qtbaseProEntry.Permissions != "100644" {
- t.Fatalf("Incorrect permissions for qtbase.pro")
- }
-
- qmakeConf, ok := tree.Entries[".qmake.conf"]
- if !ok {
- t.Fatalf("Missing .qmake.conf in tree listing")
- }
-
- content, err := repo.LookupBlob(qmakeConf.ID)
- if err != nil {
- t.Fatalf("Unexpected error looking up .qmake.conf blob: %s", err)
- }
-
- expectedContent := `load(qt_build_config)
-CONFIG += qt_example_installs
-CONFIG += warning_clean
-
-QT_SOURCE_TREE = $$PWD
-QT_BUILD_TREE = $$shadowed($$PWD)
-
-# In qtbase, all modules follow qglobal.h
-MODULE_VERSION = $$QT_VERSION
-`
- if string(content) != expectedContent {
- t.Fatalf("Unexpected blob content for .qmake.conf: %s", string(content))
- }
-}
-
-func TestIndex(t *testing.T) {
- repo, err := OpenRepository("qt/qtbase")
- if err != nil {
- t.Fatalf("Unexpected error opening qtbase repo: %s", err)
- }
-
- ref, err := repo.LookupReference("v5.5.0")
- if err != nil {
- t.Fatalf("Unexpected error looking up v5.5.0 tag")
- }
-
- index, err := repo.NewIndex()
- if err != nil {
- t.Fatalf("Could not get index.")
- }
- defer index.Free()
-
- err = index.ReadTree(ref)
- if err != nil {
- t.Fatalf("Error reading index tree: %s", err)
- }
-
- if index.EntryCount() != 21452 {
- t.Fatalf("Unexpected index entry count %v", index.EntryCount())
- }
-}
-
-func TestNewIndex(t *testing.T) {
- repo, err := OpenRepository("qt/qtbase")
- assert.Nilf(t, err, "Unexpected error opening qtbase repo: %s", err)
-
- index, err := repo.NewIndex()
- if err != nil {
- t.Fatalf("Could not get index.")
- }
- defer index.Free()
-
- sampleContent := []byte("Hello World")
-
- indexEntry := &IndexEntry{
- Permissions: "100644",
- Path: "test.txt",
- }
-
- err = index.HashObject(indexEntry, sampleContent)
- assert.Nilf(t, err, "should be able to add data to git database: %s", err)
-
- assert.Equal(t, OID("5e1c309dae7f45e0f39b1bf3ac3cd9db12e7d689"), indexEntry.ID)
-
- err = index.Add(indexEntry)
- assert.Nilf(t, err, "should be able to add entry to new index: %s", err)
-
- tree, err := index.WriteTree()
- assert.Nilf(t, err, "should be able to write tree: %s", err)
-
- assert.Equal(t, OID("4f11af3e4e067fc319abd053205f39bc40652f05"), tree)
-}
-
-func TestLog(t *testing.T) {
- repo, err := OpenRepository("qt/qtbase")
- if err != nil {
- t.Fatalf("Unexpected error opening qtbase repo: %s", err)
- }
-
- output, err := repo.LogOutput(`--pretty=format: %m %s`, "--first-parent", "v5.0.0~2..v5.0.0")
- if err != nil {
- t.Fatalf("Unexpected error calling git log: %s", err)
- }
- if len(output) != 2 {
- t.Fatalf("Unexpected length of git log output array: %v", len(output))
- }
- if output[0] != " > Fix font sizes when X11 has a forced dpi setting" {
- t.Fatalf("Unexpected first line of log output: %s", output[0])
- }
- if output[1] != " > Fix direct compilation of qtypeinfo.h and others" {
- t.Fatalf("Unexpected second line of log output: %s", output[0])
- }
-}
diff --git a/src/qtmoduleupdater/slack.go b/src/qtmoduleupdater/slack.go
deleted file mode 100644
index 6b91cc92..00000000
--- a/src/qtmoduleupdater/slack.go
+++ /dev/null
@@ -1,74 +0,0 @@
-package main
-
-import (
- "bytes"
- "encoding/json"
- "github.com/kardianos/osext"
- "io/ioutil"
- "log"
- "net/http"
- "path/filepath"
-)
-
-// SlackConfig describes the format of slack.json for the slack coin integration
-type SlackConfig struct {
- WebHookURL string `json:"WebHookURL"`
-}
-
-var (
- slackConfig SlackConfig
-)
-
-// SlackMessage is the format used to send a message into slack
-type SlackMessage struct {
- Text string `json:"text"`
-}
-
-func initSlackIntegration() {
- binFolder, err := osext.ExecutableFolder()
- if err != nil {
- log.Println("Unable to get executable folder - Slack integration disabled")
- return
- }
- configFile := filepath.Join(binFolder, "coin-secrets", "submodule_update_bot_alert_hook.json")
- jsonData, err := ioutil.ReadFile(configFile)
- if err != nil {
- log.Println("Could not read coin-secrets/submodule_update_bot_alert_hook.json - Slack integration disabled")
- return
- }
- var cfg SlackConfig
- err = json.Unmarshal(jsonData, &cfg)
- if err != nil {
- log.Println("Unable to unmarshal json data - Slack integration disabled")
- return
- }
- slackConfig = cfg
- if slackConfig.WebHookURL == "" {
- log.Println("Slack integration disabled due to missing web hook url")
- return
- }
- log.Println("Slack integration enabled")
-}
-
-func postMessageToSlack(message string) {
- if slackConfig.WebHookURL == "" {
- return
- }
-
- msg := &SlackMessage{
- Text: message,
- }
-
- buffer, err := json.Marshal(msg)
- if err != nil {
- log.Println("Error marshalling slack message", err)
- return
- }
-
- response, err := http.Post(slackConfig.WebHookURL, "application/json", bytes.NewReader(buffer))
- if err != nil {
- log.Println("Error posting to slack:", err)
- return
- }
- ioutil.ReadAll(response.Body)
-}