From aac3984217f993d1e62a8eab8e1e86b05cdaf664 Mon Sep 17 00:00:00 2001 From: Cristian Maureira-Fredes Date: Mon, 29 Mar 2021 14:36:33 +0200 Subject: Initial version of the site * Including a README that explains how to setup the project. * Adding `Procfile` to be compatible with gunicorn. * Adding a `runtime.txt` to select the Python version on heroku. * Include initial version of the text-content for "contribute", and "guidelines" * Adding box to provide a shortcut to the QUIPS * Add `requirements.txt` * Including a script to locally clone and generate the data for the dashboard. * Including script to generate the CSV files based on the content of the qt repositories. Change-Id: I8b5e5a581d7f247ead16c031adc7243ba023018f Reviewed-by: Lars Knoll Reviewed-by: Tuukka Turunen --- LICENSE.CC0 | 121 ++++++++++ Procfile | 1 + README.md | 61 +++++ app.py | 578 ++++++++++++++++++++++++++++++++++++++++++++++++ assets/favicon.ico | Bin 0 -> 5430 bytes assets/qt.png | Bin 0 -> 35321 bytes assets/style.css | 160 ++++++++++++++ assets/theqtproject.png | Bin 0 -> 7950 bytes contribute.md | 31 +++ data/get_data.py | 165 ++++++++++++++ guidelines.md | 76 +++++++ learn_more.md | 5 + quips.md | 18 ++ requirements.txt | 4 + runtime.txt | 1 + update-data.sh | 47 ++++ 16 files changed, 1268 insertions(+) create mode 100644 LICENSE.CC0 create mode 100644 Procfile create mode 100644 README.md create mode 100644 app.py create mode 100644 assets/favicon.ico create mode 100644 assets/qt.png create mode 100644 assets/style.css create mode 100644 assets/theqtproject.png create mode 100644 contribute.md create mode 100644 data/get_data.py create mode 100644 guidelines.md create mode 100644 learn_more.md create mode 100644 quips.md create mode 100644 requirements.txt create mode 100644 runtime.txt create mode 100644 update-data.sh diff --git a/LICENSE.CC0 b/LICENSE.CC0 new file mode 100644 index 0000000..0e259d4 --- /dev/null +++ b/LICENSE.CC0 @@ -0,0 +1,121 @@ +Creative Commons Legal Code + +CC0 1.0 Universal + + CREATIVE COMMONS CORPORATION IS NOT A LAW FIRM AND DOES NOT PROVIDE + LEGAL SERVICES. DISTRIBUTION OF THIS DOCUMENT DOES NOT CREATE AN + ATTORNEY-CLIENT RELATIONSHIP. CREATIVE COMMONS PROVIDES THIS + INFORMATION ON AN "AS-IS" BASIS. CREATIVE COMMONS MAKES NO WARRANTIES + REGARDING THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS + PROVIDED HEREUNDER, AND DISCLAIMS LIABILITY FOR DAMAGES RESULTING FROM + THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS PROVIDED + HEREUNDER. + +Statement of Purpose + +The laws of most jurisdictions throughout the world automatically confer +exclusive Copyright and Related Rights (defined below) upon the creator +and subsequent owner(s) (each and all, an "owner") of an original work of +authorship and/or a database (each, a "Work"). + +Certain owners wish to permanently relinquish those rights to a Work for +the purpose of contributing to a commons of creative, cultural and +scientific works ("Commons") that the public can reliably and without fear +of later claims of infringement build upon, modify, incorporate in other +works, reuse and redistribute as freely as possible in any form whatsoever +and for any purposes, including without limitation commercial purposes. +These owners may contribute to the Commons to promote the ideal of a free +culture and the further production of creative, cultural and scientific +works, or to gain reputation or greater distribution for their Work in +part through the use and efforts of others. + +For these and/or other purposes and motivations, and without any +expectation of additional consideration or compensation, the person +associating CC0 with a Work (the "Affirmer"), to the extent that he or she +is an owner of Copyright and Related Rights in the Work, voluntarily +elects to apply CC0 to the Work and publicly distribute the Work under its +terms, with knowledge of his or her Copyright and Related Rights in the +Work and the meaning and intended legal effect of CC0 on those rights. + +1. Copyright and Related Rights. A Work made available under CC0 may be +protected by copyright and related or neighboring rights ("Copyright and +Related Rights"). Copyright and Related Rights include, but are not +limited to, the following: + + i. the right to reproduce, adapt, distribute, perform, display, + communicate, and translate a Work; + ii. moral rights retained by the original author(s) and/or performer(s); +iii. publicity and privacy rights pertaining to a person's image or + likeness depicted in a Work; + iv. rights protecting against unfair competition in regards to a Work, + subject to the limitations in paragraph 4(a), below; + v. rights protecting the extraction, dissemination, use and reuse of data + in a Work; + vi. database rights (such as those arising under Directive 96/9/EC of the + European Parliament and of the Council of 11 March 1996 on the legal + protection of databases, and under any national implementation + thereof, including any amended or successor version of such + directive); and +vii. other similar, equivalent or corresponding rights throughout the + world based on applicable law or treaty, and any national + implementations thereof. + +2. Waiver. To the greatest extent permitted by, but not in contravention +of, applicable law, Affirmer hereby overtly, fully, permanently, +irrevocably and unconditionally waives, abandons, and surrenders all of +Affirmer's Copyright and Related Rights and associated claims and causes +of action, whether now known or unknown (including existing as well as +future claims and causes of action), in the Work (i) in all territories +worldwide, (ii) for the maximum duration provided by applicable law or +treaty (including future time extensions), (iii) in any current or future +medium and for any number of copies, and (iv) for any purpose whatsoever, +including without limitation commercial, advertising or promotional +purposes (the "Waiver"). Affirmer makes the Waiver for the benefit of each +member of the public at large and to the detriment of Affirmer's heirs and +successors, fully intending that such Waiver shall not be subject to +revocation, rescission, cancellation, termination, or any other legal or +equitable action to disrupt the quiet enjoyment of the Work by the public +as contemplated by Affirmer's express Statement of Purpose. + +3. Public License Fallback. Should any part of the Waiver for any reason +be judged legally invalid or ineffective under applicable law, then the +Waiver shall be preserved to the maximum extent permitted taking into +account Affirmer's express Statement of Purpose. In addition, to the +extent the Waiver is so judged Affirmer hereby grants to each affected +person a royalty-free, non transferable, non sublicensable, non exclusive, +irrevocable and unconditional license to exercise Affirmer's Copyright and +Related Rights in the Work (i) in all territories worldwide, (ii) for the +maximum duration provided by applicable law or treaty (including future +time extensions), (iii) in any current or future medium and for any number +of copies, and (iv) for any purpose whatsoever, including without +limitation commercial, advertising or promotional purposes (the +"License"). The License shall be deemed effective as of the date CC0 was +applied by Affirmer to the Work. Should any part of the License for any +reason be judged legally invalid or ineffective under applicable law, such +partial invalidity or ineffectiveness shall not invalidate the remainder +of the License, and in such case Affirmer hereby affirms that he or she +will not (i) exercise any of his or her remaining Copyright and Related +Rights in the Work or (ii) assert any associated claims and causes of +action with respect to the Work, in either case contrary to Affirmer's +express Statement of Purpose. + +4. Limitations and Disclaimers. + + a. No trademark or patent rights held by Affirmer are waived, abandoned, + surrendered, licensed or otherwise affected by this document. + b. Affirmer offers the Work as-is and makes no representations or + warranties of any kind concerning the Work, express, implied, + statutory or otherwise, including without limitation warranties of + title, merchantability, fitness for a particular purpose, non + infringement, or the absence of latent or other defects, accuracy, or + the present or absence of errors, whether or not discoverable, all to + the greatest extent permissible under applicable law. + c. Affirmer disclaims responsibility for clearing rights of other persons + that may apply to the Work or any use thereof, including without + limitation any person's Copyright and Related Rights in the Work. + Further, Affirmer disclaims responsibility for obtaining any necessary + consents, permissions or other rights required for any use of the + Work. + d. Affirmer understands and acknowledges that Creative Commons is not a + party to this document and has no duty or obligation with respect to + this CC0 or use of the Work. diff --git a/Procfile b/Procfile new file mode 100644 index 0000000..f039091 --- /dev/null +++ b/Procfile @@ -0,0 +1 @@ +web: gunicorn --timeout 600 app:server diff --git a/README.md b/README.md new file mode 100644 index 0000000..0da7ad7 --- /dev/null +++ b/README.md @@ -0,0 +1,61 @@ +# Qt Project + +Dash application behind the `qt-project.org` website. + +Currently there are two main goals that this page wants to achieve: + + 1. Help people contribute to Qt, + 2. Show statistics about the project. + +## Setup + +Create a new Python virtual environment and install the requirements: + +``` +python -m venv env # might be 'python3' depending on your system + +source env/bin/activate # macOS/Linux +env\Scripts\activate.bat # windows + +pip install -r requirements.txt +``` + +## Getting and processing the data + +The `data/` directory contains a script to get the information +from a local `qt5.git` repository, and you can use it like this: + +``` +python get_data.py --qt /path/to/my/qt5.git +``` + +this will generate many `.csv` files with the following columns + +``` +date;sha;name;original_email;email;domain;files_changed;insertions;deletions +``` + +Currently, there are also two external repositories being cloned, and used +inside the `qt5.git`, the Qt Creator and PySide repositories. + +> The difference between `original_email` and `email` is because many old +> Nokia, Digia, theqtcompany emails were transformed into `@qt.io`. + +## Starting the application + +The website is based on a framework called [Dash](https://plotly.com/dash/), +which enables the creation of Dashboard. + +We also rely on text-based information. There are `markdown` files +which contains the information displayed on the left-columns, like `guidelines.md` +and `contribute.md`. + +To start the application, run: +``` +python app.py +``` + +which you can verify on [http://localhost:8050](http://localhost:8050). + +Additionally, there is a `Procfile` to enable the site to be deployed +on servers compatible with [gunicorn](https://gunicorn.org/). diff --git a/app.py b/app.py new file mode 100644 index 0000000..2bbf4fe --- /dev/null +++ b/app.py @@ -0,0 +1,578 @@ +# Copyright (C) 2021 The Qt Company Ltd. +# Contact: https://www.qt.io/licensing/ +# +# You may use this file under the terms of the CC0 license. +# See the file LICENSE.CC0 from this package for details. + +from pathlib import Path + +import dash +import dash_core_components as dcc +import dash_html_components as html +import pandas as pd +import requests +import os +import io +import sys +import zipfile +import subprocess +from dash.dependencies import Input, Output + +external_stylesheets = [ + { + "href": "https://codepen.io/chriddyp/pen/bWLwgP.css", + "rel": "stylesheet", + }, + { + "href": "https://fonts.googleapis.com/css2?" "family=Titillium+Web&display=swap", + "rel": "stylesheet", + }, +] + +# Main application +app = dash.Dash(__name__, external_stylesheets=external_stylesheets) +server = app.server +app.title = "The Qt Project" + + +def partially_hide(x): + words = [] + x_split = x.split() + len_split = len(x_split) + + for i, name in enumerate(x_split): + # Replace with dots + # filled = '.' * (len(name) - 1) + # words.append(f'{name[0]}{filled}') + + # Use first letter + # words.append(name[0]) + + # User F. Lastime + if i != len_split - 1: + words.append(f"{name[0]}.") + else: + words.append(name) + return " ".join(words) + + +def get_domain_chart_data(df): + d = df.groupby("domain")["domain"].count() + d = d.to_frame() + d.rename(columns={"domain": "domain_count"}, inplace=True) + d = d.sort_values(by="domain_count", ascending=False) + TOP = 5 + d3 = d[:TOP].copy() + d3.loc["Other"] = d[TOP:]["domain_count"].sum() + d3 = d3.iloc[::-1] + total = d3["domain_count"].sum() + p = (d3["domain_count"] / total) * 100 + + return { + "data": [ + { + "x": d3["domain_count"], + "y": [f"{i} " for i in d3.index], + "text": p, + "type": "bar", + "orientation": "h", + # "marker": {"color": "#f9e56d"}, + "marker": {"color": "#41CD52"}, + "texttemplate": "%{value} (%{text:.2f} %)", + "textposition": "auto", + }, + ], + "layout": { + "title": "Commits per email domain", + "height": "350", + "padding": { + "r": "150", + }, + }, + } + + +def get_ranking_chart_data(df, column="commit_count"): + TOP = 10 + d4 = ( + df.groupby("name") + .agg({"name": "count", "files_changed": "sum", "insertions": "sum", "deletions": "sum"}) + .rename(columns={"name": "commit_count"}) + ) + d4 = d4.sort_values(by=column, ascending=False)[:TOP] + d4 = d4.reset_index() + d4["name"] = d4["name"].apply(partially_hide) + d4 = d4.iloc[::-1] + + titles = { + "files_changed": "Files changed", + "commit_count": "Commits", + "insertions": "Insertions", + "deletions": "Deletions", + } + + colors = { + "files_changed": "#53586b", + "commit_count": "#222840", + "insertions": "#41CD52", + "deletions": "#fb6761", + } + + return { + "data": [ + { + "y": d4["name"], + "x": d4[column], + "type": "bar", + "name": "Commits", + "orientation": "h", + "marker": {"color": colors[column]}, + }, + ], + "layout": { + "title": f"Contributors ({titles[column]})", + "yaxis": {"dtick": "1"}, + "bargap": "2", + "margin": { + "l": "100", + }, + }, + } + + +def get_commit_chart_data(df): + d = df.groupby("date_week")["date_week"].count() + d = d.to_frame() + d.rename(columns={"date_week": "date_count"}, inplace=True) + d = d.reset_index() + # d["week"] = d["date_week"].copy().apply(lambda x : int(x.split("W")[-1])) + # d["year"] = d["date_week"].copy().apply(lambda x : int(x.split("W")[0])) + + return { + "data": [ + { + "x": d["date_week"], + "y": d["date_count"], + "line": {"color": "#222840"}, + "type": "lines", + "xaxis": "x1", + }, + ], + "layout": { + "title": "Number of commits", + "height": "300", + "xaxis": {"tickangle": "45"}, + }, + } + + +def get_collab_chart_data(df): + d = df.groupby("date_week")["email"].nunique() + d = d.to_frame() + d.rename(columns={"email": "collaborators_count"}, inplace=True) + + return { + "data": [ + { + "x": d.index, + "y": d["collaborators_count"], + "line": {"color": "#41CD52"}, + "type": "lines", + }, + ], + "layout": { + "title": "Number of contributors", + "height": "300", + "xaxis": {"tickangle": "45"}, + }, + } + + +@app.callback( + [ + Output("commits-chart", "figure"), + Output("collaborators-chart", "figure"), + Output("files-changed-chart", "figure"), + Output("commit-count-chart", "figure"), + Output("insertions-chart", "figure"), + Output("deletions-chart", "figure"), + Output("domain-chart", "figure"), + ], + [ + Input("module-filter", "value"), + Input("year-filter", "value"), + Input("tqtc-filter", "value"), + ], +) +def update_charts(module, year, tqtc): + """ + This function is in charge of updating the data for all the charts, + and the connection is done by the chart-id from each. + + As Input, we get the values from the combobox: 'module', and 'year', + and then we filter the main dataframe, to re-generate the data + for all the different plots. + """ + df = data[module][data[module]["datetime"].dt.year >= int(year)] + + if "TQtC" not in tqtc: + df = df[df.domain != "qt"] + + commit_data = get_commit_chart_data(df) + collab_data = get_collab_chart_data(df) + files_changed_data = get_ranking_chart_data(df, column="files_changed") + commit_count_data = get_ranking_chart_data(df, column="commit_count") + insertions_data = get_ranking_chart_data(df, column="insertions") + deletions_data = get_ranking_chart_data(df, column="deletions") + domain_data = get_domain_chart_data(df) + + return ( + commit_data, + collab_data, + files_changed_data, + commit_count_data, + insertions_data, + deletions_data, + domain_data, + ) + + +def get_header(): + """ + This is in charge of return the divs that form the header, + both the left logo/title and the right side menu. + """ + return html.Div( + children=[ + html.Div( + children=[ + html.Img(src="assets/theqtproject.png", className="header-logo"), + ], + className="header-title-left", + ), + html.Div( + children=[ + html.A(children="Code Review", href="https://codereview.qt-project.org/"), + html.A(children="Bug Tracker", href="https://bugreports.qt.io"), + html.A(children="Wiki", href="https://wiki.qt.io"), + html.A(children="Docs", href="https://doc.qt.io"), + html.A(children="Mailing List", href="https://lists.qt-project.org"), + html.A(children="Forum", href="https://forum.qt.io"), + html.A(children="Qt.io", href="https://qt.io", style={"color": "#41CD52"}), + ], + className="header-menu-right", + ), + ], + className="header", + ) + + +def get_left_column(divs=[]): + """ + This returns the left divs which contain explanatory text + about different topics, defined by different markdown files + on this project. + """ + + def get_div(i): + return html.Div( + children=[ + dcc.Markdown(i), + ], + className="card pad", + ) + + content = [get_div(i) for i in divs if i] + + return html.Div( + children=content, + className="six columns", + ) + + +def get_services_status(): + gerrit_url = "https://codereview.qt-project.org/projects/qt%2Fqtbase/HEAD" + coin_url = "https://testresults.qt.io/coin/api/capabilities" + + def get_st(url): + response = requests.get(url) + if response.status_code == 200: + return "Online 🟩" + elif response.status_code == 404: + return "Offline 🟥" + else: + return "Undefined 🟧" + + return html.Div( + children=[ + html.H4(children="Services Status"), + html.Div( + children=[ + html.Div( + children=[ + html.B( + children="Gerrit: ", + style={"display": "inline-block", "margin-right": "5px"}, + ), + html.A( + children=f"{get_st(gerrit_url)}", + href="https://codereview.qt-project.org", + style={"display": "inline-block"}, + ), + ], + ), + ], + className="five columns", + ), + html.Div( + children=[ + html.Div( + children=[ + html.B( + children="COIN: ", + style={"display": "inline-block", "margin-right": "5px"}, + ), + html.A( + children=f"{get_st(coin_url)}", + href="https://testresults.qt.io", + style={"display": "inline-block"}, + ), + ], + ), + ], + className="five columns", + ), + ], + className="row card", + ) + + +def get_filter(modules, years): + """ + Get div containing the combobox to filter the modules + and years, to trigger the charts update. + """ + return html.Div( + children=[ + html.Div( + children=[ + html.Div( + children=[ + html.Div(children="Module", className="menu-title"), + dcc.Dropdown( + id="module-filter", + options=[{"label": m, "value": m} for m in modules], + value="qtbase", + clearable=False, + className="dropdown", + ), + ], + ), + dcc.Checklist( + id="tqtc-filter", + options=[ + {"label": "Include commits from The Qt Company", "value": "TQtC"}, + ], + value=["TQtC"], + className="tqtc-filter", + ), + ], + className="six columns", + ), + html.Div( + children=[ + html.Div( + children=[ + html.Div(children="Year", className="menu-title"), + dcc.Dropdown( + id="year-filter", + options=[{"label": m, "value": m} for m in years], + value="2018", + clearable=False, + className="dropdown", + ), + ], + ), + ], + className="six columns", + ), + ], + className="row card option-select", + ) + + +def get_filter_email(lists): + """ + Get div containing the combobox to filter the mailing lists, + to trigger the charts update. + """ + return html.Div( + children=[ + html.Div( + children=[ + html.Div( + children=[ + html.Div(children="Mailing List", className="menu-title"), + dcc.Dropdown( + id="mailing-list-filter", + options=[{"label": m, "value": m} for m in lists], + value="development", + clearable=False, + className="dropdown", + ), + ], + ), + ], + className="six columns", + ), + html.Div( + children=[ + html.Div( + children=[ + html.Div(children="Year", className="menu-title"), + dcc.Dropdown( + id="mailing-list-year-filter", + options=[{"label": m, "value": m} for m in years], + value="2018", + clearable=False, + className="dropdown", + ), + ], + ), + ], + className="six columns", + ), + ], + className="row card option-select", + ) + + +def get_markdown_content(filename): + print(f"Reading '{filename}'...") + content = "" + with open(filename, "r") as f: + content = f.read() + return content + + +# Download the data from an external source which runs a cronjob +# to keep an updated ZIP file with the processed data. +# Heroku will restart every day, so the data will be updated on +# a daily basis. +print("Downloading data...") +r = requests.get("https://qtstats.info/data_csv.zip") +if r.status_code == 404: + print("Error: Problem downloading the Data") + sys.exit(-1) + +with zipfile.ZipFile(io.BytesIO(r.content)) as z: + z.extractall(".") + +print(f"--\nCSV files found: {len(os.listdir('data/'))}\n--") + + +# Loading all the data +print("Loading all the CSV files...") +modules = [] +years = set() +lists = set() +data = {} +email_data = {} +for f in Path("data").glob("*.csv"): + if "email" in f.name: + ml = f.stem.replace("emails_", "") + email_data[ml] = pd.read_csv(f.relative_to(Path(".")), sep=";") + email_data[ml]["datetime"] = pd.to_datetime(email_data[ml]["date"], format="%Y-%m-%d") + else: + data[f.stem] = pd.read_csv(f.relative_to(Path(".")), sep=";") + data[f.stem]["datetime"] = pd.to_datetime(data[f.stem]["date"], format="%Y-%m-%d") + data[f.stem].sort_values("datetime", inplace=True) + data[f.stem]["date_week"] = data[f.stem]["datetime"].apply( + lambda x: f"{x.year} W{str(x.week).zfill(3)}" + ) + modules.append(f.stem) + years.update(set(data[f.stem]["datetime"].dt.year)) + print(f"-- Read '{f.stem}': {data[f.stem].shape}") + +# Generate entry with all the content +data["All Qt"] = pd.concat([i for _, i in data.items()], ignore_index=True) +modules.append("All Qt") + +modules = sorted(modules) +years = [i for i in sorted(years)] +lists = sorted(i.replace("emails_", "") for i in email_data.keys()) +print("Done") + +# Reading 'markdown' files +left_boxes = [ + get_markdown_content("learn_more.md"), + get_markdown_content("contribute.md"), + get_markdown_content("guidelines.md"), + get_markdown_content("quips.md"), +] + +# Layout +app.layout = html.Div( + children=[ + get_header(), + html.Div( + children=[ + html.Div( + children=[ + get_left_column(divs=left_boxes), + html.Div( + children=[ + get_services_status(), + get_filter(modules, years), + html.Div( + children=dcc.Graph(id="commits-chart"), + className="card", + ), + html.Div( + children=dcc.Graph(id="collaborators-chart"), + className="card", + ), + html.Div( + children=dcc.Graph(id="domain-chart"), + className="card", + ), + html.Div( + children=[ + html.Div( + children=dcc.Graph(id="files-changed-chart"), + className="six columns card", + ), + html.Div( + children=dcc.Graph(id="commit-count-chart"), + className="six columns card", + ), + ], + className="row", + ), + html.Div( + children=[ + html.Div( + children=dcc.Graph(id="insertions-chart"), + className="six columns card", + ), + html.Div( + children=dcc.Graph(id="deletions-chart"), + className="six columns card", + ), + ], + className="row", + ), + ], + className="six columns", + ), + ], + className="row", + ), + ], + className="wrapper", + ), + ], +) + +if __name__ == "__main__": + app.run_server(debug=False, threaded=True) diff --git a/assets/favicon.ico b/assets/favicon.ico new file mode 100644 index 0000000..47f5619 Binary files /dev/null and b/assets/favicon.ico differ diff --git a/assets/qt.png b/assets/qt.png new file mode 100644 index 0000000..f13ed3e Binary files /dev/null and b/assets/qt.png differ diff --git a/assets/style.css b/assets/style.css new file mode 100644 index 0000000..ded63f7 --- /dev/null +++ b/assets/style.css @@ -0,0 +1,160 @@ +body { + font-family: "Titillium", sans-serif; + margin: 0; + background-color: #F7F7F7; +} + +h2 { + font-size: 36px; + margin: 0px; + color: #222840; + text-align: center; +} + +h3 { + font-size: 28px; + margin: 0px; + color: #222840; + text-align: left; + padding-left: 20px; +} + +h4 { + font-size: 20px; + margin: 0px; + margin-top: 20px; + color: #222840; + text-align: left; + padding-left: 20px; +} + +ul { + margin-left: 30px; + list-style-type: square; +} + +li { + margin: 0px; +} + +p { + margin: 4px auto; + padding: 10px 20px 10px 20px; +} + +.header { + display: block; + background-color: #222840; + height: 55px; + /*max-width: 1240px;*/ + margin-right: auto; + margin-left: auto; + overflow: hidden; +} + + +.header-logo { + height: 45px; + width: auto; + vertical-align: middle; +} + +.header-title-left { + float: left; + margin: 0 auto; + padding-left: 20px; + padding-right: 20px; + padding-top: 5px; +} + +.header-title { + margin: 0; + margin-left: 5px; + display: inline-block; + color: #FFFFFF; + font-size: 32px; + font-weight: normal; + vertical-align: middle; +} + +.wrapper { + margin-right: auto; + margin-left: auto; + max-width: 1280px; + margin-top: 20px; +} + +.card { + margin-bottom: 22px; + box-shadow: 0 4px 6px 0 rgba(0, 0, 0, 0.18); +} + +.pad { + padding-top: 10px; + padding-bottom: 10px; +} + +.Select-control { + width: 220px; + height: 38px; +} + +.Select--single > .Select-control .Select-value, .Select-placeholder { + line-height: 38px; +} + +.Select--multi .Select-value-label { + line-height: 32px; +} + +.menu-title { + padding-right: 10px; + font-size: 20px; + color: #41CD52; + vertical-align: middle; +} + +.dash-dropdown { + display: inline-block; + vertical-align: middle; +} + +.tqtc-filter { + padding-top: 10px; +} + +.option-select { + padding: 10px 20px 10px 20px;; +} + +.header-menu-right { + float: right; + padding-left: 20px; + padding-right: 20px; + margin-bottom: 20px; + font-weight: bold; + color: #079A82; + overflow: hidden; +} + +.header-menu-right a { + float: left; + display: block; + display: block; + color: #ffffff; + text-align: center; + padding: 14px 16px; + text-decoration: none; + font-size: 17px; +} + +.header-menu-right a:hover { + background-color: #cecfd5; + color: black; +} + +@media screen and (max-width: 500px) { + .header-menu-right { + float: none; + } +} diff --git a/assets/theqtproject.png b/assets/theqtproject.png new file mode 100644 index 0000000..63a69a6 Binary files /dev/null and b/assets/theqtproject.png differ diff --git a/contribute.md b/contribute.md new file mode 100644 index 0000000..b43a31c --- /dev/null +++ b/contribute.md @@ -0,0 +1,31 @@ +### Contribute + +The Qt Project is an open collaboration effort to coordinate the +development of the Qt software framework and tools. + +#### Create an Account + +* [Register](https://login.qt.io/register) your Qt account, so you can + easily authenticate among all the Qt services: Bug tracker, Gerrit, Wiki, + etc. + + +#### Configure Gerrit + +* Login into [Gerrit](https://codereview.qt-project.org/) and add your desire + information to your account, but remember to accept the + [CLA](https://www.qt.io/legal-contribution-agreement-qt). +* Configure your SSH and add the proper remote to your repo. + ([read more](https://wiki.qt.io/Setting_up_Gerrit)). + + +#### Commit & Push + +Once you have your local patch ready to submit, remember a few things: + +* Check the [Commit Policy](https://wiki.qt.io/Commit_Policy) +* Remember to add a `Task-number:` or `Fixes:` entry to the commit footer. +* Since your patch will be pushed to dev, add a `Pick-to:` when a cherry pick + to another branch is necessary. +* `git push gerrit HEAD:refs/for/dev` + diff --git a/data/get_data.py b/data/get_data.py new file mode 100644 index 0000000..35cf548 --- /dev/null +++ b/data/get_data.py @@ -0,0 +1,165 @@ +# Copyright (C) 2021 The Qt Company Ltd. +# Contact: https://www.qt.io/licensing/ +# +# You may use this file under the terms of the CC0 license. +# See the file LICENSE.CC0 from this package for details. + +import argparse +import logging +import os +import re +import subprocess +import sys +from pathlib import Path + +logging.basicConfig(level=logging.INFO) +log = logging.getLogger("get_data") + + +class cd: + def __init__(self, path): + self.path = path + + def __enter__(self): + self.saved = os.getcwd() + os.chdir(self.path) + + def __exit__(self, etype, value, traceback): + os.chdir(self.saved) + + +def get_qt_modules(qt_path): + def is_qt(x): + valid_dir = (x.startswith("qt") or "pyside-setup" in x) + return valid_dir and os.path.isdir(os.path.join(qt_path, x)) + return sorted([i for i in os.listdir(qt_path) if is_qt(i)]) + + +def get_email_domain(x): + x = x.replace('"', "") + if x.count("@") == 0: + try: + v = x.split(".")[-2] + except IndexError: + v = "" + else: + v = ".".join(x.split("@")[1].split(".")[:-1]).replace("\\", "") + + if v in ("theqtcompany", "qt", "nokia", "nokiamail", "digia"): + new_email = re.sub("@.*", "@qt.io", x) + return new_email, "qt" + return x, v + + +def process_git_log_line(line): + changed = insertions = deletions = 0 + # files changed + re_changed = re.search(r"(\d+) files? changed", line) + if re_changed: + changed = re_changed.group(1) + # insertions + re_insertions = re.search(r"(\d+) insertions?", line) + if re_insertions: + insertions = re_insertions.group(1) + # deletions + re_deletions = re.search(r"(\d+) deletions?", line) + if re_deletions: + deletions = re_deletions.group(1) + + original_line = line.split("‽")[0] + # the last field is the 'email' + original_email = original_line.split(";")[-1] + email, domain = get_email_domain(original_email) + + return f'{original_line};"{email}";"{domain}";"{changed}";"{insertions}";"{deletions}"' + + +def git_log(): + def is_valid_line(x): + if ( + x.strip() + and "Qt by Nokia" not in x + and "Qt Forward Merge Bot" not in x + and "Qt Submodule Update Bot" not in x + ): + return True + return False + + # We can do this process with: + # git log --all --no-merges --date=format:'%Y-%m-%d' + # --pretty=format:'µ"%cd";"%h";"%an";"%ce"' + # --shortstat | tr '\n' ' ' | tr 'µ' '\n' | + # sed 's/\ *\(\d+\)\ files\{0,1\}/\1/g' + # But we will use only Python to perform those pipe operations. + + # This command has a trick to get the 'shortstats' on the same line + # when processing the lines. Notice the 'µ' character that depicts the + # beginning of the line. Additionally we use an interrobang '‽' + # to depict the end of the git log, so we can add the 'files changed', + # 'insertions', and 'deletions' at the end. + o = subprocess.run( + ( + "git log --all --no-merges " + "--date=format:'%Y-%m-%d' " + '--pretty=format:\'µ"%cd";"%h";"%an";"%ae"\'‽ ' + "--shortstat " + ).split(), + capture_output=True, + universal_newlines=True, + encoding="utf-8", + errors="ignore", + ).stdout + o = o.replace("\n", " ").replace("µ", "\n").replace("'", "") + return "\n".join(process_git_log_line(line) for line in o.splitlines() if is_valid_line(line)) + + +def check_arguments(options): + qt_path = Path(options.qt_dir) + if qt_path.is_dir(): + return True + log.error(f"'{qt_path}' is not a directory.") + return False + + +def is_valid_module(m): + if (m.name.startswith("qt") or str(m.name) in ("pyside-setup",)) and m.is_dir(): + return True + return False + + +def process_qt_src(options): + qt_path = Path(options.qt_dir) + HEADER = "date;sha;name;original_email;email;domain;files_changed;insertions;deletions\n" + for i in qt_path.glob("*"): + if is_valid_module(i): + log.info(f"Processing {i}...") + output_csv = f"{Path(__file__).parent}/{i.name}.csv" + out = None + with cd(i): + out = git_log() + if not out: + log.error("Empty 'git log' for i") + continue + with open(output_csv, "w") as f: + f.write(HEADER) + f.write(out) + + +if __name__ == "__main__": + parser = argparse.ArgumentParser(prog="get_data") + + parser.add_argument( + "--qt", + action="store", + dest="qt_dir", + required=True, + help="Path to a directory containing Qt modules, like the 'qt5' meta repository", + ) + + options = parser.parse_args() + if not check_arguments(options): + parser.print_help() + sys.exit(-1) + + # main process + process_qt_src(options) diff --git a/guidelines.md b/guidelines.md new file mode 100644 index 0000000..cc25ebf --- /dev/null +++ b/guidelines.md @@ -0,0 +1,76 @@ +### Contribution guidelines + +The Qt Project governs the open source development of Qt. +It allows anybody wanting to contribute to join the effort, +through a [meritocratic structure of approvers and maintainers](http://wiki.qt.io/The_Qt_Governance_Model). + +All development will be driven by the people contributing to the project. +To learn more, visit the [wiki](https://wiki.qt.io) and +[subscribe to our mailing-lists](http://lists.qt-project.org/). + +* [Qt Project Guidelines](https://wiki.qt.io/Qt_Project_Guidelines) +* [Qt source repositories](http://code.qt.io/) +* [Contribution agreement details](http://qt.io/contributionagreement) + +#### Support Users + +Helpful people are at the center of every great community, and everyone needs a bit of +help sometimes. +Whether it is a specific programming problem or more general guidance required, +there are many ways to share knowledge and support others. + +* [Forum](http://forum.qt.io) +* [Online communities](http://wiki.qt.io/OnlineCommunities) + +#### Report Bugs + +Meaningful [bug reports](http://wiki.qt.io/ReportingBugsInQt) and comments help +improve the quality of Qt in a very direct way, while voting and +[triaging](http://wiki.qt.io/Triaging_Bugs) help to prioritize tasks. We +encourage Qt users and Qt contributors to join efforts on the bug tracker. + +#### Write Documentation and Tutorials + +Qt has great official documentation. +Now everyone can contribute to it, in the same way they can contribute code to Qt itself. +In addition, there are also other ways for contributors who enjoy writing about technology. + +* [Write blogs about Qt](http://planet.qt.io/) +* [Write books about Qt](http://wiki.qt.io/books) +* Create learning material and presentations. +* [Translate Qt](http://wiki.qt.io/Qt_Localization) +* [Write documentation](http://wiki.qt.io/Category:Developing_Qt::Documentation) +* [Create demos and examples](http://wiki.qt.io/Category:Learning::Demos_and_Examples) + +#### Do Community Work + +There are plenty of important tasks to keep things going smoothly within the larger Qt community. +No matter if you prefer online or IRL interactions, there is something for you. + +* [Represent Qt at events and meetups](http://wiki.qt.io/Meet_Qt) +* [Organize events and meetups](http://www.meetup.com/QtEverywhere/) + +#### Write Qt Code + +Naturally, the Qt Project is mostly about code. +There are plenty of ways to contribute code, to learn and grow, +and to [build one's reputation](http://wiki.qt.io/Qt_Governance_Model) +To get started contributing code, get in touch with +[the relevant module maintainer](http://wiki.qt.io/Maintainers) before you start on a patch. + +* [Fix bugs](https://bugreports.qt.io) +* [Write tests](http://wiki.qt.io/Writing_Unit_Tests) +* [Review Qt code](http://wiki.qt.io/Code_Reviews) +* [Write Qt code](http://wiki.qt.io) +* [Participate in the release process](http://wiki.qt.io/Release_Management) + +### Governance model + +The main objectives of the Governance Model are to: + +* Put decision power in the hands of the community, + i.e. the people who contribute to the Project's success +* Make it easy to understand how to get involved and make a difference + +The five levels of involvement: Users, Contributors, Approvers, Maintainers and Chief Maintainer +([read more](http://wiki.qt.io/The_Qt_Governance_Model)). diff --git a/learn_more.md b/learn_more.md new file mode 100644 index 0000000..c59b6e5 --- /dev/null +++ b/learn_more.md @@ -0,0 +1,5 @@ +### General Information + +This page is focused on the contribution aspects of the Qt Project, +in case you are looking for additional information, +visit [qt.io](https://qt.io). diff --git a/quips.md b/quips.md new file mode 100644 index 0000000..ea980f7 --- /dev/null +++ b/quips.md @@ -0,0 +1,18 @@ +### Qt's Utilitarian Improvement Process (QUIPS) + + * [0001 - QUIP Purpose and Guidelines](http://quips-qt-io.herokuapp.com/quip-0001.html) + * [0002 - The Qt Governance Model](http://quips-qt-io.herokuapp.com/quip-0002.html) + * [0003 - "QUIPs for Qt" - QtCon 2016 Session Notes](http://quips-qt-io.herokuapp.com/quip-0003.html) + * [0004 - Third-Party Components in Qt](http://quips-qt-io.herokuapp.com/quip-0004.html) + * [0005 - Choosing a Branch](http://quips-qt-io.herokuapp.com/quip-0005.html) + * [0006 - Acceptable Source-Incompatible Changes](http://quips-qt-io.herokuapp.com/quip-0006.html) + * [0007 - qt_attribution.json File Format](http://quips-qt-io.herokuapp.com/quip-0007.html) + * [0009 - Evolving QList](http://quips-qt-io.herokuapp.com/quip-0009.html) + * [0010 - Reviewing API changes in preparation for release](http://quips-qt-io.herokuapp.com/quip-0010-API-review.html) + * [0011 - Qt Release Process](http://quips-qt-io.herokuapp.com/quip-0011-Release-Process.html) + * [0012 - The Qt Community Code of Conduct](http://quips-qt-io.herokuapp.com/quip-0012-Code-of-Conduct.html) + * [0013 - Qt Examples and Demos](http://quips-qt-io.herokuapp.com/quip-0013-Examples.html) + * [0014 - The Module Life-Cycle](http://quips-qt-io.herokuapp.com/quip-0014-Module-Criteria.html) + * [0015 - Qt Project Security Policy](http://quips-qt-io.herokuapp.com/quip-0015-Security-Policy.html) + * [0016 - Branch policy](http://quips-qt-io.herokuapp.com/quip-0016-branch-policy.html) + * [0017 - Change log creation](http://quips-qt-io.herokuapp.com/quip-0017-Change-log-creation.html) diff --git a/requirements.txt b/requirements.txt new file mode 100644 index 0000000..ec36fb7 --- /dev/null +++ b/requirements.txt @@ -0,0 +1,4 @@ +dash +pandas +requests +gunicorn diff --git a/runtime.txt b/runtime.txt new file mode 100644 index 0000000..30a1be6 --- /dev/null +++ b/runtime.txt @@ -0,0 +1 @@ +python-3.9.2 diff --git a/update-data.sh b/update-data.sh new file mode 100644 index 0000000..468d531 --- /dev/null +++ b/update-data.sh @@ -0,0 +1,47 @@ +# Copyright (C) 2021 The Qt Company Ltd. +# Contact: https://www.qt.io/licensing/ +# +# You may use this file under the terms of the CC0 license. +# See the file LICENSE.CC0 from this package for details. + + +# For local testing with new data + +# Approach to use Git +# If the repository doesn't exist, clone it +if [[ ! -d qt5 ]];then + git clone https://code.qt.io/qt/qt5.git || echo "Error cloning 'qt5'" +fi + +# Enter the repository +cd qt5 + +## Clone qt-creator if it doesn't exist +#if [[ ! -d qt-creator ]];then +# git clone http://code.qt.io/cgit/qt-creator/qt-creator.git || echo "Error cloning 'qt-creator'" +#fi + +# Clone pyside-setup if it doesn't exist +if [[ ! -d pyside-setup ]];then + git clone https://code.qt.io/pyside/pyside-setup || echo "Error cloning 'pyside-setup'" +fi + +# General update of all the submodules +git pull -r +git submodule foreach "(git co dev || git co master)" +git submodule foreach git pull --rebase +git submodule update --init -f + +## Update 'qt-creator' +#cd qt-creator +#git pull -r +#cd .. + +# Update 'pyside-setup' +cd pyside-setup +git pull -r +cd .. + +# Generate CSV files +cd ../data +python get_data.py --qt ../qt5 -- cgit v1.2.3