summaryrefslogtreecommitdiffstats
path: root/doc
diff options
context:
space:
mode:
authorjasplin <qt-info@nokia.com>2009-08-13 10:32:14 +0200
committerjasplin <qt-info@nokia.com>2009-08-13 10:32:14 +0200
commitdf99e9ef6e005d36aa16ceabe1838f61b6abece9 (patch)
treedebe4ed8956c1492c4a0fca2b8a29b04a5a0de1f /doc
parent3bad2f608813968d85c2ea51c1bd75dca7f7e22a (diff)
Updates.
Diffstat (limited to 'doc')
-rw-r--r--doc/bmcp.html36
-rw-r--r--doc/notes.txt179
2 files changed, 150 insertions, 65 deletions
diff --git a/doc/bmcp.html b/doc/bmcp.html
index d70ef77..28dfa26 100644
--- a/doc/bmcp.html
+++ b/doc/bmcp.html
@@ -392,8 +392,8 @@ class="alias">TESTCASERESULTS</td><td></td><td></td></tr>
<hr />
<p class="msgDescr">
-This request compares results of a particular BMCONTEXT/BRANCH combination
-since a particular SNAPSHOT.
+This request compares results as a history of a particular BMCONTEXT/BRANCH
+combination since a particular SNAPSHOT.
</p>
<h4 class="msgName">CompareHistoryRequest</h4>
@@ -487,7 +487,7 @@ request)</td></tr>
This request lists all snapshots available for a particular BMCONTEXT/BRANCH combination.
</p>
-<h4 class="msgName">ListAvailableSnapshotsRequest</h4>
+<h4 class="msgName">ListSnapshotsRequest</h4>
<table border=0>
<thead><tr><th>No. of bytes</th><th>Type</th><th>Value</th><th>Description</th></tr></thead>
<tr><td>1</td><td>U8</td><td>20</td><td>message type</td></tr>
@@ -495,7 +495,7 @@ This request lists all snapshots available for a particular BMCONTEXT/BRANCH com
<tr><td>BRANCH size</td><td class="alias">BRANCH</td><td></td><td></td></tr>
</table>
-<h4 class="msgName">ListAvailableSnapshotsReply</h4>
+<h4 class="msgName">ListSnapshotsReply</h4>
<table border=0>
<thead><tr><th>No. of bytes</th><th>Type</th><th>Value</th><th>Description</th></tr></thead>
<tr><td>1</td><td>U8</td><td>21</td><td>message type</td></tr>
@@ -519,7 +519,7 @@ entity values are empty, the output will include all available benchmarks,
even those that haven't been associated with any result yet.
</p>
-<h4 class="msgName">ListAvailableBenchmarksRequest</h4>
+<h4 class="msgName">ListBenchmarksRequest</h4>
<table border=0>
<thead><tr><th>No. of bytes</th><th>Type</th><th>Value</th><th>Description</th></tr></thead>
<tr><td>1</td><td>U8</td><td>22</td><td>message type</td></tr>
@@ -529,7 +529,7 @@ even those that haven't been associated with any result yet.
<tr><td>BRANCH size</td><td class="alias">BRANCH</td><td></td><td></td></tr>
</table>
-<h4 class="msgName">ListAvailableBenchmarksReply</h4>
+<h4 class="msgName">ListBenchmarksReply</h4>
<table border=0>
<thead><tr><th>No. of bytes</th><th>Type</th><th>Value</th><th>Description</th></tr></thead>
<tr><td>1</td><td>U8</td><td>23</td><td>message type</td></tr>
@@ -545,10 +545,10 @@ even those that haven't been associated with any result yet.
<hr />
<p class="msgDescr">
-This request behaves similarly to ListAvailableBenchmarks* (see this).
+This request behaves similarly to ListBenchmarks* (see this).
</p>
-<h4 class="msgName">ListAvailableMetricsRequest</h4>
+<h4 class="msgName">ListMetricsRequest</h4>
<table border=0>
<thead><tr><th>No. of bytes</th><th>Type</th><th>Value</th><th>Description</th></tr></thead>
<tr><td>1</td><td>U8</td><td>24</td><td>message type</td></tr>
@@ -558,7 +558,7 @@ This request behaves similarly to ListAvailableBenchmarks* (see this).
<tr><td>BRANCH size</td><td class="alias">BRANCH</td><td></td><td></td></tr>
</table>
-<h4 class="msgName">ListAvailableMetricsReply</h4>
+<h4 class="msgName">ListMetricsReply</h4>
<table border=0>
<thead><tr><th>No. of bytes</th><th>Type</th><th>Value</th><th>Description</th></tr></thead>
<tr><td>1</td><td>U8</td><td>25</td><td>message type</td></tr>
@@ -574,10 +574,10 @@ This request behaves similarly to ListAvailableBenchmarks* (see this).
<hr />
<p class="msgDescr">
-This request behaves similarly to ListAvailableBenchmarks* (see this).
+This request behaves similarly to ListBenchmarks* (see this).
</p>
-<h4 class="msgName">ListAvailablePlatformsRequest</h4>
+<h4 class="msgName">ListPlatformsRequest</h4>
<table border=0>
<thead><tr><th>No. of bytes</th><th>Type</th><th>Value</th><th>Description</th></tr></thead>
<tr><td>1</td><td>U8</td><td>26</td><td>message type</td></tr>
@@ -587,7 +587,7 @@ This request behaves similarly to ListAvailableBenchmarks* (see this).
<tr><td>BRANCH size</td><td class="alias">BRANCH</td><td></td><td></td></tr>
</table>
-<h4 class="msgName">ListAvailablePlatformsReply</h4>
+<h4 class="msgName">ListPlatformsReply</h4>
<table border=0>
<thead><tr><th>No. of bytes</th><th>Type</th><th>Value</th><th>Description</th></tr></thead>
<tr><td>1</td><td>U8</td><td>27</td><td>message type</td></tr>
@@ -603,10 +603,10 @@ This request behaves similarly to ListAvailableBenchmarks* (see this).
<hr />
<p class="msgDescr">
-This request behaves similarly to ListAvailableBenchmarks* (see this).
+This request behaves similarly to ListBenchmarks* (see this).
</p>
-<h4 class="msgName">ListAvailableHostsRequest</h4>
+<h4 class="msgName">ListHostsRequest</h4>
<table border=0>
<thead><tr><th>No. of bytes</th><th>Type</th><th>Value</th><th>Description</th></tr></thead>
<tr><td>1</td><td>U8</td><td>28</td><td>message type</td></tr>
@@ -616,7 +616,7 @@ This request behaves similarly to ListAvailableBenchmarks* (see this).
<tr><td>BRANCH size</td><td class="alias">BRANCH</td><td></td><td></td></tr>
</table>
-<h4 class="msgName">ListAvailableHostsReply</h4>
+<h4 class="msgName">ListHostsReply</h4>
<table border=0>
<thead><tr><th>No. of bytes</th><th>Type</th><th>Value</th><th>Description</th></tr></thead>
<tr><td>1</td><td>U8</td><td>29</td><td>message type</td></tr>
@@ -632,10 +632,10 @@ This request behaves similarly to ListAvailableBenchmarks* (see this).
<hr />
<p class="msgDescr">
-This request behaves similarly to ListAvailableBenchmarks* (see this).
+This request behaves similarly to ListBenchmarks* (see this).
</p>
-<h4 class="msgName">ListAvailableBranchesRequest</h4>
+<h4 class="msgName">ListBranchesRequest</h4>
<table border=0>
<thead><tr><th>No. of bytes</th><th>Type</th><th>Value</th><th>Description</th></tr></thead>
<tr><td>1</td><td>U8</td><td>30</td><td>message type</td></tr>
@@ -645,7 +645,7 @@ This request behaves similarly to ListAvailableBenchmarks* (see this).
<tr><td>HOST size</td><td class="alias">HOST</td><td></td><td></td></tr>
</table>
-<h4 class="msgName">ListAvailableBranchesReply</h4>
+<h4 class="msgName">ListBranchesReply</h4>
<table border=0>
<thead><tr><th>No. of bytes</th><th>Type</th><th>Value</th><th>Description</th></tr></thead>
<tr><td>1</td><td>U8</td><td>31</td><td>message type</td></tr>
diff --git a/doc/notes.txt b/doc/notes.txt
index 2611699..c2c9201 100644
--- a/doc/notes.txt
+++ b/doc/notes.txt
@@ -18,7 +18,7 @@ repository and a Git branch.
------------
-Definition: SNAPSHOT = -timestamp <secs since epoch> | -sha1 <sha1>
+Definition: SNAPSHOT = -timestamp <secs since 1/1 1970> | -sha1 <sha1>
A -timestamp value t has the following semantics:
@@ -29,13 +29,15 @@ Note: The -sha1 option requires exact matching and is thus less powerful than -t
----------------------------------
-
Use case 1: Add a benchmark:
./bmc add -benchmark <test case> <test function> <data tag>
-----------------------------------
+ Example:
+ ./bmc add -benchmark tst_QWidget updateOpaque 10x10x1
+
+----------------------------------
Use case 2: Add a metric:
./bmc add -metric <metric>
@@ -43,97 +45,180 @@ Use case 2: Add a metric:
Note: walltime, cputicks, callgrind, and events should be added upon installation.
----------------------------------
-
Use case 3: Add a platform:
./bmc add -platform <platform>
Notes:
- - The platform typically identifies a particular OS/compiler combination.
- - The official platforms supported by Qt (linux-g++, solaris-cc-64 etc.) should
- be added upon installation.
-----------------------------------
+ - The platform typically identifies a particular OS/compiler combination.
+ - The official platforms supported by Qt (linux-g++, solaris-cc-64 etc.) should
+ be added upon installation.
+
+----------------------------------
Use case 4: Add a host:
./bmc add -host <host> [<details>]
- Note: The host identifies a physical machine.
+ Note: The host identifies a physical machine.
-----------------------------------
+ Example:
+ ./bmc add -host barbarella
+
+----------------------------------
Use case 5: Add a branch:
./bmc add -branch <git repo> <git branch>
Note: qt/master, qt/45, qt/46, qt/... should be added upon installation.
-----------------------------------
+ Example:
+ ./bmc add -branch jasplins-qt master
+
+----------------------------------
Use case 6: Add new results:
./bmc add -results <xml file> <platform> <host> <product git repo> [<product git dir>]
Notes:
- - <product git dir> must refer to a directory accessible from the local
- file system and the git command needs to be installed (i.e. the program
- needs to run git commands from <product git dir>).
- By default, <product git dir> is the current directory.
- - The new results are assumed to have been generated using a product
- built from a clean checkout of the current head commit of the current git branch
- in <product git dir>.
- - The sha1 to be stored in the database is assumed to be the current head commit in
- <product git dir> (extracted running 'git log -1 HEAD --pretty=format:"%H"').
- - The timestamp to be stored in the database is the Unix timestamp of the committer
- date (extracted running 'git log -1 HEAD --pretty=format:"%ct"').
- - <product git repo> must be passed explicitly since it cannot in general
- be derived from <product git dir> (e.g. in the case of a local, standalone
- repository).
-
-----------------------
-
-Use case 1: Compare results for a given branch since a given snapshot:
-
- ./bmc compare -history BMCONTEXT -branch <branch> SNAPSHOT
+
+ - <xml file> must (essentially) be of the same format as the one being dumped
+ by QTestLib.
+
+ - <product git dir> must refer to a directory accessible from the local
+ file system and the git command needs to be installed (i.e. the program
+ needs to run git commands from <product git dir>).
+ By default, <product git dir> is the current directory.
+
+ - The new results are assumed to have been generated using a product
+ built from a clean checkout of the current head commit of the current git branch
+ in <product git dir>.
+
+ - The sha1 to be stored in the database is assumed to be the current head commit in
+ <product git dir> (extracted running 'git log -1 HEAD --pretty=format:"%H"').
+
+ - The timestamp to be stored in the database is the Unix timestamp of the committer
+ date (extracted running 'git log -1 HEAD --pretty=format:"%ct"').
+
+ - <product git repo> must be passed explicitly since it cannot in general
+ be derived from <product git dir> (e.g. in the case of a local, standalone
+ repository).
Example:
- ./bmc compare -history -testcase tst_QWidget -testfunc updateOpaque \
- -datatag 10x10x1 -metric walltime -platform linux-g++ -host barbarella \
- -branch qt master -timestamp first
+ ./bmc add -results results.xml linux-g++ barbarella qt master $QTDIR
+
+----------------------------------
+Use case 7: Compare results as a history of a given branch since a given snapshot:
+
+ ./bmc compare -history BMCONTEXT <branch> SNAPSHOT
+
+ Example 1:
+
+ ./bmc compare -history tst_QWidget updateOpaque 10x10x1 walltime \
+ linux-g++ barbarella qt master -timestamp first
+
+ Example 2 (1250146917 corresponds to the date Thu Aug 13 09:01:57 2009 +0200):
+ ./bmc compare -history tst_QWidget updateOpaque 10x10x1 walltime \
+ linux-g++ barbarella qt master -timestamp 1250146917
-Use case 2: Compare the results of two branches:
+ Example 3:
+
+ ./bmc compare -history tst_QWidget updateOpaque 10x10x1 walltime \
+ linux-g++ barbarella qt master -sha1 3bad2f608813968d85c2ea51c1bd75dca7f7e22a
+
+----------------------------------
+Use case 8: Compare the results of two branches:
./bmc compare -branches BMCONTEXT <branch1> SNAPSHOT1 <branch2> SNAPSHOT2
-Use case 3: Compare a result of a given branch directly to an unregistered
- result (i.e. one not yet in the DB):
+ Example:
+
+ ./bmc compare -branches tst_QWidget updateOpaque 10x10x1 walltime \
+ linux-g++ barbarella qt 45 -timestamp last qt 46 -timestamp last
+
+----------------------------------
+Use case 9: Compare a result of a given branch directly to an unregistered
+ result (i.e. one not yet in the database):
./bmc compare -direct BMCONTEXT <branch> SNAPSHOT (<value> <iterations>) | <xml file>
Note: If an xml file is specified, it is assumed to contain a
- value/iterations pair that matches the benchmark and metric in the
- BMCONTEXT.
+ value/iterations pair that matches the benchmark and metric in the
+ BMCONTEXT.
+
+ Example 1:
+ ./bmc compare -direct tst_QWidget updateOpaque 10x10x1 walltime \
+ linux-g++ barbarella qt master -timestamp last 40 1024
+
+ Example 2:
+
+ ./bmc compare -direct tst_QWidget updateOpaque 10x10x1 walltime \
+ linux-g++ barbarella qt master -timestamp last results.xml
----------------------------------
+Use case 10: List available snapshots for a given branch:
+
+ ./bmc list -snapshots BMCONTEXT <branch>
-Use case 5: List available benchmarks (subject to optional restrictions):
+ Example:
+
+ ./bmc list -snapshots tst_QWidget updateOpaque 10x10x1 walltime \
+ linux-g++ barbarella qt master
+
+----------------------------------
+Use case 11: List available benchmarks (subject to optional restrictions):
./bmc list -benchmarks [-metric <metric>] [-platform <platform>] [-host <host>] \
[-branch <branch>]
-Ditto for other entities:
+ Example 1:
+
+ ./bmc list -benchmarks
+
+ Example 2:
+
+ ./bmc list -benchmarks -platform linux-g++
+
+ Example 3:
+
+ ./bmc list -benchmarks -platform linux-g++ -host barbarella -metric walltime \
+ -branch qt master
+
+----------------------------------
+Use case 12: List available metrics (subject to optional restrictions):
+
+ ./bmc list -metrics [-benchmark <benchmark>] [-platform <platform>] [-host <host>] \
+ [-branch <branch>]
+
+ Example 1:
+
+ ./bmc list -metrics
+
+ Example 2:
- ./bmc list -metrics ...
- ./bmc list -platforms ...
- ./bmc list -hosts ...
- ./bmc list -branches ...
+ ./bmc list -metrics -benchmark tst_QWidget updateOpaque 10x10x1 \
+ -platform linux-g++ -host barbarella -branch qt master
----------------------------------
+Use case 13: List available platforms (subject to optional restrictions):
-Use case 6: List available snapshots for a given branch:
+ ./bmc list -platforms [-benchmark <benchmark>] [-metric <metric>] [-host <host>] \
+ [-branch <branch>]
- ./bmc list -snapshots BMCONTEXT <branch>
+----------------------------------
+Use case 14: List available hosts (subject to optional restrictions):
+
+ ./bmc list -hosts [-benchmark <benchmark>] [-metric <metric>] [-platform <platform>] \
+ [-branch <branch>]
+
+----------------------------------
+Use case 15: List available branches (subject to optional restrictions):
+
+ ./bmc list -platforms [-benchmark <benchmark>] [-metric <metric>] [-platform <platform>] \
+ [-host <host>]