diff options
author | jasplin <qt-info@nokia.com> | 2009-08-13 10:32:14 +0200 |
---|---|---|
committer | jasplin <qt-info@nokia.com> | 2009-08-13 10:32:14 +0200 |
commit | df99e9ef6e005d36aa16ceabe1838f61b6abece9 (patch) | |
tree | debe4ed8956c1492c4a0fca2b8a29b04a5a0de1f /doc | |
parent | 3bad2f608813968d85c2ea51c1bd75dca7f7e22a (diff) |
Updates.
Diffstat (limited to 'doc')
-rw-r--r-- | doc/bmcp.html | 36 | ||||
-rw-r--r-- | doc/notes.txt | 179 |
2 files changed, 150 insertions, 65 deletions
diff --git a/doc/bmcp.html b/doc/bmcp.html index d70ef77..28dfa26 100644 --- a/doc/bmcp.html +++ b/doc/bmcp.html @@ -392,8 +392,8 @@ class="alias">TESTCASERESULTS</td><td></td><td></td></tr> <hr /> <p class="msgDescr"> -This request compares results of a particular BMCONTEXT/BRANCH combination -since a particular SNAPSHOT. +This request compares results as a history of a particular BMCONTEXT/BRANCH +combination since a particular SNAPSHOT. </p> <h4 class="msgName">CompareHistoryRequest</h4> @@ -487,7 +487,7 @@ request)</td></tr> This request lists all snapshots available for a particular BMCONTEXT/BRANCH combination. </p> -<h4 class="msgName">ListAvailableSnapshotsRequest</h4> +<h4 class="msgName">ListSnapshotsRequest</h4> <table border=0> <thead><tr><th>No. of bytes</th><th>Type</th><th>Value</th><th>Description</th></tr></thead> <tr><td>1</td><td>U8</td><td>20</td><td>message type</td></tr> @@ -495,7 +495,7 @@ This request lists all snapshots available for a particular BMCONTEXT/BRANCH com <tr><td>BRANCH size</td><td class="alias">BRANCH</td><td></td><td></td></tr> </table> -<h4 class="msgName">ListAvailableSnapshotsReply</h4> +<h4 class="msgName">ListSnapshotsReply</h4> <table border=0> <thead><tr><th>No. of bytes</th><th>Type</th><th>Value</th><th>Description</th></tr></thead> <tr><td>1</td><td>U8</td><td>21</td><td>message type</td></tr> @@ -519,7 +519,7 @@ entity values are empty, the output will include all available benchmarks, even those that haven't been associated with any result yet. </p> -<h4 class="msgName">ListAvailableBenchmarksRequest</h4> +<h4 class="msgName">ListBenchmarksRequest</h4> <table border=0> <thead><tr><th>No. of bytes</th><th>Type</th><th>Value</th><th>Description</th></tr></thead> <tr><td>1</td><td>U8</td><td>22</td><td>message type</td></tr> @@ -529,7 +529,7 @@ even those that haven't been associated with any result yet. <tr><td>BRANCH size</td><td class="alias">BRANCH</td><td></td><td></td></tr> </table> -<h4 class="msgName">ListAvailableBenchmarksReply</h4> +<h4 class="msgName">ListBenchmarksReply</h4> <table border=0> <thead><tr><th>No. of bytes</th><th>Type</th><th>Value</th><th>Description</th></tr></thead> <tr><td>1</td><td>U8</td><td>23</td><td>message type</td></tr> @@ -545,10 +545,10 @@ even those that haven't been associated with any result yet. <hr /> <p class="msgDescr"> -This request behaves similarly to ListAvailableBenchmarks* (see this). +This request behaves similarly to ListBenchmarks* (see this). </p> -<h4 class="msgName">ListAvailableMetricsRequest</h4> +<h4 class="msgName">ListMetricsRequest</h4> <table border=0> <thead><tr><th>No. of bytes</th><th>Type</th><th>Value</th><th>Description</th></tr></thead> <tr><td>1</td><td>U8</td><td>24</td><td>message type</td></tr> @@ -558,7 +558,7 @@ This request behaves similarly to ListAvailableBenchmarks* (see this). <tr><td>BRANCH size</td><td class="alias">BRANCH</td><td></td><td></td></tr> </table> -<h4 class="msgName">ListAvailableMetricsReply</h4> +<h4 class="msgName">ListMetricsReply</h4> <table border=0> <thead><tr><th>No. of bytes</th><th>Type</th><th>Value</th><th>Description</th></tr></thead> <tr><td>1</td><td>U8</td><td>25</td><td>message type</td></tr> @@ -574,10 +574,10 @@ This request behaves similarly to ListAvailableBenchmarks* (see this). <hr /> <p class="msgDescr"> -This request behaves similarly to ListAvailableBenchmarks* (see this). +This request behaves similarly to ListBenchmarks* (see this). </p> -<h4 class="msgName">ListAvailablePlatformsRequest</h4> +<h4 class="msgName">ListPlatformsRequest</h4> <table border=0> <thead><tr><th>No. of bytes</th><th>Type</th><th>Value</th><th>Description</th></tr></thead> <tr><td>1</td><td>U8</td><td>26</td><td>message type</td></tr> @@ -587,7 +587,7 @@ This request behaves similarly to ListAvailableBenchmarks* (see this). <tr><td>BRANCH size</td><td class="alias">BRANCH</td><td></td><td></td></tr> </table> -<h4 class="msgName">ListAvailablePlatformsReply</h4> +<h4 class="msgName">ListPlatformsReply</h4> <table border=0> <thead><tr><th>No. of bytes</th><th>Type</th><th>Value</th><th>Description</th></tr></thead> <tr><td>1</td><td>U8</td><td>27</td><td>message type</td></tr> @@ -603,10 +603,10 @@ This request behaves similarly to ListAvailableBenchmarks* (see this). <hr /> <p class="msgDescr"> -This request behaves similarly to ListAvailableBenchmarks* (see this). +This request behaves similarly to ListBenchmarks* (see this). </p> -<h4 class="msgName">ListAvailableHostsRequest</h4> +<h4 class="msgName">ListHostsRequest</h4> <table border=0> <thead><tr><th>No. of bytes</th><th>Type</th><th>Value</th><th>Description</th></tr></thead> <tr><td>1</td><td>U8</td><td>28</td><td>message type</td></tr> @@ -616,7 +616,7 @@ This request behaves similarly to ListAvailableBenchmarks* (see this). <tr><td>BRANCH size</td><td class="alias">BRANCH</td><td></td><td></td></tr> </table> -<h4 class="msgName">ListAvailableHostsReply</h4> +<h4 class="msgName">ListHostsReply</h4> <table border=0> <thead><tr><th>No. of bytes</th><th>Type</th><th>Value</th><th>Description</th></tr></thead> <tr><td>1</td><td>U8</td><td>29</td><td>message type</td></tr> @@ -632,10 +632,10 @@ This request behaves similarly to ListAvailableBenchmarks* (see this). <hr /> <p class="msgDescr"> -This request behaves similarly to ListAvailableBenchmarks* (see this). +This request behaves similarly to ListBenchmarks* (see this). </p> -<h4 class="msgName">ListAvailableBranchesRequest</h4> +<h4 class="msgName">ListBranchesRequest</h4> <table border=0> <thead><tr><th>No. of bytes</th><th>Type</th><th>Value</th><th>Description</th></tr></thead> <tr><td>1</td><td>U8</td><td>30</td><td>message type</td></tr> @@ -645,7 +645,7 @@ This request behaves similarly to ListAvailableBenchmarks* (see this). <tr><td>HOST size</td><td class="alias">HOST</td><td></td><td></td></tr> </table> -<h4 class="msgName">ListAvailableBranchesReply</h4> +<h4 class="msgName">ListBranchesReply</h4> <table border=0> <thead><tr><th>No. of bytes</th><th>Type</th><th>Value</th><th>Description</th></tr></thead> <tr><td>1</td><td>U8</td><td>31</td><td>message type</td></tr> diff --git a/doc/notes.txt b/doc/notes.txt index 2611699..c2c9201 100644 --- a/doc/notes.txt +++ b/doc/notes.txt @@ -18,7 +18,7 @@ repository and a Git branch. ------------ -Definition: SNAPSHOT = -timestamp <secs since epoch> | -sha1 <sha1> +Definition: SNAPSHOT = -timestamp <secs since 1/1 1970> | -sha1 <sha1> A -timestamp value t has the following semantics: @@ -29,13 +29,15 @@ Note: The -sha1 option requires exact matching and is thus less powerful than -t ---------------------------------- - Use case 1: Add a benchmark: ./bmc add -benchmark <test case> <test function> <data tag> ----------------------------------- + Example: + ./bmc add -benchmark tst_QWidget updateOpaque 10x10x1 + +---------------------------------- Use case 2: Add a metric: ./bmc add -metric <metric> @@ -43,97 +45,180 @@ Use case 2: Add a metric: Note: walltime, cputicks, callgrind, and events should be added upon installation. ---------------------------------- - Use case 3: Add a platform: ./bmc add -platform <platform> Notes: - - The platform typically identifies a particular OS/compiler combination. - - The official platforms supported by Qt (linux-g++, solaris-cc-64 etc.) should - be added upon installation. ----------------------------------- + - The platform typically identifies a particular OS/compiler combination. + - The official platforms supported by Qt (linux-g++, solaris-cc-64 etc.) should + be added upon installation. + +---------------------------------- Use case 4: Add a host: ./bmc add -host <host> [<details>] - Note: The host identifies a physical machine. + Note: The host identifies a physical machine. ----------------------------------- + Example: + ./bmc add -host barbarella + +---------------------------------- Use case 5: Add a branch: ./bmc add -branch <git repo> <git branch> Note: qt/master, qt/45, qt/46, qt/... should be added upon installation. ----------------------------------- + Example: + ./bmc add -branch jasplins-qt master + +---------------------------------- Use case 6: Add new results: ./bmc add -results <xml file> <platform> <host> <product git repo> [<product git dir>] Notes: - - <product git dir> must refer to a directory accessible from the local - file system and the git command needs to be installed (i.e. the program - needs to run git commands from <product git dir>). - By default, <product git dir> is the current directory. - - The new results are assumed to have been generated using a product - built from a clean checkout of the current head commit of the current git branch - in <product git dir>. - - The sha1 to be stored in the database is assumed to be the current head commit in - <product git dir> (extracted running 'git log -1 HEAD --pretty=format:"%H"'). - - The timestamp to be stored in the database is the Unix timestamp of the committer - date (extracted running 'git log -1 HEAD --pretty=format:"%ct"'). - - <product git repo> must be passed explicitly since it cannot in general - be derived from <product git dir> (e.g. in the case of a local, standalone - repository). - ----------------------- - -Use case 1: Compare results for a given branch since a given snapshot: - - ./bmc compare -history BMCONTEXT -branch <branch> SNAPSHOT + + - <xml file> must (essentially) be of the same format as the one being dumped + by QTestLib. + + - <product git dir> must refer to a directory accessible from the local + file system and the git command needs to be installed (i.e. the program + needs to run git commands from <product git dir>). + By default, <product git dir> is the current directory. + + - The new results are assumed to have been generated using a product + built from a clean checkout of the current head commit of the current git branch + in <product git dir>. + + - The sha1 to be stored in the database is assumed to be the current head commit in + <product git dir> (extracted running 'git log -1 HEAD --pretty=format:"%H"'). + + - The timestamp to be stored in the database is the Unix timestamp of the committer + date (extracted running 'git log -1 HEAD --pretty=format:"%ct"'). + + - <product git repo> must be passed explicitly since it cannot in general + be derived from <product git dir> (e.g. in the case of a local, standalone + repository). Example: - ./bmc compare -history -testcase tst_QWidget -testfunc updateOpaque \ - -datatag 10x10x1 -metric walltime -platform linux-g++ -host barbarella \ - -branch qt master -timestamp first + ./bmc add -results results.xml linux-g++ barbarella qt master $QTDIR + +---------------------------------- +Use case 7: Compare results as a history of a given branch since a given snapshot: + + ./bmc compare -history BMCONTEXT <branch> SNAPSHOT + + Example 1: + + ./bmc compare -history tst_QWidget updateOpaque 10x10x1 walltime \ + linux-g++ barbarella qt master -timestamp first + + Example 2 (1250146917 corresponds to the date Thu Aug 13 09:01:57 2009 +0200): + ./bmc compare -history tst_QWidget updateOpaque 10x10x1 walltime \ + linux-g++ barbarella qt master -timestamp 1250146917 -Use case 2: Compare the results of two branches: + Example 3: + + ./bmc compare -history tst_QWidget updateOpaque 10x10x1 walltime \ + linux-g++ barbarella qt master -sha1 3bad2f608813968d85c2ea51c1bd75dca7f7e22a + +---------------------------------- +Use case 8: Compare the results of two branches: ./bmc compare -branches BMCONTEXT <branch1> SNAPSHOT1 <branch2> SNAPSHOT2 -Use case 3: Compare a result of a given branch directly to an unregistered - result (i.e. one not yet in the DB): + Example: + + ./bmc compare -branches tst_QWidget updateOpaque 10x10x1 walltime \ + linux-g++ barbarella qt 45 -timestamp last qt 46 -timestamp last + +---------------------------------- +Use case 9: Compare a result of a given branch directly to an unregistered + result (i.e. one not yet in the database): ./bmc compare -direct BMCONTEXT <branch> SNAPSHOT (<value> <iterations>) | <xml file> Note: If an xml file is specified, it is assumed to contain a - value/iterations pair that matches the benchmark and metric in the - BMCONTEXT. + value/iterations pair that matches the benchmark and metric in the + BMCONTEXT. + + Example 1: + ./bmc compare -direct tst_QWidget updateOpaque 10x10x1 walltime \ + linux-g++ barbarella qt master -timestamp last 40 1024 + + Example 2: + + ./bmc compare -direct tst_QWidget updateOpaque 10x10x1 walltime \ + linux-g++ barbarella qt master -timestamp last results.xml ---------------------------------- +Use case 10: List available snapshots for a given branch: + + ./bmc list -snapshots BMCONTEXT <branch> -Use case 5: List available benchmarks (subject to optional restrictions): + Example: + + ./bmc list -snapshots tst_QWidget updateOpaque 10x10x1 walltime \ + linux-g++ barbarella qt master + +---------------------------------- +Use case 11: List available benchmarks (subject to optional restrictions): ./bmc list -benchmarks [-metric <metric>] [-platform <platform>] [-host <host>] \ [-branch <branch>] -Ditto for other entities: + Example 1: + + ./bmc list -benchmarks + + Example 2: + + ./bmc list -benchmarks -platform linux-g++ + + Example 3: + + ./bmc list -benchmarks -platform linux-g++ -host barbarella -metric walltime \ + -branch qt master + +---------------------------------- +Use case 12: List available metrics (subject to optional restrictions): + + ./bmc list -metrics [-benchmark <benchmark>] [-platform <platform>] [-host <host>] \ + [-branch <branch>] + + Example 1: + + ./bmc list -metrics + + Example 2: - ./bmc list -metrics ... - ./bmc list -platforms ... - ./bmc list -hosts ... - ./bmc list -branches ... + ./bmc list -metrics -benchmark tst_QWidget updateOpaque 10x10x1 \ + -platform linux-g++ -host barbarella -branch qt master ---------------------------------- +Use case 13: List available platforms (subject to optional restrictions): -Use case 6: List available snapshots for a given branch: + ./bmc list -platforms [-benchmark <benchmark>] [-metric <metric>] [-host <host>] \ + [-branch <branch>] - ./bmc list -snapshots BMCONTEXT <branch> +---------------------------------- +Use case 14: List available hosts (subject to optional restrictions): + + ./bmc list -hosts [-benchmark <benchmark>] [-metric <metric>] [-platform <platform>] \ + [-branch <branch>] + +---------------------------------- +Use case 15: List available branches (subject to optional restrictions): + + ./bmc list -platforms [-benchmark <benchmark>] [-metric <metric>] [-platform <platform>] \ + [-host <host>] |