aboutsummaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
...
* Simple creation tests for a range of QtQuick typesRobin Burchell2017-04-0616-0/+318
| | | | | | | | | | | The font-using ones are a bit of a pain in that we have to try to pick a font that will likely always be there, the rest are quite straightforward. View tests are not at all comprehensive, and may deserve a set of dedicated tests of their own (with actual delegates, and so on). Change-Id: I6485e4cf62a3644519910786c32d91a490b657b4 Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Add two additional tests covering the boxing of a more complex typeRobin Burchell2017-04-062-0/+29
| | | | | | | | Basic values (like bool etc) are already covered in qml.basictypes, but we should also cover storage of more complex things like sizes. Change-Id: I81a468f1e88a92d336ef622c43d7f9745bd8add4 Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Add tests covering variant as well as varRobin Burchell2017-04-065-0/+70
| | | | | | | They have separate implementations, so we should cover them both. Change-Id: I3d3e8d950984ae88590ee18b3de1b98cb8c08860 Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Add some tests covering basic QML & Quick typesRobin Burchell2017-04-0424-0/+312
| | | | | | | | | Not full coverage no doubt, but some is better than none, and we already have some interesting information in that we can see relative cost between types. Change-Id: Iea0d1f4b8cbb2afc6ba87586ebd70ba1b262c22b Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* quick.text: Add benchmarks for decorated textRobin Burchell2017-04-044-0/+68
| | | | | Change-Id: I1ce0a72d8efced7eeb062b37ffd04d3bf09949ac Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* quick.text: Add hindi and arabic text benchmarksRobin Burchell2017-04-044-0/+67
| | | | | | | | Shows the impact of shaping, which the existing benchmarks gloss over. Text is of comparable length/complexity to the existing text benchmarks. Change-Id: Ia9fd2fdb8923fe8ae687c6e94bc7e3c3b4a0f033 Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Don't pass benchmark to BM runnerRobin Burchell2017-04-043-10/+14
| | | | | | | | It can work it out itself. This will make subsequent timing changes cleaner. Change-Id: Iff7a6abc080952180d51ec39576688b203be1fcc Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Skip manual tests unless they are explicitly runRobin Burchell2017-03-301-0/+5
| | | | | | | These are not useful for automation purposes Change-Id: I5948814c9c73aa7d21216c3081a7bbf7bcc3b254 Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* js: Port to use benchmark helpersRobin Burchell2017-03-302-36/+12
| | | | | Change-Id: I8b550e55221381eef0f508c1c2d3478e107ffab3 Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* creation: Port remaining benchmarks to use benchmark helpersRobin Burchell2017-03-305-109/+47
| | | | | Change-Id: I24ba380295eeb576d61ae28cc9db914b4522f39d Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Validators: Port to benchmark helpersRobin Burchell2017-03-303-61/+24
| | | | | Change-Id: I5eae1021f022e6cab6790c29abf80cbeecfdec4a Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Layouts: Port to using benchmark helpersRobin Burchell2017-03-307-302/+218
| | | | | Change-Id: Iabefd1817dd97485dad79f8cef4294fd4ca628c5 Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Text: Port benchmarks to use helpersRobin Burchell2017-03-3010-220/+98
| | | | | Change-Id: I3786c613c096756f2da59a12b91448f25d670b6c Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Canvas: Update benchmarks to use helpersRobin Burchell2017-03-307-185/+72
| | | | | | | | | Also remove delegates_canvas_image_immediate_with_context.qml -- this is not the right place to test performance of this, let's limit ourselves to creation for now. Change-Id: Ic59dfdff7658130525fcd7d886b61897563a6d01 Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Replace delegates_script with delegates_item_scriptRobin Burchell2017-03-273-22/+20
| | | | | | | | | | | | delegates_script was a little flawed in that it set color as a prop, and then set it again in the script handler. Simplify this by using item (not rect) to bypass rendering. This also tests a lot less stuff, and is now directly comparable with delegates_item, which is a nice bonus. Change-Id: I09a45e6dc1df2ea4cd116060a52a631bbfb9e92e Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Port more benchmarks to use built-in helpers.Robin Burchell2017-03-276-75/+51
| | | | | Change-Id: If185a15df4cb8aca312e8e16ea9cf6dd77ea39a0 Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Add a flag to mark benchmarks as having a "boolean" resultRobin Burchell2017-03-243-11/+27
| | | | | | | | | | | | | Some of our benchmarks don't have a meaningful performance indicator (in that they won't ever really improve or regress on the percentage figure), but are still of interest in that they let us know if a particular feature is totally broken or not. This flag lets us easily mark such tests for ease of discovery when looking into results / implementation of benchmarks. Change-Id: Icda0981ba32d9abe84a835c0ca6ccb8a01be374d Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Add a base Benchmark and CreationBenchmark typeRobin Burchell2017-03-247-45/+125
| | | | | | | | These will allow us to make benchmarks more succinct without adding too much complexity. Change-Id: I5b38dfc388d1d56780eb107d36060d3e1c23cdea Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Move host process away from using QGuiApplicationRobin Burchell2017-03-242-9/+31
| | | | | | | | | This makes it more robust from window system or OpenGL issues, and mean that it may stand a chance of actually reporting failures if a benchmark locks up due to a serious problem in either of those. Change-Id: I6d7372a9161f0f3565822f6990d621af34b41425 Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Move result recorder stuff to result recorderRobin Burchell2017-03-243-10/+7
| | | | | | | | | | | And at the same time, make subprocesses capable of overriding JSON output from the host. This is the last remaining step before removing QGuiApplication from the host. Change-Id: I1e8f9798e691b85441295619ca2d2c2b2238dc9c Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Split host and process mode out of mainRobin Burchell2017-03-241-90/+105
| | | | | | | | This is mostly a tidyup, but it will also be followed by making host mode use QCoreApplication to be resilient to GPU/window system issues. Change-Id: I098035b9bfdd12668b7c1e87bb83453bbba352b0 Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Compile with Qt 5.6Eskil Abrahamsen Blomfeldt2017-03-241-0/+1
| | | | | Change-Id: Ida278c76c1c268f01d5b070033f9c61f8ea2526c Reviewed-by: Robin Burchell <robin.burchell@crimson.no>
* Remove coverage for QQC1Robin Burchell2017-03-232-52/+0
| | | | | | | | | These are too slow to provide much meaningful data other than "don't use QQC1", and they take valuable runtime away from more useful benchmarks. Change-Id: I08504a8ad70aaace557dee7a17b63d325d9840f8 Reviewed-by: Gunnar Sletta <gunnar@crimson.no> Reviewed-by: Eskil Abrahamsen Blomfeldt <eskil.abrahamsen-blomfeldt@qt.io>
* Add canvas creation benchmarks.Robin Burchell2017-03-237-0/+230
| | | | | | | | | | No rendering, just testing creation (and one test of fetching context). We have multiple tests here because there are multiple configurations, and I expect that as these are exposed, there are more than likely going to be people using them. Change-Id: Id20ebb07bb6039bdd20cb3f4ddd45c796a561c3f Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Split BenchmarkRunner out into a separate fileRobin Burchell2017-03-235-177/+259
| | | | | | | | No functional changes. Change-Id: I2317331b28620320b5fa6946442341f8f442108d Reviewed-by: Eskil Abrahamsen Blomfeldt <eskil.abrahamsen-blomfeldt@qt.io> Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Split ResultRecorder out to a separate fileRobin Burchell2017-03-234-156/+250
| | | | | | | | Cleanup only, no functional changes. Change-Id: Ie9f6cfea744300b47551776ea622726a73727278 Reviewed-by: Eskil Abrahamsen Blomfeldt <eskil.abrahamsen-blomfeldt@qt.io> Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Clean up BenchmarkRunner a bitRobin Burchell2017-03-233-55/+73
| | | | | | | | | | | | | After the process split, BenchmarkRunner no longer runs multiple benchmarks, so a lot of the code here was dead - so we can remove it. While we're at it, move the benchmarks to Options - as that's where they really belong. This removes the final tie from BenchmarkRunner to the host mode process, which makes things a little tidier still. Change-Id: I180fd61d1eb7267d68f06642d78383d8c0afc55c Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* BenchmarkRunner: Merge redundant methodsRobin Burchell2017-03-231-6/+0
| | | | | | | No need to have two methods we run on completion Change-Id: Iaeb3f1ec9c45e395c973ebfb205448762695f494 Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Benchmark: Remove redundant variableRobin Burchell2017-03-231-3/+0
| | | | | Change-Id: Iedc7b9a35f574734480bcb4edbcda520bbb9a042 Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Add benchmarks for validator type creationRobin Burchell2017-03-233-0/+82
| | | | | | | | | | As expected, RegExpValidator is the slow one of the bunch (around 70% slower than the others). Surprisingly, it looks like IntValidator might be a tiny bit faster than DoubleValidator, but it's within margin of error anyway. Change-Id: I1a6ce9cf8eca1fc75d4122cc0ba6db5ad2e582c7 Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Alter standard deviation calculationRobin Burchell2017-03-221-1/+4
| | | | | | | | | | | | | | | | | | | | | | | | As with many things mathematical, there is more than one way to skin a cat. In the case of standard deviation, there is "uncorrected sample standard deviation" (or "population standard deviation"), and there is "corrected sample standard deviation". There's probably more, but the former is what we're using now, and the latter is what this change introduces. Quoting Erik: the "issue" is that the sqrt biases toward small numbers because it's a non-linear function. so the uncorrected one will give smaller x_i - avg a higher "weight" than the outliers where x_i-avg is bigger. Note: This will cause an increase in CoV values (since they are based off of stddev). This is probably acceptable due to the increased "precision" gained though. Change-Id: I0e9c8bd245de47fe7099baefc2966b744ec78028 Reported-by: Erik Verbruggen <erik.verbruggen@qt.io> Reviewed-by: Gunnar Sletta <gunnar@crimson.no> Reviewed-by: Erik Verbruggen <erik.verbruggen@qt.io>
* Add some edge case handlingRobin Burchell2017-03-221-4/+20
| | | | | | | | | | | | | | | | | | | First and most importantly, don't let a benchmark run indefinitely. At least on TX1, it seems that things can occasionally lock up (hard). After running fine for ~1.5 days, my TX1 "hung" on a benchmark, seemingly for GPU related reasons (at least, so I am informed in dmesg). To handle this, we wait only so long before giving up, logging the failure, and not attempting to start any other tests. We do this as a precaution, as if we run into this condition, it likely means either that someone is developing, or the harwdare needs a reset. Such hardware reset is left to the harness running qmlbench. In addition, we also log crashes a bit more noisily, on the offchance that they are interesting to know about. Change-Id: Id8d9eb50a29eb295669e743169a8d3e69c737776 Reviewed-by: Eskil Abrahamsen Blomfeldt <eskil.abrahamsen-blomfeldt@qt.io>
* changing_text_oneofmany: Simplify benchmarkRobin Burchell2017-03-222-18/+9
| | | | | | | | | | | | | | The original purpose of this benchmark was a little muddled. It seems like it was being used to test isolation clip (but we already have that through changing_over_isolated{*}) and changing a single piece of text. We can do this simpler, by stealing most of changing_texts' implementation but changing the text binding. This way, we gain a boolean test that tells us whether or not changing a simple text item is "cheap" or not. Change-Id: Ifd878b43e0ee6ee808784dddd12e30a8863191d8 Reviewed-by: Eskil Abrahamsen Blomfeldt <eskil.abrahamsen-blomfeldt@qt.io>
* "Get rid" of the changing_textcanvas benchmarkRobin Burchell2017-03-222-34/+28
| | | | | | | | | | | | | | | | | The original benchmark wasn't all that useful. It created a canvas for every piece of text, which meant that it wasn't really a fair comparison of methods against changing_texts. At the same time, such a comparison isn't really useful anyway: Text is always going to win. So, at the same time as we simplify that to make it *just* about text rendering, move it to a new dedicated "canvas" subfolder and rename it to make it clear that it's just benchmarking text rendering. In the future, we will likely want to add more canvas tests. Change-Id: I63ff1bd53357e0ae397afe52cfb8381e5ae2456c Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Alter benchmarks comparing anchor vs size bindingsRobin Burchell2017-03-212-4/+9
| | | | | | | | Remove the use of Rectangle, it doesn't make much sense since the anchors are what we are interested in here. Change-Id: I4baecc6296d665e2ec11559dd24c22ccbf27556f Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Remove two composite "examples"Robin Burchell2017-03-212-76/+0
| | | | | | | | | These aren't really benchmarks of anything concrete, so having them does not seem useful. Separate tests for e.g. width+height vs anchors bindings will be added instead. Change-Id: I6d5225d421529ade6c44e5c76e03b856ef2b4bd7 Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Halve the number of ops for the longtext benchmarksRobin Burchell2017-03-212-2/+2
| | | | | | | | | | These numbers are way too low right now, which leads to noise on variation. Decrease the total work to be done to get higher (less noisy) numbers. Change-Id: I05061a78c5689cb79b752f1cc38bc2a6fca115d7 Reviewed-by: Eskil Abrahamsen Blomfeldt <eskil.abrahamsen-blomfeldt@qt.io> Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Move average() to a functionRobin Burchell2017-03-211-4/+10
| | | | | | | | Fits well alongside stddev, and lets us test this stuff easier in the future if wanted/needed. Change-Id: I5d57103f94d0ba001826a3116cef9a069d3ad9a9 Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Rip out bucketingRobin Burchell2017-03-212-50/+13
| | | | | | | | | | | This was an attempt at stabilising the results from the "default" shell. However, as we are not using this for other shells, and on the limited known-good HW the default shell is happy with the results are quite stable, this seems to be an unnecessary complication. Change-Id: I683f294d8a698de86ccda09f62bd61276fc09e6a Reviewed-by: Eskil Abrahamsen Blomfeldt <eskil.abrahamsen-blomfeldt@qt.io> Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Compile with Qt 5.7 and belowEskil Abrahamsen Blomfeldt2017-03-171-0/+6
| | | | | | | The setFlags() method was added in Qt 5.8. Change-Id: I1c656e026429ded9e391fd0962d1f7032532d09a Reviewed-by: Robin Burchell <robin.burchell@crimson.no>
* Split main into smaller piecesRobin Burchell2017-03-171-30/+37
| | | | | | | | One for handling command line arguments (easy, now we stuck everything in Options), and one for setting up the surface format if required. Change-Id: I32942db53c08d38b5e504319b95869feb846edde Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Move isSubProcess to the Options structRobin Burchell2017-03-172-6/+8
| | | | | | | Will make way for some tidyups Change-Id: I565a7a40a960930a44f6e9f6085e98736d97fd3f Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Move id to Options structRobin Burchell2017-03-172-1/+3
| | | | | | | Preparing for cleanup of argument handling. Change-Id: I767c34a0e6f8378e6f23300c7f1375bfc6f3474b Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Move onlyPrintJson bool to the Options structRobin Burchell2017-03-172-95/+57
| | | | | Change-Id: I854734628eeaf7bc846e7e0b41a72de7104e4be2 Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Make options a static classRobin Burchell2017-03-172-0/+75
| | | | | | | This will allow us to clean up option parsing a bit. Change-Id: I69756f81a19293e75be103245d18e4178673b2ae Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Split qmlbench into two "process" modesRobin Burchell2017-03-171-7/+96
| | | | | | | | | | | The host mode (what you actually run) no longer runs benchmarks. It exists to proxy output from the subprocess which runs the test itself. This may help to avoid "contamination" of global state (caches and similar) on multiple levels (drivers, Qt/QML/Quick itself, etc). Change-Id: Ifc46f16cb5b02c93e48c691bc99658b9831d19d1 Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Split decidefps out to a separate toolRobin Burchell2017-03-157-111/+153
| | | | | | | | This isn't used constantly, so having it as a separate tool cleans things up a little. Change-Id: I710b57161337af26c018fa42cf8362a100df5cdd Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Rearrange source treeRobin Burchell2017-03-1511-23/+27
| | | | | | | Move qmlbench binary to src/, leaving the top directory clean. Change-Id: If0113d503dba5f567a52dfb94c76fa1b2692d992 Reviewed-by: Gunnar Sletta <gunnar@crimson.no>
* Improve the frame-count shell:Gunnar Sletta2017-03-152-52/+47
| | | | | | | | | - We no longer count in buckets as this shell is inherently more stable. - We count for --framecount-interval [ms] amount of time, defaults to 20. - Removed the label as this is potentially a tiny cost that we don't need. Change-Id: I3696492e55699d79af3ec27150d9b07393354646 Reviewed-by: Robin Burchell <robin.burchell@crimson.no>
* Do the right calculation in decidefpsGunnar Sletta2017-03-151-3/+5
| | | | | Change-Id: Iffe482f43f7e2e8d3ecfe714256903d6ebabcd9e Reviewed-by: Robin Burchell <robin.burchell@crimson.no>
generated by cgit v1.2.3 (git 2.25.1) at 2024-06-19 02:06:19 +0000