summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
...
* | | | | | Add AndCardinalPredicate and OrCardinalPredicatePrudhvi Akhil Alahari2022-11-027-3/+202
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Before this change, IndexSource derived cardinality from predicate which implement HasCardinality. Since AndPredicate and OrPredicate does not return cardinality, IndexSource defaults to 10 when query comprises more than one predicate. Due to this behavior, AndSource chooses IndexSource almost always when query contains more than one index predicate. Add AndCardinalPredicate and OrCardinalPredicate which implement HasCardinality's getCardinality(). AndCardinalPredicate will return minimum of all the cardinalities of its children. OrCardinalPredicate will return sum of all the cardinalities of its children. Enhance index rewritter to return AndCardinalPredicate when there is at least one predicate which implements HasCardinality. Similarly, return OrCardinalPredicate when all the children implement HasCardinality. This helps AndSource to choose the right source more often. Unfortunately there are no operators which are datasources other than Index in the Core as of now. We at Qualcomm have a datasource operator in a plugin which takes a bug number and queries the bug tracker's DB for changes related to that bug number. Suppose this operator (say issue:123) returns around 15 changes. On a 3.4 test site against LUCENE which contains ~20K open changes, ~2.8M merged changes, the performance of, query "issue:123 status:open age:1d" before: 4.5s, 4.4s, 4.4s after: 0.29s, 0.26s, 0.23s query "issue:123 status:merged age:1d" before: 11m, 11m, 11m after: 0.23s, 0.20s, 0.25s Release-Notes: AndSource chooses right source more often Change-Id: Id79bea6f512c7673f3183f39a187debec7ce34f7
* | | | | | Merge "Add HasCardinality interface which helps in defining a cardinality" ↵Prudhvi Akhil Alahari2022-11-0217-18/+133
|\ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | into stable-3.4
| * | | | | | Add HasCardinality interface which helps in defining a cardinalityPrudhvi Akhil Alahari2022-10-3117-18/+133
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Before this change, cardinality of the IndexSource was always set to 10. This makes AndSource to choose IndexSource over other source almost always. Add a HasCardinality interface which can be used by all the predicates which implement IndexPredicate to set a cardinality. This helps AndSource to choose the right datasource. Unfortunately there are no operators which are datasources other than Index in the Core as of now. We at Qualcomm have a datasource operator in a plugin which takes a bug number and queries the bug tracker's DB for changes related to that bug number. Suppose this operator (say issue:123) returns around 15 changes. On a 3.4 test site against LUCENE which contains ~20K open changes, ~2.8M merged changes, the performance of, query "issue:123 status:open" when status:open is datasource: 2.1s, 2.1s, 2.1s when issue:123 is datasource: 0.07s, 0.07s, 0.07s query "issue:123 status:merged" when status:merged is datasource: 6m49s, 6m38s, 6m34s when issue:123 is datasource: 0.07s, 0.07s, 0.07s The cardinality of the predicates are restored from their previous values [1] and [2]. [1] https://gerrit-review.googlesource.com/c/gerrit/+/52078/8/gerrit-server/src/main/java/com/google/gerrit/server/query/change/SqlRewriterImpl.java#b576 [2] https://gerrit-review.googlesource.com/c/gerrit/+/63214 Release-Notes: AndSource chooses right datasource more often Change-Id: I802ab592d121b01dab9557da535f99411a1c7bc0
* | | | | | | Do not set cherryPickOf on RevertSubmissionMarija Savtchouk2022-11-013-4/+8
|/ / / / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The RevertSubmission internally uses CherryPickChange operation. It creates the 'normal' revert for the first change in the relation chain, for each subsequent change it creates a revert commit that is then cherry-picked. This cherry-pick is not a cherry-pick of the reverted change. This is a cherry-pick of the revert commit of the original change. This change fixes the bug in the logic that sets cherryPickOf to the original (reverted) change, using the CherryPickChange operation. Change-Id: I27bb03987a3b0a1fde69facbda6577f58017b94c Release-Notes: skip Google-Bug-Id: b/255828001
* | | | | | Merge "Don't always rewrite And/OrPredicate to And/OrSource" into stable-3.4Prudhvi Akhil Alahari2022-10-299-67/+122
|\ \ \ \ \ \
| * | | | | | Don't always rewrite And/OrPredicate to And/OrSourcePrudhvi Akhil Alahari2022-10-289-67/+122
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Before this change OrPredicate was always rewritten to OrSource. This is not the right thing to do as OrSource#read() will eventually fail when there is at least one non-datasource child. Since change I57b97b280b, AndSource picks a source which has the lowest cardinality which makes the OrSource#read() failures more visible. For example, queries like [1] fails when cardinality of "reviewerin:<> OR project:test" is lesser than cardinality of "status:open OR status:merged". Observed this failure in FakeQueryChangesLatestIndexVersionTest while merging change I57b97b280b to stable-3.5. Rewrite OrPredicate to OrSource only when all the children in the predicate are DataSources. Also move the check in OrSource#read() to OrSource constructor which throws an exception when not all the children are a datasource. Add a test for this case and fix few tests which were creating OrSource without a source. Similarly rewrite AndPredicate to AndSource only when there is at least one child as a datasource. [1](status:open OR status:merged) AND (reviewerin:<> OR project:test) com.google.gerrit.exceptions.StorageException: No ChangeDataSource: reviewerin:bf211bb3774efeabb399a8cfa076abd922cda78a at com.google.gerrit.server.query.change.OrSource.read(OrSource.java:45) Release-Notes: skip Change-Id: Ib5eccbeba2ab150250370082701774ff2ff60c6c
* | | | | | | Merge branch 'stable-3.3' into stable-3.4Luca Milanesio2022-10-281-6/+15
|\ \ \ \ \ \ \ | |/ / / / / / |/| / / / / / | |/ / / / / | | | | | | | | | | | | | | | | | | | | | | | | * stable-3.3: Cache repository locations in LocalDiskRepositoryManager Release-Notes: skip Change-Id: Iea204a20056db01a92b5c0f399be2160df9a0750
| * | | | | Merge branch 'stable-3.2' into stable-3.3Luca Milanesio2022-10-281-6/+15
| |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-3.2: Cache repository locations in LocalDiskRepositoryManager Release-Notes: skip Change-Id: I4052ceea68399279f23a0e180227b71643e9a579
| | * | | | Merge branch 'stable-3.1' into stable-3.2Luca Milanesio2022-10-281-6/+15
| | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-3.1: Cache repository locations in LocalDiskRepositoryManager Release-Notes: skip Change-Id: I5443e5174552f2a8369f5d97162b9cb24cd2ae26
| | | * | | Merge branch 'stable-3.0' into stable-3.1Luca Milanesio2022-10-281-6/+15
| | | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-3.0: Cache repository locations in LocalDiskRepositoryManager Release-Notes: skip Change-Id: I5b6cd9e19c4a0b052da9705e0a8ddb0eced148c2
| | | | * | Merge branch 'stable-2.16' into stable-3.0Luca Milanesio2022-10-281-6/+15
| | | | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-2.16: Cache repository locations in LocalDiskRepositoryManager Release-Notes: skip Change-Id: I39713079d081e7b80f59dc36e6181f4daa565c50
| | | | | * Cache repository locations in LocalDiskRepositoryManagerKaushik Lingarkar2022-10-281-6/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Obtaining the actual location of a repository using base-path and project name can be slow as it involves some guessing to locate the repository. Cache the locations once they are obtained to avoid repeated work, thereby improving performance when opening repositories. For example, on a site with 20k repositories on NFS, ls-projects with this change takes ~60s and ~90s without. Also, a query which wraps a large (~2k) list of manifest[1] operators will take ~300ms with this change and ~2s without it. [1] https://gerrit.googlesource.com/plugins/manifest Release-Notes: skip Change-Id: I8eab3c813c4ac9433e93c7ace96d38efe332be27
* | | | | | AndSource: consider cardinality while choosing the data sourcePrudhvi Akhil Alahari2022-10-275-14/+126
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Before this change, AndSource considered cost to choose a datasource. Ideally cost should be used to determine which predicate to run #match() against first, so that queries are faster. Cardinality should be used to determine which datasource to be picked so that there are fewer changes to process. Consider cardinality of the datasource to choose the source and use cost when cardinality is same. Add a test for this case. Fix few tests which were creating AndSource without a source. This change improves performance of queries which have more than one data source and one's cardinality is lesser than the other. Unfortunately there are no operators which are datasources other than Index in the Core as of now. We at Qualcomm have a datasource operator in a plugin which takes a bug number and queries the bug tracker's DB for changes related to that bug number. Suppose this operator (say issue:123) returns around 10 changes. On a 3.4 test site against LUCENE which contains ~20K open changes, ~2.8M merged changes, the performance of, query "issue:123 status:open" when status:open is datasource: 2.1s, 2.1s, 2.1s when issue:123 is datasource: 0.07s, 0.07s, 0.07s query "issue:123 status:merged" when status:merged is datasource: 6m49s, 6m38s, 6m34s when issue:123 is datasource: 0.07s, 0.07s, 0.07s Release-Notes: skip Change-Id: I57b97b280b13e23004c3e5617ab4a6d1f9726020
* | | | | | Update git submodulesPaladox none2022-10-241-0/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Update plugins/codemirror-editor from branch 'stable-3.4' to 02b51e308e554a19800c25b288cb4b9e4601b9c7 - Improve setting height of editor Side affect is if you have the developer console open the editor will be much smaller. But this is worth the trade off. Screenshot: https://imgur.com/a/OIwISAp Change-Id: I07d923ee5038db7e0b03dddb7ccd74b019c3bf64 (cherry picked from commit 3af12c5a5e65861830b42bd07933e275c33b9159)
* | | | | | Update git submodulesPaladox none2022-10-241-0/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Update plugins/codemirror-editor from branch 'stable-3.4' to 22f4f149418f5ada6a8649319d4ac3c491574a2a - Improve performance in the editor We stop setting viewportMargin and instead use the default value. We also add support for the inbuilt search support within codemirror. This is because with the removal of viewportMargin you cannot use the browsers search to search for the code. In my view i think this is a good trade off as you'll still be able to search using the codemirror searching support whilst improving performance for large documents. In GWTUI we didn't set viewportMargin. Bug: Issue 14845 Change-Id: I84f12168c4784cf40ca102f025b7805b44bc9d9a (cherry picked from commit de723fb887100a0da85673ba9fcd7b02ae1364e3)
* | | | | | Merge branch 'stable-3.3' into stable-3.4Luca Milanesio2022-10-214-15/+90
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-3.3: Update jgit to c6b0ee04e49c96e0beec4154196c416abcf2bcc9 Limit the number of changes that can be submitted together GitwebServlet: Retrieve git path from FileRepository Make delegate() method public Release-Notes: skip Change-Id: I7a8db6b395792144cfbd0ff8c944f08cb241f69b
| * | | | | Merge branch 'stable-3.2' into stable-3.3Luca Milanesio2022-10-215-16/+92
| |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-3.2: Limit the number of changes that can be submitted together GitwebServlet: Retrieve git path from FileRepository Make delegate() method public Release-Notes: skip Change-Id: I56968d4a2cf9800e78e673042a1cc250879b3fd8
| | * | | | Merge branch 'stable-3.1' into stable-3.2Luca Milanesio2022-10-213-6/+58
| | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-3.1: Limit the number of changes that can be submitted together Release-Notes: skip Change-Id: Iab4d9bbbb44cb4e5e3db084f4350e1079f0e7195
| | | * | | Merge branch 'stable-3.0' into stable-3.1Luca Milanesio2022-10-213-6/+58
| | | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-3.0: Limit the number of changes that can be submitted together Release-Notes: skip Change-Id: I63765e81c80fa3bfb661c11ada5b7013f93f93e1
| | | | * | Merge branch 'stable-2.16' into stable-3.0Luca Milanesio2022-10-213-6/+58
| | | | |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-2.16: Limit the number of changes that can be submitted together Release-Notes: skip Change-Id: I263d636ec38f043ad5f6f8157ea5a57e12e7b145
| | | | | * Limit the number of changes that can be submitted togetherLuca Milanesio2022-10-183-6/+59
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When chaining changes together, the sequence of commits to navigate was previously unbound, causing the potential explosion to millions of changes. The explosion could have also been accidental and caused by the push of a change with a non-existent branch, which would have resulted in the full scan of the repository for changes. Introduce a new Gerrit configuration change.maxSubmittableAtOnce with a safe default of 32767, which would allow any use case that would have also worked before this change. Navigating over 32767 changes up to potentially a huge number of commits would have generated a significant CPU and memory overload and still not resulted in a submittable chain of changes anyway. Release-Notes: Limit the number of changes that can be submitted at once Bug: Issue 16322 Change-Id: Id71aed2341f72708778395359bb6e4d4c270401c
| | * | | | GitwebServlet: Retrieve git path from FileRepositoryLuca Milanesio2022-08-141-9/+32
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Instead of retrieving git base path from repository manager, open the repository and get it directly from its git directory path. Also, redirect the delegate repository wrapper to the underlying implementation, allowing to use GitWeb in combination with the multi-site plugin and the cached-refdb. The advantage of doing it: we don't need to cast GitRepositoryManager to the LocalDiskRepositoryManager, so that other implementations would also work. Release-Notes: allow using GitWeb with multi-site and cached-refdb Change-Id: If9da36214063d73953677473082cd16f8f95163a
| | * | | | Make delegate() method publicFabio Ponciroli2022-08-061-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The delegate() method was initially introduced in I8a862ac852 to allow access to the underlying repository while running GC for sites using multi-site plugin. The method was initially package private since the callers where in the same package. Changing visibility of delegate() method to public to allow plugins to access it. Accessing the wrapped repository would otherwise require hacky workarounds like this Ia3fbdc7e96. Bug: Issue 15997 Release-Notes: Make DelegateRepository#delegate() method public Change-Id: I4c8b4f97f9995a7adeca5b6f763e7913c9e8a5e5
| * | | | | Merge "Update jgit to c6b0ee04e49c96e0beec4154196c416abcf2bcc9" into stable-3.3Luca Milanesio2022-10-211-0/+0
| |\ \ \ \ \
| | * | | | | Update jgit to c6b0ee04e49c96e0beec4154196c416abcf2bcc9Matthias Sohn2022-10-211-0/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | this update to current stable-5.9 contains the following changes: fed0ab9ba Use FileSnapshot without using configs for FileBasedConfig 7828ef349 Revert "RefDirectory.scanRef: Re-use file existence check done in snapshot creation" 994434a1c Add missing @since tags 2e5110eda Add missing @since tag 8924b14d3 Add missing @since tags f8f4357d6 Remove unused import in ApacheSshTest f1547eec0 Update maven plugins fe3071f0e Ignore missing javadoc in test bundles 38db89142 storage: file: De-duplicate File.exists()+File.isFile() f829f5f83 RefDirectory.scanRef: Re-use file existence check done in snapshot creation 5606a5315 FileSnapshot: Lazy load file store attributes cache 09c923073 Update eclipse-jarsigner-plugin to 1.3.2 c213a6628 Fix p2 repository URLs 122237439 FS: debug logging only if system config file cannot be found 6aa29d116 Better git system config finding 4f8d43462 Fix target platforms d160e8a93 Fix missing peel-part in lsRefsV2 for loose annotated tags 5f8c48413 reftable: drop code for truncated reads b4782d74f reftable: pass on invalid object ID in conversion 9c3190ce7 Update eclipse-jarsigner-plugin to 1.3.2 283c23012 Fix running benchmarks from bazel c70c0acb4 Update eclipse-jarsigner-plugin to 1.3.2 d160e8a93 should fix Issue 14861 Bug: Issue 14861 Release-Notes: Update jgit to c6b0ee04e49c96e0beec4154196c416abcf2bcc9 Change-Id: Ia221acf855dfc95816def73d77af212888d700a9
* | | | | | | Merge "config-mail.txt: note about necessary restart" into stable-3.4Sven Selberg2022-10-201-0/+5
|\ \ \ \ \ \ \
| * | | | | | | config-mail.txt: note about necessary restartSven Selberg2022-10-201-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Clarify that a restart is needed before changes to email templates take effect. Release-Notes: skip Change-Id: I67ab1c59c786483e832259135ad983375fc5aa12
* | | | | | | | Reintroduce the Change-Id footer in change screenLuca Milanesio2022-10-193-0/+176
|/ / / / / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The change I971bc62bacc61 has removed the Change-Id footer of the old-style change summary, without introducing any alternative way of displaying the Change-Id value when a project does not require Change-Ids in commit messages of new changes. The removal of the Change-Id has made the change workflow unusable when a project does not require to be generated on the client side. The push of new changes without a client-side generated Change-Id is still supported but deprecated. However, it should still be supported until is completely removed. Bug: Issue 16182 Release-Notes: display the change's Change-Id underneath the commit message when not present in the commit message Change-Id: I4e7d9d35c956cb82d812ec93176278276af5ae34
* | | | | | | Fix HTTP 404 when browsing tags on GitwebLuca Milanesio2022-10-121-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The Gitweb links to tags have been broken since Gerrit v2.14.10 following the introduction of tag links with If431ad869. Gitweb supports the 'a=tags' rendering action for annotated tags only, returning 404 for all lightweight tags (see [1]). Use the generic 'a=shortlog' for Gitweb links to tags which would work for both annotated and lightweight ones. [1] https://git-scm.com/docs/gitweb#Documentation/gitweb.txt-tag Release-Notes: Fix broken links to Gitweb lightweight tags Bug: Issue 10168 Change-Id: Ibe57e774fea47ff0b00e8c9cdfa9842e7d59caa3
* | | | | | | fixup! Add an option to periodically warm the project_list cachePrudhvi Akhil Alahari2022-10-103-1/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Update project_list cache warmer to force a refresh instead of waiting till refreshAfterWrite duration to reload the cache. This allows us to run warmer less frequently. Release-Notes: skip Change-Id: I8d2af4e8b7783ca5a85650795edb08625564c2dc
* | | | | | | Add an option to periodically warm the project_list cachePrudhvi Akhil Alahari2022-10-073-1/+123
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The project_list cache is used while loading the change detail page. On a Gerrit site with ~20k projects on NFS, the project_list cache takes ~33 seconds to load. When there is no activity on the site, entries in the project_list cache are evicted after maxAge duration (possibly used to support an HA configuration). From when this happens until the cache is loaded, the cost of loading the cache is borne by all the clients who access a change detail page. Reduce the likelihood of this happening by providing an option to periodically warm the project_list cache. When this warmer is enabled, change detail page loads are consistently faster. Release-Notes: The project_list cache can be periodically warmed Release-Notes: The project_list cache warmer is enabled by default when `cache.project_list.maxAge` is set Change-Id: I2dd8eeabee5e808b584c05227fc4055555098c91
* | | | | | | Merge "prolog_rules cache: only use memoryLimit from cache.projects" into ↵Martin Fick2022-10-065-7/+36
|\ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | stable-3.4
| * | | | | | | prolog_rules cache: only use memoryLimit from cache.projectsPrudhvi Akhil Alahari2022-10-065-7/+36
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Before this change, prolog_rules was pointing to all cache.projects settings. As prolog_rules cache is not a LoadingCache and Gerrit fails to start [1] if we've set `cache.projects.refreshAfterWrite`. The prolog_rules cache never meant to copy all the settings from `cache.projects` except for `memoryLimit`. Fix this issue by using `cache.projects.memoryLimit` only when `cache.prolog_rules.memoryLimit` is not available. [1] [Guice/ErrorInCustomProvider]: IllegalStateException: refreshAfterWrite requires a LoadingCache at CacheModule.bindCache(CacheModule.java:188) \_ installed by: GerritGlobalModule -> PrologModule -> RulesCache$RulesCacheModule at RulesCache.<init>(RulesCache.java:107) \_ for 5th parameter machineCache at GerritGlobalModule.configure(GerritGlobalModule.java:248) while locating RulesCache Release-Notes: Settings in cache.prolog_rules now work if available Change-Id: If74ed1db64dd220db77995a186912a35b1ca73f0
* | | | | | | | Set version to 3.4.7-SNAPSHOTLuca Milanesio2022-10-055-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Release-Notes: skip Change-Id: Iea0604ed487b8f69afd4fbd745cb85929ceadb20
* | | | | | | | Set version to 3.4.6v3.4.6Luca Milanesio2022-10-055-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Release-Notes: skip Change-Id: Ide84b245fa1c85276bb02f0dfcd8bc6f1a6b1189
* | | | | | | | Fix javadoc summary for Google Java StyleNasser Grainawi2022-10-051-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The summary shouldn't use @return and should use "Returns". [1] https://google.github.io/styleguide/javaguide.html#s7.2-summary-fragment Change-Id: I631f0be5c048c2f93d62993c41583fb382d714ac Release-Notes: skip
* | | | | | | | Merge "Don't depend on predicate order as they get sorted based on cost" ↵Nasser Grainawi2022-10-051-8/+11
|\ \ \ \ \ \ \ \ | |/ / / / / / / |/| | | | | | | | | | | | | | | into stable-3.4
| * | | | | | | Don't depend on predicate order as they get sorted based on costPrudhvi Akhil Alahari2022-10-051-8/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ChangeIndexRewriterTest are dependent on predicate order for assertion. As the predicates are sorted based on cost, the order will not be same. Release-Notes: skip Change-Id: I4ac7b66132eed3151a0128117e049e869454ae81
* | | | | | | | Fix DefaultMemoryCacheFactory to correctly set refreshAfterWritePrudhvi Akhil Alahari2022-10-051-1/+1
|/ / / / / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change Ibc188bf663f49, unintentionally sets refreshAfterWrite value to expireAfterAccess. Release-Notes: Setting refreshAfterWrite in gerrit.config now works and does not mistakenly set expireAfterAccess Change-Id: I2d0da340805c74d1acde09c97b0c97550b6c606b
* | | | | | | Merge "Shortcut CreateRefControl#checkCreateCommit only for push processing" ↵Luca Milanesio2022-10-055-12/+130
|\ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | into stable-3.4
| * | | | | | | Shortcut CreateRefControl#checkCreateCommit only for push processingPrudhvi Akhil Alahari2022-10-045-12/+130
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The shortcut is valid when processing pushes, as the connectivity check in JGit (class ConnectivityChecker) ensures that the client can only push objects that point to objects they have access to. The CreateBranch REST API does not do a connectivity check, so we have to verify visiblity of the new commit even if the user has UPDATE access. Add a test. This fixes a data leak, where users with both UPDATE and CREATE_REF permission could obtain read access to commits by creating a branch pointing at the secret commit. Bug: Issue 16054 Release-Notes: fixes confidentiality leak in create branch API Change-Id: Icc16c2dc4d1356b9538fdc7f98b4265e32cd5e8a Co-authored-by: Han-Wen Nienhuys <hanwen@google.com>
* | | | | | | | Paginate internal change index queriesKaushik Lingarkar2022-10-044-104/+189
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Internal index queries by default have a limit of Integer.MAX_VALUE. Performance of queries against supported index backends such as Lucene and Elasticsearch is known to degrade with a high limit. Paginate these queries instead to improve performance. Release-Notes: skip Change-Id: I2f5a6aad3798365a407bdcd050c218d4aa160b0e
* | | | | | | | Fix index pagination to set 'more_changes' correctlyKaushik Lingarkar2022-10-041-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The logic which determines if results must be paginated to meet the desired query limit does not handle the case where exactly one change is not visible in the first page. Fix it to always attempt to get one result more than the limit, so that 'more_changes' is set appropriately. Release-Notes: skip Change-Id: I27751facecaabed77e0dbbe0ab6d82e70598054b
* | | | | | | | Fix Lucene change index search-after implementationKaushik Lingarkar2022-10-044-15/+77
|/ / / / / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change fixes queries (such as [1]) which search more than one change sub-index with search-after pagination type. The current search-after implementation may skip results for such queries. This can occur when we end-up with more results than the desired page-size and then trim the results down to page-size when merging results from the sub indices. Instead, keep track of the remaining results needed after each sub-index search so that, they don't exceed the page-size. [1] gerrit query project:foo --no-limit Release-Notes: skip Change-Id: I86f8bf5e475feac57c37c19e2e813397b2efcba7
* | | | | | | Elasticsearch tests: Recreate container for each test suiteNasser Grainawi2022-10-046-20/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Since Change If3f8d914d5f there are multiple index configs executed in different config suites in the tests and that means the ElasticContainer will be stopped after each suite completes. Handle that by always recreating the container at the start of a suite. Also fix a couple places that were using the default index config instead of the config from the ConfigSuite. Release-Notes: skip Bug: Issue 16300 Change-Id: I39926a3dc65a2313fbcfc566f6b2b72238c6fff6
* | | | | | | Allow different data reservoirs for metricsLuca Milanesio2022-10-0311-6/+393
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The default reservoir for collecting data with DropWizard may not be suitable for all types of metrics collection. The default ExponentiallyDecayingReservoir is typically suitable for data with a lots of values per unit whilst shows inaccurancy for smaller data sets spread over a longer period of time. Introduce a new DropWizardReservoirProvider which reads the reservoir type from gerrit.config in a new section called [metrics]. The reservoir can be subsequently configured in the subsection [metrics "<reservoir>"] and its parameters tailored to the Gerrit admin's metrics collection system. For example, when using Prometheus scraper on a 1 mins interval basis, it is possible to swap the default reservoir with a more suitable SlidingTimeWindow: [metrics] reservoir = SlidingTimeWindow [metrics "SlidingTimeWindow"] window = 1 min Release-Notes: introduce metrics configuration for different data reservoirs Change-Id: Iaf73bc5c1a73423d06f423ea813df2d96e151f9e
* | | | | | | Update git submodulesPrudhvi Akhil Alahari2022-10-021-0/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Update plugins/gitiles from branch 'stable-3.4' to 6338aee4179d932b06bcc7fda37dd8a9e0c016bb - Don't depend on update ACL to create a branch Change-Id: I8067d5885c92798fde6af289e49f7b56bfe05ae0
* | | | | | | Merge "Fix AndSource to run #match() of predicate whose cost is least" into ↵Martin Fick2022-09-225-31/+87
|\ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | stable-3.4
| * | | | | | | Fix AndSource to run #match() of predicate whose cost is leastPrudhvi Akhil Alahari2022-09-225-31/+87
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change I0785952bf2cee, unintentionally introduced a bug where AndSource no longer picks the predicate based on cost to run #match(). Fix this issue by sorting the predicates based on cost & cardinality. Add a test for this scenario. Also fix a test which depends on predicate order. With a 3.4 test site which contains ~20k open changes, the performance of query [1] where reviewerin is costlier than destination (3 samples): before: 336s, 366s, 402s after: 4s, 4s, 4s [1] status:open reviewerin:ldap/example AND destination:name=foo With this change, queries containing predicates with poorly defined costs or cardinalities could perform worse now. Release-Notes: queries with different and accurate predicate costs are always performant despite ordering Change-Id: Ib2cb25329fddf9ff931a168fa8b5b9eff8d783c7
* | | | | | | | Merge "Bump JGit to latest stable-5.13 (035e0e23f)" into stable-3.4Nasser Grainawi2022-09-221-0/+0
|\ \ \ \ \ \ \ \