summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* ReplicationQueue: Check nulls in firePendingEventsv2.15.22v2.15.19upstream/stable-2.15Marco Miller2020-02-211-2/+6
| | | | | | | | | After a sudden reboot (for unknown reason) Gerrit at startup couldn't load because of NullPointerException. There is a possibility that stored event was null at that point. Extra logging added to handle null events. Change-Id: I72f34d8def6e0246196cd865f33f6e795b21664b
* Log stack trace when an error occur while deleting eventHugo Arès2020-02-211-1/+1
| | | | | | | This will help figuring out root cause of failure to delete event file. Change-Id: I2f9774c3daf19a04f6b04414ba8145c99bb6e0fe (cherry picked from commit b62f006b1350180de0af02c82fb18fb290a2548f)
* Append LF to the json string of persisted replication eventSaša Živkov2020-02-211-1/+1
| | | | | Change-Id: I83ed3f37071125018bf23f6dcd137ef819ef3559 (cherry picked from commit 5e91925cfd391898e8e33fd149b9e1a115dafee4)
* Change default for the replicateOnStartup to falseSaša Živkov2020-02-212-2/+2
| | | | | | | | | | Now that replication events are persistent and non-finished replications rescheduled after plugin restart the replicateOnStartup feature becomes less important. We can change the default value for this option to false. Change-Id: I237d8d8514e01b8786b7db9f39bead4eb475a0a4 (cherry picked from commit 807790f7d4058235a19b2a766e84628168b64ae6)
* Don't lose ref-updated events on plugin restartSaša Živkov2020-02-1411-13/+221
| | | | | | | | | | | | | | | | | | | | When a ref-updated event is received, persist the event in the directory defined by the replication.eventsDirectory. When the updated ref is replicated deleted the persisted event. If replication queue is non-empty and plugin gets stopped, ref updates will not be replicated and, therefore, the persisted events will not get deleted. When the plugin starts it will schedule replication for all persisted events and delete them. This change provides two benefits: * no ref-updated events are lost on plugin restart * eliminate need for the replicateOnStartup=true setting which schedules replication of all refs for all projects and typically creates a humongous replication queue on every plugin restart. Change-Id: Ieacd084fabe703333241ffda11c8b6c78cced37a (cherry picked from commit bdaea910694dd5a3474dbc051b298aaee9d77950)
* Fix issue with dropping events on startv2.15.18Marcin Czech2019-09-242-1/+155
| | | | | | | | | | | | | During the configuration reload ReplicationQueue dropped reference updated events. Because replication scheduling is not created and events are not preserved replication for these events never happened. Solution is to add in memory queue which holds events until plugin is up and running and then fire those events to make sure that all replication happened. Bug: Issue 11573 Change-Id: I3f9767c3879152acc8ff4ab73233abb278fb07a7
* Merge branch 'stable-2.14' into stable-2.15v2.15.17David Pursehouse2019-09-041-0/+1
|\ | | | | | | | | | | | | * stable-2.14: Bazel: Add fixes for --incompatible_load_java_rules_from_bzl Change-Id: I73e2b89362f0e29e2e61547e33c6c36778758168
| * Bazel: Add fixes for --incompatible_load_java_rules_from_bzlv2.14.21upstream/stable-2.14David Ostrovsky2019-07-271-0/+1
| | | | | | | | Change-Id: I822955d5586174e8a0da83c0455b6e321ba44492
* | Fix creation of missing repository when replicating to a Gerrit serverv2.15.16Saša Živkov2019-07-191-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | When replicating to a Gerrit server over http, non-existence of a repository is reported as: <repo-name> unavailable This message wasn't among expected messages and creation of the missing repository wasn't executed. Bug: Issue 11204 Change-Id: I3d6c3e14573b638b17d54d1777a606b8f356f1f9
* | Revert "Remove replication event from pending when runway is allowed"v2.15.15Luca Milanesio2019-07-181-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This reverts commit e780ae61cbbba3f88558a3620065d1fcdc0768c8. Reason for revert: Creates potentially infinite Heap consumption and JVM crash when a PushOp is denied to use the runway. The problem is caused by the late removal of the PushOp from the list of pending operations, which causes the subsequent reschedule to find the same operation and double the number of replication states associated with it. If the loop happens multiple times, the generated consumption grows exponentially causing the JVM to enter a series of continuous stop-the-world GC that eventually lead to complete block of the JVM activity. The method `requestRunway` is thus not telling the truth because it is actually relying on the fact that the PushOne operation is removed also from the pending list, independently from its runway concession status. Change-Id: I6097bc7ad16c8bcc86a7d30af7d2ad331728712d
* | ReplicationFileBasedConfig: Fix setting default sshConnectionTimeoutDavid Pursehouse2019-07-181-4/+11
| | | | | | | | | | | | | | | | | | | | | | | | The default value did not get set when the replication config did not exist, was empty, or was invalid. This is not really a problem since if there is no config, no SSH connections are made anyway. A more serious issue is that the value was read with minutes as the default time unit, but then converted to milliseconds using seconds as the time unit, which resulted in an incorrect value. Change-Id: I64906e29acb56f0f53b432db61d2707dfe1963d3
* | SshHelper: Add class javadocDavid Pursehouse2019-07-171-0/+1
| | | | | | | | Change-Id: I80ab0c2225ad02628b630889c299c474e0087d70
* | Make more classes and fields public to ease extensibilityDariusz Luksza2019-07-176-19/+19
| | | | | | | | | | | | | | | | | | Make classes public, and their constructors protected, to allow them to be extended. Make event type names and fields public. Change-Id: Iba275f99b7afbf87d57dd44851d043ad0f23fbe1
* | Allow to configure timeout for SSH connection and commandsDariusz Luksza2019-07-175-5/+52
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The timeout for SSH command execution is hard-coded to 0 which means no timeout, and the timeout for SSH connections is hard-coded to 2 minutes. Introduce new configuration options to allow configuring each timeout individually: gerrit.sshCommandTimeout gerrit.sshConnectionTimeout When not set, both default to their previously hard-coded values, i.e. no limit and 2 minutes respectively. Change-Id: Ibd377d45543ef4631082e8d521aeeb99209003f7 Signed-off-by: Dariusz Luksza <dluksza@collab.net> Signed-off-by: David Pursehouse <dpursehouse@collab.net>
* | StartCommand: Fix synchronization on non-final fieldDavid Pursehouse2019-07-171-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | When all error prone warnings are enabled the SynchronizeOnNonFinalField bug pattern is reported: plugins/replication/src/main/java/com/googlesource/gerrit/plugins/replication/StartCommand.java:103: error: [SynchronizeOnNonFinalField] Synchronizing on non-final fields is not safe: if the field is ever updated, different threads may end up locking on different objects. synchronized (stdout) { ^ (see https://errorprone.info/bugpattern/SynchronizeOnNonFinalField) Change-Id: Ib2df20aa28af4edd36ce5b9dfcf7d82c409dab84
* | Destination: Suppress FutureReturnValueIgnored warningDavid Pursehouse2019-07-171-3/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When all error prone warnings are enabled the FutureReturnValueIgnored bug pattern is reported, for example: plugins/replication/src/main/java/com/googlesource/gerrit/plugins/replication/Destination.java:367: error: [FutureReturnValueIgnored] Return value of methods returning Future must be checked. Ignoring returned Futures suppresses exceptions thrown from the code that completes the Future. pool.schedule(e, now ? 0 : config.getDelay(), TimeUnit.SECONDS); ^ (see https://errorprone.info/bugpattern/FutureReturnValueIgnored) Did you mean to remove this line? Change-Id: I43b5c3c9f9bf8cda5f2d4ee701b2153e7e6f2807
* | Cancel pending replications upon shutdownLuca Milanesio2019-07-162-6/+38
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When the replication plugin is stopped or reloaded, mark all the current replications as cancelled. Having pending and unmanaged replication tasks is quite dangerous for two reasons: 1. The result of the replication isn't accounted and properly managed, because the associated objects won't be there anymore (pending replications, runway, ...) 2. The destination configuration could have changed completely (auto-reload use-case) or even not exist anymore: the replication event would thus perform an unwanted and unconfigured operation. With regards to the replication events that are already on the runway, there is no value in cancelling them. Just flag them as cancelled so that, once they finish, can be clearly recognized. Change-Id: Iabc17d375011cbc61f8c642ae07d3d018b49fc69
* | Remove replication event from pending when runway is allowedLuca Milanesio2019-07-161-1/+1
| | | | | | | | | | | | | | | | | | Consider a replication event not pending anymore *only if* the runway is free and allowed to run. Removing it too early would result in a ghost event which isn't officially pending but that will be eventually run when rescheduled. Change-Id: Ib0ffa3bec7a6d5d5be56e78a677e8b2133f91e18
* | Add missing log statement if replication was canceledEryk Szymanski2019-07-041-1/+3
| | | | | | | | Change-Id: Id06bae985cfdb000c66743ccd3c5d9f56d34d681
* | When rescheduling due to in-flight push log also the in-flight task IDSaša Živkov2019-07-043-8/+65
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When a push operation was rescheduled due to an in-flight push we logged a line like: [79bda07a] Rescheduling replication to https://host/foo.git to avoid collision with an in-flight push. This log was already useful but it didn't tell us which push task was actually in-flight. Add the ID of the in-flight push task to the log: [79bda07a] Rescheduling replication to https://host/foo.git to avoid collision with the in-flight push [80ceb18b]. This information is especially useful when a task gets repeatedly rescheduled and we need to identify the root cause of that. (cherry picked from commit f585eba0d21be222404b871bfe7a2b11e578eb14) Change-Id: I7866f964cab1e4f479b0d7d62ae4aac3019fab4b
* | Make sure to always remove in-flight pushesSaša Živkov2019-07-031-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | When a push is finished successfully or fails due to an error we have to remove it from the map of in-flight pushes. Although this was done from a finally block, the same block performed a Repository.close() operation prior to the in-flight push removal. If the Repository.close() threw an exception the pool.notifyFinished() wouldn't be called and we would end up with a zombie in-flight push. Bug: Issue 10852 Change-Id: I8d5918d5271ba74ce12153f054503adaef155c5c (cherry picked from commit 6fcdcf98fe61e8c6258fb40967f0cff6b8c38fd4)
* | Bazel: Reformat BUILD file with buildifier 0.17.2v2.15.9v2.15.8v2.15.7v2.15.14v2.15.13v2.15.12v2.15.11v2.15.10David Ostrovsky2018-11-091-1/+1
| | | | | | | | Change-Id: I8d48f0bcbdd33f5ff51fadc9cdd6a3aa623ec8c5
* | Merge branch 'stable-2.14' into stable-2.15v2.15.6v2.15.5v2.15.4v2.15.3David Pursehouse2018-06-141-1/+1
|\| | | | | | | | | | | | | * stable-2.14: Update commons-io dependency Change-Id: I8108343236c6c29cb2769297696b9871e447f490
| * Update commons-io dependencyv2.14.9v2.14.20v2.14.19v2.14.18v2.14.17v2.14.16v2.14.15v2.14.14v2.14.13v2.14.12v2.14.11v2.14.10David Pursehouse2018-06-131-1/+1
| | | | | | | | Change-Id: I79adb2cc05f6ef4f9f2ee98ea247a7fd90514a90
* | Merge branch 'stable-2.14' into stable-2.15David Pursehouse2018-06-072-2/+2
|\| | | | | | | | | | | | | * stable-2.14: Make loggers private Change-Id: I2ebbad6da3509e2d62e8fb48ebba6d0a2caf4b7f
| * Make loggers privateEdwin Kempin2018-06-072-2/+2
| | | | | | | | | | | | | | | | Loggers shouldn't be shared between classes and hence they should be private. Change-Id: Iadd54c73455665b53cf9b58918b2e8c5cf19357e Signed-off-by: Edwin Kempin <ekempin@google.com>
* | Merge branch 'stable-2.14' into stable-2.15David Pursehouse2018-06-061-3/+2
|\| | | | | | | | | | | | | * stable-2.14: StartCommand: Reformat with google-java-format 1.6 Change-Id: If85bd54040a37946cbe5aa69a80e43e5feaa6d6c
| * StartCommand: Reformat with google-java-format 1.6David Pursehouse2018-06-061-3/+2
| | | | | | | | Change-Id: I3c3420f9951b8060273d88e5bf2bcacce8266ab6
* | Merge branch 'stable-2.14' into stable-2.15v2.15.2David Pursehouse2018-04-181-12/+12
|\| | | | | | | | | | | | | * stable-2.14: Format with google-java-format 1.5 Change-Id: Ifbf944373fc3ed36422fed8e1a466c84b2d3e74f
| * Format with google-java-format 1.5v2.14.8David Pursehouse2018-04-181-12/+12
| | | | | | | | Change-Id: I9f9d55aca880c240957db2d81616e3d6a56d4898
* | GerritSshApi: strip 'gerrit+' instead of leaving only itAlon Bar-Lev2018-04-131-1/+1
| | | | | | | | | | | | | | | | | | | | | | [2018-04-12 22:30:04,038] [ReplicateTo-bitbucket-1] ERROR com.googlesource.gerrit.plugins.replication.GerritSshApi : Error creating remote repository at gerrit+ssh://gerrit-bitbucket@localhost/: Exception: org.eclipse.jgit.errors.TransportException: gerrit+: host must not be null. Command: gerrit create-project --branch refs/heads/master repo/test Output: org.eclipse.jgit.errors.TransportException: gerrit+: host must not be null. Change-Id: Ib53258625b3677f621d2e7e9d6df4b04a94a921d Signed-off-by: Alon Bar-Lev <alon.barlev@gmail.com>
* | Merge branch 'stable-2.14' into stable-2.15v2.15.1David Pursehouse2018-02-255-86/+90
|\| | | | | | | | | | | | | | | | | | | * stable-2.14: Fix creating missing repository Fix project deletion logs Use logger built-in formatter Fix project creation logs Change-Id: Idacfd3fa54d7ff5e974bc8dae41979d1f0d9b6b5
| * Fix creating missing repositoryv2.14.7Hector Oswaldo Caballero2018-02-222-2/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | When replicating to a destination where the repository does not exist, updating the HEAD reference failed because the passed reference name was not absolute. When this happened, an unchecked exception was triggered but never showed in the logs. In fact, the replication logs were showing only the message announcing the replication started, but no additional information about the success or failure of the operation was displayed. Avoid this by resolving symbolic references and checking the reference is absolute before trying to update the HEAD in the new repository. Change-Id: I18295a37a04413ba64bf7e9ee31640ee23c4c1e0
| * Fix project deletion logsHector Oswaldo Caballero2018-02-211-2/+2
| | | | | | | | | | | | Log was showing project was deleted successfully even if it failed. Change-Id: I7be19991619ce1f205884a8b6658d9a7ed4109c7
| * Use logger built-in formatterHector Oswaldo Caballero2018-02-215-82/+78
| | | | | | | | Change-Id: I6b46309f07ca4c402ae7a51cfb462c1be6bec333
| * Fix project creation logsHector Oswaldo Caballero2018-02-211-2/+2
| | | | | | | | | | | | Log was showing project was created successfully even if it failed. Change-Id: Ie37e6f2f58e1c8b4e33f61bf2d81ee43698b8aa3
* | GerritSshApi: Use Logger's string formattingDavid Pursehouse2018-02-251-5/+8
| | | | | | | | Change-Id: Ie7fef1b311f6f7fe14aa79c6023b897d91127087
* | Revert "Use proper HEAD ref value when creating project"David Pursehouse2018-02-251-20/+0
| | | | | | | | | | | | | | | | | | | | The issue fixed by this commit was fixed in a different way on the stable-2.14 branch. Reverting this on stable-2.15 will make the merge from stable-2.14 easier. This reverts commit e145010de7911623c98e6111537ffa4997d1a82e. Change-Id: I7e44f62f119b7ed26c8d96bfe9ef8442ff687b29
* | Merge branch 'stable-2.14' into stable-2.15v2.15-rc4v2.15-rc3v2.15David Pursehouse2018-02-011-0/+4
|\| | | | | | | | | | | | | * stable-2.14: Document how to exclude projects from replication Change-Id: I2189f3aabcdc5ee3ca8204886a6c7686af626c04
| * Document how to exclude projects from replicationDavid Pursehouse2017-12-131-0/+4
| | | | | | | | Change-Id: I8a12011e28be44d600a3ed63eebe596560d73e5f
* | Document which branches must be replicated to replicate the account datav2.15-rc2v2.15-rc1Edwin Kempin2017-10-091-0/+12
| | | | | | | | | | | | Change-Id: I2c9dc06927cd8e9e541eb032829029bdfac8db6e Signed-off-by: Edwin Kempin <ekempin@google.com> (cherry picked from commit 93ab1609a4ba32e9466ff0649317b84a211285ad)
* | Merge branch 'stable-2.14'v2.15-rc0David Pursehouse2017-09-150-0/+0
|\| | | | | | | | | | | | | | | | | * stable-2.14: Fix minor nits in documentation of replication.maxRetries Fix replication retries when maxRetries is set to 0 Set max retries to avoid queue congestion Change-Id: I303d3b471bc98eaead33cfa49d614b176cf34f58
| * Merge branch 'stable-2.13' into stable-2.14v2.14.6v2.14.5.1v2.14.5v2.14.4Hugo Arès2017-09-140-0/+0
| |\ | | | | | | | | | | | | | | | | | | | | | | | | * stable-2.13: Fix minor nits in documentation of replication.maxRetries Fix replication retries when maxRetries is set to 0 Set max retries to avoid queue congestion Change-Id: Icf972f3df7d3d59925db670bc1f033028306d0c1
| | * Fix minor nits in documentation of replication.maxRetriesv2.13.14v2.13.13v2.13.12v2.13.11v2.13.10upstream/stable-2.13David Pursehouse2017-06-021-3/+3
| | | | | | | | | | | | Change-Id: I80e46269e12b9e7a429634dd90ce52e90cc6ad79
| | * Fix replication retries when maxRetries is set to 0Hugo Arès2017-05-301-1/+1
| | | | | | | | | | | | | | | | | | | | | maxRetries defaults to 0 which means retry indefinitely but it was not retrying. Change-Id: I6c8d8d6b1511d67b647f25193dcfe860687eea76
| | * Set max retries to avoid queue congestionLuca Milanesio2017-05-294-4/+45
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When servers have *a lot* of remote slaves, some of them unstable and potentially offline, a maximum retry policy is needed to prevent push events to stay in the replication queue and getting rescheduled forever. Keep backward-compatible configuration by setting maxRetry by default to zero, which means disabled. Change-Id: I060cc7bc3a4d1089b0815db02d2e1430f83a2015
* | | Use proper HEAD ref value when creating projectDariusz Luksza2017-09-061-0/+20
| | | | | | | | | | | | | | | | | | | | | | | | Wrong name of HEAD reference was used when repository was created after RepositoryNotFoundException. This results in HEAD pointing to HEAD. Change-Id: I647618fc6d35a5e2ff6bcfeda5084584e591ddda Signed-off-by: Dariusz Luksza <dariusz@luksza.org>
* | | Merge "Merge branch 'stable-2.14'"David Pursehouse2017-09-065-6/+99
|\ \ \
| * | | Merge branch 'stable-2.14'David Pursehouse2017-08-235-6/+99
| |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * stable-2.14: Fix minor nits in documentation of replication.maxRetries Use rescheduleDelay instead of replicationDelay when rescheduling Fix race condition when scheduling a replication Change-Id: I45c6f5a266709e2ce85d2140f13e4ece7da69341
| | * | Fix minor nits in documentation of replication.maxRetriesv2.14.3David Pursehouse2017-08-231-3/+3
| | | | | | | | | | | | | | | | Change-Id: I80e46269e12b9e7a429634dd90ce52e90cc6ad79
generated by cgit v1.2.3 (git 2.25.1) at 2024-05-01 04:26:10 +0000