Perform Full Sync on Federated Repositories

JFrog Artifactory Documentation

Products
JFrog Artifactory
Content Type
User Guide
ft:sourceType
Paligo

Tip

Users who are running Artifactory 7.71.1 or later on all their JPDs should use the auto-healing mechanism (or perform Federation recovery manually using the REST API) before resorting to performing a Full Sync.Federation Recovery

In certain cases, it might not be possible to synchronize all the metadata events between repository Federation members. For example, suppose two members are disconnected for a long period of time (for example, more than 3 days). In that case, old events might be deleted from the queue making it impossible to copy those events to the remote member after restoring the connection.

When regular synchronization is incomplete, the solution is to perform a unidirectional Full Sync operation between the repositories. The Full Sync operation synchronizes all the events in one JPD by copying any missing files to designated remote JPDs. However, Full Sync does not synchronize any deleted events.

For example, suppose a file was deleted from JPD A, but the event wasn't propagated to the remote site (JPD B) before the event was deleted from the queue. If you run Full Sync on JPD B (where the file deleted from JPD A still exists), the operation will restore the file on JPD A.

Therefore, if you need to synchronize the entire Federation, you must run the Full Sync operation on all members.

Bear in mind that a Full Sync operation can be time-consuming if the Federated repositories contain many artifacts, as this amounts to restarting the Federation.

A Full Sync operation is performed using the Federated Repository Full Sync REST API. You can use this API to synchronize specific Federated repositories or all the Federation repositories in the Federation.Federated Repository Full Sync

How Full Sync Works

When you initiate a Full Sync operation, the query is sent to the remote member, which begins compiling a file list of the repository's contents. The local JPD polls the remote JPD every 30 seconds (this interval is configurable - see the note below) to check whether the file list is ready. The advantage of using this polling method is that it removes the need to keep the connection between the members open for a long period of time, which in the case of large repositories can lead to timeouts.

When ready, the remote JPD sends its file list to the local JPD for comparison with the local file list, and then the Full Sync operation can begin.

Note

The following property defines the polling interval:

federatedFullSyncPoolingWaitTime("federated.full.sync.pooling.wait.time.sec", TimeUnit.SECONDS.toMillis(30)),