ARTIFACTORY: How to manually synchronize a single repository

Patrick Russell
2022-05-11 21:54

In some circumstances Federated Repositories and other sync systems don't fit a migration use-case. If you need to quickly push a large amount of data, the JFrog CLI can be used to download then upload a single repository. JFrog CLI can scale up much, much more.
Please note you need to have a local disk that is at least the size of the repository you want to sync, check the "Artifact Count / Size" menu item before proceeding:
[Example: 198 MB will be downloaded]
User-added image

1. Download and install the JFrog CLI on the New Artifactory – Configure it to recognize Old and New servers:jfrog config add old #Fill out connection details for the Source Artifactory
jfrog config add new #Fill out connection details for the Target Artifactory
2. Download the current contents of the Old Artifactory's repository. "Migration-Repository" is used in this example.
#Perform this on a mounted disk location with at least the repository's size available on the disk
 mkdir -p /tmp/Migration-Repository
cd /tmp/Migration-Repository
jfrog config use old
#Adjust "--threads" setting as necessary
jfrog rt download --threads 15 Migration-Repository/*
3. (Optional) If this is a replacement of data on the New Artifactory, you have to delete the contents there to replace it effectively. Only do this if this is a replacement and not an update.
 jfrog config use new
#Will delete 15 things at a time
jfrog rt delete --threads 15 Migration-Repository 
4. Get the data up into the New Artifactory, "Migration-Repository" is used here but the upload can be to a different repository:
 jfrog config use new
ls | while read ITEM; do jfrog rt u --threads=10 --flat=false --recursive=true  $ITEM/ Migration-Repository/; done

The final upload sequence will push up all of the downloaded data to the New Artifactory.