Patrick Russell
2023-01-22 11:10

Relevant Versions: Artifactory 6.X and Artifactory 7.X

Introduction

This guide is a handy addition to the migrate Artifactory with minimal downtime article.

The challenge, which was left open ended in the article, is how to set up (potentially) thousands of Push Replication settings. This can be done with some clever scripting and the Artifactory REST API. You'll also need to download the latest JFrog CLI to take advantage of its replication configuration system.
 

Limitations of this Guide

Since this guide sets up Push Replications, the replications will only go one way. Bi-Directional Push Replications will cause serious problems, such as unintended deletions on the Source Artifactory's data. As such, this guide will only set up one way replications, an Active / Active in-sync cluster can be achieved via High Availability or other measures.

This guide has been tested on Artifactory 6.X and 7.X. The mechanisms involved have not changed significantly since Artifactory 5.X.
 

Create the Repositories and Settings on the New Artifactory

It's possible to import the Artifactory configurations from the Old Artifactory to the New Artifactory using an empty System Export even at scale. At large scales (Above 1 million artifacts) the default Full System Export takes too long to complete. For our purposes, we just need to create the repositories on the other side. 

You can exclude everything but the configurations and users by checking both "Exclude" checkboxes. This will save a nearly-empty backup of the Old Artifactory's configurations and permissions; it should complete fairly quickly.

Note: If anything goes wrong later, you should reload this exported configuration to undo the replication settings

Next, copy the export into the New Artifactory. Import it using the Import System menu to automatically create all the target repositories:

If both checkboxes are checked on the Import, nothing will be deleted. This way the import can be done again later to update the repositories list and permissions. This is also how you can recover if the replications are not set up correctly.
 

Replication Setup Step 1: Get the Repository Names

It would take a lot of work to set up Push Replications manually. For many Artifactory users, the act of going through the hundreds, perhaps thousands, of replication configurations would be impossible. However, using new tools added to the JFrog CLI, it's possible to automate this task completely.

It's suggested you create a project folder to save the template files we're about to create:

mkdir replication-setup; cd replication-setup

First, get the repositories list from the Old (Source) Artifactory. The command below saves only the repository key, which is the repository's name. The keys are all that we need to set up replications. This command will save a "repositories.list" file we'll use in the next step:
 # The curl command downloads the list, the grep pulls out the "key" we need #

curl -s -u admin "http://jfrog.7x.gcp/artifactory/api/repositories?type=local" | grep "key" > repositories.list
[Expected output syntax]

With the "repositories.list" file, we've actually gotten all of the information we need to set up replications everywhere. The next step is to create the right template files for the CLI.
 

Replication Setup Step 2: Creating the Replication Settings

The JFrog CLI offers a "replication-create" action , which requires a Replication Template file. In BASH, you can create these template files by using the repositories.list file and some clever processing tricks.

A] First, configure the JFrog CLI with both "source" and "target" Artifactories:jfrog config add source #Define the DR Source Artifactory
jfrog config add target #Define the DR Target
B] Generate the Replication template configuration files. The loop below will read each line of the "repositories.list" file and create a template file based on the key:# The JFrog CLI uses the name "target" and fills in the right URL when replication is configured

done

[Expected JSON File Contents]

After the loop finishes, please check the resulting files. They should all look like the above example image. It's important to double check the templates before we mass-deploy them to the Source to prevent misconfigured settings from being deployed into the Old Artifactory. 

C] Run the "replication-create " CLI command on each of the JSON files, this script will do this for you:jfrog config use source

ls | while read line
do
echo "jfrog rt replication-create $line"
jfrog rt replication-create $line
done
This will push each template to the Source Artifactory, and at the end all Artifactory's Local Repositories will be configured to push to the Target:

Replication Setup Step 3: Sit back and relax

Believe it or not, that's it! If the previous steps were completed successfully, over time the Production Artifactory will push all of its data to the Target and the two will have the same information. You can kick off the migration by clicking the "Run Now" button from the UI.

Enterprise Plus customers can actively synchronize security information by setting up Access Federation between the two instances.
 

Troubleshooting Issues

If the replication information used in the above steps is incorrect, the Source Artifactory will end up with a lot of bad replication settings. The fastest way to restore the application is to remove the replication settings from the Config Descriptor.

The recommended method is to use the REST API to redeploy the "artifactory.config.xml" you have saved in the empty System Export captured at the start of this process. 

The file to use is located in the export folder, for example 20211215.195625/artifactory.config.xml.​
 curl -u admin -X POST -H "Content-type:application/xml" --data-binary @artifactory.config.xml http://localhost:8081/artifactory/api/system/configuration