ARTIFACTORY: Migration Basics

Shani Attias
2022-01-20 09:15

Migration Basics

Artifactory migration is the process of copying your complete Artifactory setup from one environment into another environment.
Migration should take place only if you need to move your setup to a new location
***Migration should not be used as a replacement for an upgrade. 

Artifactory Component Overview

Artifactory is composed of 3 main components:

  • Application server (Apache Tomcat Server) – The server responsible to handled all Artifactory Activities like REST API requests, Maintenance Activities, Replication, etc.
  • Database  – The Database contains all the information about Artifactory from users, groups, permission to repositories, and information about each artifact like artifact name and full path.
  • Filestore – keep the artifact (file). Artifacts saved in the Filestore are organized by their sha1 checksum. There is no information about the artifact within the filestore (i.e like the artifact name, path, creation date, etc..). This information is saved in the database only.

System Migration Process

Full system migration includes migration of each of the above components to a new location.

  • Application Server migration is done by a fresh installation of the Artifactory server on a new location.
  • DB (Data) migration should be done by performing a full System Export & Import with “Exclude Content”. In this case, only data that is saved in the DB is migrated. The artifacts themselves will not be migrated.
  • Filestore migration is done by copying the entire data folder to the new location

*** There is an option to migrate the DB and filestore together by performing a full System Export & Import. However, if you have a large instance and require minimal downtime, the recommended migration options are described here

In the described methods, we need a new DB and manually copy the filestore to the new instance as this includes all the binaries.
The reason we are using a new DB for this process is to prevent conflicts with existing data.
We will also do a full system Export & Import (exporting with “Exclude Content” as we have already migrated the binaries). This will populate the DB with the binaries references and will also import Artifactory and Access configurations.
Artifactory configurations refer to Artifactory Skelton configurations such as repositories, replications, Authentication providers (LDAP, SAML…), etc. 
Access configuration refers to Users, Groups, Permissions, and Access Tokens.
If you only need to import Access data, you may refer to this Knowledge Base .

Migration FAQ: 

Can we follow the migration steps in order to migrate from version 6.x to 7.x?

While it is possible to migrate Artifactory 6.x to Artifactory 7.x, the best practice would be to first upgrade the existing 6.x instance to version 7.x and only then follow the migration steps.
The reason behind this is that version 7.x has introduced several major changes and breaking changes that the upgrade process can handle but it might fail the migration.

As part of the migration we are planning to switch to cloud storage, how should we copy the binaries?

Unless you are migrating to an S3 bucket, you should copy the binaries manually, however, there are external tools that can help you with that. For example, for Azure Blob you should use AzCopy.

If you are planning to use an S3 bucket, then there are 2 options to migrate the binaries.
We can either copy the filestore to the S3 bucket in the existing instance and then continue with the migration process (option A), or we can copy binaries to the S3 bucket after the new instance is set up (option B).

Option A:
In order to migrate to the S3 bucket in the existing instance, you should refer to our wiki page, Migrating Your Filestore to S3 .
In this process, we are utilizing the Eventual Binary Provider for the Automatic Filestore Migration.

Option B:
Migrating the binaries to the S3 bucket after we have installed the new Artifactory instance, requires copying the binaries manually to the bucket and is out of JFrog scope. 
However, it can be easily achieved using AWS CLI.

How to implement the sharding binary provider as part of the migration?

If you are currently using a local filestore/NFS and would like to use sharding in the new environment, you should copy the binaries to the new shards and update the binarystore.xml file accordingly.
Next, in order to balance the redundancy use the Optimize System Storage REST API.
Note that this will balance the redundancy and not the storage size/count in each shard.

We only want to migrate to a new DB, how can we achieve this without migrating the whole instance?

We have a great YouTube guide for this process, but if you prefer things written down follow the steps below;

  1. Remove the Artifactory nodes from the LB
  2. Perform full system export with exclude content enabled. Please do not check the ‘exclude metadata’ box as these are the binaries that are stored in the database.
  3. Create a new schema in the target database. The exact SQL commands for creating a new scheme and an ‘artifactory’ user can be found in our wiki (PostgreSQL, MySQL, Oracle, Microsoft SQL Server, MariaDB)
  4. Stop Artifactory 
  5. Configure your instance to connect to the new database using the system.yaml file. 
  6. Perform full system import