ARTIFACTORY: Cold Artifact Storage on AWS with Amazon S3 Glacier

Regular transfers of aging data from active storage help keep your mission-critical Artifactory performing at peak speed and availability, while also regularly releasing high-cost, performant storage for new data. With only active artifacts to search through, you can find what you need fast.

To set up a Cold Artifact Storage, you must first configure a designated Artifactory instance as the Cold Artifactory and connect to Amazon S3 Glacier.

Prerequisites
Before setting up Cold Artifact Storage, you will need to verify the following:

• Two Artifactory instances
• The Artifactory instances - Live and Cold - do not have to be the same version.
• Enterprise or Enterprise Plus licenses
• Unique license for each Artifactory instance
• Administrator access to both instances

The Artifactory instance that is set as the Live Artifactory instance must have the Mission Control microservice enabled.

Step 1: Setting up the Live Artifactory Instance

1. Download and install the Artifactory on your EC2 instance. The Live Artifactory instance can be a newly installed instance or an existing instance.
Configure inbound rules on relevant security groups to allow traffic on the internal network, for example:


My Network CIDR is 10.0.1.0/24
So I enabled traffic to this internal subnet:


In addition, assign relevant permissions to the Cold Artifactory EC2 instance to access to S3 bucket:



2. Add the below configuration to the system.yaml file present under /var/opt/jfrog/artifactory/etc/ directory to enable Mission Control.


Start the Artifactory Live instance.
Validate that you see Mission Control Map:


And Platform Management under Administration Tab


Step 2: Create a Cold Artifactory Instance
On the Artifactory instance that you are designating as the Cold instance, perform the following steps:

• Update the Artifactory system.yaml file and add the shared.jfrogColdStorage.coldInstanceEnabled property and set it as true to configure the instance as the Cold instance.:

Step 3:  Register the JPD in the Live instance

1. First, create a binding token for Mission Control by following the steps in Pairing Tokens (available from Artifactory 7.29.7).

Generate the Pairing Token in the Cold Artifactory instance for mission control and copy it

Administration → User Management → Access Tokens → +Generate Token

**


2. In the Live Artifactory instance UI, navigate to Administration -> Platform Management – > Registered JPDs -> New Platform Deployments.

Enter the below details in it:

• The name of the Cold Artifactory instance.
• Location in which Cold Artifactory instance is present.
• Provide the Cold Artifactory JFrog platform URL / Internal IP and click on the Get Server Details button.
• It is recommended that you configure a Custom Base URL for this Artifactory instance. A custom URL base is especially useful when the system is running behind a proxy. For more information, see General System Settings.

You should see the “Test connection was successful” Pop up


And Username and Password fields will be changed to Pairing Token


If you get “Unable to determine the Artifactory version”


We need to validate connectivity between the Artifactory instances

• Provide the Pairing token
• Provide Tags for Cold Artifactory instance

If the JPD already exists in our records we will get:


Step 4: Bind the Live and Cold Artifactory Instances

Available from Artifactory 7.38.4

After completing the previous two steps, your Platform Deployments will have two JPDs that are both connected to the Mission Control microservice: the Live instance and the Cold instance. The next step will be to bind the two instances to each other so that the Cold Artifactory instance will trust and receive requests from the Live Artifactory instance. You can only bind one Live instance to one Cold instance, and this is a one-way trust.

1. In Artifactory Live instance UI, navigate to Administration -> Platform Management -> Deployment Bindings


2. Click on Add Binding and select Cold Storage.


a. Select the Source JPD(Live instance) and Target JPD(Cold instance).
b. Click on Apply, you will see Selection Summary.


c. Now, click on Create. It will show the Binding successfully. Click on the OK button.


d. In the Bindings tab in the Artifactory Live instance, the cold storage which we configured will be present


e. We have successfully configured the Artifactory Cold storage Setup.

Configure the Archive Binary Provider:
When setting up the Cold instance, you will need to connect it to an Amazon S3 Glacier Cloud Storage

• Glacier, follow these steps:

1. Open the binarystore.xml configuration file located in the /opt/jfrog/artifactory/app/misc/etc/artifactory/ folder.
2. Specify the s3-storage-v3-archive chain.

Within the chain, define the standard Amazon S3 Binary Provider template as outlined in the section Amazon S3 Official SDK Template and configure the parameters as per your requirements.

Example:
s3-storage-v3-archive Template


2. Restart the Cold Artifactory instance:
After making the above changes, restart the Cold Artifactory instance for the changes to take effect.

We are uploading the files to Bucket and on the AWS level configuring Lifecycle Policy to S3 Glacier
https://aws.amazon.com/getting-started/hands-on/getting-started-using-amazon-s3-glacier-storage-classes/