Introduction
Under certain circumstances, it may become necessary to preserve the local cache of a remote or Smart Remote repository. For instance, this could occur if a public source becomes unavailable or stops serving its dependencies for an extended period, or if a remote JPD (acting as a Smart Remote repository source within your organization) is decommissioned and no longer provides private packages.
This article describes effective approaches to quickly back up such caches to a local repository. The primary goal is to safeguard these packages, ensuring their availability is no longer dependent on the lifetime or status of the original cache.
Caching mechanism
Each remote repository in Artifactory, including Smart Remote repositories, comes with its own dedicated cache. This cache is automatically created as a special service repository, typically named <remote-repo-name>-cache.
By default, this local cache stores dependencies that have been successfully resolved through the remote repository. The duration for which these cached items are considered "fresh" is determined by the Metadata Retrieval Cache Period setting in the remote repository's configuration. This parameter essentially defines the cache's lifetime for metadata.
It's important to understand that when the cache "expires," it doesn't mean the binaries are physically removed from local storage. Rather, it indicates that Artifactory will check the remote source for updated metadata or newer versions upon the next request. In common scenarios, especially when a remote repository is part of a virtual one, attempts to resolve dependencies from an expired cache and an unavailable remote source can lead to 404 errors.
Physical deletion of cached artifacts can occur if the Unused Artifacts Cleanup Period is set for the cache repository and the Cleanup Unused Cached Artifacts task is enabled within your regular maintenance operations.
Note:
Not all file types are eligible for caching; a more detailed description of this mechanism and its limitations can be found in this article.
Therefore, a situation might arise where it becomes beneficial to create a snapshot of the current state of the cache as a local repository. This could then potentially be included in a virtual repository if you need to continue serving packages that are no longer available from their original source. The main ways to move the cache to a local repository are described below.
Cache Manipulations
Important: It is assumed that you have previously created a local repository of the same package type as the cache being backed up.
- Through the Artifactory UI: Within the Artifactory repository tree, you can easily copy or move artifacts from a cache repository by right-clicking the desired repository and selecting the appropriate menu option.
- Using the REST API: Artifactory provides a comprehensive REST API that allows you to programmatically copy and move objects between repositories.
- Via Custom User Plugin: Customers with Self-Hosted deployments can also configure a custom Artifactory Remote Backup Plugin. This plugin enables the configuration of multiple cache-to-local repository pairs simultaneously and can be scheduled to run regularly.
After moving and copying, the filled local repository can be included in the virtual one, having previously switched the remote repository being replaced to Offline mode or excluded it from the virtual one. This will allow you to transparently replace the source of packages for your workflows.