How to replace .marker layers in a docker remote repository cache in Artifactory with the actual docker layers?

Nihal Reddy Chinna Choudhary
2019-06-04 23:04

Summary

When a client tries to pull a Docker image from a docker remote repository in Artifactory and if the client has some or all of the layers cached locally on the disk, then Artifactory will not download these layers and mark them as .marker files in the remote cache in Artifactory.

Affected Versions

5.X and 6.X

Details

When a client tries to pull a Docker image from a docker remote repository in Artifactory and if the client has some or all of the layers cached locally on the disk, then Artifactory will not download these layers and mark them as .marker files in the remote cache in Artifactory. 

As long as the docker remote repository in Artifactory has external access to the docker registry or any other upstream repository it is proxying, then the .marker layers will not cause any issues.

Having .marker layers in the docker remote remote cache can become an issue when the repository is marked offline or if the content is copied to a docker local repository. When this happens there is no easy way to replace these .marker layers with the actual layers. At this point the docker pull for images with marker layers will start to fail as .marker layers are only a reference of the actual layer.

 

Resolution

We have a script on our public Github page that can be used to fetch and cache actual docker layers instead of .marker layers in the docker remote repository cache. Please note that this script needs to be run on a docker remote repository in Artifactory that still has external access, so that the API calls in the script are able to retrieve and cache the actual docker layers via the Artifactory docker remote repository.

Here is the link to the script which has a detailed README:

https://github.com/jfrog/artifactory-scripts/tree/master/docker_fetch_markers