Clean Up Your Git LFS Repositories with JFrog CLI

Git has become an industry standard for source code control. However, it is not designed to store binary files efficiently, and storing large binary files in a Git repository can quickly inflate it, making it sluggish to respond. This is where Git LFS comes into play. Git LFS is a Git extension designed to accommodate large binary files by offloading their storage to an external file server, such as JFrog Artifactory, which is separate from the Git repository itself. Artifactory support for Git LFS repositories optimizes working with large files and other binary resources in several ways, including performance, security, and reliability.

Visit our Knowledge Base to learn how to setup Git LFS to work with Artifactory in 5 minutes.

However, here’s where things get a bit tricky. These large binary files can still take up a lot of space in your Artifactory repository. Git LFS stores entire files, which means that every time a large file is changed, even slightly, the entire file is stored again, alongside all the old versions of the same file. These file versions can pile up, and potentially max out storage, very quickly. JFrog CLI offers a solution.

The JFrog CLI git-lfs-clean command, which is available for use with Artifactory, allows you to quickly and easily clean up these unnecessary and outdated files from your JFrog Artifactory Git LFS repositories, freeing up valuable disk space for other artifacts. Continue reading to see how this works.

The Git LFS Clean command in action

Let’s say, for example, that we have a Git repository called “my-project”. This project has a master branch, a development branch, and a handful of feature and release branches. Each commit to any of these branches includes the build results: a handful of binary files which are stored in an Artifactory Git LFS repository called “my-project-lfs”. This repository is getting very large and needs cleanup. All we need to do is simply navigate to a local clone of “my-project”, ensure that all remote branches are fetched and up-to-date, and then run the following CLI command:

jfrog rt glc

This will delete all of the build result files that are not in any of the current remote branches, preserving only the most recent binaries that were pushed to all of the project branches, leaving you with a clean repository.

How about some options?

By default, JFrog CLI will check for a Git repository in the current working directory. If the repository is somewhere else, the path can be specified manually:

jfrog rt glc ./projects/my-project

The location of the Artifactory repository is detected automatically from the Git repo’s LFS settings. To override this behavior, the repo name can be passed directly:

jfrog rt glc --repo=my-project-lfs

By default, only LFS files in current remote branches are preserved. This can be customized to allow preservation of files from any combination of Git references. For example, to only preserve files in the master branch:

jfrog rt glc --refs=refs/remotes/origin/master

More likely, you will want to preserve files from all branches, as well as all tags:

jfrog rt glc --refs=’refs/remotes/*,refs/tags/*’

If this option is not specified, the default behavior is to only preserve all remote branches. This is equivalent to the following:

jfrog rt glc --refs=’refs/remotes/*’

Before running the cleanup, JFrog CLI will ask for confirmation that the files should really be deleted. To turn this confirmation off, pass the –quiet option. To run a test and make sure you’re really deleting the right files, pass the –dry-run option. JFrog CLI will list the files it would have deleted, but doesn’t really remove anything from your disk.

Gone means gone (almost)

Keep in mind that the Git LFS Clean command deletes files from the project’s Git LFS source, not from a local cache. Files deleted from this server will be unavailable to anyone using the project. Don’t use this tool unless you’re certain that the files it deletes won’t be missed. If the wrong files are deleted accidentally, they can usually be restored from the Artifactory trash can.

Update before you clean

When using the Git LFS Clean command, JFrog CLI uses the current state of the local Git repository to decide what needs to be preserved. If the repository is behind the origin, newer files might be cleaned by accident. To prevent this, ensure that all branches are fetched properly before running the command.

Let’s get cleaning

So if you find yourself running low on storage, it might be time to clean up your Git LFS repositories. All it takes is this one simple JFrog CLI command.

Looking for more effective CLI commands? Learn more about JFrog CLI