RubyGems Repositories

JFrog Artifactory Documentation

ft:sourceType
Paligo

Breaking Change: RubyGems Dependencies REST API

RubyGems is deprecating their support of the /dependencies REST API, and JFrog is removing the support for local repositories to align with RubyGems. For more information, please read this blog post by RubyGems: RubyGems.org Dependency API Deprecation.

This feature will reach end-of-life at the end of Q4, 2023.

Overview

Artifactory provides full support for RubyGems repositories including:

  • Local repositories with RubyGems API support

  • Caching and proxying remote RubyGems repositories

  • Virtual repositories that aggregate multiple local and remote repositories including indices of both gems and specifications

  • Support for common Ruby tools such as gem and bundler

  • Support for Bundler Compact index for Local and Remote repositories, providing you with the latest version of the package that is compatible with your installed Ruby version of the project.

For general information on configuring Artifactory to work with RubyGems, please refer to RubyGems Repositories.

General Configuration

All RubyGems repositories must be prefixed with api/gems in the path

When using the RubyGems command line to access a repository through Artifactory, the repository URL must be prefixed with api/gems in the path.

All RubyGems commands, including gem source and gem push, must prepend the repository path with api/gems .

For example, if you are using Artifactory standalone or as a local service, you would access your RubyGems repositories using the following URL:

http://localhost:8081/artifactory/api/gems/<repository key>

Or, if you are using Artifactory Cloud, the URL would be:

https://<server name>.jfrog.io/artifactory/api/gems/<repository key>

Local Repositories

Local RubyGems repositories are physical, locally-managed repositories into which you can deploy and manage your in-house Gems.

To create a local RubyGems repository, in the Administration module, under Repositories | Repositories | Local, click New Local Repository and set Gems to be the Package Type.

local_gems_repo.png

You can set up a local RubyGems repository as follows:

You need to add the repository source URL by modifying your ~/.gemrc file or using the gem source command as displayed below. You can extract the source URL by selecting the repository in the Tree Browser and clicking Set Me Up.

gems_set_me_up.png

Notice that there are two sources. First, the remote proxy, then the local one. This will effectively allow you to retrieve Gems from both of them in the specified order.

All RubyGems repositories must be prefixed with api/gems in the path

When using the RubyGems command line to access a repository through Artifactory, the repository URL must be prepended with api/gems in the path.

gem source -a http://localhost:8081/artifactory/api/gems/my-gems-local/
Default Files

Upon creation of a Local RubyGems repository, Artifactory populates it with the following set of files that are required for indexing the repository:

  • rubygems-update-*.gem (created under the gems folder of the repository)

  • quick/Marshal.*

  • latest_specs.*.gz

  • prerelease_specs.*.gz

  • specs.*.gz

The local RubyGems repository is displayed in the Application module | Artifactory | Artifacts.

gems_artifact repository manager.png
Usage

First, setup the appropriate credentials for the gem tool: either include the API key in the ~/.gem/credentials file or issue this command:

Setting Up Credentials

curl http://localhost:8081/artifactory/api/gems/<repository key>/api/v1/api_key.yaml -u admin:password > ~/.gem/credentials

Running on Linux

You may need to change the permissions of the credentials file to 600 (chmod 600)

Running on Windows

The credentials file is located under %USERPROFILE%/.gem/credentials

You also need to set the API key encoding to be "ASCII". For example:

curl http://localhost:8081/artifactory/api/gems/<repository key>/api/v1/api_key.yaml | Out-File ~/.gem/credentials -Encoding "ASCII"

API keys

You can modify the credentials file manually and add different API keys. You can later use gem push -k key to choose the relevant API key.

In order to push gems to the local repository, you can set the global variable $RUBYGEMS_HOST to point to the local repository as follows:

Setting RUBYGEMS_HOST

export RUBYGEMS_HOST=http://localhost:8081/artifactory/api/gems/<repository key>

To get this value, select your repository in the Application module | Artifactory | Artifacts | Gems-local and click Set Me Up.

gems_deploy_setmeup.png

Alternatively you could use the gem push command with --host, and optionally, --key to specify the relevant API key.

Note

Make sure you are familiar with your effective sources and their order as specified in the ~/.gemrc file.

Also, make sure you are familiar with your global $RUBYGEMS_HOST variable before you issue a gem push command or use the push --host option.

When a local repository is first created, it is populated with rubygems-update-*.gemby default. In order to disable this behavior, you must change the system properties to include:Artifactory Configuration Files

artifactory.gems.gemsAfterRepoInitHack=false

Make sure you deploy to a "gems" folder

When deploying Gems to your repositories, you need to place them in a gems folder for Artifactory to include them in its indexing calculations.

Remote Repositories

A remote RubyGems repository serves as a caching proxy for a repository managed at a remote URL such as http://rubygems.org.

Once requested, artifacts (Gems) are cached on demand. They can then be removed, but you cannot manually deploy anything into a remote repository.

To create a remote RubyGems repository, execute the following steps:

  1. in the Administration module, under Repositories | Repositories | Remote, click New Remote Repository and set Gems to be the Package Type.

  2. Set the Repository Key, and specify the URL to the remote repository. The example below references rubygems.org.

gems_new_remote_repo.png
Usage

In order to allow the integration with the gem command line tool, you must add the relevant source URL to your RubyGems configuration.

  1. In the Application module | Artifactory | Artifacts Tree Browser, select your newly created repository and click Set Me Up.

  2. Copy the source URL from the RubyGems Sources section.

    RubyGemsSourceURL.jpg
  3. Add this URL by modifying your ~/.gemrc file or using the gem source command as follows:

    All RubyGems repositories must be prefixed with api/gems in the path

    When using the RubyGems command line to access a repository through Artifactory, the repository URL must be prefixed with api/gems in the path.

    gem source -a http://localhost:8081/artifactory/api/gems/rubygems/
    

Additional Gem Commands You Can Use

You can remove previous source entries by modifying your ~/.gemrc file manually or by running gem sources -r

You can also run gem sources --list to know what your effective sources are and their order.

Overcoming Unauthorized Status Failures

Some gem/bundler commands may fail with an Unauthorized (401) status. In some cases these can be overcome by usingoneof the following options:

  • Enable the "Anonymous User" by checking Allow Anonymous Access in Security General Configuration as described in Managing Users.

  • Create a specialized Permission Target that allows anonymous access only to the remote repository.Introduction to Permissions

  • Use a source URL with embedded credentials, such as: gem sources -a http://user:password@host /...

Virtual Repositories

A Virtual RubyGems repository (or "repository group") can aggregate multiple local and remote RubyGems repositories, seamlessly exposing them under a single URL.

The repository is virtual in that you can resolve and retrieve artifacts from it but you cannot deploy anything to it. For more information please refer to Virtual Repositories.

The procedure for setting up a virtual repository is very similar to setting up a local or remote repository, however as a last step, you need to select the repositories that will be included in the virtual repository you are creating.

RubyGemsVirtualRepository.jpg
Usage

Using a virtual RubyGems repository you can aggregate both your local and remote repositories.

You need to set the right repository source URL, in the same way as described in Usage for a local RubyGems repository, just with the appropriate repository key as follows:

source: http://localhost:8081/artifactory/api/gems/<repository key>/

target: http:// localhost:8081/artifactory/api/gems/<repository key> ( no slash at the end!)

Using the REST API

The REST API provides access to the Gems Add-on through the repository key using the following URL:

http://localhost:8081/artifactory/api/gems/<repository key>/

In addition to the basic binary repository operations, such as download, deploy, delete etc., the following RubyGems.org API Gem commands are supported:

Gem Command

Curl Syntax Example

Remarks

Info

curl http://localhost:8081/artifactory/api/gems/<repository key>/api/v1/gems/my_gem.(json|yaml)

Optionally indicate JSON / YAML (default: JSON)

Search

curl http://localhost:8081/artifactory/api/gems/<repository key>/api/v1/search.(json|yaml)?query=[query]

Will search for gems with name containing query

Dependencies

curl http://localhost:8081/artifactory/api/gems/<repository key>/api/v1/dependencies?gems=[gem1,...]

Use a csv of gem names for the value of gems

Yank

curl -X DELETE http://localhost:8081/artifactory/api/gems/<repository key>/api/v1/yank

-d 'gem_name=gem_name' -d 'version=0.0.1' -d 'platform=ruby'

Deletes the specific gem file from the repository

Indexing is done automatically by Artifactory in the background, however if you still need to recreate or update the spec index files, the following REST API commands are also available:

REST Command

Curl Syntax Example

Remarks

ReIndex

curl -X POST http://localhost:8081/artifactory/api/gems/<repository key>/reindex

Re-creates all spec index files.

Update index

curl -X POST http://localhost:8081/artifactory/api/gems/<repository key>/updateIndex

Updates all spec index files if needed.

Viewing RubyGems Artifact Information

If you select a RubyGems artifact in the Tree Browser you can select the RubyGems tab to view detailed information on the selected artifact.

rubygems_setmeup.png

Retrieving Latest RubyGems Package Compatible with Your Ruby Versions

The Bundler Compact Index feature allows you to retrieve the latest RubyGems version compatible with your installed Ruby version in the project applies to local, remote, and virtual repositories.

To apply this feature, set the artifactory.gems.compact.index.enabled=true value in the artifactory.system.properties file.