RPM Repositories

JFrog Artifactory Documentation

ft:sourceType
Paligo

Overview

Artifactory is a fully-fledged RPM repository. As such, it enables:

  • RPM metadata calculation for RPMs hosted in Artifactory local repositories.

  • Deploying RPM Modules to Artifactory local repositories.

  • Provisioning RPMs directly from Artifactory to YUM clients.

  • Detailed RPM metadata views from the web UI.

  • Providing GPG signatures that can be used by the YUM client to authenticate RPMs.

Valid for YUM also

The instructions on this page can be used for RPM repositories and YUM repositories interchangeably

RPM Metadata for Hosted RPMs

The RPM metadata generated by Artifactory is identical to the basic-mode output of the Red Hat-based Linux command createrepo.

A folder named repodata is created in the configured location within a local repository with the following files in it:

File

Description

primary.xml.gz

Contains an XML file describing the primary metadata of each RPM archive.

filelists.xml.gz

Contains an XML file describing all the files contained within each RPM archive.

other.xml.gz

Contains an XML file describing miscellaneous information regarding each RPM archive.

repomd.xml

Contains information regarding all the other metadata files.

YUM Support is Platform Independent!

Artifactory's RPM metadata calculation is based on pure Java.

It does not rely on the existence of the createrepo binary or on running external processes on the host on which Artifactory is running.

Triggering RPM Metadata Updates

When enabled, the metadata calculation is triggered automatically by some actions, and can also be invoked manually by others. Either way, the metadata produced is served to YUM clients.

Automatic

RPM metadata is automatically calculated:

  1. When deploying, removing, copying, and moving an RPM file.

  2. When performing content import (both system and repository imports).

Manual

You can manually invoke RPM metadata calculation:

  1. By selecting the local repository in the Tree Browser and clicking Recalculate Index in the Actions menu.

  2. Via Artifactory's REST API.JFrog REST APIs

Note

Metadata calculation cleans up RPM metadata that already existed as a result of manual deployment or import. This includes RPM metadata stored as SQLite database files.

Indexing the File List

The filelists.xml metadata file of an RPM repository contains a list of all the files in each package hosted in the repository. When the repository contains many packages, reindexing this file as a result of interactions with the YUM client can be resource intensive causing a degradation of performance. Therefore, from version 5.4, reindexing this file is initially disabled when an RPM repository is created. To enable indexing filelists.xml, set the Enable File List Indexing checkbox.

Note that the filelists.xml metadata file for a virtual repository may not be complete (i.e. it may not actually list all the files it aggregates) if any of the repositories it aggregates do not have file listing enabled. Note that if indexing of the filelists.xml file is disabled, it is not possible to search for a file using the YUM client to determine which package wrote the queried file to the filesystem.

Configuration

To create an RPM local repository, in the Administration module go to Repositories | Repositories | Local,click New Local Repository and select RPM as the Package Type.

rpm_new_local.png
Local Repositories

To enable automatic RPM metadata calculation on a local RPM repository, in the RPM Settings section of the Basic settings screen, set Auto-calculate RPM Metadata.

rpm_settings local repo.png

Field

Description

RPM Metadata Folder Depth

Informs Artifactory under which level of directory to search for RPMs and save the repodata directory.

By default this value is 0 and refers to the repository's root folder. In this case, Artifactory searches the entire repository for RPMs and saves the repodata directory at $REPO-KEY/repodata.

Using a different depth is useful in cases where generating metadata for a repository separates its artifacts by name, version and architecture. This will allow you to create multiple RPM repositories under the same Artifactory RPM repository.

For example:

If the repository layout is similar to that shown below and you want to generate RPM metadata for every artifact divided by name, set the Depth to 1 and the repodata directory is saved at REPO_ROOT/ARTIFACT_NAME/repodata :

REPO_ROOT/$ARTIFACT_NAME/$ARTIFACT_VERSION/$ARCHITECTURE/FILE_NAME
- or -
rpm-local/foo/1.0/x64/foo-1.0-x64.rpmm

Note

When changing the configured depth of existing repository, packages indexed in the old depth might need to be re-indexed or moved to a new depth to be available in the new configured depth, and YUM clients might need to change their configuration to point to the new depth.depth.

Auto-calculate RPM Metadata

When set, RPM metadata calculation is automatically triggered by the actions described above.

Enable File List Indexing

When set, RPM metadata calculation will also include indexing the filelists.xml metadata file.

RPM Group File Names

A comma-separated list of YUM group files associated with your RPM packages.

Note that at each level (depth), the repodata directory in your repository may contain a different group file name, however each repodata directory may contain only 1 group metadata file (multiple groups should be listed as different tags inside the XML file. For more details, see YUM Documentation).

Remote Repositories

Artifactory remote repositories support RPMs out-of-the-box, and there no need for any special configuration needed in order to work with RPMs in a remote repository.

All you need to do is point your YUM client at the remote repository, and you are ready to use YUM with Artifactory.

To define a remote repository to proxy an RPM remote repository, follow the steps below:

  1. In the Administration module under Repositories | Repositories | Remote, click New Remote Repository to create a new remote repository.

  2. Set the Repository Key value, and specify the URL to the remote repository in the URL field as displayed below.

    rpm_new_rpm_remote_repo.png

    Note

    Running docker pull centos:latest will return CentOS 8, which has reached End of Life. To use CentOS, pull a supported version: for a list of supported versions, see CentOS Mirror. If you need to use CentOS 8, you can do so by changing the URL in the URL field to https://vault.centos.org/.

  3. Click Save & Finish.

  4. Back in the Application | Artifactory | Artifacts module, in the Tree Browser, select the repository. Note that in the Tree Browser, the repository name is appended with -cache.

  5. Click Set Me Up and copy the value of the baseurl tag.

    RpmBaseURL.jpg

    Note

    For CentOS 8 users, edit or create the following files with root privileges:

    • sudo vi /etc/yum.repos.d/CentOS-Linux-AppStream.repo

    • sudo vi /etc/yum.repos.d/CentOS-Linux-BaseOS.repo

  6. Next, create the /etc/yum.repos.d/ targetCentos.repo file and paste the following configuration into it:

    [targetCentos]
    name=targetCentos
    baseurl=http://localhost:8081/artifactory/targetCentos/
    enabled=1
    gpgcheck=0
Virtual Repositories

A Virtual Repository defined in Artifactory aggregates packages from both local and remote repositories.

This allows you to access both locally hosted RPM packages and remote proxied RPM repositories from a single URL defined for the virtual repository.

To define a virtual YUM repository, from the Administration module, go to Repositories | Repositories | Virtual, set the Package Type to be RPM, and select the underlying local and remote RPM repositories to include in the Basic settings tab.

Note

When using virtual RPM repositories, Artifactory does not support using YUM or DNF commands that use the updateinfo.xml metadata file.

rpm_virtual_repo.png

To allow deploying packages to this repository, set the Default Deployment Repository.

Signing RPM Metadata

Artifactory supports using a GPG key to sign RPM metadata repositories (not packages) for authentication by the YUM client.

To generate a pair of GPG keys and upload them to Artifactory, see Managing Signing Keys.Manage Signing Keys

Installing RPM Packages Using Yum

After configuring the rpm-local repository in Artifactory, you need to configure your local machine to install software packages from it by executing the following steps:

  1. Edit the artifactory.repo file with root privileges

    sudo vi /etc/yum.repos.d/artifactory.repo
  2. Paste the following configuration into the artifactory.repo file:

    [Artifactory]
    name=Artifactory
    baseurl=http://localhost:8081/artifactory/rpm-local/
    enabled=1
    gpgcheck=0

Now, every RPM file deployed to the root of the rpm-local repository can be installed using:

yum install <package_name>

Deploying RPM Packages

Once you have configured your local machine to install RPM packages from your RPM local repository, you may also deploy RPM packages to the same repository using the UI or using the REST API.JFrog REST APIs

Through the REST API you also have the option to deploy by checksum or deploying from an archive.Deploy Artifact by ChecksumDeploy Artifacts from Archive

For example, to deploy an RPM package into a repository called rpm-local you could use the following:

curl -u<USERNAME>:<PASSWORD> -XPUT http://localhost:8080/artifactory/rpm-local/<PATH_TO_METADATA_ROOT> -T <TARGET_FILE_PATH>

where PATH_TO_METADATA_ROOT specifies the path from the repository root to the deploy folder.

Deploying RPM Modules to Your Local Repository

You can deploy RPM modules to the repodata folder using the modules.yaml file which contains all the desired modules and then proceed to trigger a repository index to update the repomd.xml and other metadata files with the modules information.

  1. You can perform the upload directly in the JFrog Platform UI or using the following command.

    curl -u <USERNAME>:<PASSWORD> -XPUT "http://localhost:8081/artifactory/rpm-local/<PATH_TO_REPODATA_FOLDER>/modules.yaml -T <TARGET_MODULES_FILE_PATH>"
  2. Trigger the reindex by running the Calculate YUM Repository Metadata API.Calculate YUM Repository Metadata

YUM Groups

A YUM group is a set of RPM packages collected together for a specific purpose. For example, you might collect a set of "Development Tools” together as a YUM group.

A group is specified by adding a group XML file to same directory as the RPM packages included in it. The group file contains the metadata of the group including pointers to all the RPM files that make up the group.

Artifactory supports attaching a YUM Group file to the YUM calculation essentially mimicking the createrepo -g command.

A group file can also be created by running the following command:

sudo yum-groups-manager -n "My Group" --id=mygroup --save=mygroups.xml --mandatory yum glibc rpm
Attaching a YUM Group

The process of attaching YUM group metadata to a local repository is simple:

  1. Create an XML file in the groups format used by YUM. You can either just type it out manually using any text editor, or run the yum-groups-manager command from yum-utils.

  2. Deploy the created group file to the repodata folder.

    Artifactory will automatically perform the following steps:

    • Create the corresponding .gz file and deploy it next to the deployed group XML file.

    • Invoke a YUM calculation on the local repository.

    • Attach the group information (both the XML and the .gz file) to the repomd.xml file.

  3. Make sure the group file names are listed in the YUM Group File Names field under the Basic tab of the repository configuration. This tells Artifactory which files should be attached as repository group information.

YUM Group Commands

The following table lists some useful YUM group commands:

Command

Description

yum groupinstall <Group ID>

Install the YUM group. The group must be deployed to the root of the YUM local repository.

yum groupremove <Group ID>

Remove the RPM group

yum groupupdate <Group ID>

Update the RPM group. The group must be deployed to the root of the YUM local repository.

yum groupinfo <Group ID>

List the RPM packages within the group.

yum grouplist | more

List the YUM groups

Setting Group Properties

YUM group properties can be set in the /etc/yum.config file as follows:

Setting

Allowed values

Description

overwrite_groups

0 or 1

Determines YUM's behavior if two or more repositories offer package groups with the same name.

If set to 1 then the group packages of the last matching repository will be used.

If set to 0 then the groups from all matching repositories will be merged together as one large group.

groupremove_leaf_only

0 or 1

Determines YUM's behavior when the groupremove command is run.

If set to 0 (default) then all packages in the group will be removed.

If set to 1 then only those packages in the group that aren't required by another package will be removed.

enable_group_conditionals

0 or 1

Determines whether YUM will allow the use of conditionals packages.

If set to 0 then conditionals are not allowed

If set to 1 (default) package conditionals are allowed.

group_package_types

optional, default, mandatory

Tells YUM which type of packages in groups will be installed when groupinstall is called. Default is: default, mandatory

Yum Authentication

Proxy Server Settings

If your organization uses a proxy server as an intermediary for Internet access, specify the proxy settings in /etc/yum.conf. If the proxy server also requires authentication, you also need to specify the proxy_username, and proxy_password settings.

proxy=<proxy server url> 
proxy_username=<user>
proxy_password=pass 

If you use the yum plugin (yum-rhn-plugin) to access the ULN, specify the enableProxy and httpProxy settings in /etc/sysconfig/rhn/up2date. In addition, If the proxy server requires authentication, you also need to specify the enableProxyAuth, proxyUser, and proxyPassword settings as shown below.

enableProxy=1
httpProxy=<proxy server url> 
enableProxyAuth=1
proxyUser=<user>
proxyPassword=<password>
SSL Setting

YUM supports SSL from version 3.2.27.

To secure a repository with SSL, execute the following steps:

  • Generate a private key and certificate using OpenSSL.

  • Define your protected repository in a .repo file as follows:

    [protected]
    name = SSL protected repository
    baseurl=<secure repo url>
    enabled=1
    gpgcheck=1
    gpgKey=<URL to public key>
    sslverify=1
    sslclientcert=<path to .cert file>
    sslclientkey=<path to .key file>

    where:

    gpgkey is a URL pointing to the ASCII-armored GPG key file for the repository . This option is used if YUM needs a public key to verify a package and the required key has not been imported into the RPM database.

    If this option is set, YUM will automatically import the key from the specific URL. You will be prompted before the key is installed unless the assumeyes option is set.

Using Yum Variables

You can use and reference the following built-in variables in yum commands and in all YUM configuration files (i.e. /etc/yum.conf and all .repo files in the /etc/yum.repos.d/ directory):

Variable

Description

$releasever

This is replaced with the package's version, as listed in distroverpkg. This defaults to the version of the redhat-release package.

$arch

This is replaced with your system's architecture, as listed by os.uname() in Python.

$basearch

This is replaced with your base architecture. For example, if $arch=i686 then $basearch=i386

The following code block is an example of how your /etc/yum.conf file might look:

[main]
cachedir=/var/cache/yum/$basearch/$releasever
keepcache=0
debuglevel=2
logfile=/var/log/yum.log
exactarch=1
obsoletes=1
gpgcheck=1
plugins=1
installonly_limit=3
[comments abridged]

Viewing Individual RPM Information

You can view all the metadata that annotates an RPM by choosing it in Artifactory's tree browser and selecting the RPM Info tab.

sample_rpm_Data.png
Metadata Fields as Properties

The corresponding RPM metadata fields are automatically added as properties of an RPM artifact in YUM repositories accessed through Artifactory:

  • rpm.metadata.name

  • rpm.metadata.arch

  • rpm.metadata.version

  • rpm.metadata.release

  • rpm.metadata.epoch

  • rpm.metadata.group

  • rpm.metadata.vendor

  • rpm.metadata.summary

Properties can be used for searching and other functions. For more details see Property Sets.

Watch the Screencast

Watch this short screencast to learn how easy it is to host RPMs in Artifactory.