Resources are one of the key building blocks of all pipelines. They are information entities that are used for storing and exchanging information across steps and pipelines.
Resources are versioned and each version is immutable. They are also global and depending on the scope defined for the pipeline source, they can be available across pipelines, which enable you to connect multiple pipelines together to create a pipeline of pipelines.
Resources are pointers and they can be used to reference:
A repository in your source code control system, such as GitHub
A file on a remote file server
A Docker image
A release bundle for JFrog Distribution
A cluster for container orchestration
 |
Using resources in your pipeline involves two main steps:
In the pipeline's YAML, in the
resourcessection, define all the resources required for running the pipeline.After a resource is defined, it is available for use in the pipeline, based on the scope defined for the pipeline source.
In the
stepssection, as per your workflow, add these resources as input and/or output.
Resource Types
See it Live
Click here to see some of these resources in action.
Resource Scope and Visibility
Pipeline resources have the same scope as the pipeline source where they are defined.
When defining resources, the recommended approach is to define them in a pipeline source that is shared across environments in the same Project. This ensures that the resources are available across environments in a project. For more information, see Creating a Project - Pipelines Resources.
Note
Currently, pipeline resources cannot be shared across Projects.
Resource Types
Resources play different roles based on where and how they are used:
Triggering Resource: Whenever there is a change in this type of resource, it triggers the dependent step. Examples: CronTrigger, GitRepo, IncomingWebhook.
Generated Resource : These are resources that are generated by a step and they can trigger successive downstream steps. Examples: Aql, BuildInfo, FileSpec, Image.
Webhook Resource: IncomingWebhook and OutgoingWebhook are resources that can be used to integrate a pipeline with third-party services.
Pipelines supports several types of resources, with each resource capable of serving a specific activity:
Type | Description |
|---|---|
An Aql resource specifies an Artifactory query using Artifactory Query Language. | |
A BuildInfo resource is the metadata associated with a build in Artifactory. | |
A CronTrigger is used as an input resource as a step to trigger execution of the step at a scheduled time. | |
A DistributionRule resource is the set of Destination rules that can be applied to distributing a release bundle using JFrog Distribution. | |
A FileSpec resource specifies a File Spec, which provides the details of files to upload or download from Artifactory. | |
A GitRepo is used to connect JFrog Pipelines to a source control repository. It creates a webhook to the repo so that future commits will automatically create a new version with the webhook payload. | |
The HelmChart resource maps to a specific chart in an Artifactory Helm Repository. | |
An Image resource is used to add a reference to a Docker image to your pipeline. | |
An IncomingWebhook resource can trigger one or more jobs in your pipeline whenever the associated URL is called using the HTTP POST method. | |
An OutgoingWebhook resource uses HTTP to send information from a step to an external API endpoint through an Outgoing Webhook Integration. | |
A PropertyBag resource is used to pass information from one pipeline to another and provide environment variables to a step in the format of a resource. | |
A ReleaseBundle resource specifies a set of artifacts in Artifactory that are distributed to Artifactory Edge nodes as a JFrog Distribution Release Bundle. | |
A RemoteFile resource enables using a file on a remote file server. | |
A VmCluster is used to represent a set of virtual machines. It is mainly used to deploy services/apps to the specified clusters and in some cases, it can be used to run certain maintenance activities on the clusters as a whole. |
Resource Versions
One of the key features of Pipelines resource is versioning. Resource versions are useful for controlling the flow of pipelines and tracking the changes that a resource undergoes over time. You can trigger runs using specific versions or skip steps in the same run when input resources are not updated.
Every resource starts with an initial version, which is updated every time the resource changes. For example, the version of an Image resource used in a pipeline is updated whenever a new tag is pushed to a Docker image. Pipelines tracks these changes by updating the resource version, which is based on the metadata received for that particular resource.
A new resource version is created when:
The resource definition is updated in the pipelines YAML file.
An output resource is updated during a run.
There is an external event, such as pushing a commit to a git repository.
Note
No new version is created for an unchanged resource. However, if you need the resource to always change (for example, to trigger another pipeline), you can add $run_id environment variable as the value of an additional property.
Each version of a resource is immutable and returns the same result every time a specific version is used for a run. By default, steps always run using the latest version of an input resource. However, because a resource is versioned and holds the entire history of all the available versions, a run can be customized to use a specific version of a resource. For more information, see Triggering a Run with Custom Parameters.
Default Behavior of Resource Versions
Resource versions have the following default behavior:
Latest version: When a run is triggered, the latest version of the input resource is used in that run.
Dependent step in the same pipeline: During a run, even when an input resource is not updated (no new version), the dependent step is triggered. While this is the default behavior, it can be changed by setting the
newVersionOnlytag astrue.Note
This is applicable only for generated resources, which are resources that connect two steps of a pipeline.
Dependent step in another pipeline: During a run, the dependent pipeline is triggered only when there is a new version of the resource that connects the pipelines.
Example - Resource Versions
This is a simple, single-step pipeline, with GitRepo as the input resource for the step.
 |
In this example:
First version: When the pipeline syncs and loads for the first time, the latest commit on the Git repository is seeded as the very first version for the Gitrepo resource.
When the first run is triggered, this resource version is used for the run.
Second version: When a new commit is pushed to the Git repository, the Gitrepo resource is updated with the new version, which becomes the latest version of the resource.
The Git commit automatically triggers the run and the second version of the resource, which is now the latest, is used for the run.