Using Jenkins With Pipelines

Pipelines provides facilities to manage the execution of a Jenkins job from your pipeline, passing information into and receiving output from the job.

For organizations that have a significant legacy investment in Jenkins, it may not be practical or cost-effective to convert many existing CI/CD workflows to Pipelines. These facilities provide you with a way to interoperate between your existing Jenkins workflows and Pipelines.

For example:

A Jenkins build job may create a base Docker image and publish it to Artifactory, then send information from that job to the subsequent steps in your Pipeline to build variant Docker images for testing and release
A Pipelines pipeline may publish a build to Artifactory, then send the build info as a parameter to a Jenkins job to deliver it for staging
A Jenkins build job may push a Docker image to Artifactory, trigger Pipelines to build supplemental images, then Pipelines may trigger Jenkins to complete the build of variants

This facility presumes the use of Artifactory repositories to store and manage all builds and metadata.

Before You Start

This facility requires that Jenkins Artifactory Plugin be installed with your Jenkins installation.Configuring Jenkins Artifactory Plug-in

Running a Jenkins Job From Pipelines

A pipeline in Pipelines can transfer control to a Jenkins job and then is returned to execution after the Jenkins job completes.

In summary, this is the sequence of events:

In Pipelines, a pipeline executes a Jenkins step to invoke a Jenkins job through the Jenkins Server Integration
When the Jenkins job completes, it returns control to Pipelines and optionally updates Pipelines resources by calling a function in the Jenkins Artifactory PluginConfiguring Jenkins Artifactory Plug-in
In Pipelines, the pipeline resumes execution with any updated resources

Jenkins Server Integration

To enable Pipelines to pass control from and return to Jenkins, an administrator user must first add a Jenkins Server Integration The Jenkins Server integration specifies the Jenkins URL, as well as the username/token credentials for a valid Jenkins user. In addition, the Jenkins Server Integration must generate a bearer token to authenticate messages sent from Jenkins to Pipelines through the Jenkins Artifactory Plugin.

The Jenkins Artifactory plugin must be configured with the Callback URL, and the generated bearer token must be added to the plugin's credentials. See the Jenkins Server Integration for details.

Pipelines DSL

If the Jenkins job will update Pipelines resources, you must declare those resources in your Pipelines DSL. For example, a PropertyBag resource can be used to signal Jenkins job results:

resources

resources:
  - name: app_test_results
    type: PropertyBag
    configuration:
      passing: 0
      failing: 0

To initiate the execution of a Jenkins job from within your pipeline, use the Jenkins native step. You must specify the Jenkins Server integration in your integrations block. If the Jenkins job updates Pipelines resources, you must specify them in the outputResources block.

Jenkins Step

- name: test_app
  type: Jenkins
  configuration:
    jenkinsJobName: basic-api
    inputResources:
      - name: app_docker
    buildParameters:
      imageName: ${res_app_docker_imageName}
      imageTag: ${res_app_docker_imageTag}
    integrations:
      - name: myJenkins
    outputResources:
      - name: app_test_results

When loaded into Pipelines, the pipeline diagram shows the Jenkins step with a Docker image as an input resource, and the PropertyBag updated by the Jenkins job as an output resource.

Sending Build Parameters to Jenkins

The Jenkins step can pass parameters to the Jenkins job through the buildParameters property. These values can be fixed or you can use Pipelines environment variables to send dynamic values.

In this example, we specify an Image resource called "app_docker" in the inputResources of our Jenkins Step. Our buildParameters property references the resource's environment variables.

Jenkins Parameters

buildParameters:
  imageName: ${res_app_docker_imageName}
  imageTag: ${res_app_docker_imageTag}
  environment: "test"

Note

Run state environment variables are not available to buildParameters.

Jenkins Build Job

The Jenkins job must be prepared to recognize incoming buildParameters from Pipelines. When our Jenkins job executes, our example build parameters are received from Pipelines and available for use.

Jenkins Job

steps {
  echo "environment: ${params.environment}"
  echo "Running tests on image ${params.imageName}:${params.imageTag}"
}

At the end of your Jenkins build job, call the plugin's function jfPipelines() to signal its completion and return control back to Pipelines.

Jenkins Job

jfPipelines()

Your build job may optionally pass information back to Pipelines by updating the properties of Pipelines resources. To do so, declare each resource by name with the new property values in you call to jfPipelines().

For example, the following call references a Pipelines PropertyBag resource we have called app_test_results and provides new values for two of its properties:

Jenkins Job

jfPipelines(
  outputResources: """[
    {
      "name": "app_test_results",
      "content": {
        "passing": 1234,
        "failing": 0
      }
    }
  ]"""
)

As noted above, these Pipelines resources must be declared as outputResources of the invoking Jenkins step.