Pipelines Extensions Tutorial

Pipelines Extensions enable users to extend the capabilities of Pipelines by adding their own custom steps and resources to the Pipelines DSL.

The example in this tutorial can be found in the JFrog GitHub repository. You can fork this jfrog-pipelines-extensions-sample repo to your own GitHub account to try on your own installation of Pipelines.

The repository provides the following simple examples, both in the namespace tutorials:

tutorials/HealthCheck - A step that performs a health check action to a website, with optional notification to a Slack instance on success and failure.

stepModel.yml

configuration:
  healthCheckUrl:
    type: Url
  notifyOnSuccess:
    type: Boolean
  notifyOnFailure:
    type: Boolean
  stuff:
    type: String[]

onExecute.sh

checkHealth() {
  local success=true
  local url=$(find_step_configuration_value "healthCheckUrl")
  {
    local statusCode=$(curl --silent --output /dev/stderr --write-out "%{http_code}" "$url")
  } || exitCode=$?
  if test $statusCode -ne 200; then
    export success=false
    echo "Health check failed with statusCode: $statusCode & exitCode: $exitCode for url: $url"
  else
    echo "Health check succeeded"
  fi

  $success
}

execute_command checkHealth

tutorials/test - A resource type that holds some example data types

resourceModel.yml

configuration:
  alpha:
    type: String
    validate:
      isRegex: "^[a-zA-Z_][a-zA-Z0-9_]*$"
  number:
    type: Int
  flag:
    type: Boolean

Add and Sync Extensions

To add your Pipelines DSL extension definitions so they can be used in your pipelines, you must add the forked repository to Pipelines as an extension source.

Add an Administration Integration

In the Administrationmodule, add an administration integrationfor the version control system account (e.g., GitHub, Bitbucket Server, or other) that holds the source repository for your extensions.GitHub Integration

Note

This procedure can only be performed by an administrator user or resource manager user.Introduction to Users and Groups

To add an administrator integration for Pipelines DSL extensions:

1. From the Administration module go to Pipelines | Integrations.

2. From the Integrationsview, click Add an Integration.

3. In the resulting Add New Integration display, click the Admin radio button and then complete the entries for your VCS account and click Create.

4. After entering all the relevant details for the integration, click Test Connection to validate all the details you have entered, including the URL, user credentials, and the token for sufficient permissions on the repository.

Add an Extension Source

Once the VCS account has been added to the administration integrations, you can add your source code repository as an extension source.

Your source code repository for your extensions must be structured according to the requirements for the step model and resource model.

For example, our sample repository folder structure is:

jfrog-pipelines-extensions-sample -- steps ------ tutorials –- HealthCheck |

– resources -- tutorials -- test

where tutorials identifies the namespace for each set of steps and resources.

To add an Extension Source:

1. From the Application module go to Pipelines | Reusables.

2. Select the Sources tab and click Add Source.

3. In the resulting Add Extension Source display, select the VCS account integration added above, enter the repository path and branch, then click Create Source.

Pipelines syncs the extension source to load the extensions.

Click the Logscolumn to view the results of the sync.

Confirm Extensions

View the Steps/Resources tab to see the list of successfully loaded Pipelines DSL extensions.

The example pipeline MyHealthCheck is in the pipeline branch of the repository. The MyHealthCheck pipeline will test the newly added step and resource extensions.

pipeline.yml

resources:
  - name: MyGitHub
    type: GitRepo
    configuration:
      path: jfrogtw/jfrog-pipelines-extensions-sample    # <-- Change to your repository path
      gitProvider: tw_github                             # <-- Change to your GitHub integration
      
  - name: MyTest
    type: tutorials/test
    configuration:
      alpha: "JFrog"
      number: 42
      flag: true

pipelines: 
  - name: MyHealthCheck
    configuration:
      environmentVariables:
        readOnly:
          my_env_var: "hello" 
    steps:
      - name: Health_Check_Step
        type: tutorials/HealthCheck
        configuration:
          healthCheckUrl: "https://jfrog.com"
          notifyOnSuccess: false
          notifyOnFailure: false
          stuff: 
           - "This is a string"
           - "This is another string"
          inputResources:
           - name: MyTest
           - name: MyGitHub

Add the Pipeline Source

To load the example pipeline in your Pipelines deployment, add the pipeline branch of the repository as a Single Branch Pipeline Source.

If you have not already, you will need to add an integration for your GitHub account. Note that this is in addition to the administration integration added above (even though it is for the same account).

When you click Create Source, the Pipelines DSL file will be synced from the newly added pipeline source. When completed, you can view the Logs for the sync.

Run the Example Pipeline

The newly loaded pipeline MyHealthCheck can be seen in the My Pipelines view.

Click on MyHealthCheck to view its Pipeline History. You can click on the pipeline's single Health_Check_Step to trigger a run of the pipeline.Pipeline History