My Build, My Way | JFrog Pipelines Extensions

TL;DR

Once my new projects are almost ready to share with the team and I can build and test them locally, I’ll need a CI automation tool to test and deploy each release. As a Principal Consultant at Declarative Systems, I’ve been recommending JFrog Artifactory to clients looking to bullet-proof their deployments since 2016. After considering different CI solutions, we found that JFrog Pipelines has the best integration with Artifactory which made choosing this platform a no-brainer. It also addresses our security and license compliance goals stretching years into the future on day one.

Designing your CI/CD Environment

One of the most important decisions you will need to make in designing your CI/CD pipeline is how to match your Continuous Integration environment to your local one. JFrog Pipelines is a DevOps CI/CD automation solution for building, testing and deploying software as part of your CI/CD pipeline. Pipelines includes native steps to get up and running quickly. For example, you could use DockerBuild to build a Docker image and then push to a registry with DockerPush:

Basic Docker Pipeline in JFrog Pipelines

If the native steps do everything you want, they can be assembled together like Lego bricks to make a pipeline and you’re done. Here’s how I used JFrog Pipelines to align our existing local environments with the CI delivery process.

My Build, My way

I needed more flexibility for using existing build scripts since these had already been created and I wanted a repeatable process between CI and workstation. To make my build scripts run in pipelines I needed a way to setup the containerized build environment so that I had:

Every tool out there has its own way of configuring Artifactory and while JFrog Pipelines has a Bash step that lets me do anything in Bash I know this approach would quickly result in a lot of hard to manage, completely custom build scripts.

We can do better than this.

JFrog Pipelines Extensions

JFrog Pipelines Extensions let me abstract away common tasks and re-use them across projects. They come in two flavors:

1. Resources
   Things you need to make your step work. They are a great place to do setup, such as configuring a tool to work with Artifactory. The details of resources are invisible when you view your pipeline and each resource name must be unique across builds.
2. Steps
   Discrete things you want to do as part of your build and are typically more constrained in how they work. Each step is clearly shown when you view your pipeline.

Resources or Steps?

Which one to use depends on what you are trying to achieve. Steps let you build a workflow that does one thing and does it well. Resources, on the other hand, are the right place to setup your environment so you can build software in Pipelines the same way you build locally.

The challenge: Setup JFrog Pipelines environment to replicate local build

Resources were the right way to set up the environment for my build.

Extensions Registry

I got all of my builds working in JFrog Pipelines just the way I wanted them to and I’ve shared my Extensions Registry on GitHub for the benefit of the DevOps community. This registry is evolving to focus on using resources instead of steps where possible to give developers the greatest flexibility. It was technically reviewed with Tal Yitzhak, a Solution Engineer from the JFrog DevOps Acceleration team. Here are some of the example repositories it includes (see links in the following section).

OCI Container images

Name: declarativesystems/ContainerEnv

Description:

• Pull images from artifactory
• Push images to artifactory
• Full control of image building

Tools configured:

• buildah
• podman

Custom image should provide:

• buildah
• podman

Node.js

Name: declarativesystems/NpmEnv
Description:

• Resolve dependencies from Artifactory
• Publish to Artifactory

Tools configured:

• npm
• yarn

Custom image should provide:

• Node.js
• npm
• yarn

Python

Name: declarativesystems/PythonEnv
Description:

• Resolve dependencies from Artifactory
• Publish to Artifactory

Tools configured:

• pip
• poetry

Custom image should provide:

• Python
• pip
• poetry

Summary

JFrog Pipelines extensions make it simple to create new pipelines by configuring your environment and then running your existing build scripts. To use these extensions within your own organization, fork the repository and you can start using the resources it provides in your own pipelines.

Leap forward with JFrog Pipelines. Start for free >