Creating Custom VM Images

JFrog Pipelines Documentation

ft:sourceType
Paligo

A custom VM image enables you to use your own image as a node in Pipelines. Nodes launched using this new custom VM image would include all the customizations you made when you created the image.

Prerequisites

To perform the steps outlined in this section, you will need to have some experience with the following:

  • Building VM images

  • Hashicorp Packer or a cloud provider-specific VM image process

Building a Custom Pipelines Build Plane VM Image using Hashicorp Packer

A custom Pipelines build plane VM image can be built to include your own artifacts. In this example we'll reference Hashicorp packer and Ubuntu 20, but any VM image build process (AWS cloud formation, PowerShell, or others) can be used with any OS that we support.

To build the custom build plane VM image:

  1. Download the buildplane package (RPM for Centos or a tar file for Windows) to the machine that will be used to build the VM image.

    Note

    This API can be used by authenticated users only.

    Example

    curl --request GET  --url https://my-pipelines-api-url/v1/passthrough/artifacts/buildPlane-x86_64-Ubuntu_20.04.deb --output buildPlane-x86_64-Ubuntu_20.04.deb
    buildPlane package names:
    buildPlane-x86_64-Ubuntu_18.04
    buildPlane-x86_64-Ubuntu_20.04.deb
    buildPlane-x86_64-Centos_7.rpm
    buildPlane-x86_64-CentOS_8.rpm
    buildPlane-x86_64-WindowsServer_2019.tar.gz
  2. Update your packer (or cloud formation) template to:

    1. Copy the buildplane file to /tmp/

    2. Install it (using dpkg, yum, or untar, depending on the type)

    3. Run the boot.sh (or boot.ps1 if creating a W19 image). Path to this file is /var/opt/jfrog/pipelines/nodeInit/x86_64/<OS>/boot.sh

  3. Build your VM image.

  4. Use the resulting VM image id in the next step.

    This step should be performed with every new pipelines version.

Example 'provisioners' section of a terraform template file using the Ubuntu 20 buildplane

  "provisioners": [
    {
      "type": "file",
      "source": "buildPlane-x86_64-Ubuntu_20.04.deb",
      "destination": "/tmp/buildPlane-x86_64-Ubuntu_20.04.deb"
    },
    {
      "type": "shell",
      "inline": [
        "sudo dpkg -i /tmp/buildPlane-x86_64-Ubuntu_20.04.deb",
        "sudo /var/opt/jfrog/pipelines/nodeInit/x86_64/Ubuntu_20.04/boot.sh"
      ]
    }
Adding the VM Image to Pipelines using API

To add the VM image to Pipelines, use the POST /v1/buildPlaneImagesPipelines REST APIAPI.

Sample Request:

curl --location --request POST 'http://<pipelines_url>:8082/pipelines/api/v1/buildPlaneImages' \
--header 'Authorization: Bearer <token>' \
--header 'Content-Type: application/json' \
--data-raw '{
 "name": "eu-centrall-1 image for x86_64 Ubuntu_20.04",
 "masterIntegrationId": 46,
 "imageId": "ami-07f469736e6545c8a",
 "region": "eu-central-1",
 "isAvailable": true,
 "architecture": "x86_64",
 "operatingSystem": "Ubuntu_20.04"
}'

Sample Response:

  {
        "metadataPropertyBag": {},
        "id": 8,
        "name": "eu-centrall-1 image for x86_64 Ubuntu_20.04",
        "masterIntegrationId": 46,
        "imageId": "ami-07f469736e6545c8a",
        "region": "eu-central-1",
        "isAvailable": true,
        "architecture": "x86_64",
        "operatingSystem": "Ubuntu_20.04",
        "createdAt": "2021-01-06T00:00:00.000Z",
        "updatedAt": "2021-01-06T00:00:00.000Z"
  }
Adding the VM Image to Pipelines using UI

Perform the following steps to add an VM image to Pipelines using the UI:

  1. Go to AdministrationPipelinesNode Pools and click the Build Images tab.

  2. Click Add Build Image.

    In the resulting Create Build Image screen, provide the following information:

    Field

    Description

    Name

    An alphanumeric friendly name for the node pool

    Cloud Provider Type

    AWS, Azure or GCP

    Image ID

    VM image id. In AWS, this would be the AMI id

    Region

    Cloud provider region that the VM image is available in

    Architecture

    x86_64 architecture

    Operating System

    An OS we support

    createBuildImage_24nov21.png
  3. Click Save.

    The newly added build image appears in the list.

    buildImage_24nov21.png
  4. Now create a node pool using the above buildplane image Id. The build image you created is available for selection in the Build Image drop-down. For more information, see Managing Pipelines Node Pools.

    build_image_23sep2021.png