How to Accelerate Software Delivery with Hybrid Cloud CI/CD

Are you looking for solutions to deliver rapid application development and iterations? You’re not alone. To accomplish this, many organizations are embracing cloud native containers across multiple cloud providers. The reason? This strategy reduces the risk of vendor lock-in, and helps you scale the application infrastructure horizontally. 

In their recent swampUP 2021 talk “Going Serverless, with Artifactory and Containers on Cloud Run,” two Google cloud experts outlined best practices and provided advice on building a hybrid multi-cloud DevOps CI/CD solution.

In this blog post, we’ll describe a compelling use case detailed by Guillaume Laforge, Developer Advocate for Google Cloud Platform, and Ayrat Khayretdinov, Hybrid Cloud Specialist at Google Cloud and CNCF Ambassador, during this presentation.

Specifically, we’ll walk through a web storefront application, highlighting its microservices architecture and the cloud container service infrastructure, with an example of updating the web front end and product catalog to quickly address the surging demand for the latest product.

But first, let’s make sure we have a shared understanding of DevOps concepts. If you’re familiar with it just jump to the next section. Let’s start by defining what a ‘hybrid multi-cloud CI/CD solution’ breaks down into:

1. ‘Hybrid’ in this context refers to the application simultaneously residing on premises (eg. data center) and within a public cloud infrastructure (in this case, Docker containers.) 
2. ‘Multi-cloud’ refers to the application spanning multiple public cloud providers, which in this example includes: Google Cloud, Amazon Web Services and Microsoft Azure. 
3. ‘CI/CD’ is a common acronym in DevOps-oriented organizations, and translates to ‘Continuous Integration and Continuous Delivery’ of software. 
4. Continuous Integration (CI) is a DevOps best practice that automates code changes,  versioning, branch workflows and software bill of materials (SBOM) pipeline promotion criteria from a central repository for application teams to build, test and deploy updates and releases.
5. Continuous Deployment (CD) is a DevOps best practice that automates the software release process of new and verified builds from CI that pass the test criteria, and are stable for deployment to a production environment. 

Microservices Application Architecture

A modern software architecture such as microservices enables larger scale enterprise web applications and services by decoupling independent services into smaller (micro) tasks and code snippets. These loosely coupled microservices can share resources and exchange data using APIs to deliver the online shopping service, with the flexibility to be independently updated and iterated on. In this example we will highlight an update to the CITY BIKE product in the service catalog shown in green below.

This reference storefront application include web services and components that are typical for many digital businesses:

• The Front End user interface
• Product Catalog Service
• Checkout Service
• Payment Services
• Shipping Services

Customer Experience (CX)

Bikes have become super popular with newer electric versions, lighter materials and time to get away. As people look to biking as a fun and healthy activity in their life, the CITY BIKE category is receiving a lot of web page views. To improve the customer experience (CX)  and make it easier to purchase, the web storefront highlighted in green requires updates to the: image, description and pricing.

Guillaume and Ayrat demonstrated the workflow to update the web front end image in the online store application and push the update to the Docker container for the front end  with the latest model of the CITY BIKE. This example highlights an end-to-end workflow for developers to quickly respond to changing usage patterns.

Service Architecture

Behind every amazing front end is the magic of the back end. This service architecture interconnects three key components for a cloud-native CI/CD solution: Cloud Run for Anthos, the JFrog DevOps Platform and GitHub for source control. With this multi-cloud solution, developers are presented with a powerful set of DevOps tools to create and build CI/CD pipelines for and with different cloud providers. 

In this environment, everyone working on the release gets a common development environment with local caching for quick access to the latest version of the build for the next review and merge request. The data architecture includes on-premises data centers with Kubernetes Clusters interconnected with remote development teams and their resources.

Architecture Components

Anthos, Google Cloud’s application management platform, is built on a Google Kubernetes Engine (GKE) foundation for managing containers across the hybrid multi-cloud infrastructure. This  provides developers with a consistent development and operations experience for Google Cloud Platform (GCP), AWS or on-premises environments. 

Cloud Run is available as a fully managed Google Cloud service, and as Cloud Run for Anthos deployed into an Anthos GKE cluster. Cloud Run focuses on stateless HTTP-driven containers and abstracts away all infrastructure management, so you can focus on what matters most — building applications and services in your favorite language.

Knative easily plugs into existing build and CI/CD toolchains as the underlying platform to build, deploy, autoscale, and monitor workloads wherever they run best. 

JFrog Artifactory’s open approach to CI and CD integration provides a universal solution for managing and governing the software ’bill of materials’ (SBOM): binaries, software artifacts and dependencies for each release and distribution.  JFrog Artifactory supports all major package managers such as Docker, Maven, npm, PyPi, NuGet, Gradle, Go, and many more. It is also the leading Kubernetes registry to manage application packages, operating system component dependencies, open source libraries, Docker containers, and Helm charts with full visibility of all dependencies. 

JFrog Pipelines provides workflow automation of all key processes for your CI and CD pipelines. With JFrog Pipelines integrating with Jenkins, Artifactory supplies artifacts and resolves dependencies during the creation of the build, and it’s also a local repository for final build distribution. 

JFrog Xray is fully integrated with the JFrog DevOps Platform as your SCA Tools to scan and monitor the security and compliance of your binaries from the same pane of glass that you manage your binaries. 

CI/CD Workflow Automation

Software Bill of Materials (SBOM) – Source of Truth 

The concept of a source of truth for binaries, artifacts and dependencies is that to accelerate automation, repeatability of processes and workflows is key. If each branch is developing or testing on a different version, or if an external dependency introduces a vulnerability, then it’s mostly wasted effort. As a best practice, each developer should have access to the latest version of the build, and they should be scanned for vulnerabilities and license violations before being released to production.

With JFrog Artifactory as the source of truth, all project team members have  access, based on their permissions, to the most recent verified, scanned and traceable components for the release, regardless of their geographical location.

The JFrog Platform provides a complete solution for CI/CD (Continuous Integration/Continuous Delivery) workflows on Google Cloud with binary repository management, security vulnerability scanning and secure Software Distribution to end points.

Continuous Integration (CI)

To ensure that application changes are built and tested with the latest version of the entire codebase, and to avoid challenges from a large merge, the DevOps best practice of CI aims to validate incremental changes as they are made. As most defects are found after the code change is committed, fixing them quickly leads to better quality.

Here is a sample sequence of the CI workflow to produce a build with the CITY BIKE image changes to the front end application :

• The new image is committed to a repository, such as in GitHub for source code or Artifactory for binaries, images, artifacts and dependencies.
• GitHub notifies JFrog Pipelines through a webhook of the change.
• Pipelines commences automatic execution of the CI workflow:
  • Builds Docker image
  • Scans with Xray for vulnerabilities
  • And finally pushes docker image to JFrog Artifactory

To ease the process, the ‘Graph’ view of ‘My Pipelines’ provides a helpful visualization of the execution steps and their status, all from one web portal into the JFrog DevOps Platform.

• The workflow distributes the steps that create a build in a runtime container.
• All binaries produced by the steps are sourced from Artifactory repositories. This includes build information, or Software Bill of Materials (SBOM), and it’s promotion through the CI/D pipeline.

Continuous Delivery (CD)

In this example the CD workflow distributes the SBOM, which consists of the updated image tag of a new CITY BIKE, including it’s product description and pricing. The image tag is updated to the production Docker container, and Anthos application management completed build to automate the delivery and distribution. 

This CD workflow performs the following sequence for a containerized build:

• A CI workflow in Pipelines produces a new build (see below for successful completion).
• The CI workflow triggers execution of the CD workflow. In this case it would be the inclusion of a new CITY BIKE image to the product catalog repository that triggers the Pipelines workflow to promote the build to a production repository.
• Automatically update the production online web store with the new CITY BIKE image.

The first step is to build the jfrog.io/boutique/frontend Dockerfile;

The build is scanned by Xray to identify security vulnerabilities and compliance with license policies, and once the workflow is completed, the new style of JFrog CITY BIKE is available in the digital storefront for customers to view and purchase.

With this high-level walk through of the major components that make up a hybrid cloud CI/CD solution, DevOps-led organizations now have a reference solution for rapid application development to containerized infrastructure. 

Sign up for JFrog on Google Cloud to start for free. For more information, check out the blog post “5 Steps to Starting DevOps with a JFrog Free Subscription.”