The Top 6 CI/CD Best Practices

The most recent State of DevOps report from Google’s DevOps Research & Assessment (DORA) team found that “elite” performers recover from incidents 2,604 times faster and have a failure rate that is 7 times lower than “low” performers. In the world of DevOps continuous integration/continuous development (CI/CD), these statistics that show an extreme gap between those who do things well and those who don’t are common.

With that in mind, it is no wonder that following an introduction to DevOps CI/CD, the next thing DevOps pros want to know is how to be one of the elite performers. Fortunately, a big part of getting the upside of CI/CD is following the same best practices the elite performers do.

Here, we’ll take a look at 6 CI/CD best practices that can help you reap the benefits of automation and land among the “elite”.

1. Start where you are and iterate

One of the hardest parts of implementing DevOps CI/CD practices is knowing where to start. For teams developing modern applications, there is so much you could do it becomes intimidating. Trying to go from minimal automation to automation across the board is too drastic of a shift and can set your team up for failure.

Instead, identify business processes that are good candidates for automation and start there. By starting small, you can rack up quick wins and learn. As you gain experience with CI/CD and automation – and see the benefits in action – your organizational culture will also begin to shift to a DevOps mindset and scaling up your efforts will become easier. Good candidates to help you ease into CI/CD include:

Automating your build process
Automating unit tests
Automating functional tests

Of course, every team is different so be sure to look at your end-to-end development processes today and identify the right areas for your team. For example, maybe you already automate your builds and unit tests and your Quality Assurance (QA) team then runs manual tests. Converting some or all of those QA tests to run immediately after your builds might be an excellent candidate for your next step towards a fully automated CI/CD pipeline. Alternatively, maybe you’ve already reached the point of software continuous integration and can begin iterating towards continuous delivery.

2. Shift security left

Shifting security left means identifying security issues as early in the development process as possible and integrating security into your pipelines. In practical terms, shifting security left means following best practices such as:

Scanning dependencies for vulnerabilities (e.g. using Xray to analyze artifacts)
Integrating security scanning into integrated development environments (IDEs)
Automating security-related tests
Educating all members of your team on security principles
Implementing change management and compliance monitoring processes

3. Use asynchronous pipelines

Most teams have ambitions of a single all-in-one CI/CD pipeline that encompasses all their development, testing, and application delivery processes. In theory, that is a great idea. In practice, it usually leads to unnecessarily complex workflows and pipelines that aren’t as efficient or performant as they could be.

To avoid this problem, follow our third CI/CD best practice and use asynchronous pipelines and use multiple purpose-built pipelines. Using asynchronous pipelines simply means using multiple pipelines with each dedicated to a specific workflow.

As a simple example, many teams have long CI/CD pipelines with multiple testing and approval stages. The wait for approvals and test completion creates interdependencies and delays that can slow things down. Breaking the pipeline down into smaller individual pipelines (e.g. 1 build & unit tests, 1 for QA testing, and 1 for deployment) can remove unneeded workflow dependencies and improve performance.

By using multiple asynchronous pipelines, you eliminate unnecessary friction and complexity in your processes. Each purpose-built pipeline will do its job efficiently. Of course, you still need to be able to manage all these pipelines to ensure your processes work end-to-end, track the progress of workflows, and store records and artifacts. That’s where a purpose-built DevOps pipeline solution (e.g. JFrog Pipelines) — that enables you to build and manage a “pipeline of pipelines” — comes in.

User-added image

JFrog Pipelines is a DevOps pipeline solution that allows you to manage and orchestrate all your pipelines in one place.

4. Choose end-to-end solutions

Complexity is the enemy of efficiency. When you’re just getting started with a CI/CD strategy and realize how many tools are available to you, it can be easy to lose sight of this. However, as you progress in your DevOps journey, you will quickly realize that managing an environment that uses a patchwork of tools from dozens of vendors can bog down performance.

Of course, the reason there are so many popular CI/CD tools is that plenty of them do a few specific things well. The hard part is developing your strategy in a way that gets you the functions you need without a patchwork of dozens of tools to manage.

Choosing end-to-end solutions helps you get the best of both worlds: all the functionality without all the different vendors. For example, on the JFrog Platform, you can build, test, and deploy software, store artifacts, manage all your pipelines, implement security and compliance scans, and more.

However, while end-to-end solutions are ideal, most teams are already using multiple tools and a lift-and-shift to a single solution isn’t practical. If you find yourself in this situation, look for solutions that have a rich ecosystem of integrations. By integrating with your existing infrastructure, you can iterate towards using an end-to-end solution long-term.

5. Don’t forget about observability

Despite the common misconception: monitoring is NOT observability. Monitoring is something you do to your systems (e.g. regular polling of an API) while observability is a characteristic of a system. In simple terms, observability describes how well you can infer the health of a system given its outputs (e.g. syslog messages, responses to polls, etc.). If you do the act of monitoring well, you can improve the observability of your system.

With that out of the way, let’s discuss best practice number five. To ensure availability and help optimize performance, you need to design your pipelines with observability in mind. This means using solutions that proactively monitor the health of the components that make up your infrastructure and help you automate responses and understand root causes when issues arise.

6. Implement configuration as code

Configuration as code is the process of using managed configuration files in a repository in the same way source code is stored. Adopting configuration as code is a CI/CD best practice because it:

Enables automation: By storing configuration as code, you eliminate the need for repetitive and error-prone manual workflows, and enable automation and GitOps.
Improves reliability: Version control allows you to ensure the same configuration will be used across environments, leading to repeatable, predictable, and reliable performance.
Enforces standardization: With configuration as code, you can easily enforce standardization on caching policies, naming conventions and more.

By implementing configuration as code, you reduce the risk of human error and increase your ability to automate workflows. As a result, you can build more efficient and performant pipelines.

Final thoughts: getting started

Each of the best practices covered here can help you improve the overall performance of your pipelines, and that’s why they’re best practices. Pipeline performance is key to delivering business value. The more performant your pipelines, the more successful deployments you can fit into a day.

Of course, you won’t be able to implement every best practice at once, and that’s fine. We made “start where you are and iterate” best practice number one for a reason. Starting early is the best way to get where you want to be.

If you want to get your hands on a modern CI/CD platform, take the JFrog Platform for a spin with your own perpetually free hosted instance on your choice of public cloud.