Top DevOps Experts offer Key Insights at swampUP

With five keynotes and 15 breakout sessions in one day, there was no shortage of important industry knowledge and key insights from this year’s JFrog swampUP DevOps and DevSecOps user conference. Presenters discussed the role of DevOps at Netflix, how Fidelity migrated to the Cloud, the trend of shifting further left than left, and more.

In this post we highlight the three presentations below that challenged attendees to rethink the status quo and reassess their own DevOps and security practices.

• The Operationalism of DevSecOps by John Willis
• Supply Chain Robots, Electric Sheep, and SLSA by Brett Smith
• Secure Your DevOps Pipeline with New Security Tooling by Tracy Ragan

swampUP 2023 San Jose 2023 Highlights

The Operationalism of DevSecOps by John Willis

John Willis is a technologist, founder of DevOps, great friend of the swamp, and regular presenter at swampUP. His frank and conversational style always leaves you with some valuable nuggets of insight.

In this session on Operationalism, John shared how Dr. W. Edwards Deming would likely feel about some of the DevSecOps practices we hold dear.  John started by introducing us to Deming’s System of Profound Knowledge, which includes four elements, fourteen points of management and seven deadly diseases of management. The four elements of Profound Knowledge are:

1. A theory of Knowledge – How do we know what we believe we know?
2. A theory of Variation – How do we analyze and understand what we know?
3. A theory of Psychology – How do we account for human behavior?
4. An appreciation of Systems/Systems thinking – Are we seeing the bigger picture?

Understanding these four ways of seeing the world allows us to drive transformational change in processes.

John also discussed Deming’s “14 Points for Management”, with two that really stuck out as it relates to DevOps.

#9: Break down silos

In general, silos are bad – no surprise there. It’s up to all of us to break down walls that hamper the sharing of information. In March of 2017, when Equifax was breached via an Apache Struts vulnerability, part of the issue was that the CISO reported to the Head of Legal and didn’t communicate about the breach to the CIO or CEO.

#11: No quotas or numerical goals

Everyone loves DORA metrics. But how relevant are they to your organization? John raised the question – If organization A releases 10 times a day and organization B only releases once a week, does that mean Organization A is better than Organization B? What if Organization B’s business needs don’t warrant publishing more than once or twice a month?

This is where understanding the four elements can help us look at numbers or goals through the right lens. To further drive the point home, John brought up counting lines of code. How often are developers evaluated based on some mystical quantity of code? Isn’t this just an incentive for developers to write bloated code?

As John covered in his presentation, understanding Demming’s work can help any leader, manager or individual contributor adopt a way of thinking that can be applied to many areas of life and work. For more on this topic, check out John’s recent book, A Journey to Profound Knowledge.

Supply Chain Robots, Electric Sheep, and SLSA by Brett Smith

Brett Smith is a Principal Software Developer at the SAS Institute (An Analytics, AI/ML and ISV Company). Brett led the afternoon sessions with a look at different ways to secure the software supply chain (SSC), while trying to adhere to recent security standards and frameworks.

Brett spoke about securing the software supply chain, with specific reference to the recent Executive Order (EO) 14028, combined with the Supply Chain Levels for Software Artifacts (SLSA) framework introduced by Google in 2021. Both of these take significant steps towards ensuring the integrity and security of software development and deployment.

Brett outlined some of the key components of EO 14028 and SLSA, shedding light on their importance and the benefits they bring to the ever-evolving landscape of software security. One of his key takeaways for attendees was ‘Seperate & Protect Each Environment’ – which is part of the EO 14208 Executive Order. This means establishing clear boundaries between development, testing, staging, and production environments. By doing so, potential vulnerabilities can be identified and addressed before they make their way into other critical systems and ensure the integrity of each stage of the software development lifecycle.

Brett also touched on the importance of some of the other requirements of EO 14028 including:

• Secure & Hardened Endpoints – Implementing robust security measures on endpoints helps defend against threats like malware, phishing attacks and unauthorized access.
• Improved Executable Security – Implementing measures to ensure that only authorized and authenticated code can execute in a given environment reduces the risk of malicious code infiltrating the system.
• Least Privilege Access to Code – This ensures that only individuals or systems with a genuine need to access specific code or resources are granted permission, thereby minimizing the potential attack surface.
• Provenance Data & Software Release Integrity – This refers to the importance of maintaining accurate provenance data to track and verify the authenticity and integrity of each release and establishing trust in the software supply chain.

Supply Chain Levels for Software Artifacts or (SLSA)

Hopefully you are already aware of  the “Supply Chain Levels for Software Artifacts”, better known as  SLSA or “salsa”. Brett explained how SLSA can fit into the SSC security story as a framework designed to provide end-to-end integrity for the software supply chain, encompassing standards, attestations, and technical controls.  It also documents and ensures the integrity and security of software artifacts from development to deployment.

SLSA Build Levels

The SLSA framework defines build levels 0-3 that serve as benchmarks for software supply chain security. These levels range from basic integrity checks to advanced security measures, allowing organizations to tailor their security practices to their specific needs and risk profiles. He believes achieving SLSA level 3 is extremely difficult, and sees most companies being able to achieve somewhere around levels 1-2.

Software Supply Chain Inventory Management & GitOps

Effective inventory management within the SSC is essential for tracking and verifying the authenticity of software components. GitOps, a set of practices that leverage version control systems like Git, plays a crucial role in managing and automating the deployment of software components. This includes, storing of source code, documents and policies which further enhance security by offering an audit trail, continuous validation and guaranteed reviews of changes.

Shift Left – Automate Right

EO 14028 emphasizes the importance of shifting security to the early stages of the software development lifecycle. By “shifting left,” organizations can catch vulnerabilities and security flaws at an earlier stage, reducing costs and the likelihood of security incidents. Automation complements this approach by streamlining security processes and ensuring consistent, reliable security measures. He also emphasized the importance of automation, such as using robots, for securing source code through security scanning, building and packaging the release artifacts.

Supply Chain Attack Vectors & Attestations

Brett outlined many of the potential attack vectors you can face across the SSC. Understanding the potential attack vectors is crucial for developing effective security measures. One way in which you can mitigate those attacks is with automation of attestation, by recording and certifying events to validate the integrity of the artifacts. An attestation, as outlined in EO 14028, serves as proof of an event, helping to validate supply chain integrity, record critical events, and certify the integrity of software environments, materials, processes and artifacts.

In conclusion, here is a quote from Brett, midway through his presentation, about the best way to comply with EO 14028 and achieve the highest possible SLSA level:

“I’ve got some ideas on how you can solve [EO 14028 and SLSA compliance challenges] … one of them is to go buy the JFrog platform, move all of your stuff to it and sleep at night!”

Thanks for a great session at swampUP Brett.

Secure Your DevOps Pipeline with New Security Tooling by Tracy Ragan

Tracy Ragan, CEO of DeployHub & CDF Board Member, presented on the topic of securing the DevOps pipeline with a new generation of open-source security tools that every DevOps professional should consider as part of their security strategy.

Tracy identified five key phases of the DevOps Pipeline, and the tools to consider at each phase to build the basic security guardrails needed across the development-to-release lifecycle.

Phase 1: Code and Pre-Build

Critical security steps include code signing, scanning files for code weaknesses, and scanning the entire code base.

Tools to consider

• Git Commit Signing: Git GPG
• Git Commit Signatures: Git Signed-off-by
• Repo Scanning:
  • Git Dependabot
  • Git CodeQL
  • JFrog FrogBot

Phase 2:  Build

Actions include generating an image SBOM, image signing, and pre-package verification.

Tools to consider

• Image SBOM Tools: Docker BuildX
• Build Signing Tools: Sigstore Cosign
• Hosted Build Systems:
  • GitHub Actions
  • Tekton
  • Jenkins
  • Google Cloud Build
• SLSA – Standards for improving your build process:
  • Basic Security Steps
  • After the Build
  • Back to Source
  • Across the Chain

Phase 3: Post Build

If the build step in Phase 2 did not include creating an SBOM image, a post-build effort is needed to add security actions for generating an SBOM for the build.

Tools to consider

• Post Build SBOM Tools:
  • Syft
  • Microsoft SBOM tool
  • OpenSSF SPDX

Phase 4: Publish

This phase includes registering containers, collecting security evidence to show an organization’s security profile, and discovering CVEs.

Tools to consider

• Registries:
  • Artifact Hub
  • Docker Hub
  • Quay
• CVE Database:
  • OSV.Dev
  • GitHub Advisories
• Evidence Catalogs:
  • Ortelius

Phase 5: Pipeline Audit

Auditing the pipeline further hardens the application lifecycle process and has become an integral part of pipeline management.

Tools to consider

• Jenkins Audit Trail
• Tekton Chains

Lastly, if you feel the need to quantify your security results, Tracy noted the OpenSSF Security Scorecard check available at: https://securityscorecards.dev/

Kudos, shout-out, and thank you to Tracy for helping to make JFrog swampUp a huge success, plus congratulations on winning the Carl Quinn swampUp Speaker Award, for the second year in a row! See you at next year’s swampUp for possibly a three-peat?