Automatically Assess and Remediate the SolarWinds Hack

With software supply chain attacks on the rise, are you wondering how you can recover quickly from the recent SolarWinds breach at your company? Months after its discovery, the devastating SolarWinds hack remains a top concern for business, government and IT leaders. This destructive supply chain attack put the spotlight on software development security — a critical issue for the DevOps community. While new details about its scope and impact continue to surface, public and private sector researchers keep probing it, as key questions about the attack still hang in the air.

In this blog post, we’ll address three critical questions that are top of mind for CxOs, board members and IT chiefs about the SolarWinds breach, and explain how DevOps teams can answer them today with the JFrog DevOps Platform.

• Am I impacted by the breach?
• Where am I impacted?
• How can I remediate the breached libraries/dependencies?

A bit of background

Hackers breached the systems of SolarWinds, an IT monitoring and management vendor, and injected malware into the software build process of its Orion Platform. For several months in 2020, SolarWinds inadvertently shipped product updates with the vulnerability, which was designed to help hackers compromise customers’ Orion servers using a backdoor. 

An estimated 18,000 customers received tainted updates, and several dozen got breached, including high-profile multinationals and large U.S. federal government agencies. The SolarWinds breach is an example of a supply chain attack — an increasingly popular type of breach in which hackers hide malware in legitimate software that’s distributed through official, trusted channels from vendors to their customers.

Read on to learn how JFrog can help you quickly and precisely assess if you’re impacted by this hack and how to undo the damage.

Q1: Am I impacted by the SolarWinds breach?  

Generally, there are two methods to determine if you’re impacted by the breach, and both are extremely difficult to implement:

• Examine every system within your enterprise to see if you currently have or previously had Orion Platform versions 2019.4 HF 5, 2020.2 with no hotfix installed, or 2020.2 HF. 
• Monitor your organization’s current DNS queries and mine the historic ones to determine if any DNS queries to hostnames contain the domain “avsvmcloud[.]com”.

If one or both of these conditions are met, your enterprise is impacted by the breach. However,  you could have so many systems that checking each one could take years. Or you could have a size limitation on the DNS query logs, which would provide an incomplete history.  

With JFrog Artifactory, the JFrog Platform’s universal repository manager, you can determine if you’re impacted by the SolarWinds breach easily within seconds.

Artifactory is used extensively within enterprises as their artifact database. After acquiring a commercial software product or an Open Source project, an enterprise typically will store the acquired binary in a repository within Artifactory, where it can be versioned and managed. In the SolarWinds case, the three affected software binaries could be stored in a specific Generic Repository. Since Artifactory provides the single source of truth for all the binaries within your enterprise, the question of whether you’re impacted by this breach can be easily answered by a simple API call/query within Artifactory. 

The JFrog Platform Unified User Interface (GUI) is built on top of the JFrog Platform REST API. Below is a visual representation for this result. In this hypothetical case, the GUI provides you an interface to search for all the artifacts with the “orion*” pattern, and it looks like we are impacted by all the three infected SolarWinds libraries.

Furthermore, you have visibility into how extensively each library is used (number of downloads), and by which user. The JFrog Platform provides extensive logging, which has been integrated with widely used Log Analytics tools.

Q2: Where am I impacted?

There have been a ton of blogs written about this very topic and a lot of high-priced consultants have been hired to help you answer this question. However, there is no definite answer from the industry, yet. At best, the solution is consistent with mining a sea of logs, monitoring network traffic and the targeted software behavior. This approach is more art rather than science to say the least. More importantly, it is hardly practical, especially for large enterprises. 

Remember, all these efforts are just to fix one breach. The reality is that there could be multiple breaches going on at the same time, and the primitive, brute-force method is just not sustainable. 

JFrog Xray, the JFrog Platform’s software composition analysis (SCA) tool, gives you a precise and quick answer to this question, again with a simple API call/query.

Xray scans all of your artifacts recursively to create a binary interconnection database, also called a component graph, which maintains a holistic view of each artifact being used or deployed — whether it’s been deployed independently or as an included dependent of another deployed artifact. 

Below is a visual representation from the JFrog Platform UI. In this hypothetical case, we show a widely used, vulnerable, Maven Jackson Package “com.fasterxml.jackson.core: jackson-core:2.11.0”. 

The JFrog Platform introduces an “Ancestors” concept, which tracks all artifacts, including this specific Jackson version. The red rectangle provides exhaustive, nested interconnection information: This Jackson package is part of the Maven Spring Package, which is, in turn, part of “ui_server1.jar” — your own Build. This Build is, in turn, part of a specific Docker image layer, which is itself part of the “pet_clinic” Docker image. This Docker image is included in 3 different “pet_clinic” Release Bundles. A Release Bundle is a JFrog Distribution construct to secure the transition of your software package across networks, which we will explain in more detail later on. In the current context, the Release Bundle is equivalent to a fully contained package ready to be installed or deployed.

Q3: How do I remediate the breached libraries/dependencies?

Of the three questions, this is the easiest, most straightforward one to answer: Remediation involves rebuilding the impacted servers and starting with a fresh updated install of SolarWinds Orion. However, the million-dollar question is: Where do you get the list of all the affected servers?

Armed with the holistic knowledge from Xray’s binary interconnection database, you now know exactly which artifact is infected within your enterprise. To have full traceability of your runtime deployment, the JFrog Platform provides additional innovative technologies. JFrog Distribution’s Release Bundles, which are signed and immutable through GPG, make artifacts available just-in-time, and closest to your runtime environment across unsecured networks. 

The JFrog Platform provides end-to-end immutability to your binaries from the moment the binary is created or introduced to your enterprise environment. It also participates natively within the GitOps methodology, which provides the full traceability of every single change within your runtime environment in a version-controlled, immutable fashion within your Git/VCS. Combining the JFrog Platform’s end-to-end binary management capability with your GitOps practices, the breached SolarWinds servers can be remediated in an automated manner.

This diagram demonstrates how the JFrog Platform can seamlessly participate in the GitOps ecosystem — in this case, with Flux CD.

Conclusion

The SolarWinds incident taught us a lesson: enterprise software is never secure. Either intentionally or unintentionally, vulnerabilities are always part of the software lifecycle. If we accept the fact that enterprise software is never secure, the agility to remediate the affected software is the best security posture any enterprise should aspire for. 

Detecting, tracking and remediating the SolarWinds breach requires a centralized solution for enterprise binary lifecycle management that’s able to track each artifact and its dependencies within the entire organization. Without such a solution, you’re left with time-consuming, manual processes that don’t scale and aren’t precise, and ultimately fail to give you clear, definitive answers. Without clear visibility into this problem, your risk of getting breached heightens. 

As we’ve explained in this blog post, the JFrog Platform automates, streamlines and simplifies this process, giving you clarity into your status, along with actionable insights, and effective capabilities to identify the problem and solve it.

The JFrog Platform is the only enterprise binary lifecycle management solution that builds on top of a set of time-tested foundational technologies, including: 

• a Software Bill Of Materials (SBOM) with exhaustive metadata from Artifactory
• a component graph or binary interconnection database of your enterprise software ecosystem managed by Xray
• an immutable Release Bundle audit trail from JFrog Distribution
• immutable CI/CD workflows

Each binary artifact’s upload, download, inclusion, runtime deployment, retirement, archival and deletion are comprehensively recorded and more importantly, can be queried through an easy to use query language, REST API and user interface. 

In summary, the JFrog Platform gives you comprehensive capabilities to trace any binary artifact in its complete lifecycle context within your enterprise.

Want to know more? Please schedule a 1:1 Technical Session with JFrog.