Open source software supply chain management and security in the wake of SolarWinds
Software supply chain attacks are rising in both open source and in proprietary software.
Tidelift CEO Donald Fischer will brief application development teams on the state of software supply chain security, including an overview of common vulnerabilities and an analysis of new government policies that impact the software supply chain.
He’ll share best practices for supply chain management that can help development teams move fast and stay safe.
Video Transcript
Hi, I’m Donald Fisher, co-founder and CEO of Tidelift and I’m excited to be here today to speak with you about open source software supply chain management and security in the wake of solarwinds. This is a pre-recorded talk, but I will be available on the day of the event to answer any questions that you have in the chat.
So the recent high profile solarwinds breach is just the latest example of vulnerabilities in software supply chains.
These vulnerabilities impact both commercial software and open source software.
We’re all familiar with some of the famous historical breaches such as the Equifax breach, but more recently, there’s been an escalating number of these attacks.
For example, a new type of supply chain attack that hit both Microsoft,
Apple as well as 33 other companies.
This has gotten so bad that CSO magazine recently called it a perfect storm.
And this has also become an urgent government policy priority in a number of jurisdictions.
A number of weeks ago, the White House in the United States issued an executive order directing a long list of agencies to conduct a supply chain review, including risks posed by supply chains reliance on digital products that may be vulnerable to failures or exploitation.
More recently, just in the last couple of days, Jeff Green acting Senior Director for cybersecurity at the National Security Council said, we’re going to need all developers who are selling software to the government to implement more rigorous and predictable mechanisms to ensure their products and their software behave both as intended and designed.
Really whether or not your organization is doing business with the US federal government or not, it’s time for every organization to get serious and get its house in order with respect to its software supply chain.
And because almost every application that’s built today includes open source software in its code base, increasingly, software supply chain security is the same or includes open source software supply chain security.
Unfortunately, most organizations don’t currently have a great handle, even on what open source they’re using, much less its security status, especially when you take into account not just the direct dependencies that your applications build on but the hundreds and often thousands of indirect dependencies that get pulled in from open source community package managers.
Now, if you look at a typical application, often 70% or more of the code that comprises it will be third party open source code.
And while of course, it’s your application development team’s job at your organization, to create the business logic of your application, implement the user interface and sort of the business processes that you’re there to create.
And oftentimes, especially these days, these applications will be deployed on managed services platforms, whether that’s a third party cloud provider or an internal managed service at your organization.
Somehow for most of the application, that 70% of open source that comes from third parties through these community package managers, it’s kind of been nobody’s job to actively maintain that.
And that can become a problem if we don’t take that seriously.
Open source is really amazing, it’s incredible that we’ve been able to create this rich, technical and intellectual commons that all of us can build on in our organizations and individually.
But there are also some disturbing truths about open source maintenance.
The vast majority of open source projects that are used in application development are basically driven by a best efforts volunteer effort, you know, folks
may be working on these, oftentimes related to their day job or as a side effect of their primary work.
But those open source packages will be maintained to varying standards, even in important areas, such as security.
That creates challenges for organizations that rely on this,
you know, patrolling all of that territory can be very time consuming and leaving it unpatrolled can leave you vulnerable to software supply chain attacks, like those that we’ve discussed.
In fact, for organizations that are building with open source, which is most software development teams today, it’s hard to answer even basic questions about the third party open source that they’re using.
And that can just have the impact of slowing down development teams.
If your organization isn’t able to easily answer questions like, is this package, okay to use?
Has anyone looked at the license terms on this open source package and confirm that they work for our business?
And most importantly, is this package currently secure and actively maintained?
And what happens in the future if there’s a new security vulnerability?
Again, open source software is just software, there’s going to be new issues that come up, we just have to have a plan and a process and tools in place to respond to those well.
So when we talk to organizations that build with open source about what challenges they face from a management perspective, it typically comes down to these big three.
Maintenance – making sure that somebody is keeping that software in good working order going forward.
Security – ensuring that there’s no known security vulnerabilities and that when vulnerabilities become known, there’s a path forward that resolves that vulnerability across the board and sort of plugs the gap.
And licensing, of course. Open source software can be fractally complex
from a licensing perspective. And so, it requires some process and discipline there.
But even more challenging is oftentimes the license status of an individual open source project you might want to use might be ambiguous. You know, it’s sort of unclear whether there’s even clean data to start with.
So all of those can be, you know, significant challenges for an organization to deal with in an ad hoc manner.
And when we talk to teams about how they’re doing, when we ask them, how confident are you that the open source components in use today at your organization are up to date, secure and well maintained?
We typically hear that organizations don’t feel they’ve really got control of this.
39% of larger organizations that we asked told us they were not very or not at all confident that they have their open source components, up to date, secure and well maintained.
Only 16% told us that they were extremely confident.
And when we ask organizations to investigate how they’re currently tackling this problem today, typically we find that they fall into one or the other of two extreme scenarios.
One is the distributed approach or the move fast crowd.
This is sort of the, you know, startup mentality move fast and break things.
Organizations that are pursuing this approach, they typically have very few barriers for application development teams to adopt a new open source component.
The philosophy is sort of, you know, let people use what they need, don’t block
the developers let them build quickly.
Of course, that’s great from a developer velocity perspective but it has a significant downside that it can create maintenance, security and licensing headaches down the road, even existential problems when it comes to something like security.
Other organizations take a more conservative centralized approach.
This is what we call the stay safe crowd.
Organizations adopting this approach, they typically have some kind of central review process, maybe it’s a implemented through a ticketing system, or just plain
old email or spreadsheets or Wiki pages.
And developers in this style are required to solicit approval for a new license, or even just an individual release of a package before using it.
That has the upside that it introduces some processes and controls into the situation, but has a significant downside in that it becomes very, very onerous for application development teams to contend with that,
it slows them down and, frankly, we often see that in the real world, it causes development teams to route around to this control point.
So why can’t we have the best of both worlds? Why can’t we both move fast and stay safe when using open source?
We think that there is a way to approach this and this is where Tidelift has been focused is building better tools and processes and a new way of tackling this problem that gives you the best of both worlds.
Now, our approach is inspired by the way that we’ve seen some of the best tech companies in the business approaching their use of open source.
If you look at organizations like Google, LinkedIn, Netflix,
of course, they’re all predominantly built out of open source as well, just like your organization and they’ve put in place a series of processes to ensure that the open source that they rely on is well managed.
We find that they typically all take a pretty similar approach, which is to create a centralized catalog of pre-vetted, known good open source package releases.
Some of these organizations call this the paved path in the sense that it’s an easy road to go down where application development teams internally, if they’re using components from that centralized pre-vetted catalog, they know that it’s going to meet the requirements of their organization from a security and licensing and maintenance perspective.
So by having a paved path, that means they can go faster, they don’t have to be bushwhacking through the weeds in the trees, they can kind of move quickly, but know that they’re also not going to, you know, run into a ditch or fall off the road.
Only some of these largest organizations have really pursued this approach to its complete fruition because frankly, it’s an expensive approach to take.
It requires having not just the tooling and the process, but a team to do the active management, the investigation and the active management of every new release of every new component that’s used in the organization.
It’s a lot of work to get done.
So how can your organization learn from these approaches taken by these large tech companies to create centralized catalogs of known good open source?
We recommend that about it in terms of the analogy of a jet stream for your developers.
So if you think about, you know, planes flying across the country or internationally, there’s a huge advantage to getting into a jet stream going with the airflow, if you will, it makes the journey a lot more efficient and faster.
So if you think about some of those legacy traditional stay safe approaches that we talked about, those are really processes that while attempting to, you know, ensure the safety and security of organizations, they really create a headwind, either through bureaucratic red tape style of governance, or by putting the enforcement’s work back on application development teams by, say, running a scanner tool that has lots of false positives that that team needs to wade through.
What you really want to do is move into a situation where you’re providing your application development teams with a tailwind where they have a
set of open source components where they don’t really have to worry about all of this research and figuring out, you know, does it meet the standards of our organization, what’s going to happen to it in the future.
They’re reassured, and they can pull from that catalog of pre-approved components, and just go.
And it also requires having some ability to extend that catalog in the situations where maybe there’s a component to do some unique functionality that needs to get added to the set, that’s not already in there.
It can’t be an onerous process, and it can’t block application development.
So when we see organizations undertaking this journey to create a jet stream for their application development teams, we typically see them go through a number of stages of that journey.
The first stage of that journey is to inventory what’s already being used.
So let’s build a list of the open source components that are being used in application development across the organization, including all of the indirect and transitive dependencies, and create a build of materials for the open source that’s in use today.
The next step of that journey is to start contending with what are the standards and policies that make sense for your organization related to those open source components and future open source components that you want to use.
What open source licenses make sense for your organization’s business model or deployment scenarios?
What security standards do you want to or need to enforce?
Codifying those into a set of rules that can be automatically applied against your software development process.
And then what are the components that make sense for your organization to be part of your paved path, it could be a very broad set of open source components across many different programming language, ecosystems, for example.
Or maybe you want to also enforce some technological standards to focus your application development teams to use certain development tools or platforms.
Once you have that in place, in this build phase, you can start paying down some of the technical debt that your organization’s probably already accumulated by aligning your usage of open source towards these open source components that comply with these defined standards.
Once you’ve done that, and you have this sort of platform, a well functioning process and some tooling in place, you can accelerate your management of open source across your organization, bring more development teams, more additional open source components under that umbrella.
And finally, what we find is that organizations end up transformed into this mode where open source has become a competitive advantage to the organization as a whole, allowing development teams to move faster because they have this tailwind.
Now, at Tidelift, our commercial service helps your organization get this done.
Our service is called the Tidelift subscription.
It is a way for your organization to better manage your open source supply chain by reducing the complexity of going along this journey.
That allows you to accomplish a number of bottom line business goals.
One is cutting costs, because you don’t have to take on all of this burden yourself to root cause and investigate and govern all of this open source.
We do a lot of the work for you and feed it to you through the tools that we provide.
It allows your teams to move faster, so you’re building and shipping applications more quickly and you’re making sure that you’re mitigating the risk from your open source supply chain.
Now, the way the tide lifts subscription works is pretty unique.
So there’s three components that come together to comprise the solution.
The first of those is a set of tools.
It’s a multi tenant SaaS application that connects to your software development lifecycle and works together with tools like the JFrog platform which I’ll talk about shortly, and basically what this does is it makes it easy for your application development teams to always ensure that the components that they’re building with in any given application, comply with your organization’s policies and standards.
Now to, again, offload a lot of that work from your team having to do it all yourself, Tidelift is working upstream of your organization to research and respond to issues related to third party open source.
We currently cover thousands of open source components that are frequently used
in enterprise application development and we organize those into tight lipped managed catalogs of open source components and specific version releases of those components that we have verified, meet certain security, licensing and maintenance standards.
Those types of managed catalogs cover a whole bunch of different application development ecosystems, including JavaScript, Java, PHP, Ruby, Python, dotnet, rust and go.
Now the most unique part of the type of solution are the maintainers.
So Tidelift has a very, very different business model than other commercial open source companies that have full time internal engineers and developers who are doing all of the commercialization work to ensure that software meets the standards.
The way that Tidelift approaches this problem is, we recognize that the creation and maintenance of these thousands of open source components that we rely on is pretty distributed, right? There’s all these folks that are doing that across the internet.
So we work with a huge variety of these open source maintainers.
We allow them to partner with Tidelift to ensure that their software meets these defined security, licensing and maintenance standards, and receive a share of income for doing so.
So we turn a lot of folks who might have been doing this on an ad hoc basis side of open source, spirit of goodwill, we allow them to do it for all of those reasons and we give them a good excuse to take it to the next level, make sure it’s all fully enterprise grade.
And for doing that work, because they’re creating value for organizations that rely on that software, they get compensation through the Tidelift platform, a really unique way of solving this problem.
Now I want to turn to Tidelift’s joint solution with JFrog, and give you a sense for how that works.
So Tidelift and JFrog have been working together for almost two years, to make sure that the title of solution compare very naturally with JFrog’s artifactory platform.
And when these two components come together, it gives development teams the ability to take advantage of enterprise grade open source technologies at scale, but also safely so they get the best of both worlds, they can move fast, and stay safe.
Now here’s how Artifactory and Tidelift come together.
So the JFrog Artifactory platform serves its traditional role as the database of DevOps, the single source of truth within the organization for its binaries, and cloud native artifacts throughout its software development pipeline.
Now, tide lift complements that as the single source of truth for known good proactively maintained third party open source components that are part of the organization’s, or our inputs to the organization’s cloud native artifacts and binaries.
Your organization is responsible for the code that you write, think back to that diagram I showed with the 20%, you know, that’s your business logic.
Tidelift and our network of maintainers steps in to cover the 70% of third party open source that comprises a typical application.
And that means that your organization is covered top to bottom, ensuring that your
software meets these enterprise standards.
Here’s a quick preview of how you can integrate that into your JFrog artifactory platform workflows.
Tidelift is working through active management to ensure that these catalogs of thousands of open source application development components meet defined security licensing and maintenance standards.
You can create a custom catalog using Tidelift’s tools that implements whatever your organization’s specific policies are, in a sense, you’re kind of filtering the master Tidelift catalog based on verified clean data, you can say, well, our organization only approves these specific open source licenses or these licenses in this deployment context.
So you can customize the feed.
And then once you have that list of open source packages and releases, you can plug that to a couple of places.
One of those can be an artifactory repository that’s for your production applications.
So that has only application package releases that have been formally approved, meet all of the standards have gone through any additional review internally that you want to put in place.
And many organizations we find also have a second artifactory repository that’s sort of the development repository feeding developer desks.
That repository can include open source package releases that have been requested to be added to the master catalog to your organization’s custom Tidelift’s catalog.
That means that your application development teams again, are not blocked, they can move forward, but it also makes sure that only the packages that have been formally officially approved and demonstrated to be in compliance with your organization standards ever go to production.
So again, the best of both worlds.
Move fast in development, and stay safe in production.
Now, it’s actually pretty simple when it all comes together.
In fact, it’s so simple, we wrote a children’s book about it.
And I encourage you to check it out.
Here’s the URL here or just go ahead and do a search for cooking with Tidelift and JFrog.
It’s a pretty fun, different way of thinking about things, you know, it’s a great way to get familiar with the concepts and potentially entertain some young folks in your life at the same time.
With that, I’ll say thank you and again, pre recorded talk but I’m available to answer questions on the day of the event in the chat.
Thank you.