Trusting your Software Supply Chain Security with DevOps Agility

Speaker 1:
Excellent. Well, hey everybody, welcome today to Trusting Your Software Supply Chain Security with DevOps Agility. The talk today is going to be encapsulating some of the great and exciting new features that we have to go ahead and show you some of the exciting stuff that we showed off recently at RSA in terms of our new advanced security features that we have and the product that will help you with things like expediting the ability to remediate issues so you can spend more time innovating and less time remediating. But instead of just talking about it, let’s go jump into it. And so my name, just so you’re aware, my name is William Manning. I am one of the solution architects and also the solution engineering manager here for the Americas. My team works with customer bases and does a lot of also public speaking and things like that around best practices, the best way to secure your companies so this way you don’t get any sort of issues around things like software supply chain security, which we’ll discuss today.

But also things such as managing your SDLC, your software development lifecycle, end to end DevOps, such as developer to device, code to cloud. You name it, we’re there. If you’re not familiar with JFrog, there’s plenty of that stuff out there. We’ve been around since 2008. We went public a couple of years ago or three years ago. We have almost 8,000 customers, 89% of the Fortune 100. We’re a CNA, so we’re CVE number authority and I thought I’d start right there by explaining that we’re giving back to the community and discovering new and exciting CVEs, which are terrible, but at the same time, relatively exciting. But why are we here? And the thing is, we see these headlines all the time. I give this talk a lot and I can’t stress the importance of software supply chain security because we’re inundated all the time.

Attacks are eminent, they are happening, they are consistent. This is not something new. This has been going on for a long time. As I mentioned, we are actually giving back to the community. And the thing is that when you are looking at software supply chain and security, this is an everyone issue. This is not just simply a security person’s issue. If you want the maximum ROI from this, your developers are your best defense on protecting your organization. And the thing is that we see these numbers all the time too. We see it on Twitter, we see it on Hacker News, we see it on Reddit, we see it on devops.com, all the places, all the usual suspects that things aren’t always constantly moving forward in terms of being terrible.

But there is ways to do this. I mean, your company is bleeding money and trying to remediate and fix these issues. And the thing is that software supply chain attacks, the longer they take to identify, the more it actually puts your company in jeopardy and the more money it could cost you in the long run. And the thing is is that it’s not that the amount has changed, which it has, but also too, the amount of time in which these exploits are happening has decreased. Think about that. In the past, there was a tremendous amount of these things happening, but they were slow growing.

There were things that were coming out every once in a while, you would discover them, and whatever. Now it’s actually shortening and it’s getting shorter. And the thing is that remediation is taking longer, which is insane. Now think about it, because when we talk about remediation today, it’s not usually just a point solution. I always jokingly call any of these issues that we’re going to discuss today, the pebble and the pond, just because it’s a little pebble, you put it in, you always see the concentric circles afterwards that can extend all the way across the lake and they increase in frequency. And the thing is is that as we’re going through these, most of these exploits, most of the things that are out there are happening before CVE is created. If you’re not familiar with the way CVEs are, all you need to be familiar with is kind of like the scientific method.

Somebody discovers something and hypothesizes, they test their hypothesis, they go ahead and they try to reproduce it. If it’s reproducible, they go ahead and submit some information for it to be investigated. Others pick up this investigation and look into it too. And then you go through and they come up and they say, here is the exploit. And once the exploit has been discovered, even before CVE is created, there’s some sort of peer review amongst other CNAs, a CVE number authority, like I said, Jfrog, we have a huge research team. By the way, just so you’re aware, you can always go to one of our great sites called research@jfrog.com. This is actually where our security team publishes a lot of this information, including things that are in progress of being evaluated.

Or recently we went in and we found some great information. We found some payload attacks around things like NuGet, and we actually produced some information around NuGet to say, hey, by the way, you’re exploitable too, and here’s what we found. And the thing is that the CVEs, like I said, once they go through the peer review and they get out, they’re already out there in the wild. Now, you have to go in and figure out are you being exploited by this? And also too, how can you remediate it or is it remediable? Can you actually fix this? And the thing is is that, just to give you an idea. In 2022, 21,000 CVEs were put into the registry. That’s insane. And it’s growing all the time. I mean, I think in our database alone, we have over 16,000 MPM violations. And actually MPM is one of the examples I’m going to use today.

I’m an MPM developer. I’ve been a developer for a long time. And the thing is too is as we go through this, this one quote I always need to keep in mind, and the thing is that every time you do a PIP install, go get, or mvn fetch, basically, like I said, it’s the equivalent of finding a USB key on the street and plugging into your production server. Are you going to go do that? Of course you’re not. But at the same time, when we see these scary stats, we see these things, the problem is is that how much do you actually put into it to say, we want to block our developers from bringing in this stuff, but at the same time we want them to have responsibility. But let’s face it, most developers do what they do the best. And I always say coming from being a developer, developers are artisans, code is their craft and their medium is the language they choose and the libraries they do.

But how do you ensure that the stuff they’re using is safe and not going to be detrimental to the company while still allowing them to operate without hindering their ability to deliver? These are real major concerns because yes, security is paramount. You need to address it. You have no choice. This could ruin your company in terms of reputation, in terms of revenue, you name it. And the thing is though is one of the things we’re going to address today is as we go, is the fact that 85 to 90% of your software is someone else’s software, the equivalent of that USB key that you found in the gutter and you brought inside to your data center. And the thing is that we scanned and we looked at the code bases out there and we looked at the dependency graphs of how software development is being done and we found out that 75% of these projects out there are using at least one major component that’s vulnerable.

That’s amazing. So what we’re going to talk today is about how do you go ahead and analyze all these things that you’re utilizing, but do it in such a way that it doesn’t hinder your ability to create code and progress as a company while still ensuring security. And the thing is is that, like I said, as we’re looking at these code bases, 49% contain at least one major vulnerability or 90% of the components you use out there, one of the things we’re ready to discuss today is one of my favorite topics is, and we’re going through this, it’s called operational risk, which means, and if I just kind of show you is we’re taking the time and effort to tell you, are the libraries using end of life? What’s the version age that you’re using? Are there new versions available? And what’s the health of the open source project while you’re doing this?

And the thing is is that in 2021, there was a 650% increase in supply chain attacks and it just went up from there. So susceptibility is something you have to deal with. This is not a nice to have. This is now you need to have this to protect the org that you have. So when we talk about our platform, if you’re familiar with it, we have artifactory the cornerstone of our technology. It’s the way you maintain and manage those third party transitive dependencies, that 85 to 90% that you rely on so well. It also gives you the ability to do this across multiple sites, and there’s plenty of talks on that. But it’s also an SDLC enablement tool, so it allows you to have your software development lifecycle by using our promotion API so that you can promote the binaries, the bills, the things you create, what matter most.

And there’s over 32 package types to support it. And the thing is is that we have our x-ray product. Now, people talk about security products that are out there and there are tons of security products out there that will detect CVEs. But today I’m going to show you a couple of things. So number one, our x-ray product is continuous. It’s always scanning, it’s symbiotic to artifactory itself. And the thing is that where we’re different is that not only have we detected the components, but since you’re using artifactory, you own the components so you can track and trace its usage. One of the things is that one of the things I’ll show you is that by doing this, I had a guy last year tell me after I taught him some best practices, he said to me, he goes, “Hey, you know what? My root cause analysis, whenever we had an issue before using Artifactory and x-ray used to take us days and weeks to come up with a report and attack.”

And in an industry where we have milliseconds to work with days and weeks, like he said, it’d take him days and weeks that that’s an eternity. And he says, “Now with the best practices you taught me about, I was able to knock that down to minutes and hours. We’re saving time, we’re saving effort, we’re making it so that companies can attack faster.” But the thing is also with x-ray is this allows you to get there closer. And since we are continuously scanning, because of the way we store the binaries and artifactory and the way artifactory and x-ray were designed together, when a new zero-day happens, we can tell you where and when it happened.

Then we have our distribution and our connect side. So distribution is a way to get your software out to where it needs to be, whether it’s a web service or devices or to your customers. Connect is our IOT platform for remote device updates. It’s a way to also set updates to the devices for remote diagnostics and management. And then our pipelines product, which actually has security built into it too with things like signed pipelines. So this way if you have a regulation where you need to understand how things are being built, it’s template-izable, it’s also secure. And on top of that, you can actually have accountability on it by creating a blockchain style ledger for everything that you build. But when you have this all together holistically, as you know, you can do everything from curation, curating those third party transitive dependencies, that 85 to 90% of the things you do so that when you are creating it, you can protect yourself from end to end.

We have ID plugins, which I’m going to show you today, which is the quickest way for your developers to evaluate whether or not they’re potentially putting a risk into your system. It also integrates into our bill processes that we integrate with all major bill systems out there, whether you’re using our pipelines product, Azure DevOps, GitLab, or Jenkins or any of those, you can integrate the security into it also. But we also give the ability for your developers to go ahead and do what they do and also still pre-evaluate the actual libraries they’re bringing in so you can actually stop it before it even starts. And on top of that, we also have things like software build and materials for regulation and regulatory purposes. And we also carry this all the way down to the actual side where we’re sending it out to the devices, production, or whatnot.

So the thing is, what I’m going to show you today is a couple of ways that we can save you money, time, and effort and give you the ability to address it quicker and more rapidly. This is a very short talk, so let’s go ahead. But the things we’re going to talk about today is software bill of materials. Things like license detection. Licenses are really a good way to go ahead and actually see if a binary is potentially threatening or malicious. We also have operational risk, like I said before, this is a key component, what I just talked about. How old, outdated are these libraries that you’re using? Also too, we can detect things like malicious packages and other vulnerabilities. With our advanced features, contextual analysis is one of the most exciting things I’ve seen in forever. I am so excited every time I show it and every person I’ve showed it to absolutely just looks at me and goes, oh my god, this will save me so much.

Secret detection, you’d be surprised. Secret detection is amazing. I actually recently downloaded a docker image. I looked at it, I found API keys, AWS certificates and stuff, and they were all buried in the root history. Everything else was wiped clean, but nobody bothered to go in and clean out the root history and it was all laid out. We also could look at things like misuse and server configurations with our… We now do infrastructure as code analysis. But you know what? Instead of just talking about it, let me show you an example. So the example in this case, one of the things I want to show you is, I’m going to show you something complicated. I love showing complicated things because complicated things are complicated. And the thing is that I’m going to show you today is I want to talk about a docker image. Okay? So in this case, our contextual analysis is evaluating a docker container.

You have an OS, you have a runtime, you have an application layer. And this actually is going to hold true when I talk about the next segment of this is how you track and trace this through. But let’s take a look at this, say docker app 150. So this is an application I built. It’s got 355 violations. It doesn’t have any malicious packages. We’ll talk about that in a minute. It has 288 possible vulnerabilities and also in this case it has six exposures. Then we do application and service level exposures. You’d be surprised, but let’s go take a look. And why is this important? Because immediately I’m going to go ahead and I’m going to go ahead and jump to the vulnerabilities because this is the part, whenever I show this to anybody, they go, this is absolutely amazing.

I ask every company the same thing. And we have 288 vulnerabilities here, 288. And the thing is that you as an organization, how many engineers, time, effort, delayed releases, things like that, is it going to take you just to simply just investigate these? Are you being threatened by this? Because let’s face it, most vulnerabilities, just so you understand, happen on a particular site function and these libraries that you’re using, these third party transitive dependencies… And by the way, most of the time it is not the implicit ones you state, it’s usually the indirect transitive dependencies that come along for the ride. The dependencies of the dependencies of the dependencies, you get it ad nauseum. But the thing is is that how do you go and evaluate this? It would take… Most companies are like, we’ll probably ignore it. We’ll probably just look at the highest ones and the other ones will kind of put it aside and it’s going to take a lot of time.

Contextual analysis is magic. This one will actually, I can tell you right now that of that 288 vulnerabilities, 74% of these are not applicable. Think about that. We just quickly within seconds just by simply resorting this list after the scan was complete, we’re actually able to go in and tell you with accuracy that this issue and we use a scanner, we actually tell you the magic on how we did it and what we looked for that we looked through this container and our research team with the information we provided to it and the automatic scanning behind the scenes determined that this issue is not applicable. You don’t have to worry about it. That means you can focus on the ones that matter. And when I say that, we also tell you where the actual real transitive issues are. So if I go click on this one, I’ll just go click on this one.

It says it’s applicable. We actually tell you it’s applicable and we go in and we show you where it is. So not only do we tell you, first of all, you have an issue, secondly, it’s applicable to you and here’s how you can fix it. And we actually provide you with amazing amount of detail. And let’s be honest, if you’ve ever worked with CVEs, and I’ll show you the actual CVE here, you can see that somebody got lazy and did a cut and paste of the summary into description and provides real no context behind it or information on how you can fix it. It just tells you it’s broken. Where we actually go in and provide you with all this level of detail and we even give you our justification on why we researched it. We are slowly building out in the massive amount of data that we’re providing for people to go ahead and fix this faster.

And we also tell you where it’s impacting. Now what about these others that say undetermined? And we’ll go look at some of the other features in this in a minute. What about undetermined, the ones that you need to go and investigate? Well, I mentioned we do an IDE plugin. I’m going to show you in this case, VS Code right now. So if you take a look here, you can see as a developer, I can see in this case this is MPM. This is a package like JSON, where I define my transitive dependencies implicitly. You can see where I can go in and see all the vulnerabilities just by simply hovering over them and by us flagging them. But it goes much further than that because we can also go in and show you all the other vulnerabilities that we have here.

And we can even go one step further because if I click on this CVE here, and if you look, there’s associated to a file. This is amazing. So I click on here, we bring up the information on the CVE for the developer, we tell them it’s applicable. We actually tell them, yeah, we found it in this file and we did, and here it is right here. And we show you the information on how you could fix it. And we provide amazing amounts of remediation data and also massive amounts of detail so you understand how it could be affected. But the thing is is on top of that, we also tell you, is this a direct transitive dependency? And I know that there’s some other ones in here. I can’t remember which ones they are right now. But the thing is… Oh, here you go.

Here’s an example of a library that I’m using and we tell you what level. Look at this. This is like a fifth or sixth level transitive dependency that has an issue. I mean, that’s incredible. But as a developer now, I can go ahead and address these so I’m not overwhelmed. I have the information, I don’t have to go much further. If I look at this one right here, I can click in here and see all the information on what I need to do. I can look at the public sources. Once again, I can see which library I’m using. In this case that’s going to go ahead and allow me to see what level it is and how it’s actually correlated. But we also bring the same information into the UI and we have that all right here. But you can go ahead and also say, I want to address these.

On top of that, we have things like software bill of materials where we actually show you. And just so you know, if you have a regulatory standard, we have SB, DX, and cycle and DX formats, and we can also produce things like reports on all the things that I’m showing you. We also, like I said, I could show you secrets. Here’s an example where I have a weak password and I can go ahead and address it and I know exactly where it is. I could also look at things like service level exposure. I don’t think I have any in here. Let’s see. No, but an application… Oh no, I have some service level exposures right here. Here’s an example. I’m not enforcing TLS. And on top of that, I have an example that I’m going to show you right here where I have another example where I’m actually detecting things like a malicious package.

Look at this. We’re actually going ahead and we’re telling you, hey, by the way, just so you know, actually seem to have a critical ecopower, it’s called, package. And we give you all the remediation information so you can attack it. And as I stated, we’re also going in and doing things such as infrastructure as code analysis. Here’s Terraform. We’re able to go in and let you know as a team, are you doing anything malicious? Now, this one’s hilarious by the way. If you follow the tutorial on using Terraform, one of the funny things about it is I will go through, you put all your ACLs in for how you want things actually delegated when you just deploy your service. And mine weren’t working because I could be an idiot and I forgot to remove authorization equals none. This let me know. This was a simple rookie error and it found it for me and embarrassed me beyond belief.

But the thing is is that we don’t just stop there. We have all this information we present. But the other part of this is is other best practices too. And one of the things I want to talk about, and this is the thing that really floors a lot of people when we discuss it, is how do you track and trace things? How do you debug? How do you go ahead and ensure that the things that you’re doing are correct? Here’s an example of a docker image again. I’m going to go click in here and I’m going to show you that every standard docker registry I show looks like this. And I ask the question all the time. I tell you, you know what? This image here is hosting a Node app and it’s hosting a Java app. And I say, please tell me what version is running here.

And this is a very standard scenario and everybody has the same answer. I’ll have to do a docker poll, a docker run, or I’ll check the manifest. But most of the time that’s not available. There’s a lot of different ways, but it’s going to take you time. And one of the ways that we tell people, that we tell companies when I talk to them is, I can show you in seconds. Here’s my Node front end, here’s my Java backend. There’s ways to go ahead and help remediate. So in other words, say you deploy bill number 152 under production, users are complaining and somebody, a release manager comes to you and says, that was terrible. What changed between this and the previous version? Age old question, what changed? Simple things. Well, if you’re using artifactory, x-ray, and best practices, I can go ahead and show you in seconds how something changed. Well, it looks like the Node front end and then the Java backend changed here.

But I can also go even one step further than that. I can even do complete divs of images. Did the artifacts change? Meaning that any of the application level stuff. Did any of the base OS or any of the base components change? Did I even change things in the way it’s running the environmental and system information on how it was constructed and what it’s running. But also being able to go right from here and being like, well, that’s fine, but let’s see, how long do we run the other one for, just so I understand? So we’re going to go from the application layer of the docker container to the application and we have all this generalized information. I won’t go into this right now, but if I go into here, I could show you, here’s the bill that produced it and I could show you every container that’s ever used it.

Think about this. In seconds, I was able to do this. This comes back to that question I had from the customer that one time it was like, look, guy, you know what? You actually knocked our root cause analysis report from days and weeks down to minutes and hours. But we can go one step further. Let’s go ahead and take a look at the bill. So we went from the docker image, the application layer of the docker image to the application, to the bill that produced set application. And I can show you here’s the tar.gz. We can see how it was produced. We can hook up Jira to it. So just to let you know, one of the great things is you can actually create Jira tickets from our x-ray products so they’re actionable.

Or if you want real live results, we even have things for like MS Teams and Slack where you can get instant notifications on a is something working. Here’s an example where looks like we were actually a security violation for a, oh, looks like actually in this case, this is actually a remote third party. This just happened now in one of my testing systems. But also too, not only that, but I can show you here’s all 453 transitive dependencies and I could select any of these and let’s just say this one right here. And if I go in, I could see once again all its information, when it was first brought in, how long it’s been around, does this one have any dependencies, things like that.

But check this out. I could show you, here’s all 105 bills that have ever use this. Think about that. I could show you in seconds the exact blast radius of everything you do. Because once again, it’s not just the one bill that I was using. As you can see here, there’s a lot of other bills that are actually utilizing the same library. I now know the effective blast radius throughout the entire product. So all I wanted to say is is that when you look at what we’re doing, don’t just look at it as a security solution that detects and actually shows you something, there are tons that do that.

This gives you 10 times more information. And the more information you have, the better you can actually go ahead and combat things. And that’s all baked into the product. Whether it’s our third party transitive dependency management, through our remote repositories, adding in x-ray as a layer of security to pre-evaluate and consistently evaluate. Remember, we’re constant. We’re always skin. You hear about CI, CD, well CS, continuous security. We’re constantly going and If there’s a new vulnerability, we notify you right away. And the ability for you to have all the lineage laid out in a way that’s super easy to find and actually correlate, so this way when you are attacking an issue or if you want to go and remediate, you can actually go ahead and see everything that’s affected or it’s affected over time or how long you’ve been affected by it.

Well, thank you for your time today. I hope this has been very insightful. Please, if you have any more questions, please let us know. Contact if you want a longer demonstration, always available for our team. There’s plenty of ways to do that. But remember, a lot of this responsibility is yours and your corporations. I know developers don’t want to hear it, but we do provide the tools to make your life much easier so you’re not being harassed by the security team. As security teams, this allows you to go ahead and do a lot more. Instead of investigating little details, this will allow you to actually go ahead and get bigger affected details. Thank you so much today and be safe, be wonderful, and be well.