CTO Corner with Yoav Landman, JFrog Episode 1: Build Info

Courtney Gold: Good morning, good afternoon and good evening, everyone, my name is Courtney gold, I will just get a quick intro to our CTO corner series today. I just want to do a couple housekeeping items before we kick everything off just as a reminder, this is being recorded, so we will send this out post event. And if you have any questions, please, please don’t hesitate to ask them in the q&a chat box right below you, but other than that I will kick it off to our wonderful speakers, we have today, thank you very much, everyone, thank you for joining.

Moran Ashkenazi: Hi. Let’s introduce ourselves. Okay hi everyone so glad to be here with you, my name is Moran Ashkenazi and I am JFrog CSO (Chief Security Officer) and VP Security Engineering.

Eyal Ben Moshe: I’m Eyal Ben Moshe and I run the system engineering in R & D at JFrog. Happy to be here.

Yoav Landman: Yoav Landman CTO and co-founder of JFrog excited to run this webinar with you.

Moran Ashkenazi: Sounds great, so let’s start from the beginning, the real beginning. What is Build Info and when did it all start? 

Yoav Landman: So today we’re speaking about a technology that is almost ancient in the in our domain. So, Build Info is actually a very, very cool technology, but it preempted it’s time it started there as early as 2011 actually. We started with the with the implementation in Java, with the idea of being able to record everything that goes in your build in terms of the modules created the dependencies all kind of environment variables and properties and so on. 

The initial goal was very simple, it was in order to save the developer, that is running a build, this manual chore of looking up all the dependencies and creating a bill of material that he can go to legal with us. Because we found ourselves doing it and customers were asking for a solution. So, we developed this technology initiative for Java, implemented in Java. And along the years it went through a couple of incarnations, we re-implemented it in Go, we expanded the supported technologies. We’ll show some examples and some demos of how it’s integrated in Artifactory.

Essentially, the technology we kind of joke and call it the best kept non secret of JFrog because it was always open, always under an open source license, and we are making it more approachable today as we show you in a moment. So this is Build Info. 

Moran Ashkenazi: 2011, it’s like 11 years ago, it’s amazing. What does the Build Info schema look like? What kind of information can I get out of it? Properties? What kind of properties does it contain? 

Yoav Landman: So I kind of touched on it, but let me just share my screen for a moment. And show you the GitHub page. 

So, we can look at the schema. Basically it assumes a build and then there are the build properties, environment variables are going to be here too, and you can filter out secrets, there’s the name of the build, the number, the type, the actual build agent. 

Moran Ashkenazi: Can I see the dependencies?

Yoav Landman: Yeah, sure. Let’s skip over to the part, so you can see the version control system. And then this is the real meaty stuff where you have all the modules and all the properties about them, and all the artifacts created with the checksums and all the dependencies. We’ll show you some real world examples. 

As part of the dependencies, you also get a list of how the dependency came about, who requested it as a dependency. Transitive dependencies, you can also build the actual graph. Let me just scroll down real quick. 

Here you can see some real example that was generated for Go, with all the go modules their sums and all the modules that created the dependencies of this module. So, this is the module, these are the dependencies of the module and you can see how they came about.

Moran Ashkenazi: Actually, this is the SBOM, software bill of materials. An inventory of the software components and dependencies. I remember that almost a year ago, the US government published the executive order on improving national cybersecurity. Joe Biden actually mentioned that he wanted the United States Government, just in 2021, to set up an industry standard to protect it from the supply chain attacks. It touched on each and every one of our developers around the world exactly three weeks ago with the Log4j. So my question is, as a developer, why is it important to me, what does this SBOM give me, why should I use it?

Yoav Landman: Sure, that’s a very good question.

So as a developer, you want to be able to know what materials went into your build. Essentially, those materials that went into your build, all the third party dependencies and every package that you have created and assembled into your final product, you want to know how it came about, you want to know what’s the status of the system at the time of the build was, and you want to be able to record it as close to reality. It has to match the conditions that were there at the time of the build. I can speak about it in a moment, but it’s not always that easy to reconstruct this information. 

Now, when you have this artifact, you can think about it as a binary artifact. This SBOM which correlates to a build has a name, has a version, has a check some, and this is how we manage it also you know in Artifactory. This is something that you can sign this is something that you can use for provenance. If you have multiple builds that make up your end product, you can bundle those build information together, even if they come from different technologies, and that gives you, as a developer or as a security person or somebody who just wants to be able to go back in time and see exactly what went into your product, it gives you this this proof. And also allows you to verify that what ended up in your final distribution, actually came from that build. 

Moran Ashkenazi: So the responsibility, aware of the dependencies, and even you know from a legal perspective understand what you actually have.

Yoav Landman: Right. It’s a very, very basic building block, but if you assemble it, and if you I mean in the JFrog product we make we add value on top of it, but you don’t have to use it as part of any JFrog technology it’s an open technology that you can construct this provenance in this proof of origin completely in separation just by attaching it to any build.

Moran Ashkenazi: Yeah, we’re going to see it soon in the DEMO. Thanks. In comparison to other tools with trying to create the SBOM from the dependency scanning what is the, why is the Build Info so unique?

Yoav Landman: Yeah, so we are not married to any specific format, I mean Build Info is just a format. When it started it was very unique I mean and I don’t think anyone was capturing this kind of graph and this kind of information about the build. But you can also export Build Info to SPDX or CycloneDX and other formats. What’s really unique about it is that it tries to capture this information as part of the build. It depends on the technology, but if you think about it, when you go to any bill that you can think that you can just take the dependency declaration, so a log file or a POM xml or a go mode, and you can reconstruct the information about the builds from that file. It’s virtually impossible to guarantee it’s one to one mapping between this declarative file and the actual result of the bid. Why? because there is a dependency, there is the dynamicity that is always going to prevail in the bill. And this dynamicity can be version ranges, it can be environment variables or system properties that impact how your build is going to run. It can even access remote repositories that may be different between your CI environment and the developer environment in the production environment. And it can be even the tool that changed some logic, some algorithm in the conflict resolution for dependencies.

So that’s part of the real world problems that are happening, it’s part of the dynamicity. And therefore, you cannot just look at the origin and reconstruct back in time the same bill. Also, if you try to repeat it, if you will have different checks arms, maybe different dependency dates that are injected into your bill. So, it’s very important to accurately capture all this information.

Moran Ashkenazi: wow amazing! Eyal, which technologies are supported today?

Eyal Ben Moshe: yeah so we try to support every popular technology. So right now we’re supporting npm, Maven, Gradle, Docker, nuget, .net core, peep and we are constantly expanding our support for other package managers.

Moran Ashkenazi: How are you following new technologies?

Eyal Ben Moshe: Yeah, so practically every technology which becomes popular is being prioritized and we’re adding Build Info support for it. You know we’re integrating right into the technology right into the package manager and, as Yoav mentioned we are practically capturing everything that happens during the build. And in order to do that, we are tightly integrated into those package managers, we need to know and record exactly what they’re doing and what they’re downloading and everything that happens around. 

So yeah so definitely the aim is to support everything eventually.

Moran Ashkenazi: Quite amazing that it has been here with us there for 11 years oh. My question is what has been changed, why does it appear again, it’s always been OSS, so what’s the news, what’s coming? 

Yoav Landman: So firstly it started 11 years ago, which is a long time ago, but it went through many versions until it reached this level of maturity. And it’s battle proofed, it’s being used internally in JFrog and by many of the JFrog customers to get a record for all the builds in a fully transparent way.

And, in general, we also have the JFrog CLI, which is using Build Info, it’s like the Swiss army knife of JFrog for integrating with CI servers and integrating with the REST API of all our products. And what we have done this time is we decided to make the technology itself the extraction of the Build Info more approachable and more visible contributions. 

So what we have done is we took all the extractables, we pulled them out from the JFrog CLI as a completely independent Go project. And we run we’re starting to move we’re in the middle of the process we’re starting to move all the extractors for all the technologies into separate go modules that you can use either by invoking them directly for a minimal tool that we will show in a second, which the CLI is using also, all through an API.

So this is the big change just making these extractors available for anyone to use to export this format from any type of build and any technology.

Moran Ashkenazi: Amazing support for the Community. So we talked about the CLI, I think I’m ready, yeah, let’s see the DEMO.

Eyal Ben Moshe: Sure, so let me share my screen and show you a demo. Yeah I believe that you can see my screen right now so let’s. Start yeah so what you’re seeing right now is the new website that we created, that describes the Build Info and basically touches the main points around it. 

This is the Build Info repository. This is where we’ve been extracting, as Yoav mentioned the actual extractor logic, so you can actually go and download the CLI directly, the CLI binary, and in a minute we’re going to see how it’s being used. 

And if you wish, you can also use this as a library. So it’s written in Go. So here we are putting your for you the actual API’s that you can integrate with and just create the Build Info and use it.

So let me show you how it’s done. So here, I have already downloaded the Build Info CLI. And here I’m going to show you how it can be used.

This is a Maven project for example. Okay, this is actually the Jenkins Artifactory plugin project, so what I’m gonna do is I’m going to create a Build Info for it so I’ll just run a bi and then mvn.

And actually what is happening right now there’s a build running, okay so there’s a real build because the Build Info as we said records everything that happened during the build and then the build finished and then we can see right here the real Build Info that has been created. 

You can see here right now I’m actually inside, towards the end of the files, that you cannot see that section, but I’m telling you this is the dependency section, and you can see here, for example, the idea of the dependency. And you can see, the type and the scope of course and also, we can see how this build ended up downloading these dependencies because it was requested by those two dependencies.

So just to give you a sense how easy it is with this tool to create Build Info, I can do the same for Go, so this is our JFrog CLI which is written in Go and I can just run similarly bi and then go and then again it will just build my code what and here.

Yeah, it’s fast, because you know, most of the dependencies have already cashed.

And here you can see it’s the exact same format, the idea just in this case it’s these are Go dependencies and something else that we’ve added, which is nice. We are actually supporting exporting or converting the Build Info into a CycloneDX SBOM format, so I can just add minus minus format, and I can either choose xml or json and here you see there you go, you have it, the exact same data and the SBOM format, you see the dependencies and if we go up we’ll see other data, the other components and I can do, of course, the same for the json format of the CycloneDX and there you go.

This is the Build Info, this is actually the essence of or the core of you know, the actual code that generates the Build Info.

And we are also using Build Info inside our own products. So as Yoav mentioned, Build Info started as something that we needed and Artifactory and some of our other products are using building for Xray as well. 

Just to give you an idea of actually what we are doing, or what our products are doing with the Build Info, so this is Artifactory. 

Moran Ashkenazi: When you login to the JFrog Platform and move to Artifactory so you get the sense of what it looks like in the UI

Eyal Ben Moshe: Yeah definitely I just logged in and then inside here under Artifactory, you have your build section okay. So these are all the builds that we published to this or the Artifactory instance of the JFrog platform and here, for the sake of this DEMO I just created a sample build and, as you can see, there are a few build runs I run three build runs off of this build and if I just go into one of the build runs basically, here, you can actually see.

Moran Ashkenazi: same format right.

Eyal Ben Moshe: yeah, this is the Build Info json, as you can see here all the environment variables are actually recorded and if you scroll down you’ll see here the dependencies the artifact if this will publish artifacts and everything else, and then you know we created a UI, which is based on this information so here you can see the actual published modules. So if I go into this specific project it has only one module and I can just look at all the dependencies of the build and everything that is actually inside the json. The environment variables that were recorded as part of the build. 

Moran Ashkenazi:  I couldn’t ignore the fact that you have vulnerabilities in your modules as well, can you jump into the Xray data. Let’s see.

Eyal Ben Moshe: Let’s do it, so I made sure that this build includes real security vulnerabilities and, as you can see here, there are four security vulnerabilities inside this build.

Moran Ashkenazi: You’re in big trouble with this build, Eyal.

Eyal Ben Moshe: Two critical vulnerabilities, one high vulnerability and yeah I can actually go and drill down and see more about the data visually. My point being is, you have to Build Info, now you can decide what to do with it, so we have a record of the bill, and now scanning becomes easy.

Yeah, and this is the real data, we are all for shifting left, and this is what we do, our developers are running the JFrog IDE integrations inside their IDEs to make sure that you know vulnerabilities are found as soon as possible. But, as Yoav mentioned, there’s no real guarantee that what the developer is actually running is what ends up being built by the CI and then goes out tp the world. 

Yoav Landman: Just one comment about this, the nice thing is that you have the information you capture the information, it’s an immutable information that’s not going to ever change. It’s being frozen here in this case, there are some products around it, but you can freeze it in an s3 bucket if you wish or you can sign it. And you can scan it because it’s a system of record of everything that went on in your build, and now you can scan it, this is the JFrog Xray scan but in theory, you can scan it with any tool, because you just have to parse the package name or the sha, and based on that you can do a static component analysis scanning and being fair about the health of the security of this build.

Moran Ashkenazi: Super important. Yoav, you mentioned signing, so what about the authenticity of the build? 

Yoav Landman: I think Eyal wanted to show you.

Eyal Ben Moshe: Yeah so you’re right you want to make sure that this bill of materials this Build Info is actually signed, because what you want to do is you basically want to attach it to your software,  attack to attach it to your software to your code and you know so that so that anyone can see basically what it contains and you want it signed, so I can show you something that we did.

I’m going to use a tool called cosign, which is part of the.

Yoav Landman: Part of the open ss, yeah, it’s out of six to 16 project yeah it’s a tool for signing I think initially containers, but in more general just any blob.

Eyal Ben Moshe: Yeah, exactly so it’s for signing containers but we’re going to show you how you know when you can actually sign the Build Info so I created something here.

Here I have the building for json okay so that’s the building for Jason that we show you how we can create. And I just created here a script that can practically sign it so I’m going to run this cosign command the design blog command and, as you can see here, you see here the last argument it’s basically going to sign this building for json after we sign it, people can basically verify that it is signed, so let me just run.

Yoav Landman: A caveat they are that it’s you’re using experimental flags so not everything is still completely baked in in cosign but it’s it for the sake of the DEMO it works.

Eyal Ben Moshe: Yeah, it worked for us, for the sake of the DEMO, so let us just run this and now what’s going to happen is.

My browser is going to open, and the tool is going to ask me to authenticate so let me authenticate with my Google account. I’m going to authenticate with my JFrog Google account and now once the authentication is done I can basically close the browser. And here, you can see that the tool created for me a certificate. For the signing and I also received an ID, this is the transparency log ID that the registry is using the cosign or the record the actual registry that is being used. And now that I have this information I’m going to use this ID here, let me just open my script. I’ll just put the ID here. And then I’m going to go ahead and run these two commands. Those commands, as you can see, this is the record coi that we’re using for the signing. And these two commands are going to create for us a signature file and a public certificate yeah. Let’s do this so let’s run all of those three commands now.

That’s it then we’re done. Those files are actually created, and now what I can do, I can you know anyone practically can verify this Build Info so imagine that now you received this Build Info you found attached to the project, and now you want to verify the authenticity of authenticity.

You just run the entire script. And yeah you see the first line said that it’s verified, this error, I assume it’s because we are using an experimental version of this functionality. But yeah it’s verified, because if I go now, and let me just tamper with my Build Info json. Just open it, this is the Build Info json that we signed, and let me just add a number like one and that basically changes this Build Info and then I’m going to run the verification again, you see I’m not getting this verification. So. yeah.

Yoav Landman: If we look at the scenario basically if you’re building an Open-Source project, you can record the build info. Say you’re building some Java or a Node.js package you can verify the Build Info as the developer that builds the package, you can identify yourself using your public identity like your GitHub or Google identity, and then you can use this identity to sign your Open-Source package and it to the transparency log which not only verifies the package and the fact that you’ve created it but also everything about how you built this package, what are the dependencies back at the time, and what environment you were building on and so on.

Yeah, great, very nice.

Moran Ashkenazi: Thank you, Eyal, great demo. Let’s talk about a roadmap from the building, how do you see it?

Eyal Ben Moshe: Yeah so we mentioned earlier, the goal is to support more technologies and more package managers.

Because the format is the same so it’s basically just integrating with the technology, making sure that we extract the data correctly, correct the build and yeah and then, of course, you know, anyone can create their own applications on top of the standard.

You know, once the tool is handling the heavy lifting of actually recording the data, then you know it’s just up to anyone just you know to.

Moran Ashkenazi: As a developer, if I want to contribute to this project, how can I do that?

Oh you’re definitely welcome to contribute. So the Build Info go is an open source project. It’s a new project we just started it recently. And we are continuously adding code to it and improving it so anyone who wants to join us and contribute is invited to create a pull request and collaborate with us and that goes not only for this project, I mean basically everything.

You know all of our ecosystem integrations if it is the ID plugins that recreating the CI integrations the JFrog CLI our SDK is you know they’re all open and we see a lot of collaboration and we encourage it and we love it so you’re definitely invited. 

Yoav Landman: I think we’ve created a new repo now, but for the original building for the over 60 different collaborators that improved it over the years, so. 

Eyal Ben Moshe: Yeah, definitely and now. 

Moran Ashkenazi: Open questions yeah, sorry go ahead.

Eyal Ben Moshe: Now that the actual logic of the extraction is separated, it’s easier like to contribute it’s it’s practically now decoupled from the use cases of from the implementations so it’s easier, you can right now, contribute to the extraction part or, of course, to the part or to the libraries that actually use it. JFrog CLI, for example, uses now Build Info go as a dependency. So, you know it’s easy now like to see you know to see them for the most part, separated and you know for us. 

You know, continue on contributing and developing it. 

Moran Ashkenazi: Okay that’s great I think we’re ready for the question is Courtney are you with us. 

Courtney Gold: Yeah, I’m with you, I didn’t want to interrupt the conversation you guys are having. So we did a few questions come in and as a reminder for everybody else, please don’t hesitate to put your questions in there we’re here to answer them, but the first question we have is how does Build Info help us with Log4j Log4shell zero day vulnerability, we all know, that’s been a hot topic lately.

Moran Ashkenazi: Definitely. 

Yoav Landman: Yeah, so I’m not surprised that this question came up, I mean like the mandatory question to ask these days so building for. 

First of all, it will not help you to avoid Log4shell, but what it will help you with is first, since you have the information condensed and in a very easily parsable format, you can very easily scan and identify where in your builds, because you’re scanning full builds, so if you scan Build Info you will be able to identify where you’re using Log4j. You will be able to do so by identifying dependencies whether transitive or direct of Log4j it’s all there. 

Also there are some as we found out there are some libraries that embed Build Info and we’re actually going to publish, in the coming days, a blog about that, and if you know those libraries, you can also identify them very, very easily. 

And it’s also easier to relate the usage of Build Info to real build jobs coming from your CI and then relate them to the project and fix them in the project. Rather than trying to say Okay, I have this blob now that contains a Log4j file and how it came about and who built it and when was the class built. So, it gives you a lot more clarity and a lot more than tracking back to the source of where you use a bad version of Log4j. That’s mainly it.

Moran Ashkenazi: Very important.

Courtney Gold: Alright, so the next question that just came in, is how does JFrog support the Community with new technology? 

Yoav Landman: Okay, so I think I’ll cover this. 

Eyal Ben Moshe: yeah I think I practically covered it so in terms of you know, Community contribution. All of our ecosystem projects are open source and yeah and there, this is, you know, the way that we manage them and invite collaborations and with others, is something that is really important for us. 

I think that we’re maintaining today around 15 or so open source projects, some of them are extremely popular. I think JFrog CLI is at the top. 

We are seeing thousands of downloads a day of these tools and yeah and then we get collaboration from developers. Out of JFrog CLI a few years ago we extracted a library JFrog client go, which is the library that allows you, basically a wrapper around the JFrog API’s and allows easy development over the JFrog product, and those are just two examples we are doubling down on our ID integrations which are focusing on security, but not only security. 

Moran Ashkenazi: Okay, yes. 

Eyal Ben Moshe: Yeah, so I was just saying that you know our IDE integrations are also an example and we’re doubling down on our effort of making sure developers have the best. 

Yoav Landman: Okay, I’m not sure that you can hear us. I’ll add to it very quickly so first of all, these are all great projects and what we’re making sure is that we’re supporting open format and we’re making sure that the format that we have created is interchangeable with other formats. And also what I mentioned before, we are making the project such as that it’s decoupled from any specific product. You can use it and you we would like to foster this adoption in order to increase the security level and the visibility for things that are being used as part of your builds. And we have other CI servers and build technologies, adopting Build Info and integrating with it.

JFrog is part of the openSSF and we are working closely with other partners there to foster the standards around the software supply chain and bill of materials. We’re really excited to see that it’s getting heavy traction nowadays. And we will have a very strong component of it before, but then now the market is starting to get it, probably for the wrong reasons because the risk became too immediate, and too easy for people to anonymously do harm. But it promoted a much better code hygiene and dependency hygiene so we’re excited to be part of that.

Moran Ashkenazi: Developers and security come together and it’s super important, thank you for that party we have.

Courtney Gold: Yeah, we have some more questions coming in.

And I’ll go ahead and ask it now how scalable is the built info tool and creating an accurate SBOM by capturing build level processes, considering that there could be variations in the build process build tool, etc., and that was a long question.

Eyal Ben Moshe: If you can hear me now clearly then, then I can take it. 

Yoav Landman: Yeah, we can hear you.

Eyal Ben Moshe: Oh OK cool so yeah, so I think the question was how does Build Info handle you know, different package managers are different, I think. 

Courtney Gold: I’ll repeat the question, one more time. How scalable is the Build Info tool and creating an accurate SBOM by capturing the build level process, considering that there could be variations in that process? 

Eyal Ben Moshe: Yeah, so the answer is this Build Info aims to be as accurate as possible, because, as we mentioned earlier, it runs as part of the build, so basically imagine that the actual logic that generates the Build Info is running together with a build and the goal is what building for does it just record it it’s like a recorder it records everything that happens.

So imagine that right now you’re running two builds, one after the other, and in the second build something is different, like a different environment variable the repository is different, it actually doesn’t matter.

If the second bill downloaded a slightly different version of a dependency, it will be recorded.

Which is different than an approach that says hey let’s look at the source code and try and figure out what it will build. What we’re doing with the Build Info, we’re just recording at runtime. And that’s how we get the accuracy.

Yoav Landman: Yeah, it’s an accurate report that will give you everything that happened in the build. So it’s very trustable. It captures everything that happened in the build. If you want to reconstruct the build, we’re at JFrog are kind of against trying to reconstruct the build. There is a place for it, which is, if you want to prove the authenticity of a certain package to its source, then you can run the build from several sources and see that you’re getting the same result.

But, in theory, there is no reason for you to rebuild a binary from scratch. And once you have the binary once you store it once they have the provenance for this binary, you shouldn’t waste your time and energy to rebuild it and and also it’s very hard to reconstruct it later in time. But for the question it’s super reliable as it embodies everything that happened in the original build.

Courtney Gold: Right, awesome, alright, well the next question we have is which CI servers are we supporting and how easy is it to support new see I workflows? 

Eyal Ben Moshe: Yeah, so I can take this question so what the quick answer is, with support to everything that we’ll support any CI server and this is done as following.

We have CI integrations for the popular ci servers Jenkins, GitHub, Azure and all the popular CI servers are supported with their dedicated integrations. For example, for GitHub, we have a a JFrog CLI GitHub action, which practically uses JFrog CLI to create integration, so it generate the Build Info for you. We have a very popular Jenkins Artifactory plugin which does all of that. And I can say that we support any CI server because you can just you know embed JFrog CLI there, it includes features that are aimed at easily integrating with the CI server APIs.

And yeah, generating the Build Info for any CI is very easy.

Courtney Gold: Wonderful and that was the last question, but I want to just quickly thank everyone for joining today. This was an amazing conversation. Moran, I’ll let you go ahead and close it out, but thank you for all the attendees for joining, we appreciate your time, and we look forward to seeing you more during these soon.

Moran Ashkenazi: Yeah, sure. Thank you Courtney, and thank you for attending the first episode of the CTO corner series with JFrog co-founder and CTO Yoav Landman. Thank you to our panelist Eyal Ben Moshe, head of ecosystems at JFrog. I’m Moran Ashkenazi, JFrog CISO. We look forward to seeing you all in the next episode so stay tuned and thank you.

Yoav Landman: Thanks.

Eyal Ben Moshe: It’s been a pleasure bye bye.