JFrog Container Registry: There’s a New Container Registry in Town
New to the DevOps Universe is the new community tool, JFrog Container Registry! Catch the demo and discussion on the newest container registry on the market, how it integrates with your existing tool stack, and how to get started. Want to take the JFrog Container Registry for a test drive?
Try it for FREE here
The Agenda:
- What is JFrog Container Registry?
- How JFrog Container Registry is different Metadata, Metadata, Metadata Security?
- Yes! JFrog Container Registry with security powered by JFrog Xray Multiple Registries, one seamless promotion
- Demo
- Q&A
Webinar Transcript
Today’s topic is obviously JFrog Container Registry.
So, here’s our agenda for today. We are going to talk about, yes, what is JFrog Container Registry? Why you need JFrog Container Registry? We are going to talk about what technologies a container register supports. What type of repository it supports. We’re going to talk about multi-repository promotion, which is a very, very important aspect. We’re going to talk about the importance of metadata. We’re going to about build integration and JFrog CLI. About Artifactory Query Language. About security. I’m going to show you a demo. And we are going to talk a little bit about how JFrog Container Registry stack against competition. And then, we will have time for questions.
That’s me. My name is Baruch Sadogursky. I’m the head of DevOps Advocacy at JFrog. But we can continue our conversation on Twitter. My hashtag is… Sorry my Twitter handle is @jbaruch.
So, JFrog Container Registry. First and most important thing that you need to know is JFrog Container Registry is based on JFrog Artifactory. So it is JFrog Artifactory under the hood, and it means that you get the same quality. You get 10 years of experience of managing binary files. You have proven scalability to amazing scale. And all that kind of built in, in JFrog Artifactory, in JFrog Container Registry.
It supports two main technologies of the container ecosystem and that’s container images, namely Docker, and Helm repositories. Another very important feature that we kind of threw in, and we will see how useful it is, is the support for generic repositories, and generic… Sorry, and generic repositories basically means you can use it for whatever you like, and do whatever work needs to be there.
It’s intended for teams to get started with the artifact management and containerization. It’s, obviously, fully operational, and you use it for any state of your team. But the main purpose is in getting started you look for a container registry that’s a great and free option. It’s hybrid. It means that you can have it on-prem, or you can have it in the cloud. It’s up to you. You can combine the two. If you need more than one container registry installed, you can combine in the cloud, set up with on-prem, so that’s what hybrid means.
The next question is why JFrog Container Registry over all the others? So, first and foremost, it’s free. You can download it, you can use it. There are no limitations on number of users, or number of servers, on hardware, on anything basically. And it’s absolutely free to use without any limitation.
It simplifies the flows… It fits the simple flows with no requirements on managing intermediate binaries. So, if you are good with a version in only the images that you produce, if you don’t need to save, and manage the intermediate binaries, then JFrog Container Registry covers your scenario completely.
It’s good for co-located teams without requirements for artifact replication. So, if you have an organization which works in one place locally, this is great. And if you don’t require any kind of complicated distribution scenarios, and this kind of stuff. Basically, if you only need a container registry by itself, JFrog Container Registry, we believe, is actually the best option for you. And, as I already mentioned, it’s free.
As I mentioned, it supports Docker Registry. Soon enough we will have a full support for container registry specs for a [inaudible 00:05:09] and for now it supports Docker completely. So, that works out-of-the-box. It also supports Helm, as it defines in Helm 2 and Helm 3, Helm chart repositories built without the extension of the container registry, Helm repositories. And, as I mentioned, it supports generic repository. And generic repository is super helpful for managing other type of artifacts.
For repository types it supports a local repository. By local repository we mean it’s a place when you deploy your own Docker images and Helm charts. So, this is like a physical storage on Artifactory itself.
Remote repository. Remote repository is a proxy for remote registries and repositories. So, you can proxy Docker Hub, you can proxy any other Docker registry in the world, and Artifactory will easily store the artifacts that come… Sorry, JFrog Container Registry will easily store images that come through JFrog Container Registry. So, everything you resolved through this remote repository will be saved and will be there. You will see during the demo how it works. And that, obviously, makes your life easier in terms of both resolution times, and also you’re protected against outages, and what not.
And a third part of the repositories is virtual repository. And virtual repository is very useful for providing a single URL, a single registry view for multiple registries, or repositories behind it. So, you can have any number of different registries defined local, or remote, or virtual. And then, you can have a single virtual registry that will give a single URL for Docker to connect. Again, this is very important and unique. And combination of three of them is unique across the board of different container registries. And you will see, again, the importance on each and every one of them during the demo that you are going to see.
Another super important feature that kind of comes out of this multi-repository setup is the multi-repository promotion. There is a trade-off of how you build promotion across your Docker images. You can promote the Docker file and then rebuild from scratch. This is, obviously, very dangerous because you might end up having a different Docker container than the one that you intended in production. Another option is having multiple registrars installed. And then, how do you promote between these multiple registries? Usually you will have to do something like pull, or retag, and push, which it doesn’t make any with Docker images, which are big and slow, so that won’t work as well.
With JFrog Container Registry all that is solved because you actually set up multiple registries that [inaudible 00:08:57] from the same backend, from the same storage. It looks like different registries for Docker, which provides rigid security and quality gates. And the promotion actually is free. And immediate actually, not even fast, but immediate, because the files are not only they are not downloaded and uploaded from network that you will have in the push, retag, pull scenario. They are not even moved in your local file system because the file system remains the same. So, you get all the benefits of having multiple registries, rigid gates, but without any costs, and the promotion is actually free. And, again, we’re going to talk about that in the demo as well.
Metadata. JFrog Container Registry shines with metadata and, obviously, JFrog Container Registry inherited that because of the metadata works. All three types of metadata that exist. Implicit metadata ,like the name of the files, or the Docker images. The path of the layers, the checksums, sizes, these kind of information. Explicit metadata, for example, how it was created, what was the build name, the build number? And any custom metadata that you want to add. Stuff like QA state, target architecture, or any other type of metadata is available. And then, once you have this metadata, you can set it through stuff like build integration, and JFrog CLI, REST API and matrix params, or even the UI. Obviously, you probably won’t do it through the UI, but that exists as well.
And then, when you consume the metadata, you can consume it by filtering by matrix params. So, you can construct a query that will only refer to the images in a certain state, or a certain maturity.And another option, obviously, is using JFrog CLI. That also is a great way, but you can even do it with REST API, for example. Another very, very powerful option would be using Artifactory Query Language. And just for you, to give an example how Artifactory Query Language looks like, that’s a JSON query that here, you can see how it queries for a certain file that comes from a certain build with the build number. And you can imagine how it actually works with other metadata. So you can query, for example, a file that was downloaded, or the number of files, or was created by a certain user, or had some target architecture, or, obviously, any combination of those as well.
Another very important aspect of any container registry is security image scanning. And you need it to prevent to your next heartbleed to [inaudible 00:12:37] you. And there are two aspects. The first is control. And this is built in, obviously, and having a generic repository provides you more control because it allows you to control not only your images, but also your dependencies. I will show you an example how you can make sure that what you pull into your Docker images is under your control as well.
And actually JFrog Container Registry provides you information from a JFrog Xray, which is a security tool from JFrog that features the best security renewability database in the world [inaudible 00:13:32] security. And your JFrog Container Registry will be able to show you vulnerabilities on your Docker images that come from this source, from [inaudible 00:13:46]. It’s coming soon. I have a preview on the demo. But, in the end of the day, you will get it as well, and you will get it for free.
And so with that, let’s get to the demo. So, what I wanted to show you is, let’s say, I have a Docker image, and it misbehaves, I need to troubleshoot it, something doesn’t work. And the problem is that I don’t know much about this Docker image. The name is latest, and it doesn’t really help me. And I don’t know what’s going on. So, what I can try to do is I can run docker inspect on this image, and actually see if that gives me more information.
So, what do I see here? I see layers, which are a bunch of checksums, and then more checksums, and then some flags. Okay, here are environment variables, it’s all Java. Well, not really useful. The only useful piece of information here is the sha256 of this image. So, I’m going to take it, and I’m going to go to JFrog Container Registry. And see if I can get any information about it. So, I have a checksum search here that I can use, and I can see what’s going on. So here, this layer, is actually bunch of Docker images. So I have latest, but I also know that the same image is actually a bunch of others. So, for example, 21 and 19, 23 and 26. It was rebuilt multiple times. And this actually is already very, very important information from which I know what’s going on.
So, I can go to any of them. Let’s go to 26, for example. And I can see here, what’s going on with my Docker image. I can see here, the Docker info, I can see the layers, what actually happened on each and every slide. And I could also get the same information in different views. So, there is a package view that can show me all the packages. Here’s my JFrog Docker app. And I can see here, latest, that’s the one that I have, but also 26, the real version number. And, again, here I can see the layers and I can see what’s going on.
So, getting back to look at what we are looking and all. The Docker image that we looked at is here. It’s inside a repository that’s called docker-prod-local. docker-prod-local26, that’s the one that we are looking. And now, let’s see how we actually got there. So, we can look at the metadata, the properties here and see a bunch of useful stuff like build name, and build number, and also build URL. Build URL, that’s a link to the CI server that created this Docker image. Let’s go and see.
The CI server that we used was a different pipelines, and that’s CI and CD2 from JFrog it’s in preview release, but I know a guy that knows a guy that gave me access, so I can show you how it looks. But what we actually are interested about is seeing this pipeline. And let’s look at it and try to understand what happened, and how this Docker image ended up where it’s.
So, we can see a number of steps. The first one is buildDockerApp. So, we can see here, what we do. We actually just run DockerBuild, and build our image. Nothing interesting here. pushDockerApp. That’s a more interesting step. So, you can see here that we perform DockerPush, and we can form DockerPush into a repository, into a registry that’s called docker, right here. It’s called docker. What is this docker? docker, if we look at the list of our repositories is this one. This repository is a Docker registry and it’s called docker. And it’s actually where our CI actually pushed our image. And what we can see here, that we have a list of included repositories. That means that docker is a virtual repository. So, if you are going to list of virtual repositories, we can see our docker registry right here. That’s our repository and that’s our registry. And you can see here that it includes four other Docker repositories, Docker registries, docker-prod-local, docker-dev-local, and then docker-remote and bintray-docker-remote, two local store remotes.
Now, virtual means that it actually hides multiple registries under the single URL. And when pipelines pushes an image to this registry, the question is out of those four, where actually it goes? And for that, we have a setting here, which is the default deployment repository. Default Deployment Repository, this is where files that are pushed into this repository will end up to. And we can see that they will end up in docker-dev-local.
Now, going back to our list of repositories, this is our docker-dev-local. And we can see it’s pretty empty. There is only one here, 26 is not here, local is not here. So, the question is, what’s going on? Why it’s not here? Because, as you remember, we saw it pushed here and push was successful. The answer is there is an additional two steps in this pipeline. The next one will be publishBuildInfo. And the publishBuildInfo is the publication of the metadata about this build. So, if we can go to our Docker registry, we will see that the builds that our CI server creates actually got recorded in the Build Browser of JFrog Container Registry.
So, you can see here, we have the demo_kubecon builds, a lot of builds here. And you can see how they’re registered. And you can see stuff like which modules is published, which environment variables were available. So, for example, we can see what version of Java we used. I think, yeah. And what else is here. All this information actually comes as a JSON file that’s been pushed, and this is what publishBuildInfo step does.
The next step is promote the build. And promote the build is after we run the test… here, there are no tests, but in your environment you will, obviously, have tests. You can promote this build from docker-dev-local to docker-prod-local using just REST API, or using JFrog CLI, or using the native capabilities of the CI server. So, this is an example of using REST API, so we just use curl.
And you can see here that what we run is promote, and we promote it from docker-dev-local to docker-prod-local, and we promote our build. And this is why when we look at the container registry, we will actually see our image not in docker-dev-local, but in docker-prod-local. Here it is number 26.
Now, why this is critical? This is critical because each and every Docker repository in Artifactory, like docker-dev-local or docker-prod-local is actually a full Docker registry from the perspective of Docker client. And if you build this promotion pipeline of moving files from one repository to another in Artifactory, you will build those quality and security gates from the perspective of Docker client. So, for example, using this kind of promotion, you can guarantee that your runtime environment won’t even see anything before it was promoted to production, because it will only see the docker-prod-local as it’s Docker registry. And until the images end up in docker-prod-local, your production environment won’t be able to even see them. And moving them there is just a question of one CPI, which is very simple and actually move them.
We have two more REST APIs here. So, we look at them first. This one actually does something very interesting as well. You can see that it’s, again, a call for the same promotion API. But this time it actually keeps it in the same repository from docker-prod-local to docker-prod-local, but it renames the image from the build number to latest and does a copy. This is why we have here, the same images under do two different [inaudible 00:24:28]. This is the 26, and the latest they actually refer to the same image. And this is very powerful because although we all know the problems of latest and that latest might not refer to latest at all, having the ability to use latest is actually very convenient because you just do docker pull, and you will get one, which is the latest at the moment. How can we make sure that the latest refers to latest at the moment? Exactly how you saw it. We are going to retag every new build as latest.
But now, there is another question, how to know what latest refers to? And we provide this information as additional metadata, as I told you, the custom metadata we spoke about. And here’s an example of how we do that. We add properties and this property called docker.refersTo, and I put it on the latest and I put here the build number. So, you can see, if I look at the latest and I look at its properties, I can see here an entry, docker.refersTo 26, and this is how I know that it actually refers to the image number 26. I know the latest is actually this one. And this is obviously very, very powerful.
Okay, now, after you show how it works, let’s take a look for a second at the Docker file that we created it from. So, you can see here… Where is it? Well, can just show you in GitHub. Okay, that’s the Docker file, it’s docker-app. And you can look at the Docker file. This is all it does. It basically takes some WAR file and adds it to our Docker image. And then, we’re good to go.
Now, the interesting part to see here is the from. The from the Docker image that is a basic dependency for our image is also an image that comes from JFrog Container Registry. And this is an image that is called JFrog Docker Framework. And you can obviously see it here. Here it is, jfrog-docker-framework, and we rely on number 5. That’s the latest one that is checked and proven to be good to rely. And this jfrog-docker-framework is also built using JFrog pipelines. And also went through the same promotion and stuff.
So, if you look here at the pipelines that are available, we can see here that we have the buildDockerFramework. The last one was… Let’s see, that’s the Docker App. Let’s find number five. Yeah, here we go. buildDockerFramework. That’s the framework number 5 that we relate. And you can see here that what happened is it was built, and then it was pushed, then it was promote, very much alike. What I wanted to show you here is how that the from and builds from another one and here… Okay, that’s already copying it.
I wanted to show you where the dependencies come from. Here, artifactory_configure, docker_login. Okay. I cannot see it. Fetching pipelines. Okay, this is where we build the container. So here, all those files, the Tomcats, and the JDK, they come from the JFrog Container industry as well. And they come from a generic repository that we have. So, here we have a generic file in which we have the dependencies that our framework needs. So, you can see here that we have the Java JDK that we use and the Apache Tomcat that we use. And they also come from JFrog Container Registry, they also come from a controlled resource. This is how you can guarantee that you don’t download a bunch of unknown files from unknown resources over the internet. But instead, you have everything controlled in one repository.
And speaking about control and how you get everything under control, you can actually see the integration with the security part. If we are going back to look at our docker-prod-local and look at our application, we can see here… And sorry, [inaudible 00:30:23]. You can see here an Xray tab. And this is a preview on how integration with JFrog Xray actually would work. So, you can see here, a bunch of vulnerabilities. You can click on any one of them, and see all information about what’s wrong. So, you can see here that we have vulnerability in Tar. And this vulnerability is actually in this layer. And this layer is in this Docker image. And then, you can see the component, the summary, the description. And comes from us information about the CVE, the CVE it contains, and then all the information you need in order to see, and understand what’s the issue with this image. And, obviously, you will see all the vulnerabilities for all those files.
So just to recap, JFrog Container Registry, you can install it for free on prem. You can grab it for free under Free Tier on any of your major cloud providers, Amazon Web Services, Google Cloud Platform, and Microsoft Azure. It’s all for free. You can create unlimited number of registries inside the JFrog Container Registry, and they will be either local that will be storage for the images that you deploy. Or a remote that will be the proxy for remote repositories. So, I didn’t show you the remote. docker-remote, you can see here that it actually cached three different images from Docker Hub. And that’s one that we used for our base image. So, they’re also cached, and you have control over there as well. Or virtual, which allows you to define one unified URL, or unified view for more than one Docker repositories which means Docker Registry’s underneath. And then, once you do that, you can build promotions from one registry to another registry that are immediate and free.
And, obviously, the importance of metadata. You saw how properties on Docker image actually give you any kind of information about it. You can add custom properties here, and then query them with REST API, or Artifactory Query Language. It works the same for Docker, and it works the same for Helm. So, you can see here that we have actually a Helm package, of Artifactory JFrog Container Registry right here. And that’s a Helm repository that you can see all the charts and dive into any chart to see any details.
So, with that, let’s go to our slides and talk about and talk about competition. So, the competition you can see how JFrog is different. And that has to do with the demo that I showed you. So, I spoke about how hybrid it is. It’s on-prem and in the cloud, support Docker Registry, Helm Repository, generic repositories, remote and virtual, and the support for metadata. If you compare it with Amazon ECR, well, it is obviously in the cloud on AWS. They have some on-prem capabilities, but they are very limited. And it, basically, supports only Docker Registry. There is some kind of metadata that you can apply on the images, but nothing as powerful as JFrog CLI, or Artifactory Query Language that comes with it.
Google Container Registry, absolutely the same. It’s in the cloud, but nothing more. And it only supports Docker and nothing except that. Same with the Azure Container Registry, it does support Helm repositories, but that’s the only thing. Everything else is pretty much not existent. GitLab Container Registry. They are on-prem and in the cloud, so they are hybrid. But, again, they only support Docker and nothing else in any other option. And Harbor has the most advanced of them all. Installs on-prem, but doesn’t really have any good cloud solution. It supports Docker. It supports Helm. There is some kind of remote repositories, but they are not integrated. And they are no promotion. Obviously no virtual repositories. And the metadata capabilities are very limited and nothing that can compare to Artifactory Query Language. So, I hope I convinced you that JFrog Container Registry is the best container registry out there at the moment.
And with that, we have about 10 minutes to answer your questions. So now, will be a good time to ask them if you didn’t already. Wow, there are a lot of questions. This is great. So, let’s get to it and get through those questions.
Okay so, the first question is, “The relationship between JFrog Container Registry and Artifactory open source?” So, I would say those are different editions of JFrog Artifactory and they support different subset of technologies that are supported in the full version of JFrog Artifactory. Open source version supports, Java repositories, and generic repositories. And JFrog Container Registry supports Docker, Helm and generic. There is no way to combine them in any way, obviously. And if you want the support across the board, I would recommend getting JFrog Artifactory. If you still look for a free solution, you might very well install two of them. There is no license limitation on doing that. I would personally say that the hassle of managing more and more tools, in some point of time, will more expensive than just having one Artifactory instance that supports all of them. I hope that makes sense.
Another question is, “Can I proxy Docker Container Registry that requires license authentication to access?” The answer is yes. When you set up a remote repository, there is a way to provide authentication. And this is how you connect to other container registries, which require authentication to [inaudible 00:38:14].
Another question. “Last time I looked promoting Docker image [inaudible 00:38:24] repositories required API call, which is fine for [inaudible 00:38:27] but a small project might like manual process. Is there a button in the UI?” So, there is no button in the UI. It’s an API call. What I would suggest for smaller teams that don’t want to deal with all those flags on curl and what not is writing one line of a shell script that takes two parameters and simplifies the work. There is no button in the UI. And from my best knowledge, it is not planned as well.
“Did I hear it right, Xray will be offered for free with JFrog Container Registry?” Yes, you did. You heard it right. Information from Xray will be or free within JFrog Container Registry. You’ll have a view that allows you to see all the vulnerabilities on your images inside JFrog Container Registry from JFrog Xray, yes. Glad, that’s exciting. That’s also exciting for us.
“Please show how to take Docker build to use the generic rep.” Ah, okay. So, I think I understood the question. The question is, “How can I provide information about what I do in my build info? How can I provide…” Sorry, “What I do with my generic repo? How can I provide this information inside the build info?” And for that, I will share my screen again, because I want to show you where to find it in the documentation. It’s very, very easy and straightforward.
So, if you just Google for JFrog CLI you’ll be able to find the documentation. And here in JFrog CLI Core Artifactory, you will see all the build integration part. So, you can see here, for example, [inaudible 00:40:53] file specs. So you can see here, how can you provide a file spec, which is a group of files. And then, you can add build name, and build number to the commands for downloading and uploading those file specs. And those files then, will be recorded in build info as dependencies or artifacts. File that you download will become dependencies. File that you upload will become artifacts. And that’s exactly how it works.
So, basically, when you run commands such as, let me find it quick, download files and upload files. Here, upload files. So, when you run an upload command, you can provide a file spec, and then build name and build number. Once you do that, those files you download will become dependencies for your build. So, I hope that helps.
“How the data is stored? We’re looking for a way to share registry across regions, can we do it with JFrog?” So JFrog Artifactory has a number of different data providers, which range from local file system through NFS, and all the way to S3 data provider. They are different between different subscriptions. JFrog Container Registry provides a local file storage, which means the files will be saved on a local disc.
If you need more scalable solution, like all the way to S3, then it might make sense to look on different subscriptions, on different license of JFrog Artifactory like Pro and Enterprise or even Enterprise+ for real big installations. Also, what provides you with sharing registries across regions is replication, which is also available in Artifactory Pro, Enterprise, and Enterprise+. Replication will provide you making Docker images available in other instances of JFrog Artifactory. At the moment, they are pushed into local instances of JFrog Artifactory.
“Can you add metadata to the image after it’s pushed or only during build time?” Obviously, we can add the metadata whenever we like. What we saw in pipelines that it was a separate step. So, it’s not really related to the build. We first, built it. And then, we added metadata, but can be much later. There is some metadata that we want to add during the push, and that will be build name, build number, and environment variables during the build. All this information, which is relevant to the build itself, but we can definitely easily add more and more metadata after, actually in any point in time.
Another question is what free means with cloud business model. And this means that we have a free tier, like any other provider of a container register in the cloud. You can use it for free up to a certain storage and traffic limitations. And so, the answer is yes, it’s free even in the cloud.
Another question is, “If I’m already using Artifactory, do I need to use JFrog Container Registry? And what is the relationship?” So, the good news is, if you already use Artifactory, unless it’s the other free editions, like the open source version, or the Conan edition, Community edition the full JFrog Artifactory already contains all the functionality of JFrog Container Registry. So, there is no need for you to do anything. You already have your JFrog Container Registry built into any JFrog Artifactory Docker Artifactory instance.
I think that’s pretty much all. If you have any additional questions I have probably couple more minutes. And if not, thank you very much for coming. I hope you found this webinar useful. And please go ahead and use JFrog Container Registry, we will be more than happy, if you just grab it, install it, start using it. No questions asked. This is from us to you free and unlimited to use.
Okay, with that, thank you very much.
