DevOps: 8 Reasons for DevOps to Use a Binary Repository Manager on the Cloud

Cloud DevOps tools offer greater flexibility, rapid deployment, cloud automation, reduced IT costs, and low upfront costs with subscription pricing. Setting up your environment with Artifactory on the cloud on your choice provides unlimited scalability allowing you to grow according to your needs and is easily achieved by using cloud storage providers (Amazon AWS, Google GCP or Microsoft Azure) in your environment with Artifactory.

Together with the great advantages, moving to the cloud introduces these new DevOps challenges:

  • Supporting multiple package types and technologies
  • Increasing costs in hardware and infrastructure due to the growth and demands in storage and bandwidth
  • Applications have to be up and running at all times. Zero tolerance for software downtime has become the de facto standard
  • Providing stable and reliable access to repositories
  • Supporting a large number of common binaries across different environments
  • Security and access control
  • Tracing any action done to a file back to the user
  • Supporting a large number of common binaries across different environments
  • Transferring a large number of binaries to a remote location
  • Managing infrastructure configuration across different environments

This white paper describes how an Artifact Repository Manager such as Artifactory gives a DevOps team the tools to efficiently manage an ever-growing matrix of binaries, environments and geographically distributed sites on the cloud, while ensuring a smooth application development workflow.

1. A Universal, End-to-End Solution For All Binaries

No single tool or technology is enough to support development in a modern organization. There is a multitude of packaging formats, a variety of build tools, different continuous integration systems and other technologies that go into building a flexible and maintainable software development ecosystem. Managing binaries for all the different packaging formats and integrating with all the moving parts of the ecosystem can become a maintenance nightmare.

Artifactory as a universal solution supports Kubernetes helm, Go, npm, Docker, and in fact all major package formats (over 20 and growing) in common use today. So whether you need a Docker registry, or a Kubernetes registry to deploy containerized apps to your Kubernetes cluster, or use any of those other supported technologies,  Artifactory is the tool that will manage your binaries.

Artifactory was designed from the ground up to fit in with any development ecosystem. Uniquely built on checksum-based storage, Artifactory supports any repository layout and can, therefore, provide native-level support for any packaging format. Essentially, regardless of the packaging format you are using, Artifactory can store and manage your binaries, and is transparent to the corresponding packaging client. The client works with Artifactory in exactly the same way it would work with its native repository.

But development is only one end of the software delivery pipeline. Before a package makes it into a product, it needs to go through processes of build and integration. There are many build and integration tools on the market, but there is only one product that works with them all. Through a set of plugins, Artifactory provides tight integration with popular CI systems available today such as Jenkins, Bamboo and TeamCity. These systems use Artifactory to supply artifacts and resolve dependencies when creating a build, and also as a target to deploy build output.

And to support cloud-based CI systems on which you are not able to apply plugins, Artifactory provides plugins for the build tools you use (such as Maven, Gradle, TFS and Azure DevOps) which ultimately provides the same level of build automation.

CI/CD in the cloud with JFrog Artifactory

That takes care of development and deployment, but what about distributing your software once it’s ready for consumption.

That’s where Bintray comes in. Bintray is the JFrog download center in the cloud offering rapid downloads, fine-grained access control, detailed stats and logs and an extensive REST API. Promoting releases for distribution from Artifactory is a matter of a single-click or API call. Like Artifactory, Bintray is package-agnostic and works seamlessly with all the different package clients, so it can be fully integrated into any continuous integration/continuous delivery ecosystem.

2. Scalability and Redundancy

A huge advantage of going to cloud computing is the decrease in hardware costs whereby you pay only for exactly what you use. Instead of purchasing in-house equipment, hardware needs are now left to the cloud vendor. Adding new hardware every release or every quarter to meet your increasing needs can be very expensive and inconvenient, on the other hand, cloud computing alleviates these issues because resources can be acquired quickly and easily.

And the most important thing you gain is redundancy. A number of cloud storage providers can ensure your data is stored on multiple machines and complies with any regulations that need to be applied (like keeping two copies of all the data in two different regions).

3. Unmatched Stability and Availability

You can easily deploy, manage and scale an Artifactory Enterprise high availability cluster in the cloud. Artifactory HA architecture presents a Load Balancer connected to a cluster of two or more Artifactory servers that share a common database where all the Artifactory configuration files are maintained.  Binaries may be stored on a Network File System, or using a zoned sharded binary provider as described in Configuring Sharding for High Availability. One of the Artifactory cluster nodes is set as the “Primary” node that will execute cluster-wide tasks such as cleaning up unreferenced binaries. One of the Artifactory cluster nodes is set as the “Primary” node that will execute cluster-wide tasks such as cleaning up unreferenced binaries.

4. Managing Many Binaries across Different Environments

Artifactory lets you streamline your DevOps toolchain and solves all the following challenges that arise when managing your binaries on the cloud:

  • Higher maintenance cost to manage multiple configurations and providers
  • Lack of traceability to figure out the origin (CI job/source code revision) of the binary file
  • Lack of context in the form of metadata
  • Difficulty to enforce policies

To add to the above, you have the need to manage cross-project binaries of multiple types coming from multiple repositories adds to the overall complexity. The lack of a single source of truth or a universal binary repository makes the implementation of policy enforcement difficult.

The solution to resolve these challenges is to use Artifactory that will act as a single source of truth for all your binaries. It’s the most pragmatic approach to managing multiple binaries of different types and their associated metadata in a single place. And, that’s where JFrog Artifactory comes into the solution!
A recommended cloud-based CI/CD pipeline scenario would be to use Artifactory to manage the entire pipeline and create separate environments for development, staging and production.

 

Tip! We recommend applying the “Shift left principle”, whereby you move back tasks typically performed at later stages to earlier stages in the pipeline. The aim is to move faster to reduce delivery time while improving the quality of each release.

5. Security, Access Control and Traceability

Authenticated Users and Access Control

Keeping your binaries secure, ensuring that all access to them is only by authenticated users with the right permissions is not something that starts in production systems. JFrog Access, packaged and installed together with JFrog Artifactory gives you all the capabilities needed to ensure your binaries are safe, and only exposed according to your access policies, from the moment they are created until they are deployed to your production systems.  

Artifactory provides consistent and reliable access to remote artifacts. Those dependencies that your builds need, and the remote public repositories that host them may not always be available. Remote public resources can go down, dependencies can be removed, and even the network can sometimes be spotty – all of which can cause your builds to grind to a halt. Once artifacts have been downloaded, Artifactory caches them locally eliminating the need to download them again and removing your dependence on the remote public repository and network.

Full Traceability of Your Artifacts

A substantial amount of valuable metadata is emitted during the CI process and is captured by Artifactory. One of the major advantages of leveraging this metadata is traceability. The build-info is critical to make your build traceable and fully reproducible, so if a bug is found at any time in the application lifecycle you can precisely recreate the build and debug it. With the help of JFrog CLI, JFrog Artifactory collects and stores exhaustive build information.

Securing Your Binaries

JFrog Xray works with Artifactory to perform universal analysis of binary software artifacts at any stage of the application lifecycle. Xray helps identify vulnerabilities in images by scanning  and analyzing artifacts and their metadata, recursively going through dependencies at any level, before the images are run.

6. Reliable Remote Repositories

Artifactory provides consistent and reliable access to remote artifacts. Those dependencies that your builds need, and the remote public repositories that host them may not always be available. Remote public resources can go down, dependencies can be removed, and even the network can sometimes be spotty – all of which can cause your builds to grind to a halt. Once artifacts have been downloaded, Artifactory caches them locally eliminating the need to download them again and removing your dependence on the remote public repository and network.

7. Artifactory as Your Docker Registry

Artifactory can serve as a secure, robust Docker registry; a single access point to manage and organize your Docker images. With full support for the Docker registry API, Artifactory works transparently with the Docker client allowing you to boost your organization’s productivity by removing the overhead connected to managing your applications and images that are developed to run in Docker containers.
Artifactory provides:

  • Reliable, consistent and efficient access to remote Docker registries
  • Full integration with your build ecosystem
  • Security and access control
  • Distribution and sharing images across your organization
  • Smart Search for images

8. Artifactory as Your Kubernetes Registry

Artifactory as your “Kubernetes Registry” lets you gain insight on your code-to-cluster process while relating to each layer for each application. It provisions your k8s cluster with the charts and images needed to orchestrate your application and supports:

  • Secure and Private Provisioning: Artifactory can serve as a secure private Kubernetes Docker Registry that hosts local Docker images. These give you fine-grained access control as you securely share packages and images across your organization for provisioning to your k8s cluster. And to keep you independent of the network and remote resources, Artifactory uses remote repositories to proxy public Docker resources, caching Docker images that are downloaded so they are available on the local network. Finally, virtual Docker registries aggregate local and remote repositories giving you a single URL through which you can manage all your Docker image.
  • Kubernetes Docker registry: Use Artifactory as your main Kubernetes Docker registry to gain insight and full control on your code-to-cluster process while relating to each layer for each application. You get to collect and manage the artifacts for your containerized app, trace the content, dependencies and relationships with other Docker images which cannot be done using a simple Docker registry.

Summary

This white paper has shown how DevOps can use a Binary Repository Manager such as Artifactory to support the development cycle in a continuous integration environment on the cloud.

To get our new whitepapers subscribe to our newsletter

Release Fast Or Die