Infrastructure as Binaries with Chef and Artifactory: Five Best Practices

Infrastructure as Binaries became a reality several weeks ago when we announced that Artifactory supports infrastructure management platforms such as Chef. Both we, here at JFrog, and our infrastructure champions at Chef, believe that establishing a canonical Artifactory repository for all artifacts used in a company’s infrastructure such as Ruby Gems, NuGet packages, RPMs, and especially Chef Cookbooks, is essential to achieving DevOps velocity while maintaining safety and security. So we’d like to offer the following five best practices you should follow to make the best use of this integration between Artifactory and Chef.

1. Ensure consistent and reliable access to Cookbooks on the community Chef Supermarket

When creating your own Cookbooks, you are likely to use community Cookbooks downloaded from Chef Supermarket as dependencies. However, working directly with Chef Supermarket leaves you dependent on this public repository always being up, and also on the external network that must always be available. If either of these resources becomes unavailable, your work may grind to a halt.

You can mitigate this risk by a using remote Chef Cookbook Repository in Artifactory to proxy the public Chef Supermarket. This removes your dependence on the public Chef Supermarket and the web in general because a remote Chef Cookbook repository in Artifactory downloads Cookbooks on demand and then caches them locally. Once a Cookbook has been downloaded it is available on your local network and is not subject to availability of the community Chef Supermarket or the external network.

2. Use local repositories to distribute internal Cookbooks and other artifacts

Some of the environments you want to spin up are defined in internal Cookbooks that your DevOps team has written. These may also need tarballs of application binaries, JARs, WARs, RubyGems and other  internal artifacts that are often needed for a Chef run. Using the file system to distribute these artifacts is tedious, unreliable and insecure. In the case of Chef, putting large artifacts in the “files” directory of your Cookbooks makes them difficult to audit and update, and ties their change management to that of the Cookbook. The right way to share these internal artifacts is to use local repositories.

A local repository in Artifactory is a physical, type-specific,locally managed repository into which you can deploy artifacts. It gives you a central location to store your internal binaries giving access (with fine-grained access control) to other teams through a single URL. Artifactory supports all common package formats, and now, with Artifactory’s support for Chef Cookbook repositories, you can build your own internal local Chef Supermarket in Artifactory.  According to the permissions you define, your Chef clients can access the internal Cookbooks stored in the local Chef Cookbook repositories, and your Cookbooks can access your other internal artifacts via Chef resources like remote_file. If necessary, you can use the headers parameter to perform HTTP basic authentication. If you want to go a step further, you can use Artifactory’s replication capabilities to distribute artifacts and internal Cookbooks to teams in geographically distant sites. By replicating these repositories, globally distributed teams can easily collaborate while keeping all of their collective resources synchronized.

3. Use a virtual repository to provide a single unified view of internal and community Cookbooks

Using a single resource for both uploading internal Cookbooks and accessing Cookbooks hosted on remote public repositories is very convenient. Your developers only need to be aware of one URL they need to work with. Virtual repositories in Artifactory provide this capability.

With a virtual Chef Supermarket in Artifactory, you can aggregate all the remote sites that you want to access for community Chef Cookbooks, add a local Chef Supermarket to where your developers can upload your internal Cookbooks, and present them all under a single logical view.

4. To maintain security and access control, disable anonymous access to your local Chef Cookbook repositories in Artifactory

Whether your organization is large or small, you probably want to provide different access privileges to your internal Cookbooks for the different users in your organization. For example, a developer may be able to upload Cookbooks to a development repository, but only QA team members can promote Cookbooks from the development repository to a staging repository.

To enable fine-grained access control, disable anonymous access to your Chef Cookbook repositories. This will apply several layers of security that Artifactory offers. Your first line of defense is virtual repositories that we mentioned in the previous best practice since you can ensure that your developers only access external resources that have been approved such as the community Chef Supermarket. Then, Artifactory lets you use naming patterns with wildcards to define “Includes” and “Excludes” for download. This flexible mechanism lets you define anything from a whole repository to be excluded from access, down to including a single Cookbook that may be critical for your development. Once you have controlled what can be downloaded, you now have a flexible, permission-based authorization system that lets you define which users can access what repositories or artifacts, and what they can do with them (download, upload, delete, move, etc.). Finally, through integration with common access protocols such as LDAP, Active Directory, SAML, Crowd and others, you can control access at the level of your servers.

To apply access control to your Chef Cookbook repositories in Artifactory, first disable anonymous access. Then, your users install the knife-art.gem plugin using:

chef gem install knife-art

…and access to your repositories will then require basic authentication which means that all the layers of security offered by Artifactory come into play.

5. Use Chef’s Artifactory RubyGem to communicate with Artifactory from a Cookbook

Some of your Cookbooks may need to communicate with Artifactory during their execution. For example, they may need to download protected artifacts or modify metadata of artifacts in one of Artifactory’s repositories. This is exactly what Artifactory’s extensive REST API is for.

Chef has created the Artifactory RubyGem that authenticates you and makes REST API calls to Artifactory from your Cookbook. Be sure to read the documentation since it doesn’t provide complete coverage for Artifactory’s REST API. You should also consider that many of Artifactory’s REST API endpoints are only available in the Pro version. As a perk, you can also use this gem outside of Chef to interact with Artifactory from any Ruby program.

Infrastructure as binaries – controlled, auditable and reliable

Using an artifact repository is an essential component of a highly functioning and modern DevOps team, especially when artifacts must be change-controlled, versioned and audited. Now, with Artifactory’s support for infrastructure management, it is the perfect place to store and manage your Chef Cookbooks along with all the other artifacts that your Cookbooks depend on. By using Chef with Artifactory and following these best practices, the entire lifecycle of your infrastructure will be properly controlled, auditable and independent of the internet.

Ready to start managing infrastructure as binaries?

Learn all about managing Chef Cookbooks with Artifactory in our webinar. Register now.