Process is Critical So Are the Systems of Record.

You probably read the awesome post in the Netflix Engineering Tools team blog about the build process that Netflix uses to continuously deploy the service that streams movies and TV shows to more than 75 million global Netflix members.

And while the post concentrates on the build process – the build tool, the CI server and the deployment platform, there is another layer of infrastructure that doesn’t get much attention — the systems that take care of your files at rest – the Repositories. Being the Systems of Record for the process, they keep the data and the metadata that is constantly used and produced by the process. The System of Record shouldn’t be taken for granted; you can’t just dump your files into a shared drive, and even uploading them to S3 won’t cut it – you need your System of Record to be:

1. Highly available (your mission-critical artifacts are there),
2. Connected to every other system in the development organization (the artifacts come from those systems and get deployed to them),
3. Equipped with powerful and versatile query capabilities (because you don’t really know what the “future-you” might want to know about the artifacts you produce or use today).

Those are just a few examples of the long list of requirements from an enterprise-grade repository. And this is equally true for your source code repository and your binary repository.

As mentioned in the comments, the repositories are a whole different “layer” of the infrastructure, and deserve a blog post of their own. Or maybe a conference talk?
Jon Schneider and Nadav Cohen (both with the same Engineering Tools team) are going to talk about how JFrog Artifactory is used as the System of Record in Netflix’s JAGG stack. Come hear how they do it at the JFrog User Conference — swampUP on May 23rd in Napa. Ah, and raise a toast with Jon and Nadav for the great work they do at Netflix so we can watch House of Cards and other great series.