Blog Home

Does Ruby Need a Mature Binary Repository?

By Shay Yaakov

5 min read

At some point in time, a Ruby developer realized the need to serve gems within a private network. The main reasons why:
  • You can’t rely onRubyGems.org
  • You need a place to host the gems is not available in RubyGems. Those can be of two flavors:
    • Something not hosted at RubyGems. For example, Vagrant.
    • Something internal (neither open source nor public); something you want to share with your colleagues, but as a gem rather than source.

Then the question arises – where can those gems be stored? Natural answer: Git(Hub) or any other source control. So…

Source control is the way to share files, right? No, it isn’t.

Of course, source control works just fine for sources – we’ve used it for ages. But gems aren’t source, they are binaries.  And source control is, well, for sources. So what? Why won’t it fit? Some reasons include:
  • First and foremost – a version control system (VCS) is not a gem repository! It can’t calculate indexes on the server and it doesn’t support any dynamic REST API, such as the dependency resolution API used by Bundler (which makes resolution much faster).
  • Versioning mismatch. Source files are versioned by their content. VCSs know how to differentiate them and understand what changed. Binaries, on the other side, are usually versioned by their name. From the VCS point of view, they are different entries, each one without any version history.
  • Some very popular VCSs (like Subversion) can’t obliterate files. That means – once a file is added, it stays in the repository forever. That’s not a big issue for small source files, but can become quite a pain when it comes to obsolete, large binaries.
  • Source control knows how to search sources. And, of course, the most important type of search is by content. However, searching for binaries is different: what matters most is the file metadata, the location, structure of the file name and, in case of archived artifact, the contents of the archive.
  • The permissions scheme of VCSs is tailored for versioning sources (again!). For example, there is no override permission. That’s because overriding sources is something we do all the time in VCS – it’s the same security level as, let’s say, adding a new source file. However, the situation is very different with binaries. While adding new binaries is fine, overriding released binary is something that shouldn’t be done (one should have a special permission for it).
  • Distributed VCSs (yes, Git, I am looking at you!) are awesome by themselves, but particularly unsuited for handling big binary files. When cloning a remote repository to your machine, you are bringing all the history of its files. Now just think about all the huge binaries sitting there…
By now, you should be convinced that source control isn’t a good place for binaries. What we actually need is an installable RubyGems server! And guess what? There are a couple of options:

Go get yourself a RubyGems server

  • Gem in a box is a Sinatra application that provides, well, a gems server. It’s nice, but a bit naive: no built-in authentication, no authorization, no repositories separation, and no other servers (i.e. RubyGems.org) proxy.
  • GemFury is a very basic, subscription-based cloud-hosted gems server. You get a private repository, protected with an obscure URL. Again, pretty basic stuff here – no proxy for RubyGems.org (or any other repo), no authentication model for collaborators, and no virtual aggregation of repositories in case you have more than one.

What can I say? The Ruby universe is not very sophisticated when it comes to managing binaries – and that’s OK (after all, Ruby is about source, usually open source). But there’s something the Ruby community can borrow from the “dark Java Enterprise” side – the proper binary repository. And we have one to offer…

Welcome to the dark side and see the light. Meet Artifactory, with RubyGems support:

Let’s start with the basics. The binary repository serves two main goals:
  1. Proxy of remote RubyGems repositories. First and foremost, RubyGems.org; but also any instance of GemFury, Gem in a Box, etc. out there. These are called “remote repositories.”
  2. Deployment target for your gems. Everything you don’t want to put on RubyGems.org for any reason, and everything you need but other repositories don’t have. These are called “local repositories.”
On top of that we add:
  • Virtual repositories to aggregate any number of remote, local and virtual repositories under a single URL.
  • Authentication and authorization schemes which allow controlling permissions on repositories per user and/or group, including integration with external authorization services.
  • Searching and browsing hosted and remote gems.
  • REST API with Info, Search, Dependencies list and Yank commands.
  • Powerful user plugin framework.
You can get all of this goodness installed on your servers or in the cloud with Artifactory Online, where JFrog will babysit it, upgrade it, and keep it running.

How can you begin using Artifactory with RubyGems support? Simple! The full documentation is available in our user guide:

  1. Install Artifactory on your server (RPM or just an unzipped folder) or get your own instance in a cloud.
  2. Set up some repositories:
    1. Set up a RubyGems.org proxy.
    2. Create some local repositories for your gems.
    3. Aggregate them under virtual repositories.
  3. Set up your client to work with the virtual repository you created by running the “gem source” command.
  4. Enjoy your build using the tools you are used to, e.g. Bundler.

You are more than welcome to give Artifactory with RubyGems support a try today – download itor create a cloud instance. We will appreciate your feedback.

Welcome aboard!

Popular Tags