Build Times Matter – Speed is Everything
By
6 min read
UPDATE: As of May 1, 2021 – GoCenter central repository has been sunset and all features deprecated. For more information on the sunsetting of the centers read the deprecation blog post
After GoCenter was first released the team at JFrog said it was faster to get Go modules from there than it was getting modules “the old way”. To be honest, when they said that, I did not believe them. Following the old proverb “seeing is believing”, I set out to test the speed and see whether those claims were accurate.
TL;DR: It turns out the team was right! GoCenter doesn’t only save time, it also saves CPU cycles. As your project gets bigger, and your dependency tree grows, the savings will increase too. You don’t have to take my word for it, check out the “Now it’s your turn!” section below to see my test script and start your test in two minutes flat!
To test, I took two projects and tested them on two different networks. The engineering team only heard about my test after all tests were completed, to avoid any chance of them tampering with my data. I chose one of my personal projects and my favorite static site generator, Hugo. The GoCenter team had no influence on the projects or the modules and versions used. The two locations were to make sure that testing at the JFrog office couldn’t favor GoCenter unfairly.
The test
Building a Go project usually consists of two parts. First of all, you’d download the dependencies and second you’d build the executable. With all of the tests done on the same machine, the results only focused on the first part, downloading the dependencies. To make the test as fair and uninterrupted as possible, I wrote a little script that removes all modules before running “go get”. For both using GitHub and GoCenter, the commands are run ten times and the result logged in a file.
The numbers
The complete list of tests and results can be found here. To make comparing the results a little easier the below tables mark the difference between the fastest and slowest runs for each of the projects.
Comparison table for my personal project using the JFrog network
| CPU seconds spent in user mode (less is better) | CPU seconds spent in kernel mode (less is better) | The CPU percentage (less is better) | Total elapsed time in seconds (less is better) | |
|---|---|---|---|---|
| Comparing the slowest runs | GoCenter spends 17.79 times fewer CPU seconds | GoCenter spends 7.03 times fewer CPU seconds | GoCenter uses 6.12 times less CPU | Using GoCenter is 2.14 times faster |
| Comparing the fastest runs | GoCenter spends 9.37 times fewer CPU seconds | GoCenter spends 5.64 times fewer CPU seconds | GoCenter uses 2.64 times less CPU | Using GoCenter is 2.94 times faster |
Comparison table for my personal project using my home network
| CPU seconds spent in user mode (less is better) | CPU seconds spent in kernel mode (less is better) | The CPU percentage (less is better) | Total elapsed time in seconds (less is better) | |
|---|---|---|---|---|
| Comparing the slowest runs | GoCenter spends 6.79 times fewer CPU seconds | GoCenter spends 3.31 times fewer CPU seconds | GoCenter spends 2.21 times fewer CPU | Using GoCenter is 2.38 times faster |
| Comparing the fastest runs | GoCenter spends 5.73 times fewer CPU seconds | GoCenter spends 3.49 times fewer CPU seconds | GoCenter spends 1.07 times fewer CPU | Using GoCenter is 4.45 times faster |
Looking at the total elapsed time, which is the time the machine spends getting the modules and putting them into the right location, using GoCenter is a lot faster than getting them from GitHub.
I’ll accept that this is a personal project with only 6 directly used modules and 12 indirectly used modules. What would be the impact of using GoCenter with a much larger project, like Hugo with 43 directly used modules and 11 indirectly used modules?
Comparison table for Hugo using the JFrog network
| CPU seconds spent in user mode (less is better) | CPU seconds spent in kernel mode (less is better) | The CPU percentage (less is better) | Total elapsed time in seconds (less is better) | |
|---|---|---|---|---|
| Comparing the slowest runs | GoCenter spends 7.76 times fewer CPU seconds | GoCenter spends 5.17 times fewer CPU seconds | GoCenter spends 0.79 times fewer CPU | Using GoCenter is 8.34 times faster |
| Comparing the fastest runs | GoCenter spends 7.51 times fewer CPU seconds | GoCenter spends 4.97 times fewer CPU seconds | GoCenter spends 0.66 times fewer CPU | Using GoCenter is 9.77 times faster |
Comparison table for Hugo using my home network
| CPU seconds spent in user mode (less is better) | CPU seconds spent in kernel mode (less is better) | The CPU percentage (less is better) | Total elapsed time in seconds (less is better) | |
|---|---|---|---|---|
| Comparing the slowest runs | GoCenter spends 6.71 times fewer CPU seconds | GoCenter spends 4.65 times fewer CPU seconds | GoCenter spends 0.75 times fewer CPU | Using GoCenter is 7.83 times faster |
| Comparing the fastest runs | GoCenter spends 7.64 times fewer CPU seconds | GoCenter spends 5.61 times fewer CPU seconds | GoCenter spends 0.67 times fewer CPU | Using GoCenter is 10.10 times faster |
Even for larger projects, it makes sense to use GoCenter to get Go modules.
The results
As it turns out the team was right! GoCenter isn’t only useful to eliminate vendoring and making your builds repeatable using immutable modules. Using GoCenter to get your Go modules is also faster than getting them from a Version Control System. There are two interesting results from the above numbers. First of all, on slower networks, you’ll see a bigger performance increase as downloading modules takes a lot less bandwidth than downloading source code. Secondly, as the complexity of the project grows (larger dependency trees), you’ll also see a bigger performance increase.
Now it’s your turn!
You don’t have to take my word for it, in fact, I challenge you to try it yourself! This is the script I used to test the project:
I challenge you to try it on your project and tweet the results to us using the hashtags #golang and #GoCenter. After you do that, drop us an email on gocenter@jfrog.com with an address we can ship this awesome shirt to!
