UPDATE: As of May 1, 2021 – GoCenter central repository has been sunset and all features deprecated. For more information on the sunsetting of the centers read the deprecation blog post

Ever since the introduction of Go’s new dependency management system in Go 1.11 and later, GoLang developers have embraced the package versioning solution. Those that do can use the GoCenter repository of immutable public Go modules, and gain faster Go builds with a more robust and reliable Go pipeline.

But converting existing projects to use Go modules isn’t always easy, especially if the project has evolved through GoLang’s many earlier attempts at package management solutions.

To help the GoLang community adopt Go modules correctly, we’ll take the open-source etcd, the key-value datastore used by Kubernetes, through the conversion process. This is a good real-world example for showing best practices because it’s complex enough to present some common challenges, including its use of code generators and static analysis tools that need to be properly module-aware.

Go Modules Conversion Best Practices

This conversion journey was validated by successful tests, and the results can be seen in a pull request updating 544 files.

Step 1: Preparing the go.mod File

For a project that has never used modules before (so there is no go.mod file), or any of the now-deprecated dependency management solutions, the procedure would be very simple. You would only need to run go mod tidy in the project’s root directory. This generates a new, populated go.mod file with the module’s dependencies.

If, however, the project had used one of those older solutions, such as dep, glide, govendor, or godep, then you would need to run go mod init to generate the populated go.mod file. This command honors the dependency versions described in the older formats.

The etcd project, it turns out, does have a go.mod file, although it had never been enabled in the project’s build system. The catch is that the module name doesn’t have the correct version identifier since the current version tag is v2+. This needs to change to v3 due to the implications of semantic import versioning. 

That involves performing these procedures:

1. Update etcd’s go.mod file to fix the module name to include the v3 suffix.
2. Update all imports to include the version number. We wrote a script to make it easier on ourselves to change all the references. When finished, this change appears as follows:

Step 2: Enable Go Modules

To enable the go client to use go modules, you need to set GO111MODULE=on 

Since, as we noted, the etcd project already had a go.mod file, one might expect that this had already been done. But it has not, and that absence confirms that the project isn’t already using go modules. 

Note: From Go 1.13 onwards, this step won’t be required since Go modules will be enabled by default.

Step 3: Update Imports in Tests

In the procedure above, we updated our go.mod for the files that compose the main module of etcd to use the v3 version tag., With the main module now tagged as v3, we also need to update the imports in the etcd project’s tests for v3 as well, to ensure they import the correct version of the main module. 

Step 4: Other Updates

After those changes, you might expect to be in good shape — after all, the application modules have now all been converted to use go modules, and use the proper version tag.

Not so fast, though. As soon as you start running tests you will find two additional scenarios that needs to be handled:

1. Static analysis tools such as golint, gosimple, staticcheck, ineffassign are used by etcd, but some of them aren’t module-aware and will fail to recognize the module path, and not pass necessary checks. In the case of etcd, there is no v3 folder under etcd-io/etcd but the imports (or module path) contains v3 such as etcd-io/etcd/v3. Other tools are module-aware, but must be available in recent releases of Go 1.12. If the build systems are on 1.11, they also need to move to 1.12.
2. Code generators such as protobuf were used. Update the .proto files so that the code is generated with correctly versioned imports.

Step 5: Bring in GoCenter

In the process of building, you might notice that a lot of go get commands are executed at various stages of the etcd pipeline. 

To speed up GoLang app build times, and to guarantee the immutability and availability of Go module versions used in the etcd pipeline, update the etcd build to make use of GoCenter — which is as simple as setting GOPROXY=https://gocenter.io.  

Go For More

As you can see, converting your Go projects to use Go modules is straightforward, but there are some details that might slow you down. By selecting this project with such a rich legacy to demonstrate the process, we believe we hit most of the scenarios that need to be handled, providing you a good example of what you may face.