The Go Blog

Using Go Modules

19 March 2019

Introduction

Go 1.11 and 1.12 include preliminary support for modules, Go’s new dependency management system that makes dependency version information explicit and easier to manage. This blog post is an introduction to the basic operations needed to get started using modules. A followup post will cover releasing modules for others to use.

A module is a collection of Go packages stored in a file tree with a go.mod file at its root. The go.mod file defines the module’s module path, which is also the import path used for the root directory, and its dependency requirements, which are the other modules needed for a successful build. Each dependency requirement is written as a module path and a specific semantic version.

As of Go 1.11, the go command enables the use of modules when the current directory or any parent directory has a go.mod, provided the directory is outside $GOPATH/src. (Inside $GOPATH/src, for compatibility, the go command still runs in the old GOPATH mode, even if a go.mod is found. See the go command documentation for details.) Starting in Go 1.13, module mode will be the default for all development.

This post walks through a sequence of common operations that arise when developing Go code with modules:

  • Creating a new module.
  • Adding a dependency.
  • Upgrading dependencies.
  • Adding a dependency on a new major version.
  • Upgrading a dependency to a new major version.
  • Removing unused dependencies.

Creating a new module

Let's create a new module.

Create a new, empty directory somewhere outside $GOPATH/src, cd into that directory, and then create a new source file, hello.go:

package hello

func Hello() string {
    return "Hello, world."
}

Let's write a test, too, in hello_test.go:

package hello

import "testing"

func TestHello(t *testing.T) {
    want := "Hello, world."
    if got := Hello(); got != want {
        t.Errorf("Hello() = %q, want %q", got, want)
    }
}

At this point, the directory contains a package, but not a module, because there is no go.mod file. If we were working in /home/gopher/hello and ran go test now, we'd see:

$ go test
PASS
ok      _/home/gopher/hello    0.020s
$

The last line summarizes the overall package test. Because we are working outside $GOPATH and also outside any module, the go command knows no import path for the current directory and makes up a fake one based on the directory name: _/home/gopher/hello.

Let's make the current directory the root of a module by using go mod init and then try go test again:

$ go mod init example.com/hello
go: creating new go.mod: module example.com/hello
$ go test
PASS
ok      example.com/hello    0.020s
$

Congratulations! You’ve written and tested your first module.

The go mod init command wrote a go.mod file:

$ cat go.mod
module example.com/hello

go 1.12
$

The go.mod file only appears in the root of the module. Packages in subdirectories have import paths consisting of the module path plus the path to the subdirectory. For example, if we created a subdirectory world, we would not need to (nor want to) run go mod init there. The package would automatically be recognized as part of the example.com/hello module, with import path example.com/hello/world.

Adding a dependency

The primary motivation for Go modules was to improve the experience of using (that is, adding a dependency on) code written by other developers.

Let's update our hello.go to import rsc.io/quote and use it to implement Hello:

package hello

import "rsc.io/quote"

func Hello() string {
    return quote.Hello()
}

Now let’s run the test again:

$ go test
go: finding rsc.io/quote v1.5.2
go: downloading rsc.io/quote v1.5.2
go: extracting rsc.io/quote v1.5.2
go: finding rsc.io/sampler v1.3.0
go: finding golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c
go: downloading rsc.io/sampler v1.3.0
go: extracting rsc.io/sampler v1.3.0
go: downloading golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c
go: extracting golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c
PASS
ok      example.com/hello    0.023s
$

The go command resolves imports by using the specific dependency module versions listed in go.mod. When it encounters an import of a package not provided by any module in go.mod, the go command automatically looks up the module containing that package and adds it to go.mod, using the latest version. (“Latest” is defined as the latest tagged stable (non-prerelease) version, or else the latest tagged prerelease version, or else the latest untagged version.) In our example, go test resolved the new import rsc.io/quote to the module rsc.io/quote v1.5.2. It also downloaded two dependencies used by rsc.io/quote, namely rsc.io/sampler and golang.org/x/text. Only direct dependencies are recorded in the go.mod file:

$ cat go.mod
module example.com/hello

go 1.12

require rsc.io/quote v1.5.2
$

A second go test command will not repeat this work, since the go.mod is now up-to-date and the downloaded modules are cached locally (in $GOPATH/pkg/mod):

$ go test
PASS
ok      example.com/hello    0.020s
$

Note that while the go command makes adding a new dependency quick and easy, it is not without cost. Your module now literally depends on the new dependency in critical areas such as correctness, security, and proper licensing, just to name a few. For more considerations, see Russ Cox's blog post, “Our Software Dependency Problem.”

As we saw above, adding one direct dependency often brings in other indirect dependencies too. The command go list -m all lists the current module and all its dependencies:

$ go list -m all
example.com/hello
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c
rsc.io/quote v1.5.2
rsc.io/sampler v1.3.0
$

In the go list output, the current module, also known as the main module, is always the first line, followed by dependencies sorted by module path.

The golang.org/x/text version v0.0.0-20170915032832-14c0d48ead0c is an example of a pseudo-version, which is the go command's version syntax for a specific untagged commit.

In addition to go.mod, the go command maintains a file named go.sum containing the expected cryptographic hashes of the content of specific module versions:

$ cat go.sum
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c h1:qgOY6WgZO...
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:Nq...
rsc.io/quote v1.5.2 h1:w5fcysjrx7yqtD/aO+QwRjYZOKnaM9Uh2b40tElTs3...
rsc.io/quote v1.5.2/go.mod h1:LzX7hefJvL54yjefDEDHNONDjII0t9xZLPX...
rsc.io/sampler v1.3.0 h1:7uVkIFmeBqHfdjD+gZwtXXI+RODJ2Wc4O7MPEh/Q...
rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9...
$

The go command uses the go.sum file to ensure that future downloads of these modules retrieve the same bits as the first download, to ensure the modules your project depends on do not change unexpectedly, whether for malicious, accidental, or other reasons. Both go.mod and go.sum should be checked into version control.

Upgrading dependencies

With Go modules, versions are referenced with semantic version tags. A semantic version has three parts: major, minor, and patch. For example, for v0.1.2, the major version is 0, the minor version is 1, and the patch version is 2. Let's walk through a couple minor version upgrades. In the next section, we’ll consider a major version upgrade.

From the output of go list -m all, we can see we're using an untagged version of golang.org/x/text. Let's upgrade to the latest tagged version and test that everything still works:

$ go get golang.org/x/text
go: finding golang.org/x/text v0.3.0
go: downloading golang.org/x/text v0.3.0
go: extracting golang.org/x/text v0.3.0
$ go test
PASS
ok      example.com/hello    0.013s
$

Woohoo! Everything passes. Let's take another look at go list -m all and the go.mod file:

$ go list -m all
example.com/hello
golang.org/x/text v0.3.0
rsc.io/quote v1.5.2
rsc.io/sampler v1.3.0
$ cat go.mod
module example.com/hello

go 1.12

require (
    golang.org/x/text v0.3.0 // indirect
    rsc.io/quote v1.5.2
)
$

The golang.org/x/text package has been upgraded to the latest tagged version (v0.3.0). The go.mod file has been updated to specify v0.3.0 too. The indirect comment indicates a dependency is not used directly by this module, only indirectly by other module dependencies. See go help modules for details.

Now let's try upgrading the rsc.io/sampler minor version. Start the same way, by running go get and running tests:

$ go get rsc.io/sampler
go: finding rsc.io/sampler v1.99.99
go: downloading rsc.io/sampler v1.99.99
go: extracting rsc.io/sampler v1.99.99
$ go test
--- FAIL: TestHello (0.00s)
    hello_test.go:8: Hello() = "99 bottles of beer on the wall, 99 bottles of beer, ...", want "Hello, world."
FAIL
exit status 1
FAIL    example.com/hello    0.014s
$

Uh, oh! The test failure shows that the latest version of rsc.io/sampler is incompatible with our usage. Let's list the available tagged versions of that module:

$ go list -m -versions rsc.io/sampler
rsc.io/sampler v1.0.0 v1.2.0 v1.2.1 v1.3.0 v1.3.1 v1.99.99
$

We had been using v1.3.0; v1.99.99 is clearly no good. Maybe we can try using v1.3.1 instead:

$ go get rsc.io/sampler@v1.3.1
go: finding rsc.io/sampler v1.3.1
go: downloading rsc.io/sampler v1.3.1
go: extracting rsc.io/sampler v1.3.1
$ go test
PASS
ok      example.com/hello    0.022s
$

Note the explicit @v1.3.1 in the go get argument. In general each argument passed to go get can take an explicit version; the default is @latest, which resolves to the latest version as defined earlier.

Adding a dependency on a new major version

Let's add a new function to our package: func Proverb returns a Go concurrency proverb, by calling quote.Concurrency, which is provided by the module rsc.io/quote/v3. First we update hello.go to add the new function:

package hello

import (
    "rsc.io/quote"
    quoteV3 "rsc.io/quote/v3"
)

func Hello() string {
    return quote.Hello()
}

func Proverb() string {
    return quoteV3.Concurrency()
}

Then we add a test to hello_test.go:

func TestProverb(t *testing.T) {
    want := "Concurrency is not parallelism."
    if got := Proverb(); got != want {
        t.Errorf("Proverb() = %q, want %q", got, want)
    }
}

Then we can test our code:

$ go test
go: finding rsc.io/quote/v3 v3.1.0
go: downloading rsc.io/quote/v3 v3.1.0
go: extracting rsc.io/quote/v3 v3.1.0
PASS
ok      example.com/hello    0.024s
$

Note that our module now depends on both rsc.io/quote and rsc.io/quote/v3:

$ go list -m rsc.io/q...
rsc.io/quote v1.5.2
rsc.io/quote/v3 v3.1.0
$

Each different major version (v1, v2, and so on) of a Go module uses a different module path: starting at v2, the path must end in the major version. In the example, v3 of rsc.io/quote is no longer rsc.io/quote: instead, it is identified by the module path rsc.io/quote/v3. This convention is called semantic import versioning, and it gives incompatible packages (those with different major versions) different names. In contrast, v1.6.0 of rsc.io/quote should be backwards-compatible with v1.5.2, so it reuses the name rsc.io/quote. (In the previous section, rsc.io/sampler v1.99.99 should have been backwards-compatible with rsc.io/sampler v1.3.0, but bugs or incorrect client assumptions about module behavior can both happen.)

The go command allows a build to include at most one version of any particular module path, meaning at most one of each major version: one rsc.io/quote, one rsc.io/quote/v2, one rsc.io/quote/v3, and so on. This gives module authors a clear rule about possible duplication of a single module path: it is impossible for a program to build with both rsc.io/quote v1.5.2 and rsc.io/quote v1.6.0. At the same time, allowing different major versions of a module (because they have different paths) gives module consumers the ability to upgrade to a new major version incrementally. In this example, we wanted to use quote.Concurrency from rsc/quote/v3 v3.1.0 but are not yet ready to migrate our uses of rsc.io/quote v1.5.2. The ability to migrate incrementally is especially important in a large program or codebase.

Upgrading a dependency to a new major version

Let's complete our conversion from using rsc.io/quote to using only rsc.io/quote/v3. Because of the major version change, we should expect that some APIs may have been removed, renamed, or otherwise changed in incompatible ways. Reading the docs, we can see that Hello has become HelloV3:

$ go doc rsc.io/quote/v3
package quote // import "rsc.io/quote"

Package quote collects pithy sayings.

func Concurrency() string
func GlassV3() string
func GoV3() string
func HelloV3() string
func OptV3() string
$

(There is also a known bug in the output; the displayed import path has incorrectly dropped the /v3.)

We can update our use of quote.Hello() in hello.go to use quoteV3.HelloV3():

package hello

import quoteV3 "rsc.io/quote/v3"

func Hello() string {
    return quoteV3.HelloV3()
}

func Proverb() string {
    return quoteV3.Concurrency()
}

And then at this point, there's no need for the renamed import anymore, so we can undo that:

package hello

import "rsc.io/quote/v3"

func Hello() string {
    return quote.HelloV3()
}

func Proverb() string {
    return quote.Concurrency()
}

Let's re-run the tests to make sure everything is working:

$ go test
PASS
ok      example.com/hello       0.014s

Removing unused dependencies

We've removed all our uses of rsc.io/quote, but it still shows up in go list -m all and in our go.mod file:

$ go list -m all
example.com/hello
golang.org/x/text v0.3.0
rsc.io/quote v1.5.2
rsc.io/quote/v3 v3.1.0
rsc.io/sampler v1.3.1
$ cat go.mod
module example.com/hello

go 1.12

require (
    golang.org/x/text v0.3.0 // indirect
    rsc.io/quote v1.5.2
    rsc.io/quote/v3 v3.0.0
    rsc.io/sampler v1.3.1 // indirect
)
$

Why? Because building a single package, like with go build or go test, can easily tell when something is missing and needs to be added, but not when something can safely be removed. Removing a dependency can only be done after checking all packages in a module, and all possible build tag combinations for those packages. An ordinary build command does not load this information, and so it cannot safely remove dependencies.

The go mod tidy command cleans up these unused dependencies:

$ go mod tidy
$ go list -m all
example.com/hello
golang.org/x/text v0.3.0
rsc.io/quote/v3 v3.1.0
rsc.io/sampler v1.3.1
$ cat go.mod
module example.com/hello

go 1.12

require (
    golang.org/x/text v0.3.0 // indirect
    rsc.io/quote/v3 v3.1.0
    rsc.io/sampler v1.3.1 // indirect
)

$ go test
PASS
ok      example.com/hello    0.020s
$

Conclusion

Go modules are the future of dependency management in Go. Module functionality is now available in all supported Go versions (that is, in Go 1.11 and Go 1.12).

This post introduced these workflows using Go modules:

  • go mod init creates a new module, initializing the go.mod file that describes it.
  • go build, go test, and other package-building commands add new dependencies to go.mod as needed.
  • go list -m all prints the current module’s dependencies.
  • go get changes the required version of a dependency (or adds a new dependency).
  • go mod tidy removes unused dependencies.

We encourage you to start using modules in your local development and to add go.mod and go.sum files to your projects. To provide feedback and help shape the future of dependency management in Go, please send us bug reports or experience reports.

Thanks for all your feedback and help improving modules.

By Tyler Bui-Palsulich and Eno Compton

The New Go Developer Network

14 March 2019

A sense of community flourishes when we come together in person. As handles become names and avatars become faces, the smiles are real and true friendship can grow. There is joy in the sharing of knowledge and celebrating the accomplishments of our friends, colleagues, and neighbors. In our rapidly growing Go community this critical role is played by the Go user groups.

To better support our Go user groups worldwide, the Go community leaders at GoBridge and Google have joined forces to create a new program called the Go Developer Network (GDN). The GDN is a collection of Go user groups working together with a shared mission to empower developer communities with the knowledge, experience, and wisdom to build the next generation of software in Go.

We have partnered with Meetup to create our own Pro Network of Go Developers providing Go developers a single place to search for local user groups, events, and see what other Gophers are doing around the world.

User groups that join the GDN will be recognized by GoBridge as the official user group for that city and be provided with the latest news, information, conduct policies, and procedures. GDN groups will have Meetup fees paid by the GDN and will have access to special swag and other fun items. Each organizer of a GDN local group will continue to own the group and maintain full admin rights. If you currently run a user group, please fill out this application to request to join the GDN.

We hope you are as excited about the GDN as we are.

By GoBridge Leadership Team

What's new in the Go Cloud Development Kit

4 March 2019

Introduction

Last July, we introduced the Go Cloud Development Kit (previously referred to as simply "Go Cloud"), an open source project building libraries and tools to improve the experience of developing for the cloud with Go. We've made a lot of progress since then -- thank you to early contributors! We look forward to growing the Go CDK community of users and contributors, and are excited to work closely with early adopters.

Portable APIs

Our first initiative is a set of portable APIs for common cloud services. You write your application using these APIs, and then deploy it on any combination of providers, including AWS, GCP, Azure, on-premise, or on a single developer machine for testing. Additional providers can be added by implementing an interface.

These portable APIs are a great fit if any of the following are true:

  • You develop cloud applications locally.
  • You have on-premise applications that you want to run in the cloud (permanently, or as part of a migration).
  • You want portability across multiple clouds.
  • You are creating a new Go application that will use cloud services.

Unlike traditional approaches where you would need to write new application code for each cloud provider, with the Go CDK you write your application code once using our portable APIs to access the set of services listed below. Then, you can run your application on any supported cloud with minimal config changes.

Our current set of APIs includes:

  • blob, for persistence of blob data. Supported providers include: AWS S3, Google Cloud Storage (GCS), Azure Storage, the filesystem, and in-memory.
  • pubsub for publishing/subscribing of messages to a topic. Supported providers include: Amazon SNS/SQS, Google Pub/Sub, Azure Service Bus, RabbitMQ, and in-memory.
  • runtimevar, for watching external configuration variables. Supported providers include AWS Parameter Store, Google Runtime Configurator, etcd, and the filesystem.
  • secrets, for encryption/decryption. Supported providers include AWS KMS, GCP KMS, Hashicorp Vault, and local symmetric keys.
  • Helpers for connecting to cloud SQL providers. Supported providers include AWS RDS and Google Cloud SQL.
  • We are also working on a document storage API (e.g. MongoDB, DynamoDB, Firestore).

Feedback

We hope you're as excited about the Go CDK as we are -- check out our godoc, walk through our tutorial, and use the Go CDK in your application(s). We'd love to hear your ideas for other APIs and API providers you'd like to see.

If you're digging into Go CDK please share your experiences with us:

  • What went well?
  • Were there any pain points using the APIs?
  • Are there any features missing in the API you used?
  • Suggestions for documentation improvements.

To send feedback, you can:

Thanks!

By The Go Cloud Development Kit team at Google, Twitter #GoCDK

Go 1.12 is released

25 February 2019

Today the Go team is happy to announce the release of Go 1.12. You can get it from the download page.

For details about the changes in Go 1.12, see the Go 1.12 release notes.

Some of the highlights include opt-in support for TLS 1.3, improved modules support (in preparation for being the default in Go 1.13), support for windows/arm, and improved macOS & iOS forwards compatibility.

As always, we also want to thank everyone who contributed to this release by writing code, filing bugs, providing feedback, and/or testing the betas and release candidates. Your contributions and diligence helped to ensure that Go 1.12 is as stable as possible. That said, if you do notice any problems, please file an issue.

Enjoy the new release!

By Andrew Bonventre

Go Modules in 2019

19 December 2018

What a year!

2018 was a great year for the Go ecosystem, with package management as one of our major focuses. In February, we started a community-wide discussion about how to integrate package management directly into the Go toolchain, and in August we delivered the first rough implementation of that feature, called Go modules, in Go 1.11. The migration to Go modules will be the most far-reaching change for the Go ecosystem since Go 1. Converting the entire ecosystem—code, users, tools, and so on—from GOPATH to modules will require work in many different areas. The module system will in turn help us deliver better authentication and build speeds to the Go ecosystem.

This post is a preview of what the Go team is planning relating to modules in 2019.

Releases

Go 1.11, released in August 2018, introduced preliminary support for modules. For now, module support is maintained alongside the traditional GOPATH-based mechanisms. The go command defaults to module mode when run in directory trees outside GOPATH/src and marked by go.mod files in their roots. This setting can be overridden by setting the transitional environment variable $GO111MODULE to on or off; the default behavior is auto mode. We’ve already seen significant adoption of modules across the Go community, along with many helpful suggestions and bug reports to help us improve modules.

Go 1.12, scheduled for February 2019, will refine module support but still leave it in auto mode by default. In addition to many bug fixes and other minor improvements, perhaps the most significant change in Go 1.12 is that commands like go run x.go or go get rsc.io/2fa@v1.1.0 can now operate in GO111MODULE=on mode without an explicit go.mod file.

Our aim is for Go 1.13, scheduled for August 2019, to enable module mode by default (that is, to change the default from auto to on) and deprecate GOPATH mode. In order to do that, we’ve been working on better tooling support along with better support for the open-source module ecosystem.

Tooling & IDE Integration

In the eight years that we’ve had GOPATH, an incredible amount of tooling has been created that assumes Go source code is stored in GOPATH. Moving to modules requires updating all code that makes that assumption. We’ve designed a new package, golang.org/x/tools/go/packages, that abstracts the operation of finding and loading information about the Go source code for a given target. This new package adapts automatically to both GOPATH and modules mode and is also extensible to tool-specific code layouts, such as the one used by Bazel. We’ve been working with tool authors throughout the Go community to help them adopt golang.org/x/tools/go/packages in their tools.

As part of this effort, we’ve also been working to unify the various source code querying tools like gocode, godef, and go-outline into a single tool that can be used from the command line and also supports the language server protocol used by modern IDEs.

The transition to modules and the changes in package loading also prompted a significant change to Go program analysis. As part of reworking go vet to support modules, we introduced a generalized framework for incremental analysis of Go programs, in which an analyzer is invoked for one package at a time. In this framework, the analysis of one package can write out facts made available to analyses of other packages that import the first. For example, go vet’s analysis of the log package determines and records the fact that log.Printf is a fmt.Printf wrapper. Then go vet can check printf-style format strings in other packages that call log.Printf. This framework should enable many new, sophisticated program analysis tools to help developers find bugs earlier and understand code better.

Module Index

One of the most important parts of the original design for go get was that it was decentralized: we believed then—and we still believe today—that anyone should be able to publish their code on any server, in contrast to central registries such as Perl’s CPAN, Java’s Maven, or Node’s NPM. Placing domain names at the start of the go get import space reused an existing decentralized system and avoided needing to solve anew the problems of deciding who can use which names. It also allowed companies to import code on private servers alongside code from public servers. It is critical to preserve this decentralization as we shift to Go modules.

Decentralization of Go’s dependencies has had many benefits, but it also brought a few significant drawbacks. The first is that it’s too hard to find all the publicly-available Go packages. Every site that wants to deliver information about packages has to do its own crawling, or else wait until a user asks about a particular package before fetching it.

We are working on a new service, the Go Module Index, that will provide a public log of packages entering the Go ecosystem. Sites like godoc.org and goreportcard.com will be able to watch this log for new entries instead of each independently implementing code to find new packages. We also want the service to allow looking up packages using simple queries, to allow goimports to add imports for packages that have not yet been downloaded to the local system.

Module Authentication

Today, go get relies on connection-level authentication (HTTPS or SSH) to check that it is talking to the right server to download code. There is no additional check of the code itself, leaving open the possibility of man-in-the-middle attacks if the HTTPS or SSH mechanisms are compromised in some way. Decentralization means that the code for a build is fetched from many different servers, which means the build depends on many systems to serve correct code.

The Go modules design improves code authentication by storing a go.sum file in each module; that file lists the cryptographic hash of the expected file tree for each of the module’s dependencies. When using modules, the go command uses go.sum to verify that dependencies are bit-for-bit identical to the expected versions before using them in a build. But the go.sum file only lists hashes for the specific dependencies used by that module. If you are adding a new dependency or updating dependencies with go get -u, there is no corresponding entry in go.sum and therefore no direct authentication of the downloaded bits.

For publicly-available modules, we intend to run a service we call a notary that follows the module index log, downloads new modules, and cryptographically signs statements of the form “module M at version V has file tree hash H.” The notary service will publish all these notarized hashes in a queryable, Certificate Transparency-style tamper-proof log, so that anyone can verify that the notary is behaving correctly. This log will serve as a public, global go.sum file that go get can use to authenticate modules when adding or updating dependencies.

We are aiming to have the go command check notarized hashes for publicly-available modules not already in go.sum starting in Go 1.13.

Module Mirrors

Because the decentralized go get fetches code from multiple origin servers, fetching code is only as fast and reliable as the slowest, least reliable server. The only defense available before modules was to vendor dependencies into your own repositories. While vendoring will continue to be supported, we’d prefer a solution that works for all modules—not just the ones you’re already using—and that does not require duplicating a dependency into every repository that uses it.

The Go module design introduces the idea of a module proxy, which is a server that the go command asks for modules, instead of the origin servers. One important kind of proxy is a module mirror, which answers requests for modules by fetching them from origin servers and then caching them for use in future requests. A well-run mirror should be fast and reliable even when some origin servers have gone down. We are planning to launch a mirror service for publicly-available modules in 2019. Other projects, like GoCenter and Athens, are planning mirror services too. (We anticipate that companies will have multiple options for running their own internal mirrors as well, but this post is focusing on public mirrors.)

One potential problem with mirrors is that they are precisely man-in-the-middle servers, making them a natural target for attacks. Go developers need some assurance that the mirrors are providing the same bits that the origin servers would. The notary process we described in the previous section addresses exactly this concern, and it will apply to downloads using mirrors as well as downloads using origin servers. The mirrors themselves need not be trusted.

We are aiming to have the Google-run module mirror ready to be used by default in the go command starting in Go 1.13. Using an alternate mirror, or no mirror at all, will be trivial to configure.

Module Discovery

Finally, we mentioned earlier that the module index will make it easier to build sites like godoc.org. Part of our work in 2019 will be a major revamp of godoc.org to make it more useful for developers who need to discover available modules and then decide whether to rely on a given module or not.

Big Picture

This diagram shows how module source code moves through the design in this post.

Before, all consumers of Go source code—the go command and any sites like godoc.org—fetched code directly from each code host. Now they can fetch cached code from a fast, reliable mirror, while still authenticating that the downloaded bits are correct. And the index service makes it easy for mirrors, godoc.org, and any other similar sites to keep up with all the great new code being added to the Go ecosystem every day.

We’re excited about the future of Go modules in 2019, and we hope you are too. Happy New Year!

By Russ Cox

See the index for more articles.