The Go Blog

Announcing App Engine’s New Go 1.11 Runtime

16 October 2018

App Engine launched experimental support for Go in 2011. In the subsequent years, the Go community has grown significantly and has settled on idiomatic patterns for cloud-based applications. Today, Google Cloud is announcing a new Go 1.11 runtime for the App Engine standard environment that provides all the power of App Engine—things like paying only for what you use, automatic scaling, and managed infrastructure—while supporting idiomatic Go.

Starting with Go 1.11, Go on App Engine has no limits on application structure, supported packages, context.Context values, or HTTP clients. Write your Go application however you prefer, add an app.yaml file, and your app is ready to deploy on App Engine. Specifying Dependencies describes how the new runtime supports vendoring and modules (experimental) for dependency management.

Along with Cloud Functions support for Go (more on that in a future post), App Engine provides a compelling way to run Go code on Google Cloud Platform (GCP) with no concern for the underlying infrastructure.

Let’s take a look at creating a small application for App Engine. For the example here, we assume a GOPATH-based workflow, although Go modules have experimental support as well.

First, you create the application in your GOPATH:

// This server can run on App Engine.
package main

import (
    "fmt"
    "log"
    "net/http"
    "os"
)

func main() {
    port := os.Getenv("PORT")
    if port == "" {
        port = "8080"
    }
    http.HandleFunc("/", hello)

    log.Fatal(http.ListenAndServe(fmt.Sprintf(":%s", port), nil))
}

func hello(w http.ResponseWriter, r *http.Request) {
    w.Write([]byte("Hello, 世界"))
}

The code contains an idiomatic setup for a small HTTP server that responds with “Hello, 世界.” If you have previous App Engine experience, you’ll notice the absence of any call to appengine.Main(), which is now entirely optional. Furthermore, the application code is completely portable—there are no ties to the infrastructure that your application is deployed on.

If you need to use external dependencies, you can add those dependencies to a vendor directory or to a go.mod file, both of which the new runtime supports.

With the application code complete, create an app.yaml file to specify the runtime:

runtime: go111

Finally, set your machine up with a Google Cloud Platform account:

With all the setup complete, you can deploy using one command:

gcloud app deploy

We think Go developers will find the new Go 1.11 runtime for App Engine an exciting addition to the available options to run Go applications. There is a free tier. Check out the getting started guide or the migration guide and deploy an app to the new runtime today!

By Eno Compton and Tyler Bui-Palsulich

Compile-time Dependency Injection With Go Cloud's Wire

9 October 2018

Overview

The Go team recently announced the open source project Go Cloud, with portable Cloud APIs and tools for open cloud development. This post goes into more detail about Wire, a dependency injection tool provided with Go Cloud.

What problem does Wire solve?

Dependency injection is a standard technique for producing flexible and loosely coupled code, by explicitly providing components with all of the dependencies they need to work. In Go, this often takes the form of passing dependencies to constructors:

// NewUserStore returns a UserStore that uses cfg and db as dependencies.
func NewUserStore(cfg *Config, db *mysql.DB) (*UserStore, error) {...}

This technique works great at small scale, but larger applications can have a complex graph of dependencies, resulting in a big block of initialization code that's order-dependent but otherwise not very interesting. It's often hard to break up this code cleanly, especially because some dependencies are used multiple times. Replacing one implementation of a service with another can be painful because it involves modifying the dependency graph by adding a whole new set of dependencies (and their dependencies...), and removing unused old ones. In practice, making changes to initialization code in applications with large dependency graphs is tedious and slow.

Dependency injection tools like Wire aim to simplify the management of initialization code. You describe your services and their dependencies, either as code or as configuration, then Wire processes the resulting graph to figure out ordering and how to pass each service what it needs. Make changes to an application's dependencies by changing a function signature or adding or removing an initializer, and then let Wire do the tedious work of generating initialization code for the entire dependency graph.

Why is this part of Go Cloud?

Go Cloud's goal is to make it easier to write portable Cloud applications by providing idiomatic Go APIs for useful Cloud services. For example, blob.Bucket provides a storage API with implementations for Amazon's S3 and Google Cloud Storage (GCS); applications written using blob.Bucket can swap implementations without changing their application logic. However, the initialization code is inherently provider-specific, and each provider has a different set of dependencies.

For example, constructing a GCS blob.Bucket requires a gcp.HTTPClient, which eventually requires google.Credentials, while constructing one for S3 requires an aws.Config, which eventually requires AWS credentials. Thus, updating an application to use a different blob.Bucket implementation involves exactly the kind of tedious update to the dependency graph that we described above. The driving use case for Wire is to make it easy to swap implementations of Go Cloud portable APIs, but it's also a general-purpose tool for dependency injection.

Hasn't this been done already?

There are a number of dependency injection frameworks out there. For Go, Uber's dig and Facebook's inject both use reflection to do runtime dependency injection. Wire was primarily inspired by Java's Dagger 2, and uses code generation rather than reflection or service locators.

We think this approach has several advantages:

  • Runtime dependency injection can be hard to follow and debug when the dependency graph gets complex. Using code generation means that the initialization code that's executed at runtime is regular, idiomatic Go code that's easy to understand and debug. Nothing is obfuscated by an intervening framework doing "magic". In particular, problems like forgetting a dependency become compile-time errors, not run-time errors.
  • Unlike service locators, there's no need to make up arbitrary names or keys to register services. Wire uses Go types to connect components with their dependencies.
  • It's easier to avoid dependency bloat. Wire's generated code will only import the dependencies you need, so your binary won't have unused imports. Runtime dependency injectors can't identify unused dependencies until runtime.
  • Wire's dependency graph is knowable statically, which provides opportunities for tooling and visualization.

How does it work?

Wire has two basic concepts: providers and injectors.

Providers are ordinary Go functions that "provide" values given their dependencies, which are described simply as parameters to the function. Here's some sample code that defines three providers:

// NewUserStore is the same function we saw above; it is a provider for UserStore,
// with dependencies on *Config and *mysql.DB.
func NewUserStore(cfg *Config, db *mysql.DB) (*UserStore, error) {...}

// NewDefaultConfig is a provider for *Config, with no dependencies.
func NewDefaultConfig() *Config {...}

// NewDB is a provider for *mysql.DB based on some connection info.
func NewDB(info *ConnectionInfo) (*mysql.DB, error) {...}

Providers that are commonly used together can be grouped into ProviderSets. For example, it's common to use a default *Config when creating a *UserStore, so we can group NewUserStore and NewDefaultConfig in a ProviderSet:

var UserStoreSet = wire.ProviderSet(NewUserStore, NewDefaultConfig)

Injectors are generated functions that call providers in dependency order. You write the injector's signature, including any needed inputs as arguments, and insert a call to wire.Build with the list of providers or provider sets that are needed to construct the end result:

func initUserStore() (*UserStore, error) {
    // We're going to get an error, because NewDB requires a *ConnectionInfo
    // and we didn't provide one.
    wire.Build(UserStoreSet, NewDB)
    return nil, nil  // These return values are ignored.
}

Now we run go generate to execute wire:

$ go generate
wire.go:2:10: inject initUserStore: no provider found for ConnectionInfo (required by provider of *mysql.DB)
wire: generate failed

Oops! We didn't include a ConnectionInfo or tell Wire how to build one. Wire helpfully tells us the line number and types involved. We can either add a provider for it to wire.Build, or add it as an argument:

func initUserStore(info ConnectionInfo) (*UserStore, error) {
    wire.Build(UserStoreSet, NewDB)
    return nil, nil  // These return values are ignored.
}

Now `go generate` will create a new file with the generated code:

// File: wire_gen.go
// Code generated by Wire. DO NOT EDIT.
//go:generate wire
//+build !wireinject

func initUserStore(info ConnectionInfo) (*UserStore, error) {
    defaultConfig := NewDefaultConfig()
    db, err := NewDB(info)
    if err != nil {
        return nil, err
    }
    userStore, err := NewUserStore(defaultConfig, db)
    if err != nil {
        return nil, err
    }
    return userStore, nil
}

Any non-injector declarations are copied into the generated file. There is no dependency on Wire at runtime: all of the written code is just normal Go code.

As you can see, the output is very close to what a developer would write themselves. This was a trivial example with just three components, so writing the initializer by hand wouldn't be too painful, but Wire saves a lot of manual toil for components and applications with more complex dependency graphs.

How can I get involved and learn more?

The Wire README goes into more detail about how to use Wire and its more advanced features. There's also a tutorial that walks through using Wire in a simple application.

We appreciate any input you have about your experience with Wire! Go Cloud's development is conducted on GitHub, so you can file an issue to tell us what could be better. For updates and discussion about the project, join the project’s mailing list.

Thank you for taking the time to learn about Go Cloud's Wire. We’re excited to work with you to make Go the language of choice for developers building portable cloud applications.

By Robert van Gent

Participate in the 2018 Go Company Questionnaire

4 October 2018

The Go project wants to hear from you!

We need your help to create the best programming language for developing simple, reliable, and scalable software. To do this, we need to better understand how companies are using Go. Please help by participating in a 7-minute company questionnaire.

Who: If you are in a position to share details like “company name,” “if your company is hiring Go developers,” and “reasons your team or company adopted Go” then please help us by taking this company questionnaire. We only need one response per company (or per department for larger companies). If you aren’t the right person, please forward this onto the right person at your company.

Please note: this is different from our annual anonymous Go user survey, which will be announced in November.

Where: Please take this 7-minute questionnaire by October 30th: Go Company Questionnaire 2018.

The questionnaire is confidential, but not anonymous. For more information, please refer to Google’s privacy policy here.

The Go project leadership will use your responses to better understand how companies use Go and in what ways we can improve their experience.

Spread the word!

We would like as many companies as possible to participate to help us better understand our global user base. Please help us spread the word by sharing this post on your social network feeds, at meetups, and in other communities.

By Ran Tao, Steve Francia

Go 2 Draft Designs

28 August 2018

Yesterday, at our annual Go contributor summit, attendees got a sneak peek at preliminary drafts of possible designs for changes to error handling and generics. The development of Go 2 was announced last year and we are excited to share updates with you today.

For a quick overview, watch this short message we just played at Gophercon 2018:

We invite everyone in the Go community to learn more about the designs and help us improve them.

Go 1.11 is released

24 August 2018

Who says releasing on Friday is a bad idea?

Today the Go team is happy to announce the release of Go 1.11. You can get it from the download page.

There are many changes and improvements to the toolchain, runtime, and libraries, but two features stand out as being especially exciting: modules and WebAssembly support.

This release adds preliminary support for a new concept called “modules,” an alternative to GOPATH with integrated support for versioning and package distribution. Module support is considered experimental, and there are still a few rough edges to smooth out, so please make liberal use of the issue tracker.

Go 1.11 also adds an experimental port to WebAssembly (js/wasm). This allows programmers to compile Go programs to a binary format compatible with four major web browsers. You can read more about WebAssembly (abbreviated “Wasm”) at webassembly.org and see this wiki page on how to get started with using Wasm with Go. Special thanks to Richard Musiol for contributing the WebAssembly port!

We also want to thank everyone who contributed to this release by writing code, filing bugs, providing feedback, and/or testing the betas and release candidates. Your contributions and diligence helped to ensure that Go 1.11 is as bug-free as possible. That said, if you do notice any problems, please file an issue.

For more detail about the changes in Go 1.11, see the release notes.

Have a wonderful weekend and enjoy the release!

By Andrew Bonventre

See the index for more articles.