go fmt your code

Gofmt is a tool that automatically formats Go source code.

Gofmt'd code is:

1. easier to write: never worry about minor formatting concerns while hacking away,
2. easier to read: when all code looks the same you need not mentally convert others' formatting style into something you can understand.
3. easier to maintain: mechanical changes to the source don't cause unrelated changes to the file's formatting; diffs show only the real changes.
4. uncontroversial: never have a debate about spacing or brace position ever again!

Format your code

We recently conducted a survey of Go packages in the wild and found that about 70% of them are formatted according to gofmt's rules. This was more than expected - and thanks to everyone who uses gofmt - but it would be great to close the gap.

To format your code, you can use the gofmt tool directly:

 gofmt -w yourcode.go

Or you can use the "go fmt" command:

 go fmt path/to/your/package

To help keep your code in the canonical style, the Go repository contains hooks for editors and version control systems that make it easy to run gofmt on your code.

For Vim users, the Vim plugin for Go includes the :Fmt command that runs gofmt on the current buffer.

For emacs users, go-mode.el provides a gofmt-before-save hook that can be installed by adding this line to your .emacs file:

 (add-hook 'before-save-hook #'gofmt-before-save)

For Eclipse or Sublime Text users, the GoClipse and GoSublime projects add a gofmt facility to those editors.

And for Git aficionados, the misc/git/pre-commit script is a pre-commit hook that prevents incorrectly-formatted Go code from being committed. If you use Mercurial, the hgstyle plugin provides a gofmt pre-commit hook.

Mechanical source transformation

One of the greatest virtues of machine-formatted code is that it can be transformed mechanically without generating unrelated formatting noise in the diffs. Mechanical transformation is invaluable when working with large code bases, as it is both more comprehensive and less error prone than making wide-sweeping changes by hand. Indeed, when working at scale (like we do at Google) it often isn't practical to make these kinds of changes manually.

The easiest way to mechanically manipulate Go code is with gofmt's -r flag. The flag specifies a rewrite rule of the form

 pattern -> replacement

where both pattern and replacement are valid Go expressions. In the pattern, single-character lowercase identifiers serve as wildcards matching arbitrary sub-expressions, and those expressions are substituted for the same identifiers in the replacement.

For example, this recent change to the Go core rewrote some uses of bytes.Compare to use the more efficient bytes.Equal. The contributor made the change using just two gofmt invocations:

 gofmt -r 'bytes.Compare(a, b) == 0 -> bytes.Equal(a, b)'
 gofmt -r 'bytes.Compare(a, b) != 0 -> !bytes.Equal(a, b)'

Gofmt also enables gofix, which can make arbitrarily complex source transformations. Gofix was an invaluable tool during the early days when we regularly made breaking changes to the language and libraries. For example, before Go 1 the built-in error interface didn't exist and the convention was to use the os.Error type. When we introduced error, we provided a gofix module that rewrote all references to os.Error and its associated helper functions to use error and the new errors package. It would have been daunting to attempt by hand, but with the code in a standard format it was relatively easy to prepare, execute, and review this change which touched almost all Go code in existence.

For more about gofix, see this article.

Concurrency is not parallelism

If there's one thing most people know about Go, is that it is designed for concurrency. No introduction to Go is complete without a demonstration of its goroutines and channels.

But when people hear the word concurrency they often think of parallelism, a related but quite distinct concept. In programming, concurrency is the composition of independently executing processes, while parallelism is the simultaneous execution of (possibly related) computations. Concurrency is about dealing with lots of things at once. Parallelism is about doing lots of things at once.

To clear up this conflation, Rob Pike gave a talk at Heroku's Waza conference entitled Concurrency is not parallelism, and a video recording of the talk was released a few months ago.

The slides are available at talks.golang.org (use the left and right arrow keys to navigate).

To learn about Go's concurrency primitives, watch Go concurrency patterns (slides).

The App Engine SDK and workspaces (GOPATH)

When we released Go 1 we introduced the go tool and, with it, the concept of workspaces. Workspaces (specified by the GOPATH environment variable) are a convention for organizing code that simplifies fetching, building, and installing Go packages. If you're not familiar with workspaces, please read this article or watch this screencast before reading on.

Until recently, the tools in the App Engine SDK were not aware of workspaces. Without workspaces the "go get" command cannot function, and so app authors had to install and update their app dependencies manually. It was a pain.

This has all changed with version 1.7.4 of the App Engine SDK. The dev_appserver and appcfg tools are now workspace-aware. When running locally or uploading an app, these tools now search for dependencies in the workspaces specified by the GOPATH environment variable. This means you can now use "go get" while building App Engine apps, and switch between normal Go programs and App Engine apps without changing your environment or habits.

For example, let's say you want to build an app that uses OAuth 2.0 to authenticate with a remote service. A popular OAuth 2.0 library for Go is the goauth2 package, which you can install to your workspace with this command:

     go get code.google.com/p/goauth2/oauth

When writing your App Engine app, import the oauth package just as you would in a regular Go program:

     import "code.google.com/p/goauth2/oauth"

Now, whether running your app with the dev_appserver or deploying it with appcfg, the tools will find the oauth package in your workspace. It just works.

Hybrid stand-alone/App Engine apps

The Go App Engine SDK builds on Go's standard net/http package to serve web requests and, as a result, many Go web servers can be run on App Engine with only a few changes. For example, godoc is included in the Go distribution as a stand-alone program, but it can also run as an App Engine app (godoc serves golang.org from App Engine).

But wouldn't it be nice if you could write a program that is both a stand-alone web server and an App Engine app? By using build constraints, you can.

Build constraints are line comments that determine whether a file should be included in a package. They are most often used in code that handles a variety of operating systems or processor architectures. For instance, the path/filepath package includes the file symlink.go, which specifies a build constraint to ensure that it is not built on Windows systems (which do not have symbolic links):

    // +build !windows

The App Engine SDK introduces a new build constraint term: "appengine". Files that specify

    // +build appengine

will be built by the App Engine SDK and ignored by the go tool. Conversely, files that specify

    // +build !appengine

are ignored by the App Engine SDK, while the go tool will happily build them.

The goprotobuf library uses this mechanism to provide two implementations of a key part of its encode/decode machinery: pointer_unsafe.go is the faster version that cannot be used on App Engine because it uses the unsafe package, while pointer_reflect.go is a slower version that avoids unsafe by using the reflect package instead.

Let's take a simple Go web server and turn it into a hybrid app. This is main.go:

    package main
    
    import (
        "fmt"
        "net/http"
    )
    
    func main() {
        http.HandleFunc("/", handler)
        http.ListenAndServe("localhost:8080", nil)
    }
    
    func handler(w http.ResponseWriter, r *http.Request) {
        fmt.Fprint(w, "Hello!")
    }

Build this with the go tool and you'll get a stand-alone web server executable.

The App Engine infrastructure provides its own main function that runs its equivalent to ListenAndServe. To convert main.go to an App Engine app, drop the call to ListenAndServe and register the handler in an init function (which runs before main). This is app.go:

    package main
    
    import (
        "fmt"
        "net/http"
    )
    
    func init() {
        http.HandleFunc("/", handler)
    }
    
    func handler(w http.ResponseWriter, r *http.Request) {
        fmt.Fprint(w, "Hello!")
    }

To make this a hybrid app, we need to split it into an App Engine-specific part, an stand-alone binary-specific part, and the parts common to both versions. In this case, there is no App Engine-specific part, so we split it into just two files:

app.go specifies and registers the handler function. It is identical to the code listing above, and requires no build constraints as it should be included in all versions of the program.

main.go runs the web server. It includes the "!appengine" build constraint, as it must only included when building the stand-alone binary.

    // +build !appengine
    
    package main
    
    import "net/http"
    
    func main() {
        http.ListenAndServe("localhost:8080", nil)        
    }

To see a more complex hybrid app, take a look at the present tool.

Conclusions

We hope these changes will make it easier to work on apps with external dependencies, and to maintain code bases that contain both stand-alone programs and App Engine apps.

Two recent Go talks

Late last year I wrote a couple of Go talks and presented them at Strange Loop, Øredev, and various other venues. The talks are designed to give insight into the practice of Go programming, each describing the construction of a real program and demonstrating the power and depth of the Go language and its libraries and tools.

The following videos are, in my opinion, the best recordings of these talks.

Go: a simple programming environment

Go is a general-purpose language that bridges the gap between efficient statically typed languages and productive dynamic language. But it’s not just the language that makes Go special – Go has broad and consistent standard libraries and powerful but simple tools.

This talk gives an introduction to Go, followed by a tour of some real programs that demonstrate the power, scope, and simplicity of the Go programming environment.

See the slide deck (use the left and right arrows to navigate).

Go: code that grows with grace

One of Go's key design goals is code adaptability; that it should be easy to take a simple design and build upon it in a clean and natural way. In this talk I describe a simple "chat roulette" server that matches pairs of incoming TCP connections, and then use Go's concurrency mechanisms, interfaces, and standard library to extend it with a web interface and other features. While the function of the program changes dramatically, Go's flexibility preserves the original design as it grows.

See the slide deck (use the left and right arrows to navigate).