The Go Blog

Package names

4 February 2015

Introduction

Go code is organized into packages. Within a package, code can refer to any identifier (name) defined within, while clients of the package may only reference the package's exported types, functions, constants, and variables. Such references always include the package name as a prefix: foo.Bar refers to the exported name Bar in the imported package named foo.

Good package names make code better. A package's name provides context for its contents, making it easier for clients to understand what the package is for and how to use it. The name also helps package maintainers determine what does and does not belong in the package as it evolves. Well-named packages make it easier to find the code you need.

Effective Go provides guidelines for naming packages, types, functions, and variables. This article expands on that discussion and surveys names found in the standard library. It also discusses bad package names and how to fix them.

Package names

Good package names are short and clear. They are lower case, with no under_scores or mixedCaps. They are often simple nouns, such as:

  • time (provides functionality for measuring and displaying time)
  • list (implements a doubly linked list)
  • http (provides HTTP client and server implementations)

The style of names typical of another language might not be idiomatic in a Go program. Here are two examples of names that might be good style in other languages but do not fit well in Go:

  • computeServiceClient
  • priority_queue

A Go package may export several types and functions. For example, a compute package could export a Client type with methods for using the service as well as functions for partitioning a compute task across several clients.

Abbreviate judiciously. Package names may be abbreviated when the abbreviation is familiar to the programmer. Widely-used packages often have compressed names:

  • strconv (string conversion)
  • syscall (system call)
  • fmt (formatted I/O)

On the other hand, if abbreviating a package name makes it ambiguous or unclear, don't do it.

Don't steal good names from the user. Avoid giving a package a name that is commonly used in client code. For example, the buffered I/O package is called bufio, not buf, since buf is a good variable name for a buffer.

Naming package contents

A package name and its contents' names are coupled, since client code uses them together. When designing a package, take the client's point of view.

Avoid stutter. Since client code uses the package name as a prefix when referring to the package contents, the names for those contents need not repeat the package name. The HTTP server provided by the http package is called Server, not HTTPServer. Client code refers to this type as http.Server, so there is no ambiguity.

Simplify function names. When a function in package pkg returns a value of type pkg.Pkg (or *pkg.Pkg), the function name can often omit the type name without confusion:

start := time.Now()                           // start is a time.Time
t, err := time.Parse(time.Kitchen, "6:06PM")  // t is a time.Time

A function named New in package pkg returns a value of type pkg.Pkg. This is a standard entry point for client code using that type:

q := list.New()  // q is a *list.List

When a function returns a value of type pkg.T, where T is not Pkg, the function name may include T to make client code easier to understand. A common situation is a package with multiple New-like functions:

d, err := time.ParseDuration("10s")  // d is a time.Duration
elapsed := time.Since(start)         // elapsed is a time.Duration
ticker := time.NewTicker(d)          // ticker is a *time.Ticker
timer := time.NewTimer(d)            // timer is a *time.Timer

Types in different packages can have the same name, because from the client's point of view such names are discriminated by the package name. For example, the standard library includes several types named Reader, including jpeg.Reader, bufio.Reader, and csv.Reader. Each package name fits with Reader to yield a good type name.

If you cannot come up with a package name that's a meaningful prefix for the package's contents, the package abstraction boundary may be wrong. Write code that uses your package as a client would, and restructure your packages if the result seems poor. This approach will yield packages that are easier for clients to understand and for the package developers to maintain.

Package paths

A Go package has both a name and a path. The package name is specified in the package statement of its source files; client code uses it as the prefix for the package's exported names. Client code uses the package path when importing the package. By convention, the last element of the package path is the package name:

import (
    "fmt"                       // package fmt
    "os/exec"                   // package exec
    "golang.org/x/net/context"  // package context
)

Build tools map package paths onto directories. The go tool uses the GOPATH environment variable to find the source files for path "github.com/user/hello" in directory $GOPATH/src/github.com/user/hello. (This situation should be familiar, of course, but it's important to be clear about the terminology and structure of packages.)

Directories. The standard library uses like directories crypto, container, encoding, and image to group packages for related protocols and algorithms. There is no actual relationship among the packages in one of these directories; a directory just provides a way to arrange the files. Any package can import any other package provided the import does not create a cycle.

Just as types in different packages can have the same name without ambiguity, packages in different directories can have the same name. For example, runtime/pprof provides profiling data in the format expected by the pprof profiling tool, while net/http/pprof provides HTTP endpoints to present profiling data in this format. Client code uses the package path to import the package, so there is no confusion. If a source file needs to import both pprof packages, it can rename one or both locally. When renaming an imported package, the local name should follow the same guidelines as package names (lower case, no under_scores or mixedCaps).

Bad package names

Bad package names make code harder to navigate and maintain. Here are some guidelines for recognizing and fixing bad names.

Avoid meaningless package names. Packages named util, common, or misc provide clients with no sense of what the package contains. This makes it harder for clients to use the package and makes it harder for maintainers to keep the package focused. Over time, they accumulate dependencies that can make compilation significantly and unnecessarily slower, especially in large programs. And since such package names are generic, they are more likely to collide with other packages imported by client code, forcing clients to invent names to distinguish them.

Break up generic packages. To fix such packages, look for types and functions with common name elements and pull them into their own package. For example, if you have

package util
func NewStringSet(...string) map[string]bool {...}
func SortStringSet(map[string]bool) []string {...}

then client code looks like

set := util.NewStringSet("c", "a", "b")
fmt.Println(util.SortStringSet(set))

Pull these functions out of util into a new package, choosing a name that fits the contents:

package stringset
func New(...string) map[string]bool {...}
func Sort(map[string]bool) []string {...}

then the client code becomes

set := stringset.New("c", "a", "b")
fmt.Println(stringset.Sort(set))

Once you've made this change, its easier to see how to improve the new package:

package stringset
type Set map[string]bool
func New(...string) Set {...}
func (s Set) Sort() []string {...}

which yields even simpler client code:

set := stringset.New("c", "a", "b")
fmt.Println(set.Sort())

The name of the package is a critical piece of its design. Work to eliminate meaningless package names from your projects.

Don't use a single package for all your APIs. Many well-intentioned programmers put all the interfaces exposed by their program into a single package named api, types, or interfaces, thinking it makes it easier to find the entry points to their code base. This is a mistake. Such packages suffer from the same problems as those named util or common, growing without bound, providing no guidance to users, accumulating dependencies, and colliding with other imports. Break them up, perhaps using directories to separate public packages from implementation.

Avoid unnecessary package name collisions. While packages in different directories may have the same name, packages that are frequently used together should have distinct names. This reduces confusion and the need for local renaming in client code. For the same reason, avoid using the same name as popular standard packages like io or http.

Conclusion

Package names are central to good naming in Go programs. Take the time to choose good package names and organize your code well. This helps clients understand and use your packages and helps maintainers to grow them gracefully.

Further reading

By Sameer Ajmani

Errors are values

12 January 2015

A common point of discussion among Go programmers, especially those new to the language, is how to handle errors. The conversation often turns into a lament at the number of times the sequence

if err != nil {
    return err
}

shows up. We recently scanned all the open source projects we could find and discovered that this snippet occurs only once per page or two, less often than some would have you believe. Still, if the perception persists that one must type

if err != nil

all the time, something must be wrong, and the obvious target is Go itself.

This is unfortunate, misleading, and easily corrected. Perhaps what is happening is that programmers new to Go ask, "How does one handle errors?", learn this pattern, and stop there. In other languages, one might use a try-catch block or other such mechanism to handle errors. Therefore, the programmer thinks, when I would have used a try-catch in my old language, I will just type if err != nil in Go. Over time the Go code collects many such snippets, and the result feels clumsy.

Regardless of whether this explanation fits, it is clear that these Go programmers miss a fundamental point about errors: Errors are values.

Values can be programmed, and since errors are values, errors can be programmed.

Of course a common statement involving an error value is to test whether it is nil, but there are countless other things one can do with an error value, and application of some of those other things can make your program better, eliminating much of the boilerplate that arises if every error is checked with a rote if statement.

Here's a simple example from the bufio package's Scanner type. Its Scan method performs the underlying I/O, which can of course lead to an error. Yet the Scan method does not expose an error at all. Instead, it returns a boolean, and a separate method, to be run at the end of the scan, reports whether an error occurred. Client code looks like this:

scanner := bufio.NewScanner(input)
for scanner.Scan() {
    token := scanner.Text()
    // process token
}
if err := scanner.Err(); err != nil {
    // process the error
}

Sure, there is a nil check for an error, but it appears and executes only once. The Scan method could instead have been defined as

func (s *Scanner) Scan() (token []byte, error)

and then the example user code might be (depending on how the token is retrieved),

scanner := bufio.NewScanner(input)
for {
    token, err := scanner.Scan()
    if err != nil {
        return err // or maybe break
    }
    // process token
}

This isn't very different, but there is one important distinction. In this code, the client must check for an error on every iteration, but in the real Scanner API, the error handling is abstracted away from the key API element, which is iterating over tokens. With the real API, the client's code therefore feels more natural: loop until done, then worry about errors. Error handling does not obscure the flow of control.

Under the covers what's happening, of course, is that as soon as Scan encounters an I/O error, it records it and returns false. A separate method, Err, reports the error value when the client asks. Trivial though this is, it's not the same as putting

if err != nil

everywhere or asking the client to check for an error after every token. It's programming with error values. Simple programming, yes, but programming nonetheless.

It's worth stressing that whatever the design, it's critical that the program check the errors however they are exposed. The discussion here is not about how to avoid checking errors, it's about using the language to handle errors with grace.

The topic of repetitive error-checking code arose when I attended the autumn 2014 GoCon in Tokyo. An enthusiastic gopher, who goes by @jxck_ on Twitter, echoed the familiar lament about error checking. He had some code that looked schematically like this:

_, err = fd.Write(p0[a:b])
if err != nil {
    return err
}
_, err = fd.Write(p1[c:d])
if err != nil {
    return err
}
_, err = fd.Write(p2[e:f])
if err != nil {
    return err
}
// and so on

It is very repetitive. In the real code, which was longer, there is more going on so it's not easy to just refactor this using a helper function, but in this idealized form, a function literal closing over the error variable would help:

var err error
write := func(buf []byte) {
    if err != nil {
        return
    }
    _, err = w.Write(buf)
}
write(p0[a:b])
write(p1[c:d])
write(p2[e:f])
// and so on
if err != nil {
    return err
}

This pattern works well, but requires a closure in each function doing the writes; a separate helper function is clumsier to use because the err variable needs to be maintained across calls (try it).

We can make this cleaner, more general, and reusable by borrowing the idea from the Scan method above. I mentioned this technique in our discussion but @jxck_ didn't see how to apply it. After a long exchange, hampered somewhat by a language barrier, I asked if I could just borrow his laptop and show him by typing some code.

I defined an object called an errWriter, something like this:

type errWriter struct {
    w   io.Writer
    err error
}

and gave it one method, write. It doesn't need to have the standard Write signature, and it's lower-cased in part to highlight the distinction. The write method calls the Write method of the underlying Writer and records the first error for future reference:

func (ew *errWriter) write(buf []byte) {
    if ew.err != nil {
        return
    }
    _, ew.err = ew.w.Write(buf)
}

As soon as an error occurs, the write method becomes a no-op but the error value is saved.

Given the errWriter type and its write method, the code above can be refactored:

ew := &errWriter{w: fd}
ew.write(p0[a:b])
ew.write(p1[c:d])
ew.write(p2[e:f])
// and so on
if ew.err != nil {
    return ew.err
}

This is cleaner, even compared to the use of a closure, and also makes the actual sequence of writes being done easier to see on the page. There is no clutter any more. Programming with error values (and interfaces) has made the code nicer.

It's likely that some other piece of code in the same package can build on this idea, or even use errWriter directly.

Also, once errWriter exists, there's more it could do to help, especially in less artificial examples. It could accumulate the byte count. It could coalesce writes into a single buffer that can then be transmitted atomically. And much more.

In fact, this pattern appears often in the standard library. The archive/zip and net/http packages use it. More salient to this discussion, the bufio package's Writer is actually an implementation of the errWriter idea. Although bufio.Writer.Write returns an error, that is mostly about honoring the io.Writer interface. The Write method of bufio.Writer behaves just like our errWriter.write method above, with Flush reporting the error, so our example could be written like this:

b := bufio.NewWriter(fd)
b.Write(p0[a:b])
b.Write(p1[c:d])
b.Write(p2[e:f])
// and so on
if b.Flush() != nil {
    return b.Flush()
}

There is one significant drawback to this approach, at least for some applications: there is no way to know how much of the processing completed before the error occurred. If that information is important, a more fine-grained approach is necessary. Often, though, an all-or-nothing check at the end is sufficient.

We've looked at just one technique for avoiding repetitive error handling code. Keep in mind that the use of errWriter or bufio.Writer isn't the only way to simplify error handling, and this approach is not suitable for all situations. The key lesson, however, is that errors are values and the full power of the Go programming language is available for processing them.

Use the language to simplify your error handling.

But remember: Whatever you do, always check your errors!

Finally, for the full story of my interaction with @jxck_, including a little video he recorded, visit his blog.

By Rob Pike

GothamGo: gophers in the big apple

9 January 2015

Last November more than two hundred gophers from all across the United States got together for the first full-day Go conference in New York City.

The diverse speaker lineup included university students, industry experts, and Go team members.

And good news, everybody! All the talks were recorded and are available:

Two more talks come from the Go meetup in New York City, which met the day before GothamGo:

  • Benchmarking Go by Brian Bulkowski - the founder of Aerospike talks about profiling tools for Go and Linux, and micro benchmarks for goroutines, channels, buffers, and and other Go features.
  • Go Static Analysis Tools by Alan Donovan - a member of the Go team at Google NY gives a guided tour of several static analysis tools designed to help Go programmers understand, navigate , and refactor their code.

Make sure to have a look at all of those in preparation for the FOSDEM Go devroom FOSDEM in Brussels (Belgium) and gophercon.in in Bengaluru (India).

By Francesc Campoy

The Gopher Gala is the first worldwide Go hackathon

7 January 2015

The Gopher Gala is the first Go hackathon at a global scale and will take place from January 23rd through the 25th. The event is organized by the community, supported by the Go team, and sponsored by Google among others.

Fancy Gopher, by Renée French

You can read about the rules of the hackathon here, but if you know about Rails Rumble or Node Knockout you already have a pretty good idea of what to expect.

During this event gophers from all around the globe will form teams to build great applications using Go as the main tool. Afterwards, all the participants will vote for their favorite applications and the 20 highest voted applications will be ranked by a jury of renowned names from the community, including some members of the Go core team. More information on the judging phase can be found here.

And in case you needed one more reason to get involved there will be prizes!

"What prizes?" you ask. Well, that’s a secret until January 10th but we’re pretty sure you won’t be disappointed.

By Francesc Campoy

Generating code

22 December 2014

A property of universal computation—Turing completeness—is that a computer program can write a computer program. This is a powerful idea that is not appreciated as often as it might be, even though it happens frequently. It's a big part of the definition of a compiler, for instance. It's also how the go test command works: it scans the packages to be tested, writes out a Go program containing a test harness customized for the package, and then compiles and runs it. Modern computers are so fast this expensive-sounding sequence can complete in a fraction of a second.

There are lots of other examples of programs that write programs. Yacc, for instance, reads in a description of a grammar and writes out a program to parse that grammar. The protocol buffer "compiler" reads an interface description and emits structure definitions, methods, and other support code. Configuration tools of all sorts work like this too, examining metadata or the environment and emitting scaffolding customized to the local state.

Programs that write programs are therefore important elements in software engineering, but programs like Yacc that produce source code need to be integrated into the build process so their output can be compiled. When an external build tool like Make is being used, this is usually easy to do. But in Go, whose go tool gets all necessary build information from the Go source, there is a problem. There is simply no mechanism to run Yacc from the go tool alone.

Until now, that is.

The latest Go release, 1.4, includes a new command that makes it easier to run such tools. It's called go generate, and it works by scanning for special comments in Go source code that identify general commands to run. It's important to understand that go generate is not part of go build. It contains no dependency analysis and must be run explicitly before running go build. It is intended to be used by the author of the Go package, not its clients.

The go generate command is easy to use. As a warmup, here's how to use it to generate a Yacc grammar. Say you have a Yacc input file called gopher.y that defines a grammar for your new language. To produce the Go source file implementing the grammar, you would normally invoke the standard Go version of Yacc like this:

go tool yacc -o gopher.go -p parser gopher.y

The -o option names the output file while -p specifies the package name.

To have go generate drive the process, in any one of the regular (non-generated) .go files in the same directory, add this comment anywhere in the file:

//go:generate go tool yacc -o gopher.go -p parser gopher.y

This text is just the command above prefixed by a special comment recognized by go generate. The comment must start at the beginning of the line and have no spaces between the // and the go:generate. After that marker, the rest of the line specifies a command for go generate to run.

Now run it. Change to the source directory and run go generate, then go build and so on:

$ cd $GOPATH/myrepo/gopher
$ go generate
$ go build
$ go test

That's it. Assuming there are no errors, the go generate command will invoke yacc to create gopher.go, at which point the directory holds the full set of Go source files, so we can build, test, and work normally. Every time gopher.y is modified, just rerun go generate to regenerate the parser.

For more details about how go generate works, including options, environment variables, and so on, see the design document.

Go generate does nothing that couldn't be done with Make or some other build mechanism, but it comes with the go tool—no extra installation required—and fits nicely into the Go ecosystem. Just keep in mind that it is for package authors, not clients, if only for the reason that the program it invokes might not be available on the target machine. Also, if the containing package is intended for import by go get, once the file is generated (and tested!) it must be checked into the source code repository to be available to clients.

Now that we have it, let's use it for something new. As a very different example of how go generate can help, there is a new program available in the golang.org/x/tools repository called stringer. It automatically writes string methods for sets of integer constants. It's not part of the released distribution, but it's easy to install:

$ go get golang.org/x/tools/cmd/stringer

Here's an example from the documentation for stringer. Imagine we have some code that contains a set of integer constants defining different types of pills:

package painkiller

type Pill int

const (
    Placebo Pill = iota
    Aspirin
    Ibuprofen
    Paracetamol
    Acetaminophen = Paracetamol
)

For debugging, we'd like these constants to pretty-print themselves, which means we want a method with signature,

func (p Pill) String() string

It's easy to write one by hand, perhaps like this:

func (p Pill) String() string {
    switch p {
    case Placebo:
        return "Placebo"
    case Aspirin:
        return "Aspirin"
    case Ibuprofen:
        return "Ibuprofen"
    case Paracetamol: // == Acetaminophen
        return "Paracetamol"
    }
    return fmt.Sprintf("Pill(%d)", p)
}

There are other ways to write this function, of course. We could use a slice of strings indexed by Pill, or a map, or some other technique. Whatever we do, we need to maintain it if we change the set of pills, and we need to make sure it's correct. (The two names for paracetamol make this trickier than it might otherwise be.) Plus the very question of which approach to take depends on the types and values: signed or unsigned, dense or sparse, zero-based or not, and so on.

The stringer program takes care of all these details. Although it can be run in isolation, it is intended to be driven by go generate. To use it, add a generate comment to the source, perhaps near the type definition:

//go:generate stringer -type=Pill

This rule specifies that go generate should run the stringer tool to generate a String method for type Pill. The output is automatically written to pill_string.go (a default we could override with the -output flag).

Let's run it:

$ go generate
$ cat pill_string.go
// generated by stringer -type Pill pill.go; DO NOT EDIT

package pill

import "fmt"

const _Pill_name = "PlaceboAspirinIbuprofenParacetamol"

var _Pill_index = [...]uint8{0, 7, 14, 23, 34}

func (i Pill) String() string {
    if i < 0 || i+1 >= Pill(len(_Pill_index)) {
        return fmt.Sprintf("Pill(%d)", i)
    }
    return _Pill_name[_Pill_index[i]:_Pill_index[i+1]]
}
$

Every time we change the definition of Pill or the constants, all we need to do is run

$ go generate

to update the String method. And of course if we've got multiple types set up this way in the same package, that single command will update all their String methods with a single command.

There's no question the generated method is ugly. That's OK, though, because humans don't need to work on it; machine-generated code is often ugly. It's working hard to be efficient. All the names are smashed together into a single string, which saves memory (only one string header for all the names, even if there are zillions of them). Then an array, _Pill_index, maps from value to name by a simple, efficient technique. Note too that _Pill_index is an array (not a slice; one more header eliminated) of uint8, the smallest integer sufficient to span the space of values. If there were more values, or there were negatives ones, the generated type of _Pill_index might change to uint16 or int8: whatever works best.

The approach used by the methods printed by stringer varies according to the properties of the constant set. For instance, if the constants are sparse, it might use a map. Here's a trivial example based on a constant set representing powers of two:

const _Power_name = "p0p1p2p3p4p5..."

var _Power_map = map[Power]string{
    1:    _Power_name[0:2],
    2:    _Power_name[2:4],
    4:    _Power_name[4:6],
    8:    _Power_name[6:8],
    16:   _Power_name[8:10],
    32:   _Power_name[10:12],
    ...,
}

func (i Power) String() string {
    if str, ok := _Power_map[i]; ok {
        return str
    }
    return fmt.Sprintf("Power(%d)", i)
}

In short, generating the method automatically allows us to do a better job than we would expect a human to do.

There are lots of other uses of go generate already installed in the Go tree. Examples include generating Unicode tables in the unicode package, creating efficient methods for encoding and decoding arrays in encoding/gob, producing time zone data in the time package, and so on.

Please use go generate creatively. It's there to encourage experimentation.

And even if you don't, use the new stringer tool to write your String methods for your integer constants. Let the machine do the work.

By Rob Pike

See the index for more articles.