At Revinate, we use PHP and Java extensively. These languages give us a good balance between developer efficiency and robustness for building frontend and backend systems, respectively, however we continuously evaluate new languages that may offer even greater developer efficiency and robustness. Enter Go. On paper Go fits the bill perfectly due to its dynamic programming ease, C-like performance, and solid garbage collection support. To get a feel of Go, I wrote a service so that I could learn as well evaluate if it is worthwhile to put into our stack. I learned a lot of things in the process and I want to share these learnings with others who are starting to use the language for real projects. Just a note that I am still a very young gopher and my suggestions might not be the most accurate, therefore I am open to improvements or suggestions from the Go community. Just a note that my code is not publically available as it was an internal project.

Go IDE

It was hard finding a great IDE for Go. Most people are using IDEs that don’t provide the entire toolset that can help you develop Go code. You can hop on here and view the list of IDEs but most of them lack in some way or the other. I started my project with Atom with Go-Plus plugin but it did not have great ctags support and it was always pretty hard to easily find out function signature using a shortcut key. Then I came across the Go-IDE plugin for IntelliJ IDEs, which was a life saver. Just use it and save yourself the trouble. If you are a vim loyalist, checkout Vim-go.

A few things you might want to setup to streamline your workflow:

  • If you are using Go 1.4, set the GOPATH environment variable to the path where you store your Go code. Read How to Write Go Code for more info

  • If you using Go 1.5, you don’t have to set GOPATH. Instead you can use any of the third party package managers. I used Glide for my project.

  • Go to "Preferences" > "Languages & Frameworks" > "Go" > "Go SDK" and set the SDK by selecting the Go install path. This will enable features like "Jump to Definition" and quick definition lookup in the IDE.

  • Install goimports from here. And then add a File Watcher to run goimports on file save. Few tips on setting up this watcher:

    • Use the full path of the binary.

    • Set the arguments to be “-w .”. It will replace the current code with formatted code

    • Set "Working Directory" to $ProjectFileDir$

  • Add another watcher for "Go Generate" which run the command automatically when certain files are saved. Read Accessing Static YML or JSON Files section for more details.

  • For info about Go tools, checkout this great post

REST API in Go

There are plenty of Routers for Go and I ended up using Gin-Gonic to create REST APIs. They all have pretty much the same interface for defining routes and handlers but be sure to pick a router than you can easily understand and that has support for adding custom middleware that you might need for authentication, logging, analytics, etc. Setting up routing is as simple as initializing the Gin router and setting up route handlers

r := gin.Default()
handler := EventHandler{}
r.GET("/event/:id", handler.Get)
r.POST("/event", handler.Post)

Gin-Gonic does not support graceful stop which means if your Go app is killed, in-flight requests will not complete prior to your app shutting down. This is an unacceptable behavior but thanks to fvbock/endless package, you can use this alternate version of http server that does shutdown gracefully when it receives kill signals.

Use:

// r is gin router
endless.ListenAndServe(":80", r)

instead of:

// r is gin router
r.Run(":80")

To know how to use Gin in testing, go to section on testing.

Goroutines

One of the coolest features of Go is how easy it makes writing asynchronous code. Appending go in front of any function call makes that function run asynchronously. Goroutines are great but you need to be careful about terminating them as they are not garbage collected if they don’t return. There is another thing you need to be aware of when working with Goroutines: before quitting the App, ensure that all Goroutines successfully return. This can be done by using sync.WaitGroup as follows:

func main() {
        wg := sync.Waitgroup{}
        for i:=1; i<=3; i++ {
                wg.Add(1) // Tell WaitGroup that we are starting a goroutine
                go func() {
                        defer wg.Done() // executes before returning and tells WaitGroup that I am done
                        // Do some useful work here
                }()
        }
        wg.Wait() // Wait for all 3 goroutines to end before quitting
}

Graceful Shutdown

Gracefully shutting down your app is very important because you don’t want to leave API requests or any other operations midway when your app is requested to be killed. How do you do that in Go? To achieve that, your app needs to listen to OS signals for any kill signals. Luckily, you can create a OS signal channel and subscribe to a list of system signals and take some action when you receive something on that channel.

func main() {
        // Create a signal channel
        exit := make(chan os.Signal)
        // Subscribe to kill signals
        signal.Notify(exit, syscall.SIGINT, syscall.SIGTERM, syscall.SIGHUP, syscall.SIGTSTP)

        go func() {
                // Do some useful work
        }()

        // Wait for Kill Signal
        <-exit

        // You will reach here only if app is requested to be killed, therefore do cleanup here
        doCleanup()
}

Okay, so now we can wait for the quit signal and then quit but what happens when your Goroutines are in the middle of something and your app receives a kill signal? We ideally want to tell Goroutines to stop gracefully and return so that we can quit the go process. This can be done by passing a channel to all of your goroutines on which you can notify them when you want to quit the app. Lets see an example:

func someUsefulWork(done <-chan bool) {
        // Assume you get a channel from somewhere which has lot of work for you
        workChan := GetWorkChannel()
        for {
                select {
                        case work := <-workChan
                                // Do work here
                        case <-done
                                // Do cleanup
                                return
                }
        }
}

func main() {
        exit := make(chan os.Signal)
        signal.Notify(exit, syscall.SIGINT, syscall.SIGTERM, syscall.SIGHUP, syscall.SIGTSTP)
        wg := sync.Waitgroup{}
        wg.Add(1)
        go func() {
                defer wg.Done()
                someUsefulWork(done)
        }()
        <-exit
        close(done) // Tell goroutines that we are done
        wg.Wait() // Wait for goroutines to cleanup before quitting
        // All Goroutines have quit now. We can safely exit.
}

In the above code, goroutines are notified via done channel if we are about to quit. That’s great!

Now, what if you want to send some data from main() to goroutines? One way to do this is by adding more function params along with done channel, which works but isn’t as extendible if we want to add more pieces of data. Also, a called goroutine will have to pass this done channel down to any other goroutines that are spawned by it. Apart from this you might want the ability to tell goroutines when to quit if they can’t do their job within a given timeout. To make things a bit cleaner, Google recommends a better way to pass done channel and data, which is by using the context package. You can read in detail in this blog post about the context package for an in-depth example. The context package has a struct called Context which contains a done channel that is similar to what we used above and it also contains a key-value data-bag where you can store any key-values. Therefore, you can just pass around this Context which has all the information needed for your goroutine. You can create various kinds of contexts:

  • Empty Context: Created using context.Background(), which is a top-level context without the ability to be cancelled or timeout.

  • Context with Timeout: Created using context.WithTimeout(…​), which is a context that can timeout. This is good to ensure an upper time bound on your goroutines

  • Context with Cancel: Created using context.WithCancel(…​), which is a context that returns a cancel function that can be called to notify goroutines to stop. cancel is similar to calling close(done) in our example above.

  • Context with Deadline: Created using context.WithDeadline(…​), which is same as context.WithTimeout except it accepts a time.Time object rather than time.Duration.

Here is an example of Context:

func someUsefulWork(ctx context.Context) {
        // Assume you get a channel from somewhere which has lot of work for you
        workChan := GetWorkChannel()
        for {
                select {
                        case work := <-workChan
                                // Do work here
                        case <-ctx.Done()
                            // Done() returns the "done" channel
                                // Do cleanup
                                return
                }
        }
}

func main() {
        // context.Background() is an empty context with no timeout or cancel support
        ctx, cancelFunc := context.WithCancel(context.Background())

        go someUsefulWork(ctx)
        time.Sleep(time.Second * 5)

        // Calling cancel is equivalent of calling close(done) in the example above. It will tell goroutines to quit.
        cancelFunc()
}

If you want to pass some data to a Goroutine, you can also create a context with a value, like:

user := User{id: 123}
infoCtx := context.WithValue(context.Background(), "user", user)
ctx, cancelFunc := context.WithCancel(infoCtx)

// Now Goroutine can access user by doing
myuser, ok := ctx.Value("user").(User)

Accessing Static YML or JSON Files

Accessing static YML or JSON files is not as simple as you might think. Go creates a static binary at the end of the build and you just can’t include static files as a part of the binary. If they need to be there, they should be included as Go Strings. I came across a nice package called aybabtme/embed that lets you embed static files as strings in your go code. To use it, do the following:

//go:generate embed file -var myConfig --source config.yml
var myConfig string

The comment above is a special comment. It starts with keyword go:generate which is read by the go generate tool. When we run go generate in this directory, it will invoke a command called embed that will take contents of file config.yml and put them in myConfig variable. You can then read this string using any YML reader.

Result:

//go:generate embed file -var myConfig --source config.yml
var myConfig := "env: dev\n  myFlag: true\n"

Note, that if you add an IntelliJ file watcher that listens on static files and runs go generate on save, it will keep these strings always in sync with the static files.

Configuration Management

Every app requires config management and I came across this amazing library called spf13/viper which is a one stop shop for configuration management. It can read formats like JSON, TOML, and YAML from locations like environment variables, files, or even config systems like Consul or Etcd.

Viper.SetConfigType("json")
Viper.AutomaticEnv() // Bind env variables
configString := "{\"foo\":\"bar\"}"
err := Viper.ReadConfig(bytes.NewReader([]byte(configString)))
fmt.Println(Viper.GetString("foo"))

Command-line Params

The ability to take command line params is really important if you are building a 12 factor app. This enables you to take in environment specific params like database ips and passwords according to where the app is running. You can easily do this in Go using the flag package.

type Flag struct {
        Env  *string
}
func init() {
        f := new(Flag)
        f.Env = flag.String("env", "dev", "Environment name (dev/prod/test)")
        flag.Parse() // This is required before reading

        // Read it
        fmt.Println(*f.env)
}

An example of running app with these params would be:

> myapp --env test

Note that running myapp --help will show help on the command listing all possible command line params it can take.

Writing Functional Tests

Functional Tests test your app and ensure its basic functionality is working properly. I am using check.v1 for writing these tests. To run a webserver in test mode, you can use the httptest package in Go. After server initialization, the running server host and port can be accessed using server.URL property. Here is an example of a simple test written against an API endpoint.

var server *httptest.Server

type MySuite struct{}

// This func should be shared between non-test and test code
func getRouter() *gin.Engine {
        r := gin.Default()
        handler := EventHandler{}
        r.POST("/event", handler.Post)
        return r
}
func init() {
        server = httptest.NewServer(getRouter())
}

func (s *MySuite) TestCreateEventViaREST(c *C) {
        b, err := getPostBody("foo.bar", time.Now().Unix())
        c.Assert(err, Equals, nil)

        resp, err := http.Post(server.URL+"/event", "application/json", bytes.NewBuffer(b))
        c.Assert(err, Equals, nil)

        defer resp.Body.Close()
        c.Assert(resp.StatusCode, Equals, http.StatusOK)

        body, err := ioutil.ReadAll(resp.Body)
        c.Assert(string(body), NotNil)
}

Useful Resources

Go community is great and I learnt a lot from blog posts written by them. Here is my reading list if you want to expand your knowledge on Go:

Summary

At the end of this project, I was really impressed with Golang and all the tooling around it. I was able learn the language and create a working app in 2-3 weeks. In writing this article, I chose to focus on the most important things that would be relevant to a majority of other Go projects, so if you have any questions, please ask them in the comment section below. Likewise, if you have any comments or suggestions, please leave them in the comment section below. I hope you found this article useful!