Evaluating Go at Revinate
At Revinate, we use PHP and Java extensively. These languages give us a good balance between developer efficiency and robustness for building frontend and backend systems, respectively, however we continuously evaluate new languages that may offer even greater developer efficiency and robustness. Enter Go. On paper Go fits the bill perfectly due to its dynamic programming ease, C-like performance, and solid garbage collection support. To get a feel of Go, I wrote a service so that I could learn as well evaluate if it is worthwhile to put into our stack. I learned a lot of things in the process and I want to share these learnings with others who are starting to use the language for real projects. Just a note that I am still a very young gopher and my suggestions might not be the most accurate, therefore I am open to improvements or suggestions from the Go community. Just a note that my code is not publically available as it was an internal project.
Go IDE
It was hard finding a great IDE for Go. Most people are using IDEs that don’t provide the entire toolset that can help you develop Go code. You can hop on here and view the list of IDEs but most of them lack in some way or the other. I started my project with Atom with Go-Plus plugin but it did not have great ctags support and it was always pretty hard to easily find out function signature using a shortcut key. Then I came across the Go-IDE plugin for IntelliJ IDEs, which was a life saver. Just use it and save yourself the trouble. If you are a vim loyalist, checkout Vim-go.
A few things you might want to setup to streamline your workflow:
-
If you are using Go 1.4, set the
GOPATH
environment variable to the path where you store your Go code. Read How to Write Go Code for more info -
If you using Go 1.5, you don’t have to set
GOPATH
. Instead you can use any of the third party package managers. I used Glide for my project. -
Go to "Preferences" > "Languages & Frameworks" > "Go" > "Go SDK" and set the SDK by selecting the Go install path. This will enable features like "Jump to Definition" and quick definition lookup in the IDE.
-
Install
goimports
from here. And then add a File Watcher to rungoimports
on file save. Few tips on setting up this watcher:-
Use the full path of the binary.
-
Set the arguments to be “-w .”. It will replace the current code with formatted code
-
Set "Working Directory" to
$ProjectFileDir$
-
-
Add another watcher for "Go Generate" which run the command automatically when certain files are saved. Read Accessing Static YML or JSON Files section for more details.
-
For info about Go tools, checkout this great post
REST API in Go
There are plenty of Routers for Go and I ended up using Gin-Gonic to create REST APIs. They all have pretty much the same interface for defining routes and handlers but be sure to pick a router than you can easily understand and that has support for adding custom middleware that you might need for authentication, logging, analytics, etc. Setting up routing is as simple as initializing the Gin router and setting up route handlers
r := gin.Default()
handler := EventHandler{}
r.GET("/event/:id", handler.Get)
r.POST("/event", handler.Post)
Gin-Gonic does not support graceful stop which means if your Go app is killed, in-flight requests will not complete prior to your app shutting down. This is an unacceptable behavior but thanks to fvbock/endless package, you can use this alternate version of http server that does shutdown gracefully when it receives kill signals.
Use:
// r is gin router
endless.ListenAndServe(":80", r)
instead of:
// r is gin router
r.Run(":80")
To know how to use Gin in testing, go to section on testing.
Goroutines
One of the coolest features of Go is how easy it makes writing asynchronous code. Appending go
in front of any function call makes that function run asynchronously. Goroutines are great but you need to be careful about terminating them as they are not garbage collected if they don’t return. There is another thing you need to be aware of when working with Goroutines: before quitting the App, ensure that all Goroutines successfully return. This can be done by using sync.WaitGroup
as follows:
func main() {
wg := sync.Waitgroup{}
for i:=1; i<=3; i++ {
wg.Add(1) // Tell WaitGroup that we are starting a goroutine
go func() {
defer wg.Done() // executes before returning and tells WaitGroup that I am done
// Do some useful work here
}()
}
wg.Wait() // Wait for all 3 goroutines to end before quitting
}
Graceful Shutdown
Gracefully shutting down your app is very important because you don’t want to leave API requests or any other operations midway when your app is requested to be killed. How do you do that in Go? To achieve that, your app needs to listen to OS signals for any kill signals. Luckily, you can create a OS signal channel and subscribe to a list of system signals and take some action when you receive something on that channel.
func main() {
// Create a signal channel
exit := make(chan os.Signal)
// Subscribe to kill signals
signal.Notify(exit, syscall.SIGINT, syscall.SIGTERM, syscall.SIGHUP, syscall.SIGTSTP)
go func() {
// Do some useful work
}()
// Wait for Kill Signal
<-exit
// You will reach here only if app is requested to be killed, therefore do cleanup here
doCleanup()
}
Okay, so now we can wait for the quit signal and then quit but what happens when your Goroutines are in the middle of something and your app receives a kill signal? We ideally want to tell Goroutines to stop gracefully and return so that we can quit the go process. This can be done by passing a channel to all of your goroutines on which you can notify them when you want to quit the app. Lets see an example:
func someUsefulWork(done <-chan bool) {
// Assume you get a channel from somewhere which has lot of work for you
workChan := GetWorkChannel()
for {
select {
case work := <-workChan
// Do work here
case <-done
// Do cleanup
return
}
}
}
func main() {
exit := make(chan os.Signal)
signal.Notify(exit, syscall.SIGINT, syscall.SIGTERM, syscall.SIGHUP, syscall.SIGTSTP)
wg := sync.Waitgroup{}
wg.Add(1)
go func() {
defer wg.Done()
someUsefulWork(done)
}()
<-exit
close(done) // Tell goroutines that we are done
wg.Wait() // Wait for goroutines to cleanup before quitting
// All Goroutines have quit now. We can safely exit.
}
In the above code, goroutines are notified via done channel
if we are about to quit. That’s great!
Now, what if you want to send some data from main()
to goroutines? One way to do this is by adding more function params along with done channel
, which works but isn’t as extendible if we want to add more pieces of data. Also, a called goroutine will have to pass this done
channel down to any other goroutines that are spawned by it. Apart from this you might want the ability to tell goroutines when to quit if they can’t do their job within a given timeout. To make things a bit cleaner, Google recommends a better way to pass done
channel and data, which is by using the context
package. You can read in detail in this blog post about the context package for an in-depth example. The context
package has a struct called Context
which contains a done
channel that is similar to what we used above and it also contains a key-value data-bag where you can store any key-values. Therefore, you can just pass around this Context
which has all the information needed for your goroutine. You can create various kinds of contexts:
-
Empty Context: Created using
context.Background()
, which is a top-level context without the ability to be cancelled or timeout. -
Context with Timeout: Created using
context.WithTimeout(…)
, which is a context that can timeout. This is good to ensure an upper time bound on your goroutines -
Context with Cancel: Created using
context.WithCancel(…)
, which is a context that returns acancel
function that can be called to notify goroutines to stop.cancel
is similar to callingclose(done)
in our example above. -
Context with Deadline: Created using
context.WithDeadline(…)
, which is same ascontext.WithTimeout
except it accepts atime.Time
object rather thantime.Duration
.
Here is an example of Context
:
func someUsefulWork(ctx context.Context) {
// Assume you get a channel from somewhere which has lot of work for you
workChan := GetWorkChannel()
for {
select {
case work := <-workChan
// Do work here
case <-ctx.Done()
// Done() returns the "done" channel
// Do cleanup
return
}
}
}
func main() {
// context.Background() is an empty context with no timeout or cancel support
ctx, cancelFunc := context.WithCancel(context.Background())
go someUsefulWork(ctx)
time.Sleep(time.Second * 5)
// Calling cancel is equivalent of calling close(done) in the example above. It will tell goroutines to quit.
cancelFunc()
}
If you want to pass some data to a Goroutine, you can also create a context with a value, like:
user := User{id: 123}
infoCtx := context.WithValue(context.Background(), "user", user)
ctx, cancelFunc := context.WithCancel(infoCtx)
// Now Goroutine can access user by doing
myuser, ok := ctx.Value("user").(User)
Accessing Static YML or JSON Files
Accessing static YML or JSON files is not as simple as you might think. Go creates a static binary at the end of the build and you just can’t include static files as a part of the binary. If they need to be there, they should be included as Go Strings. I came across a nice package called aybabtme/embed that lets you embed static files as strings in your go code. To use it, do the following:
//go:generate embed file -var myConfig --source config.yml
var myConfig string
The comment above is a special comment. It starts with keyword go:generate
which is read by the go generate
tool. When we run go generate
in this directory, it will invoke a command called embed
that will take contents of file config.yml
and put them in myConfig
variable. You can then read this string using any YML reader.
Result:
//go:generate embed file -var myConfig --source config.yml
var myConfig := "env: dev\n myFlag: true\n"
Note, that if you add an IntelliJ file watcher that listens on static files and runs go generate
on save, it will keep these strings always in sync with the static files.
Configuration Management
Every app requires config management and I came across this amazing library called spf13/viper which is a one stop shop for configuration management. It can read formats like JSON, TOML, and YAML from locations like environment variables, files, or even config systems like Consul or Etcd.
Viper.SetConfigType("json")
Viper.AutomaticEnv() // Bind env variables
configString := "{\"foo\":\"bar\"}"
err := Viper.ReadConfig(bytes.NewReader([]byte(configString)))
fmt.Println(Viper.GetString("foo"))
Command-line Params
The ability to take command line params is really important if you are building a 12 factor app. This enables you to take in environment specific params like database ips and passwords according to where the app is running. You can easily do this in Go using the flag
package.
type Flag struct {
Env *string
}
func init() {
f := new(Flag)
f.Env = flag.String("env", "dev", "Environment name (dev/prod/test)")
flag.Parse() // This is required before reading
// Read it
fmt.Println(*f.env)
}
An example of running app with these params would be:
> myapp --env test
Note that running myapp --help
will show help on the command listing all possible command line params it can take.
Writing Functional Tests
Functional Tests test your app and ensure its basic functionality is working properly. I am using check.v1 for writing these tests. To run a webserver in test mode, you can use the httptest package in Go. After server initialization, the running server host and port can be accessed using server.URL
property. Here is an example of a simple test written against an API endpoint.
var server *httptest.Server
type MySuite struct{}
// This func should be shared between non-test and test code
func getRouter() *gin.Engine {
r := gin.Default()
handler := EventHandler{}
r.POST("/event", handler.Post)
return r
}
func init() {
server = httptest.NewServer(getRouter())
}
func (s *MySuite) TestCreateEventViaREST(c *C) {
b, err := getPostBody("foo.bar", time.Now().Unix())
c.Assert(err, Equals, nil)
resp, err := http.Post(server.URL+"/event", "application/json", bytes.NewBuffer(b))
c.Assert(err, Equals, nil)
defer resp.Body.Close()
c.Assert(resp.StatusCode, Equals, http.StatusOK)
body, err := ioutil.ReadAll(resp.Body)
c.Assert(string(body), NotNil)
}
Useful Resources
Go community is great and I learnt a lot from blog posts written by them. Here is my reading list if you want to expand your knowledge on Go:
-
Great place to start: https://gobyexample.com/
-
2012 Rob Pike Keynote on Go: http://talks.golang.org/2012/splash.article
-
Go OOP Concepts: https://github.com/luciotato/golang-notes/blob/master/OOP.md and http://nathany.com/good/
-
Good list of Tips & Tricks: http://www.golangbootcamp.com/book/tricks_and_tips
-
Go Pipelines: http://blog.golang.org/pipelines
-
Go Concurrency: https://blog.golang.org/context
-
Go Gotchas: http://devs.cloudimmunity.com/gotchas-and-common-mistakes-in-go-golang/index.html
-
How to organize Go code: http://blog.golang.org/organizing-go-code
-
Go Error Handling: http://blog.golang.org/error-handling-and-go
-
List of Awesome Go Tools: http://dominik.honnef.co/posts/2014/12/an_incomplete_list_of_go_tools/
Summary
At the end of this project, I was really impressed with Golang and all the tooling around it. I was able learn the language and create a working app in 2-3 weeks. In writing this article, I chose to focus on the most important things that would be relevant to a majority of other Go projects, so if you have any questions, please ask them in the comment section below. Likewise, if you have any comments or suggestions, please leave them in the comment section below. I hope you found this article useful!