The Myth about Golang Frameworks and External Libraries

I have been writing Go in production for some time now and really enjoy how fast and reliable it is. Smaller build sizes, the concurrency provided by go-routines and the fact that you could run the builds directly on the machine make Golang really sweet.

The standard packages are so performant that you could create a production ready microservice without the need for any external libraries or frameworks. But that doesn’t mean that there are no frameworks in Go which provide more flexibility or faster, they are simply not preferred.

If you ask a Go developer about what web frameworks or libraries you could use, the typical answer is stick to the standard libraries. Ironically, the top google search result for “golang frameworks” is about why you should not use them.

The top comment to a question about Go frameworks on HN

I did some research on alternatives to standard libraries to see how they perform and benchmarked them. I categorized them into what constitutes as essential components of a microservice.

All the benchmarks are performed on the below configuration. While the standard benchmark time is just 1s , I ran all the tests for 10s to have consistent cycles for each test.

Processor - 2.7 GHz Intel Core i7

RAM - 16GB - 16 GB 2133 MHz LPDDR3

OS - MacOS High Sierra

go version - go1.11.1

Bench Time 10s instead of standard 1s

go test -bench=. -benchtime=10s

Routing

The standard http Go server provides a good router which can read query params but fails to read named parameters like,

/students/:studentID/grades/:gradeID

Any REST service which has nested resources like above would already have to use an external routing library for parsing them. Gin, Echo, Beego, Gorilla Mux, Goji are some of the popular ones (according to their Github following).Some of these are full blown middleware frameworks which also does routing and some just are routing libraries.

I ran benchmarks against these libraries for a single named parameter and below are the results. Gin has the fastest router, followed by close second Echo.

JSON Serialization & Deserialization

Once an API request is hit through the router and passed on to a controller or handler, the next step is to Decode the request JSON or Encode while returning the response.

Go has a really good encoding package which supports multiple formats like json, XML, csv, but a quick look at the alternatives show you tons of libraries. I benchmarked Jsoniter, EasyJson against the standard encoding/json package and below are the results.

Below is the result for Encoding and as the results indicate the performance difference is not significant.

But for decoding JSON though, jsoniter performs 5X faster than the standard encoding package.

Now if you have your request decoded, next step could be applying your business logic and may be do some database operations.

ORM or not?

Most of the popular languages depend on frameworks to build microservice which interact with databases. Hibernate in the Java world, Active Record for Rails and Django ORM are pretty popular. ORM’s(Object relational mapper) help sometimes to better handle transactions, relationships between tables and help avoid writing complex SQL for simple joins.

But Go again has a really good database standard library which makes connecting to a relational database really easier and it’s pretty fast too. But, one pain point is for Querying. When you query some rows, you’d have to manually map each field in the row and then assign them to a struct. This works, but soon gets messy and makes you write a lot of code. **sqlx** is a library which allows you to scan the whole row into your struct variable.

While sqlx reduces the typical number of lines you write to build a CRUD, you still end up writing repetitive code a lot of times. Using an ORM could help reduce it and focus on your business logic.

I benchmarked database, database + sqlx, gorm , go-pg for querying and below are the results. Surprisingly, go-pg, an ORM performed faster than the standard package or even sqlx. GORM while very famous in the ecosystem is relatively slow.

Querying 200K records from a postgres DB

In each step of your API call, there are better frameworks or external libraries which will make your responses faster and also provide some flexibility.

Caveats

While some of the speed improvements are quite visible, this alone shouldn’t be the reason to move away from the standard libraries. Things like how easy is to test the code, long term maintenance for the open source library, the learning curve for the team should be taken into consideration.

Despite all this, I think something like echo + jsoniter + go-pg along with all the awesomeness of the Go standard library features would be the quickest way to build your microservice and avoid redundant code.

Please feel free to share your thoughts and experiences regarding Go libraries.