The beauty of Go

I recently started exploring Go for some of my side projects and was really struck by its beauty. I realized how beautifully it made a balance between ease of use (generally associated with dynamically typed, interpreted languages), and performance and safety (type safety, memory safety) (generally associated with statically typed, compiled languages). Apart from these, two more features make it really the perfect language for modern systems development. Both these features are explained in more detail in the Strengths section below. One of them is y in the language (through goroutines and channels, explained below). Concurrency, by its design, enables you to efficiently use your CPU horsepower. Even if your processor just has 1 core, concurrency’s design enables you to use that one core efficiently. That is why you can typically have hundreds of thousands of concurrent goroutines (lightweight threads) running on a single machine. Channels and goroutines are central to distributed systems since they abstract the producer-consumer messaging paradigm. first class support for concurrenc The other feature I really like about Go is interfaces. . Meaning that a part of your code can just rely on an interface type and doesn’t really care about who implements the interface or how the interface is actually implemented. Your controller can then supply a dependency which satisfies the interface (implements all the functions in the interface) to that code. This also enables a really clean architecture for unit testing (through dependency injection). Now, your controller can just inject a mock implementation of the interface required by the code to be able to test if it’s doing its job correctly or not. Interfaces enable loosely coupled or decoupled components for your systems Keeping all these features in mind, I think Go is really a great language. Especially for use cases like cloud systems development (web servers, CDNs, caches etc), distributed systems, microservices etc. So if you’re an engineer or a startup trying to decide what language you want to explore or try out, do give Go a serious thought. In this post, I’ll talk about the following aspects of the language: a) Introduction b) Why was Go needed c) Target Audience d) Go’s strengths e) Go’s weaknesses f) Towards Go 2 g) Go’s design philosophy h) How to get started i) Who is using Go Introduction Go is an open source language, created at Google by Robert Griesemer, Rob Pike, and Ken Thompson. Open source here means that everybody can contribute to the language by opening for new features, fix bugs etc. The language’s code is available on . Documentation on how you can contribute to the language is provided . proposals GitHub here Why was Go needed The authors mention that the primary motive for designing a new language was to solve software engineering issues at Google. They also mention that Go was actually developed as an alternative to C++. Rob Pike mentions the purpose for the Go programming language: “Go’s purpose is therefore not to do research into programming language design; it is to improve the working environment for its designers and their coworkers. Go is more about software engineering than programming language research. Or to rephrase, it is about language design in the service of software engineering.” (taken from ): Issues that were plaguing the software engineering horizon at Google were https://talks.golang.org/2012/splash.article a) slow builds — builds would sometime take as long as an hour to complete b) uncontrolled dependencies c) each programmer using a different subset of the language d) poor program understanding (code hard to read, poorly documented, and so on) e) duplication of effort f) cost of updates g) version skew h) difficulty of writing automatic tools i) cross-language builds (taken from ): For Go to succeed, Go must solve these problems https://talks.golang.org/2012/splash.article a) Go must work at scale, for large teams of programmers working on them, for programs with large numbers of dependencies. b) Go must be familiar, roughly C-like. Google needs to get programmers productive quickly in Go, means that the language cannot be too radical. c) Go must be modern. It should have features like concurrency so that programs can make efficient use of multi core machines. It should have built-in networking and web server libraries so that it aids modern development. Target Audience Go is a systems programming language. Go really shines for stuff such as cloud systems (web servers, caches), microservices, distributed systems (due to concurrency support). Strengths Go is statically typed. This means that you need to declare types for all your variables and your function arguments (and return variables) at compile time. Although this may sound inconvenient, this is a great advantage since a lot of errors will be found at compile time itself. This factor plays a very big role when your team size increases, since declared types make functions and libraries more readable and more easier to understand. a) Statically typed: Go code compiles , so you don’t need to keep waiting for your code to compile. :) In fact, the ‘go run’ command fires up your Go program so quickly so that you don’t even get a feeling that your code got compiled first. It feels like an interpreted language. b) Compilation Speed: really fast Go code gets directly compiled to machine code, depending upon the OS (Linux/Windows/Mac) and the CPU instruction set architecture (x86, x86–64, arm etc) of the machine the code is being compiled upon. So, it runs really fast. c) Execution Speed: Since the code gets directly compiled to machine code, therefore, the binaries become portable. Portability here means that you can pick up the binary from your machine (let’s say Linux, x86–64) and directly run that on your server (if your server is also running Linux on a x86–64 architecture). d) Portable: This becomes possible since Go binaries are statically linked, meaning that any shared operating system libraries your program needs are included in the binary at the time of the compilation. They are not dynamically linked at the time of running the program. This has a huge benefit for deployment of your programs on multiple machines in a data center. If you have 100 machines in your data center, you can simply ‘scp’ your program binary to all of them, as long as the binary is compiled for the same OS and instruction set architecture your machines run on. You don’t need to care about which version of Linux they are running. There is no need for checking/managing dependencies. The binaries simply run and all your services are up :) Go has first class support for concurrency. Concurrency is one of the major selling points of Go. The language designers have designed the concurrency model around the ‘ ’ paper by Tony Hoare. e) Concurrency: Communicating Sequential Processes . A Goroutine is a lightweight thread of execution. The Go runtime multiplexes those goroutines over operating system threads. That means that multiple goroutines can run concurrently on a single OS thread. The Go runtime has a scheduler whose job is to schedule these goroutines for execution. The Go runtime allows you to run hundreds of thousands of concurrent goroutines on a machine There are two benefits of this approach: i) A Goroutine when initialized has a stack of 4 KB. This is really tiny as compared to a stack of an OS thread, which is generally 1 MB. This number matters when you need to have hundreds of thousands of different goroutines running concurrently. If you would run more than thousands of OS threads in parallel, the RAM obviously will become a bottleneck. ii) Go could have followed the same model as other languages like Java, which support the same concept of threads as OS threads. But in that case, the cost of a context switch between OS threads is much larger than the cost of a context switch between different goroutines. Since I’m referring to “concurrency” multiple times in this article, I would advise you to check out Rob Pike’s talk on ‘ ”. In programming, concurrency is the composition of independently executing processes, while parallelism is the simultaneous execution of (possibly related) computations. Unless you have a processor with multiple cores or have multiple processors, you can’t really have parallelism since a CPU core can only execute one thing at a time. On a single core machine, it’s just concurrency that’s doing its job behind the scenes. The OS scheduler schedules different processes (threads actually. every process has atleast a main thread) for different timeslices on the processor. Therefore, at one moment in time, you can only have one thread(process) running on the processor. Due to the high speed of execution of the instructions, we get the feeling that multiple things are running. But it’s actually just one thing at a time. Concurrency is not parallelism Concurrency is about dealing with lots of things at once. Parallelism is about doing lots of things at once. Interfaces enable loosely coupled systems. An interface type in Go can be defined as a set of functions. That’s it. Any type which implements those functions implicitly implements the interface, i.e. you don’t need to specify that a type implements the interface. This is checked by the compiler automatically at compile time. f) Interfaces: This means that a part of your code can just rely on an interface type and doesn’t really care about who implements the interface or how the interface is actually implemented. Your main/controller function can then supply a dependency which satisfies the interface (implements all the functions in the interface) to that code. This also enables a really clean architecture for unit testing (through dependency injection). Now, your test code can just inject a mock implementation of the interface required by the code to be able to test if it’s doing its job correctly or not. While this is great for decoupling, the other benefit is that you then start thinking about your architecture as different microservices. Even if your application resides on a single server (if you’re just starting out), you architect different functionalities required in your application as different microservices, each implementing an interface it promises. So other services/controllers just call the methods in your interface not actually caring about how they are implemented behind the scenes. Unlike C, you don’t need to remember to free up pointers or worry about dangling pointers in Go. The garbage collector automatically does this job. g) Garbage collection: I love the fact that Go doesn’t have the standard exception logic that other languages have. Go forces developers to handle basic errors like ‘couldn’t open file’ etc rather than letting them wrap up all of their code in a try catch block. This also puts pressure on developers to actually think about what needs to be done to handle these failure scenarios. h) No exceptions, handle errors yourself: One of the best aspects about Go is its tooling. It has tools like: i) Amazing tooling: i) It automatically formats and indents your code so that your code looks like the same as every Go developer on the planet. This has a huge effect on code readability. Gofmt: ii) This compiles your code and runs it, both :). So even though Go needs to be compiled, this tool makes you feel like it’s an interpreted language since it just compiles your code so fast that you don’t even feel when the code got compiled. Go run: iii) This downloads the library from GitHub and copies it to your GoPath so that you can import the library in your project Go get: iv) Godoc parses your Go source code — including comments — and produces its documentation in HTML or plain text format. Through godoc’s web interface, you can then see documentation tightly coupled with the code it documents. You can navigate from a function’s documentation to its implementation with one click. Godoc: You can check more here. tools Go has great built-in libraries to aid modern development. Some of them are: j) Great built-in libraries: a) — Provides HTTP client and server implementations net/http b) — For interaction with SQL databases database/sql c) — JSON is treated as a first class member of the standard language :) encoding/json d) — HTML templating library html/templates e) — Implements I/O utility functions io/ioutil There is a lot of development going on in the Go horizon. You can find all Go libraries and frameworks for all sorts of tools and use cases . here Weaknesses — Generics let us design algorithms around types-to-be-specified-later. Let’s say you need to write a function to sort a list of integers. Later on, you need to write another function for sorting a list of strings. At that moment, you realize that the code would pretty much look the same but you can’t use the original function since the function can either take a list of type integer or a list of type string as an argument. This would require code duplication. Therefore, generics let you design algorithms around types which can be specified later. You can design an algorithm for sorting a list of type T. Then, you can call the same function with integers/strings/any other type given that there exists an ordering function for that type. Meaning that the compiler can check if one value of that type is bigger than another value of that type or not (since this is needed for sorting) 1. Lack of generics It is possible to implement some sort of a generic mechanism in Go by using the empty interface (interface {}) feature of the language. However, it’s not ideal. Generics is a highly controversial topic. Some programmers swear by it. While others don’t want them included in the language since generics generally are a trade-off between compilation time and execution time. This being said, authors of Go have expressed openness towards implementing some sort of generics mechanism in Go. However, it’s not just about generics. Generics can only be implemented into the language if they work well and naturally with all the other features in the language. Let’s wait and see if Go 2 provides some sort of solution for them. — The Go1 promise means that the Go language and its library cannot change its APIs over the lifetime of Go 1. This means that your source code will continue to compile for both Go 1.5 and Go 1.9. Therefore, most 3rd party Go libraries follow the same promise as well. Since the primary way to get a 3rd party library from GitHub is through the ‘go get’ tool, therefore, when you do a ‘go get github.com/vendor/library’, you are hopeful that the latest code in their master branch does not change the library APIs. While this is cool for casual side projects since most libraries don’t break the promise, this isn’t ideal for production deployments. 2. Lack of dependency management There should ideally be some way for dependency versioning so that you can simply include the version number of a 3rd party library in your dependency file. Even if their API changes, you don’t need to worry about it since the newer API will come with a newer version. You can later go back to check what changes were made and then take a decision on whether or not to upgrade the version in your dependency file and change your client code according to the changes in the API interface. Go’s official experiment should ideally become the solution to this problem soon. Probably in Go 2 :) dep Towards Go 2 I really love the way the authors have taken such an open source approach towards the language. If you want a feature to be implemented in Go 2, you need to write a document wherein you need to: a) describe your use case or problem b) illustrate how you cannot solve the use case / problem using Go c) describe how big of a problem it really is (some problems just aren’t big enough or concerning enough to be prioritized for solving at a given moment in time). d) optionally, propose a solution to how the problem could be solved The authors will review it and link it . All discussions around issues will happen on public mediums like mailing lists and issue trackers. here The two most pressing problems for the language, in my opinion are Generics and Dependency management. Dependency management is more of a release engineering or tooling issue. Hopefully we will see (the official experiment) become the official tool to solve the problem. Given that the authors have expressed openness to generics in the language, I’m curious to see how they implement them since generics come at the cost of either compilation time or execution time. dep Go’s design philosophy A couple of things really shined to me from Rob Pike’s talk. Simplicity is Complicated Specifically, the things which I liked were: a) . This way, all of the languages are just adding bloatedness, too much complexity in their compilers and their specification. If this continues, every language will look like the same in the future because every language will keep adding features it doesn’t have. Consider, JavaScript adding object oriented features. The Go authors deliberately did not include a lot of features in the language. Traditionally, every other language wants to just keep adding new features Only those features were included for which there was consensus from the authors, for the ones which really felt like they did brought value into what can be achieved by the language. b) What’s important is the ability to pick and combine different vectors for your use case. This way, those set of features cover the whole solution space. Implementing all these features, which work very naturally with each other, brings a lot of complexity into the language implementation. But the language abstracts the complexity and provides you with a simple, easy to understand interface. Therefore, simplicity is just the art of hiding complexity :) Features are like orthogonal vectors in a solution space. And those vectors should just work with each other naturally. Means that every feature of the language should work predictably with any other. c) Readability is critical, arguably, one of the most important things in designing programming languages, since the importance and also the cost of maintaining software is high. Too many features hurt the readability of a language. The importance of readability is often underrated. d) If a language is complicated, you must understand more things to read and work on the code. Similarly, to debug it and to be able to fix it. This also means that new developers on your team will need much larger scale up times, to get their understanding on the language upto the point at which they can contribute to your codebase. Readability also means reliability. Taken from: https://blog.digitalocean.com/get-your-development-team-started-with-go/ How to get started You can download Go and follow the installation instructions from . here Here is the to get started with Go. is also a good one. official guide Go by example If you want to read a book, is an excellent one. It’s written in a similar spirit as the legendary book and is authored by and . The Go Programming Language C Programming Language Alan A. A. Donovan Brian W. Kernighan You can join the to engage with community and to participate in discussions around the language. Gophers Slack Channel Who is using Go have started investing a lot in Go. Here are some of the bigger names: A lot of companies Google — , , (Download servers) Kubernetes MySQL scaling infrastructure dl.google.com BaseCamp — Go at BaseCamp CloudFlare — , Blog ArsTechnica article CockroachDB — Why Go was the right choice for CockroachDB CoreOS — , GitHub Blog DataDog — Go at DataDog DigitalOcean — Get your development team started with Go Docker — Why did we decide to write Docker in Go Dropbox — Open Sourcing our Go Libraries Parse — How we moved our API from Ruby to Go and saved our Sanity Facebook — GitHub Intel — GitHub Iron.IO — Go after 2 years in Production/ MalwareBytes — Handling 1 million requests per minute with golang/ Medium — How Medium goes Social MongoDB — Go Agent Mozilla — GitHub Netflix — GitHub Pinterest — GitHub Segment — GitHub SendGrid — How to convince your company to go with Golang Shopify — Twitter SoundCloud — Go at SoundCloud SourceGraph — YouTube Twitter — Handling Five Billion Sessions a day in Real Time Uber — , Blog GitHub Before you go… Thank you for your time. If you liked this article, I would really appreciate if you share this post with your friends via Twitter/Facebook. You can find me on Twitter at https://twitter.com/kanishkdudeja Cross-posted on: https://kanishkdudeja.in/the-beauty-of-go/