Imagine a scenario where you need to distribute your application far and wide, across the ether. People all around the world run to their computers because they heard the news that your application is finally live. They shed a single tear as they hit download, as they have been waiting for this moment for a lifetime. But wait, how will your users know that the application they are downloading was compiled from your source code, especially if your software handles sensitive data or needs a high level of security? A checksum, of course! But hang on, you might need to compile the binary on different machines, or even have the user compile the binary themselves to independently verify the integrity of the code. This is where things become a little more complicated.
There is no built-in way to compile a reproducible binary with Golang. There are specific flags that you can use during compilation like
-trimpath which removes prefixes from recorded source file paths or
-ldflags=-buildid= , which leaves the build ID blank, but to create a reproducible binary across every computer, you will need an identical environment. That means the same operating system, the same Go version, and the same environment variables. I wrote this article because I needed a way to create a reproducible binary so that users could compile an application from the source code and independently verify the application's integrity. After searching the internet for a way to do this, I only found forum posts that led to a dead-end or approaches that seemed to work but were much more complicated than they needed to be. So after I finally found a way to do this, I decided to share the approach so that it might save someone some time and energy. With the context out of the way, here is a straightforward way to create reproducible binaries with Go, that even the average end-user can follow.
Since we will need an identical environment to compile a reproducible binary, we are going to use Docker. If you are unfamiliar, Docker is a containerization software that makes it easy to replicate environments on any machine. We will also be creating a
Makefile to abstract any Docker commands, so that all you need to do, is input one command into the terminal to compile the binary. If you are familiar with Docker and Makefiles, this entire setup will take 5 minutes.
First, make sure you have docker installed. If you are on a Linux distro based on Debian, you can type
sudo apt-get install docker.io into the terminal and you will be good to go. Otherwise, here are the official docs to install Docker.
Next, create a new file called
Dockerfile and paste in the code below. I’ll explain what each step is doing.
#Get golang 1.17-buster as a base image FROM golang:1.17-buster as builder #Set build arguments ARG BUILDOS ARG BUILDARCH ARG BUILDNAME #Define the working directory in the container WORKDIR /app #Copy all files from root into the container COPY . ./ #Use go mod tidy to handle dependencies RUN go mod tidy #Compile the binary RUN env GOOS=$BUILDOS GOARCH=$BUILDARCH go build -o $BUILDNAME -trimpath -ldflags=-buildid=
First, the script gets Golang 1.17-buster as a base image and then sets build arguments for the operating system, the system architecture and the filename that you want the compiled binary to be called. Then, the Dockerfile defines the working directory for the container and copies all of the files from the root folder of your project into the container. The container then runs
go mod tidy to make sure all dependencies required in the source code match the
go.mod file. Finally, the container runs the command to build the binary by passing in the
BUILDNAME arguments. The flags
-ldflags=-buildid= are passed in to strip any file paths and remove the build ID for good measure. You might be wondering where
BUILDNAME come from, which is a good question and a great transition into the next step!
For the uninitiated, a
Makefile contains shell commands that execute a series of predefined instructions. In our case, we will be building a makefile to compile binaries for each operating system. First, create a new file called
Makefile and paste in the code below. Again, I will explain what each step is doing.
BUILDNAME=yourBinaryName linux-binary: #Build the linux-binary image docker build -t linux-binary-image --build-arg BUILDOS=linux --build-arg BUILDARCH=amd64 --build-arg BUILDNAME=$(BUILDNAME) --no-cache . #Run the linux-binary-image in a new container called linux-binary docker run --name linux-binary linux-binary-image #Copy the compiled linux binary to the host machine docker cp linux-binary:/app/$(BUILDNAME) . #Remove the linux-binary container docker rm linux-binary #Remove the linux-binary-image docker image rm linux-binary-image
First, the script creates a docker image and passes in the environment variables for the operating system, system architecture, and output filename. Next, it tells docker to run a container with that image. Then, it tells docker to copy the binary created in the container and place it in the root project folder. Afterward, it removes the container and lastly removes the image.
To execute the script on Linux or Mac, simply type
sudo make linux-binary in the root folder of your project. For Windows, you will need nmake or equivalent installed and then you can run
nmake linux-binary. Make sure to set the
BUILDNAME to your desired filename for the binary.
Now you have a way to create fully reproducible, independently verifiable binaries with Golang! If you want to integrate this functionality into your project, you can check out the source code at github.com/0xKitsune/Reproducible-Go-Binaries where there are 30 different build configurations. Thanks for reading and I hope this helped!