For the past few weeks, I have been focusing most of my time at work on a project, which we have coined the spark. In its essence, it is a data gateway built on top of a Raspberry Pi, which bridges machine and energy metrics from existing infrastructure to our data pipeline.
As our main focus was set on high frequency read-outs at up to 100Hz, it was quickly sorted out that python, usually my go-to language, wasn’t the way to go. After some research, I quickly settled on Rust due to its cutting edge performance and ‘if i compiles, it’s safe’ mantra.
But as always, everything comes with a price: While Rust is blazingly fast and enables us to develop and ship new features and fixes with confidence, it needs to be compiled for a particular architecture — in our case armv7. Working on a MacbookPro, I initially tackled this problem by pulling the entire repo and building the application on the Raspberry Pi itself, which resulted in up to 30 minutes of impatience.
In search of a more streamlined build process, I ended up implementing a cross-compilation pipeline built on top of Docker, which I would like to share with you.
This post is designed to be a step-by-step manual for you to build your own automated cross-compilation pipeline:
Now that we’ve set our scope, let’s get ready for take off!
As this post mainly focuses on automating the cross-compilation of Rust applications, we will set up a bare scaffold and add a system dependency, which I found particularly tricky: OpenSSL.
We start by creating a new directory named hello-rpi
and cd
ing into it. Then, we initialize a new cargo project by running cargo new --bin hello-rpi
in our terminal of choice. Now, we should have a project directory called hello-rpi
with the following folder structure:
hello-rpi||--- hello-rpi* |--- Cargo.toml||--- src* |--- main.rs*
With our minimal application generated, add openssl = “0.10.5”
as a dependency in our Cargo.toml
file and declare openssl
as an external crate in main.rs
. After our small adjustments, our Cargo.toml
and main.rs
should something like this:
Cargo.toml
[package]name = "hello-rpi"version = "0.1.0"authors = ["your handle <[email protected]om>"]
[dependencies]openssl = "0.10.5"
main.rs
extern crate openssl;
fn main() {println!("Hello, world!");}
Finally, go ahead and add the following lines to your crate’s cargo config file found in hello-rpi/hello-rpi/.cargo/config
:
[target.armv7-unknown-linux-gnueabihf]linker = "arm-linux-gnueabihf-gcc"
This will tell cargo which linker to use, when we specify armv7-unkown-linux-gnueabih
as our target, which is our Pi’s architecture. With these few lines of code, we’re done on the Rust end.
With our minimal application set and ready to go, we now focus on automating the cross compilation process of our application. To this end, we will create a Docker image, which will handle cross-compiling our application as well as our dependency, i.e. OpenSSL for us.
Fortunately, we don’t have to start from scratch — Stephen Thirlwall’s Docker image raspberry-pi-cross-compiler
, which may be found here, provides an awesome boilerplate to build up on. When run without any arguments, a container based on this image returns a helper script, which in turn let’s you interact with the container and start the actual cross-compilation.
So let’s dive right in! Create a new folder in our project’s (not crate!) root named rpxc
, create a Dockerfile
and paste the following into it:
Wow, there’s a lot going on here — let’s break it down a bit: First, we point our base image to sdthirwall’s raspberry-pi-cross-compiler
and install some packages via apt-get
— nothing out of the ordinary.
We then use the ENV
expression to set environment variables pointing to our Cargo
and Rustup
installations as well as our OpenSSL lib
and include
directories.
The subsequent RUN
block is borrowed from the rust:1.26.0
image and deals with installing the Rust toolchain and adding armv7-unknown-linux-gnueabihf
as a compilation target. As indicated by its name, this is the compiler for our Raspberry’s architecture.
The second RUN
command then downloads, un-tars and cross-compiles OpenSSL for our target architecture. Note that our environment variables pointing to OpenSSL’s lib
and include
directory match up with our install’s path.
Not that complicated, right? What’s more, this setup facilitates including any additional system dependency in no time by simply extending our Dockerfile
— it doesn’t get easier than that.
With our Dockerfile
set up, we’re meremly one step short of our build pipeline. More precisely, we want to
Transforming this procedure into a shell script and adding some clean up yields six actual lines of code, which need to be placed in the rpxc
directory to work correctly.
And that’s it! We have defined custom Docker image, which handles cross-compiling system dependencies at build time and written a shell script to actually build and cross-compile the entire application. Now, you can simply cd
into rpxc
, run the script above, sit back and watch the magic happening.
In this post, we built a build pipeline, which let’s us cross-compile any Rust application for the armv7 architecture by simply executing a single shell script. To this end, we built a custom Dockerfile
, added application specific system dependencies and wrote a small shell script, which kicks off the cross-compilation process.
This setup is favourable for multiple reasons: First and foremost, you can build your application on any machine and platform, as long as Docker is installed. Thus, you can easily integrate the entire build process as the final step of your continuous integration pipeline. What’s more, you can add any additional system dependency by extending the Dockerfile
, which makes this solution scalable towards more complex applications, which may depend on a large number of system dependencies.
That’s it from my side! I hope you’ve enjoyed my first post and that I was able to get you kick started with cross-compiling Rust for Raspberry Pis. If you have any questions or critique regarding my post, let me know in the comments below!
Full disclosure: I am one of the co-founders of Enlyze. We are a startup based in Aachen, Germany, which treats power consumption as a universal indicator for any appliance’s state of health. Our team of six consists of passionate AI researchers and electrial as well as software engineers. If you’d like to get in touch, shoot us a mail to [email protected].