Hey there. I'm Mrinal Wadhwa, CTO at . In the early days of Ockam, we were developing a C library. This is the story of why, many months in, we decided to abandon tens of thousands of lines of C and rewrite in . Ockam Rust Before we begin, I was in a recorded webinar this week together with Paul Dix, the CTO of InfluxData, where we both discussed InfluxDB’s and Ockam’s rewrites in Rust. Why the two open source projects chose to re-write, why we chose Rust as our new language, lessons we learnt along the way, and more. Do check out . It was an insightful discussion. the recording Ockam enables developers to build applications that can trust data-in-motion. We give you simple tools to add end-to-end encrypted and mutually authenticated communication to - any application running in any environment. Your apps get end-to-end guarantees of data integrity, authenticity, and confidentiality … across private networks, between multiple clouds, through message streams in – over any multi-hop, multi-protocol topology. All communication becomes end-to-end authenticated and private. Kafka We also make the hard parts super easy to scale - bootstrap trust relationships, safely manage keys, rotate/revoke short-lived credentials, enforce attribute-based authorization policies, etc. The end result is - you can build apps that have granular control over every trust and access decision - apps that are private and secure-by-design. In 2019, we started building all of this in C. We wanted Ockam to run everywhere - from constrained edge devices to powerful cloud servers. We also wanted Ockam to be usable in any type of application - regardless of the language that the application is built in. This made C an obvious candidate. It can be compiled for 99% of computers and pretty much run everywhere (once you figure out how to deal with all the target-specific toolchains). And all other popular languages can call C libraries through some form of a native function interface - so we could later provide language idiomatic wrappers for every other language: Typescript, Python, Elixir, Java, etc. The idea was we’ll keep the core of our communication-centric protocols decoupled from any hardware-specific behavior and have pluggable adapters for the hardware we want to support. For example, there would be adapters to store secret keys in various HSMs, adaptors for various transport protocols, etc. Our plan was to implement our core as a C library. We would then wrap this C library with wrappers for other languages and run it everywhere with the help of pluggable hardware adapters. Simple and Safe Interfaces But, we also care deeply about simplicity - it's in our name. We want Ockam to be simple to use, simple to build, and simple to maintain. At Ockam’s core is a layered stack of cryptographic and message-based protocols like Ockam and Ockam . These are asynchronous, multi-step, stateful communication protocols and we wanted to abstract away all of the details of these protocols from application developers. We imagined the user experience to be a single one-line function call to create an end-to-end authenticated and encrypted secure channel. Secure Channels Routing Cryptography-related code also tends to have a lot of footguns, one little misstep, and your system becomes insecure. So simplicity isn't just an aesthetic ideal for us, we think it's a crucial requirement to ensure that we can empower everyone to build secure systems. Knowing the nitty-gritty of every protocol involved should not be necessary. We wanted to hide these footguns away and provide developer interfaces that are easy to use correctly and impossible/difficult to use in a way that will shoot your application in the foot. That’s where C was severely lacking. Our attempts at exposing safe and simple interfaces, in C, were not successful. In every iteration, we found that application developers would need to know too much detail about protocol state and state transitions. The Elixir Prototype Around that time I wrote a of creating an Ockam over Ockam in Elixir. prototype Secure Channel Routing Elixir programs run on BEAM, the Erlang Virtual Machine. BEAM provides Erlang Processes. Erlang Processes are lightweight stateful concurrent actors. Since actors can run concurrently while maintaining internal state, it became easy to run a concurrent stack of stateful protocols Ockam Transports + Ockam Routing + Ockam Secure Channels. I was able to hide all the stateful layers and create a simple one-line function that someone can call to create an end-to-end encrypted secure channel over a variety of multi-hop, multi-protocol routes. {:ok, channel} = Ockam.SecureChannel.create(route, vault, keypair) An application developer would invoke this simple function and multiple concurrent actors would run the underlying layers of stateful protocols. The function would return when the channel is established or if there is an error. This is exactly what we wanted in our interface. But Elixir isn’t like C. It doesn’t run that well on small/constrained computers and it's not a good choice for being wrapped in language-specific idiomatic wrappers. Exploring Rust At this point, we knew we wanted to implement lightweight actors but we also knew C would not make that easy. This is when I started digging into Rust and very quickly encountered a few things that made Rust very attractive: Compatibility with the C calling convention Rust libraries can export an interface that is compatible with C's calling convention. This means that any language or runtime that can statically or dynamically link and call functions in a C library can also link and call functions in a Rust library - in the exact same way. Since most languages support native functions in C, they also already support native functions in Rust. This made Rust equal to C from the perspective of our requirement of having language-specific wrappers around our core library. Support for lots of targets Rust compiles using LLVM which means that it can target a very large number of computers. This set is likely not as big as everything that C can target with GCC and various proprietary GCC forks but is still a very large subset and there’s work ongoing to make Rust compile with GCC. With the growing support of new LLVM targets and potential GCC support in Rust, it seemed like a good bet from the perspective of our requirement of being able to run everywhere. Strong typing and a powerful type system Rust’s type system allows us to turn invariants into compile-time errors. This reduces the set of possible mistakes that can be shipped to production by making them easier to catch at development time. Our team and the user of our Rust library become less likely to ship behavioral bugs or security vulnerabilities to production. Memory safety and the borrow checker Rust’s memory safety features eliminate the possibility of use-after-frees, double frees, overflows, out-of-bounds access, data races and many other common mistakes that is known to cause 60-70% of high-severity vulnerabilities in large C or C++ codebases. Rust provides this safety at compile time without incurring the performance costs of safely managing memory at runtime using a garbage collector. This gives Rust a serious advantage to write code that needs to be highly performant, run in constrained environments, and be highly secure. Async/await and pluggable async runtimes The final piece that convinced me that Rust is a great fit for Ockam was . We had already identified that we need lightweight actors to create a simple and safe interface Ockam's stack of protocols. meant that a lot of the hard work to create actors had already been done in projects like tokio and async-std. We could build Ockam's actor implementation on this foundation. async/await async/await Another important aspect that stood out was that in rust has one significant difference from in other languages like Javascript. In Javascript a browser engine or nodejs picks the way it will run async functions. But in Rust you can plugin a mechanism of your own choice. These are called async runtimes - tokio is a popular example of such a runtime that is optimized for highly scalable systems. But we don't always have to use tokio, we can instead chose an async runtime optimized for tiny embedded devices or microcontrollers. async/await async/await This meant that Ockam's actor implementation, which we later called Ockam , if we base it on Rust's can present exactly the same interface to our users regardless of where it is running - big computers or tiny computers. All our protocol interfaces that would sit on top of Ockam Workers can also present the exact same simple interface - regardless of where they are running. Workers async/await At this point we were convinced we should re-write Ockam in Rust. In the conversation, that I mentioned earlier, Paul Dix and I discussed what the transition looked like for our teams at Ockam and InfluxDB after each project had decided to switch to Rust. We discussed how InfluxDB moved from Go to Rust and how Ockam moved from C to Rust. In case you're interested, in that part of our journey go listen to the . webinar recording Many iterations later, anyone can now to create an end-to-end encrypted and mutually authenticated secure channel with a simple function call. use the Ockam crate in rust Here’s that one single line, we had imagined when we started: let channel = node.create_secure_channel(&identity, route, options).await?; It creates an authenticated and encrypted channel over arbitrary multi-hop, multi-protocol routes that can span across private networks and clouds. We are able to hide all the underlying complexity and foot guns behind this simple and safe function call. The code remains the same regardless of how you use it - on servers or tiny microcontrollers. scalable To learn more, check kout or try the step-by-step walk-throughs of the and . Ockam on GitHub Ockam Rust library Ockam Command Also published here. If you've been part of a project that was rewritten in Rust, come share your team's story with us on our . We're also hiring for both Rust and Elixir roles, come - we’re on a quest to make software more secure and private by design. discord server join our team of builders

The is an opinion piece based on the author’s POV and does not necessarily reflect the views of HackerNoon.

Why We Rewrote Ockam in Rust

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

A CTO’s Reflection of the 2023 RSA Conference

$DEFI Token Hits 7 Major Exchanges: A Milestone Achievement

$JTC Network To List On BitMart Exchange

$500k Presale: TG.Casino Passes Milestone with Upcoming Telegram-Powered Platform

$3 Million in Seed Funding for Web3 Founders Announced By Necto Labs

A CTO’s Reflection of the 2023 RSA Conference

$DEFI Token Hits 7 Major Exchanges: A Milestone Achievement

$JTC Network To List On BitMart Exchange

$500k Presale: TG.Casino Passes Milestone with Upcoming Telegram-Powered Platform

$3 Million in Seed Funding for Web3 Founders Announced By Necto Labs

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps