C++ to WebAssembly using Bazel and Emscripten

How to get Bazel and Emscripten to compile C++ to WebAssembly or JavaScript In my quest to generate a re-usable WebAssembly template, I discovered many ways that appear easy to implement, but don’t really work in applications beyond a simple ‘hello world’. If you’re interested in building slightly more complex WebAssembly modules, this article is for you. Ultimately, I’m looking to compile a nice C/C++ library to the JavaScript domain — I’m not looking to build a specific one-off functions, I want the entire library support (or at least the majority). Additionally, I want this to run in the browser as well as NodeJS. I don’t want to deal with instantiating the WASM or manage its memory across these environments either. These requirements mean I can rule out several alternatives… : Use with . Shortcut to the . Spoiler Emscripten Bazel github repo What’s out there? There are many starting points and many tutorials on how to get started using C/C++ with JavaScript. Here are a few tools you would find in the wild: (WAT) WebAssembly Text Format (N-API) Node Addons LLVM Wasmer Cheerp Emscripten Some of these tools are not exactly what I want… — requires a lot of low-level work and only makes sense for simple one-off functions. 👎👎👎 WAT — is not much better. I’m still writing a lot of bindings, need to worry about node-gyp, and it may not work in the browser. 👎👎 N-API — allows us to directly compile C/C++ to WASM! Unfortunately, this is still a low-level job that requires me to perform just to get it working. 👎 LLVM a lot of extra steps — actually looks great! They’re a relatively new player and support lots of integrations. Event their builds are relatively lean! Unfortunately, they still to the native C/C++ code which is not really pleasant for larger projects. However, they are working on better integration support and are moving rather fast. 👍 Wasmer require a lot of glue — another great tool. They’re similar to emscripten, but have a different that allows for automated garbage collection. The performance is quite similar, often beating emscripten in special cases. However, the community support is not quite as large and I found myself getting stuck. I’ll keep these guys on the radar. 👍👍 Cheerp memory model — just right. Integration with C++ is made extremely easy by using . I can pass non-primitive types between both domains (C++ only). They have a larger community presence. They can output into a format that is relatively straight forward to use in the browser or NodeJS with ease. 👍👍👍 Emscripten embind Getting started I’ll showcase a simple “hello world” C++ application that we will convert to WebAssembly. How do I convert an existing C++ library? This is the crux of it all. Every toolchain has some initial difficulties setting up and I’m often left scratching my head on where to even start. No one wants to manually invoke gcc so we built scripts such as , , or to automate the build process — great! configure make cmake …except, not 🙁 Sometimes I’ve needed to hack the existing make/cmake rules to avoid dependencies on shared libraries, ignore some intrinsics checks, etc. This obviously doesn’t play nice with a centralized C++ code base that attempts to build bindings for many languages. So what are our options? 💚 Bazel — . a fast, scalable, multi-language, and extensible build system While this build system can be quite daunting, it is actually powerful. Unfortunately, there’s just not that much documentation to learn to use it with emscripten. In fact — their docs are , more , and . very broken broken maybe not even supported I argue that it can be done decently well — even the reputable team has managed to get it working! So what was so difficult? What makes it so special? TensorFlow.js After converting several libraries to WebAssembly, I can tell you that the isolation Bazel offers is quite nice — no horrible breaking changes when a cmake script has been modified. No more complex logic determining the target to build, etc. Once defined it will almost always . just work First steps . You will also need to install the dev dependencies. Install Bazel yarn Fast forward a bit, so you can follow along. here is the github repo : I’ve taken a lot of inspiration from the project on how they managed to get it working. My changes revolve around compiler/linker flags, showing how to output both JS and WASM, and most important — using the emscripten release 🎉! Note TensorFlow.js latest git clone --recurse-submodules https://github.com/s0l0ist/bazel-emscripten.git cd bazel-emscripten yarn install I’ve taken the liberty to include the emsdk as a git submodule instead of managing it yourself. The first step is to get the emsdk cloned. If you’ve cloned my repo recursively, you can skip this step: yarn submodule:update Next, we need to update the release tags and then install the latest version of emscripten: yarn em:update yarn em:init Done 🎉! The layout Some important files and directories: — describes default commands for building a target .bazelrc — defines our external dependencies WORKSPACE Some files inside : hello-world/ — empty file so bazel doesn’t complain BUILD — bazel toolchain dependencies (emsdk) deps.bzl A few directories in : hello-world/ — holds the simple C++ sources cpp/ — holds all JS related material javascript/ — holds all emscripten bindings javascript/bindings/ — holds all JS wrappers javascript/src/ — the handy build scripts to shorten our statements javascript/scripts cli — the heart of the Bazel + Emscripten configuration javascript/toolchain The rest is self explanatory. The code 💻 I’ve outlined a very simple library containing and classes that have static methods for this example: Greet LocalTime class: LocalTime HelloWorld { : ; }; } HelloWorld { LocalTime::Now() { :: result = ::time( ); ( , ::asctime( ::localtime(&result))); } } //////// cpp/localtime.hpp //////// # LIB_LOCAL_TIME_H_ ifndef # LIB_LOCAL_TIME_H_ define namespace { class LocalTime public /* * Prints the current time to stdout */ static void Now () // namespace HelloWorld # endif //////// cpp/localtime.cpp //////// # include # include # include "localtime.hpp" namespace void std time_t std nullptr printf "%s" std std // namespace HelloWorld class: Greet HelloWorld { : :: ; }; } HelloWorld { :: Greet::SayHello( :: &name) { + name + ; } } //////// cpp/greet.hpp //////// # LIB_GREET_H_ ifndef # LIB_GREET_H_ define # include namespace { class Greet public /* * Greets the name */ static std string SayHello ( :: &name) const std string // namespace HelloWorld # endif //////// cpp/greet.cpp //////// # include # include "greet.hpp" namespace std string const std string return "Hello, " "!" // namespace HelloWorld Emscripten bindings 🦾 The bindings are quite short for our example. We make use of the powerful which lets us talk to C++ classes. embind You may notice that outputs directly to . Emscripten is intelligent enough to redirect our output to so we don’t need to do anything else 😎. returns a primitive that we will manually need to send to . LocalTime::Now stdout console.log Greet::SayHello string console.log emscripten; HelloWorld; EMSCRIPTEN_BINDINGS(Hello_World) { class_ ( ) .constructor<>() .class_function( , &Greet::SayHello); class_ ( ) .constructor<>() .class_function( , &LocalTime::Now); } //////// javascript/bindings/hello-world.cpp //////// # include # include "hello-world/cpp/greet.hpp" # include "hello-world/cpp/localtime.hpp" using namespace using namespace "Greet" "SayHello" "LocalTime" "Now" Now that we’ve defined our bindings, we’re ready to build! Building 🏗 You may build the native libraries, but they’re quite useless by themselves… bazel build -c opt //hello-world/cpp/... I’ve configured the file to build the with two different options: or . .bazelrc JS WASM — Specifies flags to emscripten to output a single file that does contain any WebAssembly. This is useful for environments that can’t work with WebAssembly such as React-Native, but is significantly larger and slower. JS asmjs not — Specifies flags to emscripten to output a single file containing the WebAssembly as a encoded string. This means we don’t need to manage a separate file in our bundles or figure out how to properly serve this file in the browser. The drawback is a larger file size due to the base64 encoding. WASM JavaScript base64 .wasm To make it simple, I’ve created some helper scripts so all you need to do is run the following: yarn build:js // or yarn build:wasm // or both yarn build There are some good and bad things about using emscripten here: : It generates glue code for you automatically. Good : It generates glue code for you automatically. Bad Obviously, the glue code adds some bloat but keeps me from having to deal with the intricacies of initialization 👌. : In there are a few defined compiler flags that are present for both the JS/WASM builds geared towards production use. You may feel free to modify the flags as necessary, but I wanted to show what’s possible here. Note .bazelrc If you want to have full control over instantiating the WASM to reduce the bundle size, you may generate a pure WASM build by adding the link flag inside the starlark file, . do -s STANDALONE_WASM=1 hello-world/javascript/BUILD Bundling 📦 You may have seen the files which wrap the emscripten output. Do we really need these files? — . However, I like my APIs to be abstracted from the output of emscripten. This allows for more flexibility when there are potentially breaking changes to the C++ core. javascript/src/implementation no, you don’t An important thing to note is that the outputs are quite a bit larger than you would expect. A reason for some people is that some code requires where — but our builds don’t have this problem. Then there is the glue code auto-generated to manage initialization and provide helpers for memory allocation, resizing, and the like. big a lot of code is pulled in for static constructors to initialize the iostream system even if it is not used Generate the bundles yarn rollup This gathers the files in , and produces a few output bundles in . hello-world/javascript/bin/* hello-world/javascript/src/* hello-world/javascript/dist/ You will notice two minified bundles for and for that each have two different targets for support or (for browser and NodeJS) in . js wasm ES6 module UMD hello-world/javascript/dist/ / /* Details of the rollup configuration are in . rollup.config.js Let’s run 🏃 So we’ve compiled our C++ to JS and WASM — what’s next? Run the bundle in : JS NodeJS yarn demo:js Or run the bundle in : WASM NodeJS yarn demo:wasm Or open to run the WASM bundle in the : javascript/html/index_wasm.html browser Conclusion By spending a little time with bazel, you can create a nice build system that works for many languages without breaking your other targets. We can now drive a core C++ application with bindings in several different languages all while simplifying the interoperability between them. Stay tuned for where I show a C++ library converted to JS and WASM! part 2 real Hope you enjoyed and thanks for reading! Credits for help with optimizing the bazel configuration schoppmp for the initial bazel configuration TensorFlow.js