Building a WebAssembly Module from Rust

Building a WebAssembly Module from Rust

In this article, you'll learn how to generate & build a WebAssembly module from Rust and import it into a React project

WebAssembly, the future of the web?

WebAssembly is being billed as the future technology of the web, with the ability to run code in the browser at near native speeds. WebAssembly (often shortened to wasm) is a compact binary format that is compiled from languages like C and C++, as well as Rust — the language we will be demonstrating in this article.

Rust, being a relatively new language itself, has already built up a lot of support for developing, compiling and publishing modules in WebAssembly, mostly led by the Mozilla Foundation. Development of both Rust and WebAssembly are ongoing, albeit Rust being in a more mature state. Nonetheless, we can expect the APIs talked about in this article to be changed over time, which we’ll attempt to keep up to date as releases are rolled out.

AssemblyScript, a strict subset of *[Typescript](https://morioh.com/p/e11185e87aad "Typescript*"), is another interesting language aimed at *[Javascript developers](https://morioh.com/p/f6381fc8d65a "Javascript developers*")* to start adopting Wasm, having a smaller learning curve for front-end developers than Rust or C++. This is a good example of innovation happening in the WebAssembly ecosystem.*

In this piece we will publish a wasm module that makes a fetch request to Github, and return the resulting JSON. This compiled wasm function will then be called from Javascript in a Create React App project.

The full project that coincides with this piece can be found here on Github.

The Rust WebAssembly tools

To build a wasm module with Rust, we’ll utilise two frameworks to package up the module and get the compiled WebAssembly to interact with the browser:

  • <a href="https://rustwasm.github.io/docs/wasm-pack/" target="_blank"><strong>wasm-pack</strong></a> (Github): The “one-stop shop” for compiling Rust based WebAssembly for the web. wasm-pack is a CLI tool that can build, test and publish WebAssembly modules
  • <a href="https://rustwasm.github.io/docs/wasm-bindgen/" target="_blank"><strong>wasm-bindgen</strong></a>** (Github):** A Rust library and CLI tool for facilitating interactions with Javascript and the DOM. In fact, the underlying APIs of wasm-bindgen provide bindings for all the Web APIs, making it possible to manipulate the DOM, listen to events, call fetch requests, websockets, and more — all with Rust compiled wasm

It is important to stress that WebAssembly cannot directly access the DOM — yet. This will undoubtedly be different in 1–2 years, where various proposals being developed now will resolve features like multi-threading and direct DOM manipulation, two major bottlenecks of WebAssembly adoption currently existing.

For now, wasm-bindgen acts as the bridge to interact with Javascript APIs, all of which have Rust bindings within the library. This is done with two underlying dependencies of wasm-bindgen: the <a href="https://docs.rs/js-sys/0.3.27/js_sys/" target="_blank">js-sys</a> crate and <a href="https://docs.rs/web-sys/0.3.27/web_sys/" target="_blank">web-sys</a> crate, exposing the entire Javascript standard library and Web API library respectively, to Rust and WebAssembly.

To summarise this introductory, what we can most likely expect in the near term is:

  • wasm-bindgen to dramatically speed up as more WebAssembly proposals are implemented and support rolled out in browsers. We can also expect smaller module sizes as boilerplate code becomes unnecessary
  • wasm-bindgen will attempt to maintain their top level APIs as features are rolled out, but the underlying web-sys will undoubtedly undergo major changes as direct DOM manipulation rolls out
  • js-sys will still be around for the purpose of interacting with Javascript standard library and Javascript modules, where the two languages will be working hand in hand

This last point will most likely be the main use case of WebAssembly in the short term. Where libraries of modules (and entire web apps) have already been established in Javascript, there will be little incentive to re-build entire projects into WebAssembly.

However, what we can expect is WebAssembly based modules, that do specific things very well, to be wrapped up in NPM modules and imported into a Javascript project. Modules like:

  • Computationally expensive things like crunching numbers, rendering 3D objects, or running machine learning algorithms. These tasks in wasm will be running an order of magnitude faster than their Javascript counterparts
  • Blockchain light clients and distributed network protocols. With blockchain clients, particularly Parity’s Ethereum and Substrate frameworks, it will become apparent that compiling their existing source code into WebAssembly will make total sense, having client clients run in the browser as an imported module. These will also leverage speed and efficiency, while modularising protocol level APIs from your UX code
  • With Rust’s built in memory safety and strict typing, we can also expect mission critical tools such as security features and live chat / real-time web environments to be implemented as WebAssembly modules, with a focus on stability

    Javascript is not going anywhere

What does all this mean in terms of Javascript? At this point it is hard to see a future without Javascript being the dominant language of the web. Javascript is not going anywhere, and WebAssembly will more than likely be a compliment to Javascript, rather than a replacement, speeding up certain parts of applications and providing more ways for codebases from other languages to run in the browser.

With this understanding, let’s now jump into a real-world example. We’ll be slightly modifying the fetch request example from the wasm-bindgen examples hosted on Github.

We’ll then import this module asynchronously into a Create React App project and call our wasm function within a React component, providing webpack the means to recognise wasm based modules in the process without ejecting the project.

Installing Wasm Pack

For the sake of this talk we will clone the git repository coinciding with this piece, that contains a wasm-pack generated Rust project with the fetch request code therein. We’ll also cover some basic setup steps.

A note on VS Code

Visual Studio code has great support for Rust with the RLS extension. In addition, the WebAssembly extension is also available to have wasm syntax highlighting and the ability to easily preview wasm binaries.

The following instructions assume that you already have Rust installed on your system. If not, head over to the *[Rust Installation page](https://www.rust-lang.org/tools/install "Rust Installation page*")* and download rustup.*

Let’s firstly install the required CLI tools for our wasm endeavours, starting with wasm-pack. The latest installation instructions will be on their installation page, but installing the package only requires one curl request:

# install wasm-pack

curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh

A wasm-pack binary will be installed in your ~/.cargo/bin directory. The program contains the new, build, test and publish commands, the documentation of which can be found here.

Getting familiar with wasm-bindgen

Also recommended is to clone the entire wasm-bindgen repository, and have all the example code present on your system to test:

# clone wasm-bindgen

git clone https://github.com/rustwasm/wasm-bindgen

The <a href="https://github.com/rustwasm/wasm-bindgen/tree/master/examples" target="_blank">examples/</a> folder contains a range of projects, albeit not configured to be built as a module. Each example can be viewed online and is accompanied with a dedicated documentation page. Refer to each example’s README.md file to get links to the live demo and documentation.

The simplest wasm-bindgen example is the <a href="https://github.com/rustwasm/wasm-bindgen/tree/master/examples/console_log" target="_blank">console_log</a> project, showcasing how to bind the console.log() Javascript function to a Rust function — the simplest implementation of which being the following:

#[wasm_bindgen]

extern "C" {   #[wasm_bindgen(js_namespace = console)]

   fn log(s: &str);

}

We must annotate all wasm blocks and functions with #[wasm_bindgen] to let the compiler know this block of code is to be compiled into WebAssembly. This is a type of Conditional Compilation, and wasm_bindgen is termed an attribute.

Note:* You may see some WebAssembly examples wrapped in extern {}, while others are wrapped in extern “C” {}. There is currently no significance between the two. One concerned developer [opened an issue](https://github.com/rustwasm/wasm-bindgen/issues/422 "opened an issue*")* for the discrepancy, that appears to stem from an auto-formatting feature of rustfmt*.

Visiting conditional compilation

Rust has a range of conditional compilation attributes built in, such as the <a href="https://doc.rust-lang.org/reference/conditional-compilation.html#the-cfg-attribute" target="_blank">#cfg</a> attribute to determine whether blocks of code should be compiled based on variables, some built in, such as the platform your program is being compiled to. Here are a couple of examples taken from the documentation:

// This function is only compiled when compiling for macOS 

#[cfg(target_os = "macos")] 

fn macos_only() {   

   // ... 

}// This function is only compiled when either foo or bar is defined #[cfg(any(foo, bar))] 

fn needs_foo_or_bar() {   

   // ... 

}

In the first example, <a href="https://doc.rust-lang.org/reference/conditional-compilation.html#target_os" target="_blank">target_os</a> is one of a few set configuration options, that determines which platform we’re compiling to. We can even define platform specific dependencies within Cargo.toml; if I wanted to support macOS core libraries only on that platform, I could do the following in Cargo.toml:

# Cargo.toml[target.'cfg(target_os = "macos")'.dependencies]

cocoa = "0.18.4"

core-foundation = "0.6"

core-graphics = "0.17.3"

...

You may also have noticed the #[test] attribute in your Rust projects too — preventing code annotated with #[test] from being compiled, and to be run in conjunction with cargo test.

So #[wasm] and #[wasm-bindgen] are custom annotations specifically for compiling into WebAssembly. Revisiting the console.log example, we are also using attribute arguments to specify the Javascript namespace to bind our function to:

#[wasm_bindgen(js_namespace = console)]

fn log(s: &str);

...

In the above example, we’re aiming to bind the log() function to the console.log() Javascript function.

On the subject of conditional compilation, WebAssembly projects also commonly use a feature cfg parameter, further filtering what is compiled based on features we define in Cargo.toml:

[features]

default = ["super_mode"]

These features can then be used within #[cfg]:

#[cfg(feature = "super_mode")]

fn super_execute(s: &str;) {

   ...

}

Attributes can be fun to work with, adding flexibility to your code while accommodating edge cases you may run into — e.g. custom builds for specific clients. Both wasm-pack and wasm-bindgen rely heavily on the feature for custom compilation into wasm. Read more about Attributes in Rust here.

Back to the console.log demo, the demo also documents how console.log() is polymorphic, and can accept multiple arguments. Because of this, we can also bind multiple signatures while ensuring the console.log() function is still being called, with the js_name attribute parameter:

# a log function that takes a unsigned integar#[wasm_bindgen(js_namespace = console, js_name = log)]    

fn log_u32(a: u32);

This is a great example of how Rust can bind to Javascript, being the opposite of what we are attempting with our fetch example, aiming to call Rust compiled wasm from the Javascript side.

Beyond running these examples from the repository, it is also good practice to copy functionality from them into your own wasm-pack projects, remembering to include the required dependencies from Cargo.toml to coincide with the functions.

With some Rust understanding cemented, let’s now move onto our wasm fetch demo.

Wasm Fetch Project Structure

Clone the following repository to fetch the example demo we’ll walk through next, that hosts both the Rust module and React app client:

git clone https://github.com/rossbulat/wasm-fetch-example

The wasm-module directory contains the Rust project, whereas the client directory contains the React client.

I have packaged both projects into a single repository for convenience purposes. It is recommended to separate them into separate repositories and have each project’s config files at the top level (Cargo.toml, package.json, etc).

Inside wasm-module

wasm-pack new has been run to generate the project structure, and the fetch functionality has been plugged into our lib.rs.

Upon calling the call_fetch() function (that we’ll do in Javascript), a fetch request is called to the Github wasm-bindgen repository, fetching the latest commits from the repo. The resulting JSON is returned, which will then be available to deconstruct on the Javascript side.

The folder structure is simple, with our Rust source files within the src/ directory, and Cargo.toml outlining the project dependencies (in the form of Cargo crates), and some initial configuration:

# Rust project structuresrc/

   fetch.rs

   lib.rs

   utils.rs.gitignore

Cargo.toml

...

These are the files we are interested in. Let’s drill down what this project consists of:

  • The src/ folder contains our Rust code to be compiled. The meat of the project is in <a href="https://github.com/rossbulat/wasm-fetch-example/blob/master/wasm-module/src/lib.rs" target="_blank">lib.rs</a>, with the call_fetch() as the function we wish to call in Javascript. This file contains a range of use statements to bring various libraries into scope, including the required wasm_bindgen Javascript bindings, and types, required
  • <a href="https://github.com/rossbulat/wasm-fetch-example/blob/master/wasm-module/src/fetch.rs" target="_blank">src/fetch.rs</a> simply contains some structs that will be used to store returned fetch data. In the original wasm-bindgen fetch example, these structs were also defined within lib.rs, along with the rest of the example code. For the sake of readability, I have opted to separate them from the main execution
  • <a href="https://github.com/rossbulat/wasm-fetch-example/blob/master/wasm-module/Cargo.toml" target="_blank">Cargo.toml</a> is a key file to understand, defining the dependencies and “features” to be used in the project. We will visit how these work further down

The execution flow within call_fetch() calls a Javascript fetch request from the wasm-bindgen bindings, before converting the returned Promise into a Rust Future — the Rust equivalent of a JS promise. Once a response is received from Github, it is persisted and formatted via the structs defined in fetch.rs.

This is the full execution flow:

# fetch request execution flow-> JS calls call_fetch()-> Fetch request is called via wasm-bindgen bindings, returns promise-> JS Promise converted into Rust Future-> Await Github response and store via provided structs-> Rust Future converted back to JS Promise-> Return response for use in JS

As we explored earlier, this process will become more simplified as the WebAssembly specification becomes more capable.

Building the module with wasm-pack build

When running wasm-pack build, the compiled result will be output to a pkg/ directory in your project folder. We can in fact do this now to examine the result:

# build projectcd wasm-fetch-example

wasm-pack build> [INFO]: 🎯  Checking for the Wasm target...

> [INFO]: 🌀  Compiling to Wasm...

> Compiling proc-macro2 v0.4.30

  ...

The project build time will depend on your system. The resulting pkg/ folder will contain our compiled module:

# pkg contents pkg/

   README.md   

   wasm_fetch_example.d.ts  

   wasm_fetch_example_bg.d.ts

   package.json  

   wasm_fetch_example.js  

   wasm_fetch_example_bg.wasm

We can see that wasm-pack does not simply output a .wasm binary for us:

  • A package.json has been generated, treating this directory as a module itself, ready to be published to NPM or another directory (we’ll briefly cover publishing the module to a private registry further down)
  • Type definition files .d.ts have been generated for your module and surrounding binding functions in the event you are importing the module into a Typescript based project. These files contains types for every export of our wasm module — these could be constants, functions classes etc
  • wasm_fetch_example.js contains Javascript bindings to the wasm module itself
  • Your README.md will be copied to the package also, to provide documentation about the module

wasm-pack has done the work of formatting the project ready to be published as a module, and supports both Javascript and its Typescript superset. Upon compiling, you may see this warning in the console:

> Optional fields missing from Cargo.toml: 'description', 'repository', and 'license'. These are not necessary, but recommended

wasm-pack will attempt to take this information from Cargo.toml to populate package.json. You could indeed slot in this information in Cargo.toml under the [package] section:

# Cargo.toml[package]

description = "An example Rust based WebAssembly project implementing a fetch request via wasm-bindgen."

license = "MIT"

repository = "<your_repo_url>",

...

Note:* Subsequent builds are a lot quicker, only re-compiling the changes you have made.*

Publishing wasm modules

In the event you wish to publicly publish your WebAssembly module, you can indeed do so with the <a href="https://rustwasm.github.io/wasm-pack/book/commands/pack-and-publish.html" target="_blank">pack</a> and <a href="https://rustwasm.github.io/wasm-pack/book/commands/pack-and-publish.html" target="_blank">publish</a> commands wasm-pack provides:

  • wasm-pack pack creates a tarball from the pkg/ directory
  • wasm-pack publish creates a tarball from the pkg/ directory, and then publishes it to the public NPM registry

The publish command is simply a pointer to the npm publish command, the official means of publishing to an NPM registry. So all the flags available to npm publish are also available to wasm-pack publish.

# publishing to public npm registry

wasm-pack build

It is likely that you’ll want to publish your module privately to a private registry. I have published an article on exactly how to set up such a registry, using the private proxy registry Verdaccio.

Once you have a private registry set up, simply provide the URL with the --registry flag:

# publish to a private registry

wasm-pack publish --registry "http://<your_registry_ip_or_domain>"

This provides a means of testing WebAssembly libraries internally — a more realistic scenario for organisations that are iteratively developing WebAssembly modules as the specification evolves.

wasm-pack has now played its part, its role stops after the publishing process, with our module ready to be added to projects as a dependency. Let’s now explore how to use wasm modules in a React project.

Importing wasm modules

To test the fetch functionality, I have included a base Create React App project in the accompanying repository of this piece.

To save the reader from publishing the wasm module themselves for use with this project, I have included it in within node_modules, ignoring the entire directory apart from this module:

// .gitignoreclient/node_modules/*

!/client/node_modules/wasm-fetch-example

...

Amending Create React App to support wasm modules

Create React App does not currently support WebAssembly based modules in its Webpack configuration.

Note:* [ECMA Script module integration](https://github.com/webassembly/esm-integration "ECMA Script module integration*")* is currently an active WebAssembly proposal — we can expect a more streamlined integration process once these features are finalised and rolled out.*

This again is most likely a short term issue, and will be resolved as WebAssembly gains more adoption — but there is a solution.

In order to support our newly published module, we need to amend the Webpack configuration of Create React App, adding a wasm loader. We can indeed do this, without ejecting CRA, with a package called <a href="https://www.npmjs.com/package/react-app-rewired" target="_blank">react-app-rewired</a>, along with <a href="https://www.npmjs.com/package/wasm-loader" target="_blank">wasm-loader</a>, adding WebAssembly support to Webpack.

These have been installed as dev dependencies:

yarn add react-app-rewired wasm-loader --dev

A <a href="https://github.com/rossbulat/wasm-fetch-example/blob/master/client/config-overrides.js" target="_blank">config-overrides.js</a> script has been defined in the client’s root directory, that plugs in support for wasm based modules.

The last amendment here is in package.json, where we are calling react-app-rewired instead of react-scripts when compiling and running the app:

// package.json"script": {

   "start": "react-app-rewired start",

   "build": "react-app-rewired build",

   "test": "react-app-rewired test",

   ...

}

Asynchronously importing a wasm module

App.tsx demonstrates how we can asynchronously import a wasm module and load it into a component’s state.

Here is the full solution:

import React from 'react';
import logo from './logo.svg';
import './App.css';

const App: React.FC = () => {

  // wasm module will be stored in state once loaded
  const [wasmModule, setWasmModule] = React.useState();

  // asynchronous function to fetch module and load into state
  const loadWasm = async () => {
    try {
      const wasm = await import('wasm-fetch-example');
      setWasmModule({ wasm });
      console.log('wasm set');
    } catch (err) {
      console.error(`Unexpected error in loadWasm. [Message: ${err.message}]`);
    }
  };

  // takes our module and calls the call_fetch() function
  const callFetch = async ({ wasm }: { wasm: any }) => {
    console.log('calling fetch');
    const res = await wasm.call_fetch();
    console.log(res);
  }

  // load wasm asynchronously if not yet defined
  wasmModule === undefined && loadWasm();

  // call fetch once module has imported
  if (wasmModule !== undefined) {
    callFetch(wasmModule);
  }

  return (
    <div className="App">
    </div>
  )
}

export default App;

In Summary

This article has aimed to introduce the process of building a WebAsssembly module from Rust, with wasm-pack as our means of generating and compiling the final module. We have ascertained:

  • That wasm-bindgen provides us with bindings to the Javascript standard library and standard Web API library, giving us access to call Javascript, manipulate the DOM, get window and event data, etc — all from our Rust wasm module
  • wasm-pack is a useful tool for generating a bare-bones Rust based WebAssembly project, and automates the process of compiling wasm and preparing the resulting package to be published as a module
  • Create React App by default does not currently support .wasm module extensions in its Webpack config. To get WebAssembly into your components, the react-app-rewired package can be used to plug a wasm-loader into the Webpack configuration, extending that of CRA’s configuration, without the need to eject the project
  • Importing wasm modules asynchronously via Promises will not interrupt the flow of your app. Loading indicators or prompts can be used to let the user know your module is being loaded into state

Thanks for reading

If you liked this post, share it with all of your programming buddies!

Follow us on Facebook | Twitter

Further reading

Why you should move from Node.js to Rust in 2019

The Rust Programming Language

Rust Vs. Haskell: Which Language is Best for API Design?

7 reasons why you should learn Rust programming language in 2019

An introduction to Web Development with Rust for Node.js Developers

The Rust developers should Know top 8 Rust IDE & text Editors

WebAssembly: The Future of JS and a Multi-Language Web

Using WebAssembly With Node.js

Get started with WebAssembly using JavaScript

webassembly rust javascript web-development

Bootstrap 5 Complete Course with Examples

Bootstrap 5 Tutorial - Bootstrap 5 Crash Course for Beginners

Nest.JS Tutorial for Beginners

Hello Vue 3: A First Look at Vue 3 and the Composition API

Building a simple Applications with Vue 3

Deno Crash Course: Explore Deno and Create a full REST API with Deno

How to Build a Real-time Chat App with Deno and WebSockets

Convert HTML to Markdown Online

HTML entity encoder decoder Online

Hire Web Developer

Looking for an attractive & user-friendly web developer? HourlyDeveloper.io, a leading web, and mobile app development company, offers web developers for hire through flexible engagement models. You can **[Hire Web...

Why Web Development is Important for your Business

With the rapid development in technology, the old ways to do business have changed completely. A lot more advanced and developed ways are ...

Important Reasons to Hire a Professional Web Development Company

    You name the business and I will tell you how web development can help you promote your business. If it is a startup or you seeking some...

Rust & WebAssembly para JavaScripters

A lo largo de la charla descubriremos las características más destacables de Rust, sus similitudes y diferencias con JavaScript y veremos qué aporta Rust al futuro de la Web gracias a WebAssembly. Rust es un lenguaje tipado, rápido y seguro, que ha sido diseñado por Mozilla como lenguaje de sistemas, aunque en los últimos tiempos ha ganado mucha popularidad en el terreno del desarrollo Web gracias a WebAssembly, su amplio ecosistema y gran comunidad

Hire Dedicated eCommerce Web Developers | Top eCommerce Web Designers

Build your eCommerce project by hiring our expert eCommerce Website developers. Our Dedicated Web Designers develop powerful & robust website in a short span of time.