Web Assembly: The Future of JS and a Multi-Language Web

Web Assembly: The Future of JS and a Multi-Language Web

Let's talk about WebAssembly; why it came about, what it's good at, and why it really doesn't spell doom for JS but instead allows you to truly enhance web experiences by augmenting JS with the ability to use modules written in other languages that can perform tasks faster or more efficiently.

80% of users are running browsers capable of running languages other than JS in their browser.

It's a new change, but an important one; we're looking at a future where the web will be built not just on JS, but on C, Rust, even GoLang.

Let's talk about WebAssembly; why it came about, what it's good at, and why it really doesn't spell doom for JS but instead allows you to truly enhance web experiences by augmenting JS with the ability to use modules written in other languages that can perform tasks faster or more efficiently.

We'll go over what WebAssembly is, how to build a WebAssembly module, and why this may be a force that will shape not only how we think about JS, but the entire web experience.

Mobile App Development Company India | Ecommerce Web Development Company India

Mobile App Development Company India | Ecommerce Web Development Company India

Best Mobile App Development Company India, WebClues Global is one of the leading web and mobile app development company. Our team offers complete IT solutions including Cross-Platform App Development, CMS & E-Commerce, and UI/UX Design.

We are custom eCommerce Development Company working with all types of industry verticals and providing them end-to-end solutions for their eCommerce store development.

Know more about Top E-Commerce Web Development Company

Building a WebAssembly Module from Rust

Building a WebAssembly Module from Rust

In this article, you'll learn how to generate & build a WebAssembly module from Rust and import it into a React project

WebAssembly, the future of the web?

WebAssembly is being billed as the future technology of the web, with the ability to run code in the browser at near native speeds. WebAssembly (often shortened to wasm) is a compact binary format that is compiled from languages like C and C++, as well as Rust — the language we will be demonstrating in this article.

Rust, being a relatively new language itself, has already built up a lot of support for developing, compiling and publishing modules in WebAssembly, mostly led by the Mozilla Foundation. Development of both Rust and WebAssembly are ongoing, albeit Rust being in a more mature state. Nonetheless, we can expect the APIs talked about in this article to be changed over time, which we’ll attempt to keep up to date as releases are rolled out.

AssemblyScript, a strict subset of Typescript, is another interesting language aimed at Javascript developers to start adopting Wasm, having a smaller learning curve for front-end developers than Rust or C++. This is a good example of innovation happening in the WebAssembly ecosystem.

In this piece we will publish a wasm module that makes a fetch request to Github, and return the resulting JSON. This compiled wasm function will then be called from Javascript in a Create React App project.

The full project that coincides with this piece can be found here on Github.

The Rust WebAssembly tools

To build a wasm module with Rust, we’ll utilise two frameworks to package up the module and get the compiled WebAssembly to interact with the browser:

  • wasm-pack (Github): The “one-stop shop” for compiling Rust based WebAssembly for the web. wasm-pack is a CLI tool that can build, test and publish WebAssembly modules
  • wasm-bindgen (Github): A Rust library and CLI tool for facilitating interactions with Javascript and the DOM. In fact, the underlying APIs of wasm-bindgen provide bindings for all the Web APIs, making it possible to manipulate the DOM, listen to events, call fetch requests, websockets, and more — all with Rust compiled wasm

It is important to stress that WebAssembly cannot directly access the DOM — yet. This will undoubtedly be different in 1–2 years, where various proposals being developed now will resolve features like multi-threading and direct DOM manipulation, two major bottlenecks of WebAssembly adoption currently existing.

For now, wasm-bindgen acts as the bridge to interact with Javascript APIs, all of which have Rust bindings within the library. This is done with two underlying dependencies of wasm-bindgen: the js-sys crate and web-sys crate, exposing the entire Javascript standard library and Web API library respectively, to Rust and WebAssembly.

To summarise this introductory, what we can most likely expect in the near term is:

  • wasm-bindgen to dramatically speed up as more WebAssembly proposals are implemented and support rolled out in browsers. We can also expect smaller module sizes as boilerplate code becomes unnecessary
  • wasm-bindgen will attempt to maintain their top level APIs as features are rolled out, but the underlying web-sys will undoubtedly undergo major changes as direct DOM manipulation rolls out
  • js-sys will still be around for the purpose of interacting with Javascript standard library and Javascript modules, where the two languages will be working hand in hand

This last point will most likely be the main use case of WebAssembly in the short term. Where libraries of modules (and entire web apps) have already been established in Javascript, there will be little incentive to re-build entire projects into WebAssembly.

However, what we can expect is WebAssembly based modules, that do specific things very well, to be wrapped up in NPM modules and imported into a Javascript project. Modules like:

  • Computationally expensive things like crunching numbers, rendering 3D objects, or running machine learning algorithms. These tasks in wasm will be running an order of magnitude faster than their Javascript counterparts
  • Blockchain light clients and distributed network protocols. With blockchain clients, particularly Parity’s Ethereum and Substrate frameworks, it will become apparent that compiling their existing source code into WebAssembly will make total sense, having client clients run in the browser as an imported module. These will also leverage speed and efficiency, while modularising protocol level APIs from your UX code
  • With Rust’s built in memory safety and strict typing, we can also expect mission critical tools such as security features and live chat / real-time web environments to be implemented as WebAssembly modules, with a focus on stability
Javascript is not going anywhere

What does all this mean in terms of Javascript? At this point it is hard to see a future without Javascript being the dominant language of the web. Javascript is not going anywhere, and WebAssembly will more than likely be a compliment to Javascript, rather than a replacement, speeding up certain parts of applications and providing more ways for codebases from other languages to run in the browser.

With this understanding, let’s now jump into a real-world example. We’ll be slightly modifying the fetch request example from the wasm-bindgen examples hosted on Github.

We’ll then import this module asynchronously into a Create React App project and call our wasm function within a React component, providing webpack the means to recognise wasm based modules in the process without ejecting the project.

Installing Wasm Pack

For the sake of this talk we will clone the git repository coinciding with this piece, that contains a wasm-pack generated Rust project with the fetch request code therein. We’ll also cover some basic setup steps.

A note on VS Code

Visual Studio code has great support for Rust with the RLS extension. In addition, the WebAssembly extension is also available to have wasm syntax highlighting and the ability to easily preview wasm binaries.

The following instructions assume that you already have Rust installed on your system. If not, head over to the Rust Installation page and download rustup.

Let’s firstly install the required CLI tools for our wasm endeavours, starting with wasm-pack. The latest installation instructions will be on their installation page, but installing the package only requires one curl request:

# install wasm-pack

curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh

A wasm-pack binary will be installed in your ~/.cargo/bin directory. The program contains the new, build, test and publish commands, the documentation of which can be found here.

Getting familiar with wasm-bindgen

Also recommended is to clone the entire wasm-bindgen repository, and have all the example code present on your system to test:

# clone wasm-bindgen

git clone https://github.com/rustwasm/wasm-bindgen

The examples/ folder contains a range of projects, albeit not configured to be built as a module. Each example can be viewed online and is accompanied with a dedicated documentation page. Refer to each example’s README.md file to get links to the live demo and documentation.

The simplest wasm-bindgen example is the console_log project, showcasing how to bind the console.log() Javascript function to a Rust function — the simplest implementation of which being the following:

#[wasm_bindgen]

extern "C" {   #[wasm_bindgen(js_namespace = console)]

   fn log(s: &str);

}

We must annotate all wasm blocks and functions with #[wasm_bindgen] to let the compiler know this block of code is to be compiled into WebAssembly. This is a type of Conditional Compilation, and wasm_bindgen is termed an attribute.

Note: You may see some WebAssembly examples wrapped in extern {}, while others are wrapped in extern “C” {}. There is currently no significance between the two. One concerned developer opened an issue for the discrepancy, that appears to stem from an auto-formatting feature of rustfmt.

Visiting conditional compilation

Rust has a range of conditional compilation attributes built in, such as the #cfg attribute to determine whether blocks of code should be compiled based on variables, some built in, such as the platform your program is being compiled to. Here are a couple of examples taken from the documentation:

// This function is only compiled when compiling for macOS

#[cfg(target_os = "macos")]

fn macos_only() {  

   // ...

}// This function is only compiled when either foo or bar is defined #[cfg(any(foo, bar))]

fn needs_foo_or_bar() {  

   // ...

}

In the first example, target_os is one of a few set configuration options, that determines which platform we’re compiling to. We can even define platform specific dependencies within Cargo.toml; if I wanted to support macOS core libraries only on that platform, I could do the following in Cargo.toml:

# Cargo.toml[target.'cfg(target_os = "macos")'.dependencies]

cocoa = "0.18.4"

core-foundation = "0.6"

core-graphics = "0.17.3"

...

You may also have noticed the #[test] attribute in your Rust projects too — preventing code annotated with #[test] from being compiled, and to be run in conjunction with cargo test.

So #[wasm] and #[wasm-bindgen] are custom annotations specifically for compiling into WebAssembly. Revisiting the console.log example, we are also using attribute arguments to specify the Javascript namespace to bind our function to:

#[wasm_bindgen(js_namespace = console)]

fn log(s: &str);

...

In the above example, we’re aiming to bind the log() function to the console.log() Javascript function.

On the subject of conditional compilation, WebAssembly projects also commonly use a feature cfg parameter, further filtering what is compiled based on features we define in Cargo.toml:

[features]

default = ["super_mode"]

These features can then be used within #[cfg]:

#[cfg(feature = "super_mode")]

fn super_execute(s: &str;) {

   ...

}

Attributes can be fun to work with, adding flexibility to your code while accommodating edge cases you may run into — e.g. custom builds for specific clients. Both wasm-pack and wasm-bindgen rely heavily on the feature for custom compilation into wasm. Read more about Attributes in Rust here.

Back to the console.log demo, the demo also documents how console.log() is polymorphic, and can accept multiple arguments. Because of this, we can also bind multiple signatures while ensuring the console.log() function is still being called, with the js_name attribute parameter:

# a log function that takes a unsigned integar#[wasm_bindgen(js_namespace = console, js_name = log)]   

fn log_u32(a: u32);

This is a great example of how Rust can bind to Javascript, being the opposite of what we are attempting with our fetch example, aiming to call Rust compiled wasm from the Javascript side.

Beyond running these examples from the repository, it is also good practice to copy functionality from them into your own wasm-pack projects, remembering to include the required dependencies from Cargo.toml to coincide with the functions.

With some Rust understanding cemented, let’s now move onto our wasm fetch demo.

Wasm Fetch Project Structure

Clone the following repository to fetch the example demo we’ll walk through next, that hosts both the Rust module and React app client:

git clone https://github.com/rossbulat/wasm-fetch-example

The wasm-module directory contains the Rust project, whereas the client directory contains the React client.

I have packaged both projects into a single repository for convenience purposes. It is recommended to separate them into separate repositories and have each project’s config files at the top level (Cargo.toml, package.json, etc).

Inside wasm-module

wasm-pack new has been run to generate the project structure, and the fetch functionality has been plugged into our lib.rs.

Upon calling the call_fetch() function (that we’ll do in Javascript), a fetch request is called to the Github wasm-bindgen repository, fetching the latest commits from the repo. The resulting JSON is returned, which will then be available to deconstruct on the Javascript side.

The folder structure is simple, with our Rust source files within the src/ directory, and Cargo.toml outlining the project dependencies (in the form of Cargo crates), and some initial configuration:

# Rust project structuresrc/

   fetch.rs

   lib.rs

   utils.rs.gitignore

Cargo.toml

...

These are the files we are interested in. Let’s drill down what this project consists of:

  • The src/ folder contains our Rust code to be compiled. The meat of the project is in lib.rs, with the call_fetch() as the function we wish to call in Javascript. This file contains a range of use statements to bring various libraries into scope, including the required wasm_bindgen Javascript bindings, and types, required
  • src/fetch.rs simply contains some structs that will be used to store returned fetch data. In the original wasm-bindgen fetch example, these structs were also defined within lib.rs, along with the rest of the example code. For the sake of readability, I have opted to separate them from the main execution
  • Cargo.toml is a key file to understand, defining the dependencies and “features” to be used in the project. We will visit how these work further down

The execution flow within call_fetch() calls a Javascript fetch request from the wasm-bindgen bindings, before converting the returned Promise into a Rust Future — the Rust equivalent of a JS promise. Once a response is received from Github, it is persisted and formatted via the structs defined in fetch.rs.

This is the full execution flow:

# fetch request execution flow-> JS calls call_fetch()-> Fetch request is called via wasm-bindgen bindings, returns promise-> JS Promise converted into Rust Future-> Await Github response and store via provided structs-> Rust Future converted back to JS Promise-> Return response for use in JS

As we explored earlier, this process will become more simplified as the WebAssembly specification becomes more capable.

Building the module with wasm-pack build

When running wasm-pack build, the compiled result will be output to a pkg/ directory in your project folder. We can in fact do this now to examine the result:

# build projectcd wasm-fetch-example

wasm-pack build> [INFO]: 🎯  Checking for the Wasm target...

> [INFO]: 🌀  Compiling to Wasm...

> Compiling proc-macro2 v0.4.30

  ...

The project build time will depend on your system. The resulting pkg/ folder will contain our compiled module:

# pkg contents pkg/

   README.md  

   wasm_fetch_example.d.ts 

   wasm_fetch_example_bg.d.ts

   package.json 

   wasm_fetch_example.js 

   wasm_fetch_example_bg.wasm

We can see that wasm-pack does not simply output a .wasm binary for us:

  • A package.json has been generated, treating this directory as a module itself, ready to be published to NPM or another directory (we’ll briefly cover publishing the module to a private registry further down)
  • Type definition files .d.ts have been generated for your module and surrounding binding functions in the event you are importing the module into a Typescript based project. These files contains types for every export of our wasm module — these could be constants, functions classes etc
  • wasm_fetch_example.js contains Javascript bindings to the wasm module itself
  • Your README.md will be copied to the package also, to provide documentation about the module

wasm-pack has done the work of formatting the project ready to be published as a module, and supports both Javascript and its Typescript superset. Upon compiling, you may see this warning in the console:

> Optional fields missing from Cargo.toml: 'description', 'repository', and 'license'. These are not necessary, but recommended

wasm-pack will attempt to take this information from Cargo.toml to populate package.json. You could indeed slot in this information in Cargo.toml under the [package] section:

# Cargo.toml[package]

description = "An example Rust based WebAssembly project implementing a fetch request via wasm-bindgen."

license = "MIT"

repository = "<your_repo_url>",

...

Note: Subsequent builds are a lot quicker, only re-compiling the changes you have made.

Publishing wasm modules

In the event you wish to publicly publish your WebAssembly module, you can indeed do so with the pack and publish commands wasm-pack provides:

  • wasm-pack pack creates a tarball from the pkg/ directory
  • wasm-pack publish creates a tarball from the pkg/ directory, and then publishes it to the public NPM registry

The publish command is simply a pointer to the npm publish command, the official means of publishing to an NPM registry. So all the flags available to npm publish are also available to wasm-pack publish.

# publishing to public npm registry

wasm-pack build

It is likely that you’ll want to publish your module privately to a private registry. I have published an article on exactly how to set up such a registry, using the private proxy registry Verdaccio.

Once you have a private registry set up, simply provide the URL with the --registry flag:

# publish to a private registry

wasm-pack publish --registry "http://<your_registry_ip_or_domain>"

This provides a means of testing WebAssembly libraries internally — a more realistic scenario for organisations that are iteratively developing WebAssembly modules as the specification evolves.

wasm-pack has now played its part, its role stops after the publishing process, with our module ready to be added to projects as a dependency. Let’s now explore how to use wasm modules in a React project.

Importing wasm modules

To test the fetch functionality, I have included a base Create React App project in the accompanying repository of this piece.

To save the reader from publishing the wasm module themselves for use with this project, I have included it in within node_modules, ignoring the entire directory apart from this module:

// .gitignoreclient/node_modules/*

!/client/node_modules/wasm-fetch-example

...

Amending Create React App to support wasm modules

Create React App does not currently support WebAssembly based modules in its Webpack configuration.

Note: ECMA Script module integration is currently an active WebAssembly proposal — we can expect a more streamlined integration process once these features are finalised and rolled out.

This again is most likely a short term issue, and will be resolved as WebAssembly gains more adoption — but there is a solution.

In order to support our newly published module, we need to amend the Webpack configuration of Create React App, adding a wasm loader. We can indeed do this, without ejecting CRA, with a package called react-app-rewired, along with wasm-loader, adding WebAssembly support to Webpack.

These have been installed as dev dependencies:

yarn add react-app-rewired wasm-loader --dev

A config-overrides.js script has been defined in the client’s root directory, that plugs in support for wasm based modules.

The last amendment here is in package.json, where we are calling react-app-rewired instead of react-scripts when compiling and running the app:

// package.json"script": {

   "start": "react-app-rewired start",

   "build": "react-app-rewired build",

   "test": "react-app-rewired test",

   ...

}

Asynchronously importing a wasm module

App.tsx demonstrates how we can asynchronously import a wasm module and load it into a component’s state.

Here is the full solution:

import React from 'react';
import logo from './logo.svg';
import './App.css';

const App: React.FC = () => {

// wasm module will be stored in state once loaded
const [wasmModule, setWasmModule] = React.useState();

// asynchronous function to fetch module and load into state
const loadWasm = async () => {
try {
const wasm = await import('wasm-fetch-example');
setWasmModule({ wasm });
console.log('wasm set');
} catch (err) {
console.error(Unexpected error in loadWasm. [Message: ${err.message}]);
}
};

// takes our module and calls the call_fetch() function
const callFetch = async ({ wasm }: { wasm: any }) => {
console.log('calling fetch');
const res = await wasm.call_fetch();
console.log(res);
}

// load wasm asynchronously if not yet defined
wasmModule === undefined && loadWasm();

// call fetch once module has imported
if (wasmModule !== undefined) {
callFetch(wasmModule);
}

return (
<div className="App">
</div>
)
}

export default App;

In Summary

This article has aimed to introduce the process of building a WebAsssembly module from Rust, with wasm-pack as our means of generating and compiling the final module. We have ascertained:

  • That wasm-bindgen provides us with bindings to the Javascript standard library and standard Web API library, giving us access to call Javascript, manipulate the DOM, get window and event data, etc — all from our Rust wasm module
  • wasm-pack is a useful tool for generating a bare-bones Rust based WebAssembly project, and automates the process of compiling wasm and preparing the resulting package to be published as a module
  • Create React App by default does not currently support .wasm module extensions in its Webpack config. To get WebAssembly into your components, the react-app-rewired package can be used to plug a wasm-loader into the Webpack configuration, extending that of CRA’s configuration, without the need to eject the project
  • Importing wasm modules asynchronously via Promises will not interrupt the flow of your app. Loading indicators or prompts can be used to let the user know your module is being loaded into state

Thanks for reading

If you liked this post, share it with all of your programming buddies!

Follow us on Facebook | Twitter

Further reading

Why you should move from Node.js to Rust in 2019

The Rust Programming Language

Rust Vs. Haskell: Which Language is Best for API Design?

7 reasons why you should learn Rust programming language in 2019

An introduction to Web Development with Rust for Node.js Developers

The Rust developers should Know top 8 Rust IDE & text Editors

WebAssembly: The Future of JS and a Multi-Language Web

Using WebAssembly With Node.js

Get started with WebAssembly using JavaScript

Building full-stack web apps with Go, Vecty and WebAssembly

Building full-stack web apps with Go, Vecty and WebAssembly

In this article, you'll learn how to build full-stack web apps with Go, Vecty and WebAssembly

Many of us have heard of — and maybe written — full-stack web applications. We do them in a variety of different ways, but the common denominator is usually JavaScript and Node.js.

Today, we’re going to break with that tradition and write a complete web applicationfront end and backend – without writing a line of JavaScript. We’ll be comparing the developer experience to JavaScript along the way, but we’re going to be only writing Go for this entire project.

We’ll learn how to build a single page link shortener application with just Go, and we’ll end up with working code that shows it in action.

Prerequisites

Today, we’re going to be focusing on Go so make sure you’ve installed the tooling on your machine. I’m going to assume you have basic knowledge of Go, but check out the free Tour of Go to brush up (or learn!) if you need to.

All the shell commands that I’m going to be showing work on a Mac, but should also work on most Linux systems (including WSL!).

Finally, make sure to clone the repository with the code for this article.

And then you’re good to go, let’s get started!

Getting started

First, we’re going to get the application running locally.

Coming from Webpack and surrounding technologies — which you’d use to build a web app with JavaScript — building and running this application is embarrassingly easy. There’s a front-end and a backend part (more on that below), and you compile both of them with the go tool, which requires no configuration.

First, run the backend server:

$ go run .

Next, build the front end in a new terminal window:

$ cd frontend
$ GOOS=js GOARCH=wasm go build -o ../public/frontend.wasm

Finally, go to https://localhost:8081 in your browser to see the app in action.

How this all works

Like most web apps, our link shortener has a front-end and backend piece. In our app, the backend is just a static server written in Go. All of the magic is in the front-end directory, so let’s start there!

If you’re familiar with React or the DOM, you’ll recognize a lot of the concepts we’ll cover. If not, this stuff will come naturally.

We’re using a new Go framework called Vecty to organize our application. Vecty forces you to break down your app into components and arrange them into a tree. The whole scheme is really similar to HTML and the DOM or React.

Here’s what our app’s high-level components would look like if they were HTML:

  • A h2 for the title of the page
  • A form to enter the link to shorten
  • A div to hold the shortened link (this value is dynamically updated as the user types the link into the above)
  • An a to save the short link

Vecty components are so similar to React that they look like the Go equivalent of JSX, except that they have more parentheses.

Let’s zoom in on one and see how it works. Here’s the code for the form component:

elem.Form(
    elem.Input(vecty.Markup(
        event.Input(func(e *vecty.Event) {
            short := uuid.NewV4().String()[0:5]
            h.shortened = short
            vecty.Rerender(h)
        }),
    )),
)

First, elem.Form and elem.Input on lines 1 and 2 are for the <form> and <input> tags, respectively. Those are both Go functions that take one or more arguments. Each argument is something that goes between the opening and closing HTML tags. For example, the stuff we pass to elem.Form goes in between <form> and </form>. This is what the above Go code would look like in HTML:

<form>
    <input>
    </input>
</form>

Pretty simple, right?

The last piece of code we didn’t look at is that event.Input function. This is an event handler just like in HTML/JavaScript. This function takes in another function, which in this case is roughly an onchange handler. Just like you’d expect, that *vecty.Event argument the handler takes in is roughly the same as the JavaScript event.

The logic to actually shorten the link is all inside this handler, and it’s fairly simple. Here is that code commented thoroughly:

// First, make a new UUID and take the first 5 characters of it.
// This will be our new shortcode
short := uuid.NewV4().String()[0:5]
// Next, write the shortcode to a variable. This variable is shared
// with the <div>, so when we re-render this component, the <div> will
// get updated
h.shortened = short
// Finally, re-render the component so that the <div> gets the new shortcode.
// Unlike React, there's no automatic diff functionality. We tell Vecty
// explicitly which components to re-render.
vecty.Rerender(h)
You get web assembly for free

Vecty can scale to big applications because of this component structure, and we can scale our app as big as we want by adding more components as needed. For example, we can add a component above our current top-level to dynamically route to different sub-components based on the URL. This would be similar to some of the popular react-router implementations.

One final thing to keep in mind is that WASM is not HTML, it’s a full departure from the DOM and everything HTML.

I compared all the components in the last section to HTML tags, but they aren’t! That’s where the big difference between Vecty / WASM and React comes in. We’re compiling our Go code straight to WASM, which represents these components differently from the DOM.

Conclusion

At the end of the day, you get some big benefits from using Go and Vecty to build apps:

  1. You get to think in terms of components and nesting, just like with React and the DOM
  2. You can write as much dynamic logic as you want, right next to your components, all in pure Go
  3. You can share code between the server and client, similar to writing a React client and a Node.js server
  4. You get to take advantage of WASM
  5. Or you can compile your Vecty code to HTML too if you want! That’s a whole other article

Thanks for reading

If you liked this post, share it with all of your programming buddies!

Follow us on Facebook | Twitter

Further reading about WebAssembly

WebAssembly: The Future of JS and a Multi-Language Web

Using WebAssembly With Node.js

Get started with WebAssembly using JavaScript

WebAssembly for Web Developers

What is WebAssembly?

Speed, Speed, Speed: JavaScript vs C++ vs WebAssembly

WebAssembly Disrupting JavaScript

WebAssembly: Expectation vs. Reality