In this overview of WebXR technologies and the Babylon.js framework, we’ll embark on a journey through the past, present, and future of mixed reality both on the web and on immersive headsets. We’ll inspect the underpinnings of WebXR and the most important aspects of the WebXR Device API before turning our attention to Babylon.js, a framework for building immersive applications in JavaScript for web, mobile, and headset. This tutorial is geared toward web and JavaScript developers who build web applications but are looking to dip their toes into immersive experiences.
Immersive experiences, especially those governed by mixed reality (XR), which encompasses both augmented and virtual reality, are quickly gaining new attention among developers and architects interested in reaching users and customers in novel ways. For many years, the lack of adoption of mixed reality experiences came down to hardware — too expensive and unwieldy — and software — too complex and finicky to use.
But the Coronavirus pandemic may be scrambling all of those old calculations by encouraging the sorts of experiences that are mostly limited to the gaming world, which is seeing immense growth in playing time during the current crisis. The math behind three-dimensional spaces can also present barriers to developers, but fortunately, you need only a little vector geometry and matrix math to succeed with XR experiences, not a college course in linear algebra and multivariate calculus.
Though browser support for WebXR is widening, building immersive experiences into browsers or headsets can be complicated due to shifting specifications and APIs as well as rapidly evolving frameworks and best practices. But incorporating immersion into your next web application can also introduce new dimensionality and richness to your user experience — all without the need to learn a new programming language.
- What Is WebXR?
- The WebXR Specification And Browser Support
- Field Of View (FOV) And Degrees of Freedom (DoF)
- WebXR Session Modes
- Setting A Scene With WebXR And Babylon.js
- Introducing Babylon.js
- Lights, Camera, Action!
- A Quick Aside On WebXR Geometry
- Cameras
- Lights
- Light Sources
- Light Parameters
- Taking Shape: Set And Parametric Shapes
- Putting It All Together: Rendering The Scene
- Next Steps: Supporting And Managing User Input
- Debugging, Extending, And Bundling Babylon.js
- Debugging Babylon.js With The Inspector
- Integrating And Bundling Babylon.js With Other JavaScript
- Immersing Yourself In WebXR
What Is WebXR?
Simply put, WebXR is the grouping of standards responsible for supporting rendered three-dimensional scenes in virtual and augmented reality, both experiential realms known together as mixed reality (XR). Virtual reality (VR), which presents a fully immersive world whose physical elements are entirely drawn by a device, differs considerably from augmented reality (AR), which instead superimposes graphical elements onto real-world surroundings.
WebXR-compatible devices run the gamut from immersive 3D headsets with built-in motion and orientation tracking and names like Vive, Oculus, and Hololens to eyeglasses with graphics placed over real-world images and smartphones that display the world — and additional elements — on their native cameras.
The WebXR Specification And Browser Support
The WebXR Device API is the primary conduit by which developers can interact with immersive headsets, AR eyeglasses, and AR-enabled smartphones. It includes capabilities for developers to discover compatible output devices, render a three-dimensional scene to the device at the correct frame rate, mirror the output to a two-dimensional display (such as a 2D web browser), and create vectors that capture the movements of input controls.
Currently a working draft, the WebXR specification is a combination of the foregoing WebVR API, which was designed solely for virtual reality use cases, and the all-new WebXR Augmented Reality Module, which remains highly experimental. WebVR, formerly the predominant and recommended approach for virtual reality experiences, is now superseded by WebXR, and many frameworks and libraries offer migration strategies between WebVR and the newer WebXR specification.
Though WebXR is now witnessing adoption across the industry, browser support remains spotty, and it’s not yet guaranteed that a mixed reality application built according to the WebXR specification will work off the shelf in production.
Chrome 79, Edge 79, Chrome for Android 79, and Samsung Internet 11.2 all offer full WebXR support. But for unsupported browsers like Firefox, Internet Explorer, Opera, Safari, or certain mobile browsers (Android webview, Firefox for Android, Opera for Android, and Safari on iOS), there’s a WebXR Polyfill available thanks to WebXR community members that implements the WebXR Device API in JavaScript so that developers can write applications according to the latest state of the specification. On Firefox for web and Firefox for Android, you can enable the experimental feature flag by navigating to about:config and setting dom.vr.webxr.enabled to true in your browser’s advanced settings.
Installing the WebXR API Emulator on Chrome or Firefox on a personal computer will introduce additional tools to help you with debugging and testing.
The WebXR Device API depends on WebGL (Web Graphics Library), the rendering engine that supports three-dimensional graphics, and therefore employs many WebGL concepts when it performs the rendering, lighting, and texturing needed for a scene. Though the deepest reaches of WebGL are well beyond the scope of this article, those already familiar with WebGL will benefit from existing expertise.
Several open-source JavaScript frameworks are available to interact with WebGL and WebXR, namely Three.js and Babylon.js. A-Frame, a browser-based, markup-focused approach to WebXR, is built on top of Three.js. In this tutorial, we spotlight Babylon.js, which has gained attention as of late due to its large API surface area and relative stability. But these aren’t like the JavaScript libraries and frameworks we use to build two-dimensional web applications; instead, they play in the sandbox of three-dimensional spaces.
Field Of View (FOV) And Degrees of Freedom (DoF)
In this article, we’ll focus on building a simple immersive experience with limited input and a static object, which means our need for deep knowledge about WebGL is minimal. But there are critical WebXR concepts outside of WebGL that are fundamental not to three-dimensional graphics themselves but how to interact with three-dimensional spaces. Because WebXR is rooted in a viewer’s experience, everything revolves around the immersive headset or viewport the user faces.
All headsets and smartphones have a camera serving as the user’s viewport into an immersive experience. Every camera has a certain field of view (FOV) that encompasses the extent of the viewer’s surroundings that is visible at any given time in a device. A single human eye has an FOV of 135º, while two human eyes, with overlapping FOVs, have a combined FOV of 220º wide. According to MDN, most headsets range between 90º to 150º in their field of view.
The virtual or augmented world seen through the camera’s field of view can be adjusted through movement, which occurs along degrees of freedom whenever a device is shifted in certain ways while the user remains stationary. Rotational movement occurs along three degrees of freedom (3DoF), which is a baseline for most basic immersive headsets:
- Pitch is movement incurred by looking up and down. In pitch, the user’s head pivots on the x-axis, which extends horizontally across the viewport.
- Yaw is movement incurred by looking left and right. In yaw, the user’s head pivots on the y-axis, which extends vertically across the viewport.
- Roll is movement incurred by tilting left and right. In roll, the user’s head pivots on the z-axis, which extends forward into the viewport and into the horizon.
Rotational movement occurs along three degrees of freedom (3DoF): pitch (pivoting on the x-axis), yaw (pivoting on the y-axis), and roll (pivoting on the z-axis). (Large preview)
Though three degrees of freedom are sufficient for simpler immersive experiences, users typically wish to move through space rather than merely changing their perspective on it. For this, we need six degrees of freedom (6DoF), whose latter three degrees define translational movement through space — forward and reverse, left and right, up and down — to pitch, yaw, and roll. In short, 6DoF includes not just pivoting along the x-, y-, and z-axes but also moving along them. Due to the frequent requirement of external sensors to detect translational movement, only high-end headsets support all six degrees.
Rotational and translational movement occur along six degrees of freedom (6DoF). (Large preview)
WebXR Session Modes
With WebXR superseding the foregoing WebVR specification, it now provides one API as a single source of truth for both augmented and virtual reality. Each WebXR application begins by kicking off a session, which represents an immersive experience in progress. For virtual reality, WebXR makes available two session modes: inline, which deposits a rendered scene into a browser document, and immersive-vr, which depends on a headset. For augmented reality, because rendering is only possible into smartphone cameras and transparent glasses or goggles instead of browsers, immersive-ar is the only available mode.
Because many of us don’t have an immersive headset handy at home, and because the WebXR Augmented Reality Module remains in heavy development, we’ll focus our attention on an immersive VR experience that can be rendered into a browser canvas.
Setting A Scene With WebXR And Babylon.js
In this section, we’ll learn how to create and render a WebXR scene with Babylon.js, the surroundings of our environment and the setting of our experience, before turning any attention to actions like user input or motion. Babylon.js is a free and open-source web rendering engine based on WebGL that includes support for WebXR and cross-platform applications in the form of Babylon Native. Babylon.js offers a slew of additional features, including a low-code Node Material Editor for shader creation, and deep integration with WebXR features like session and input management. The Babylon.js website also provides playground and sandbox environments.
Though selecting between Babylon.js and Three.js comes down to developer preference, Three.js focuses on extensibility over comprehensiveness, with a host of interchangeable modules that add supplemental functionality. Babylon.js, meanwhile, provides a more full-featured suite that may prove overkill for smaller projects but does offer the needed surface area for many implementations.
Introducing Babylon.js
Though you can handle all of the interactions with the WebXR Device API yourself, Babylon.js provides an optional Default Experience Helper that can set up and shut down sessions on your behalf. The Default WebXR experience helper also includes input controls and other features, as well as a rudimentary HTML button to enter the immersive experience. To experiment with the default experience helper, let’s write an HTML page that provides a canvas for XR display and serves the Babylon.js source from a CDN. You can find this HTML page at the GitHub repository for this tutorial on the main branch.
Open index.html in your preferred code editor and in a browser. For this first portion of the tutorial, we’ll inspect the file instead of adding code. In a WebXR-enabled browser like Chrome or Firefox (with the WebXR feature flag enabled in the case of Firefox), you’ll see a canvas containing the initial Babylon.js playground — a “Hello World” of sorts — and you can drag your mouse on the screen to reorient yourself. The screenshot below depicts this initial state.
The initial state of the Babylon.js default playground. (Large preview)
First, we’ll embed the latest versions of Babylon.js from the Babylon CDN as well as other helpful dependencies. We’ll also add some styles for our scene canvas element in <body>, which is where our immersive experience will render.
<!-- babylon-webxr/index.html -->
<!DOCTYPE html>
<html>
<head>
<title>Babylon WebXR Demo</title>
<!-- Embed latest version of Babylon.js. -->
<script src="https://cdn.babylonjs.com/babylon.js"></script>
<!-- Embed Babylon loader scripts for .gltf and other filetypes. -->
<script src="https://cdn.babylonjs.com/loaders/babylonjs.loaders.min.js"></script>
<!-- Embed pep.js for consistent cross-browser pointer events. -->
<script src="https://code.jquery.com/pep/0.4.3/pep.js"></script>
<style>
html, body {
overflow: hidden;
width: 100%;
height: 100%;
margin: 0;
padding: 0;
box-sizing: border-box;
}
#render-canvas {
width: 100%;
height: 100%;
touch-action: none;
}
</style>
</head>
<body>
<canvas id="render-canvas"></canvas>
Now it’s time for our Babylon.js implementation. Inside a <script> element just before the terminal </body> tag, we start by identifying our canvas element to Babylon.js during the instantiation of a new Babylon engine with default configuration.
<!-- Our Babylon.js implementation. -->
<script>
// Identify canvas element to script.
const canvas = document.getElementById('render-canvas');
// Initialize Babylon.js variables.
let engine,
scene,
sceneToRender;
const createDefaultEngine = function () {
return new BABYLON.Engine(canvas, true, {
preserveDrawingBuffer: true,
stencil: true
});
};
#javascript #babylon #webxr #programming #developer
