How to Read the RxJS 6 Sources: Map and Pipe

Disclaimer: This series is just my notes as I read through the RxJS sources. I’ll provide a summary of the main points at the end of the article, so don’t feel too bogged down with the details

Welcome back. Today I’m very excited, because I’m finally going to dig into how pipe is implemented in RxJS. This article will start with an overview of how map and pipe work, and then will delve into the RxJS sources.

Previously

In the last article, I looked into the of method for creating an observable. I’ll continue working off of that simple Stackblitz example, except this time, I’ll uncomment map and pipe. You don’t have to be familiar with the previous article to follow this one. Here’s the excerpt from Stackblitz:

map attack!

Here’s a link to the Stackblitz.

Before I dive into the sources, let’s talk about map and pipe. Before trying to read any source, it’s best to have a high-level understanding of how everything works. Otherwise, it’s too easy to get lost in the details.

I know these two things before going in:

• map is an operator that transforms data by applying a function
• pipe composes operators (like map, filter, etc)

Map

Map’s job is to transform things

map is a pretty simple operator. It takes a projection function, and applies it to each value that comes from the source observable.

In this example, the observable returned by of('World’) is the source observable, and the single value 'World' is going to be pipe’d through to map’s projection function, which looks like this:

x => `Hello ${x}!` // projection function
// It's used like this:
of('World').pipe(map(x => `Hello ${x}!`));

The projection function will receive 'World' as its input parameter x, and will create the string Hello World!.
map wraps the project function in an observable, which then emits the string value Hello World!. Remember, operators always return observables.

map wraps the projection function in an observable, and starts emitting string values.

I’ve written about the basics of map and other operators pretty extensively in this article. I’ll cover some of that material again here.

Basically, if you understand how Array.prototype.map works, most of that knowledge will carry over to observables.

We’ll see more on map later in this article. Let’s look at pipe next.

Pipe

pipe is the star of this article. Unlike map, which is an operator, pipe is a method on Observable which is used for composing operators. pipe was introduced to RxJS in v5.5 to take code that looked like this:

of(1,2,3).map(x => x + 1).filter(x => x > 2);

and turn it into this

of(1,2,3).pipe(
  map(x => x + 1),
  filter(x => x > 2)
);

Same output, same concept (composing operators), different syntax.
pipe offers the following benefits:

• It cleans up Observable.prototype by removing operators
• It makes the RxJS library more tree-shakeable
• It makes it easier to write and use third-party operators (since you don’t have to worry about patching Observable.prototype).

Nicholas Jamieson provides a great explanation of the benefits of using pipe for composition in this article.

Quick detour (skip this section if you are comfortable with pipe)

If you’re unfamiliar with using pipe for composition, it’s worthwhile to see how it works on regular functions before seeing how it works with operators. Let’s look at a simplified version of pipe which acts on normal functions:

const pipe = (...fns) => 
           initialVal => 
           fns.reduce((g,f) => f(g), initialVal);

In this example, pipe is a function which accepts functions as arguments. Those arguments are collected into an array called fns through use of ES6 rest parameters (…fns). pipe then returns a function which accepts an initialValue to be passed into reduce in the following step. This is the value which is passed into the first function in fns, the output of which is then fed into the second function in fns, which is then fed into the third…and so on. Hence, a pipeline.
For example:

pipe.ts

const pipe = (...fns) => initialVal => fns.reduce((g,f) => f(g), initialVal);
const add1 = x => x + 1;
const mul2 = x => x * 2;

const res = pipe(add1,mul2)(0); // mul2(add1(0)) === 2

You can experiment with a simple pipe at this stackblitz link.

In RxJS, the idea is that you create a pipeline of operators (such as map and filter) that you want to apply to each value emitted by a source observable, of(1,2,3) in this example.

This approach lets you create small, reusable operators like map and filter, and compose them together when needed using pipe.

Composition is a pretty fascinating topic, although I can hardly do it justice.
I recommend Eric Elliott’s series on the topic if you want to learn more.

Enough talk! Get to the Sources!

I’ll start by adding a debugger statement into map. This will give me access to map within the dev tools debugger, as well as a way to step up into pipe.

and, in the dev tools:

Now that I’m oriented in the call stack, and I can start to dig around.

Notice that in the call stack, it’s Observable.subscribe that’s kicking everything off. Because observables tend to be lazy, no data will flow through the pipe and map until we subscribe to the observable.

var sub = source.subscribe(...)

Looking inside of map, I notice that MapOperator and MapSubscriber look interesting:

On line 55, source is the observable produced by of('World'). It is subscribed to on line 56, causing it to emit its one value, 'World', and then complete.

On line 56, an instance of MapSubscriber is created, and passed into source.subscribe. We’ll see later that the projection function is invoked inside of MapSubscriber’s _next method.

On line 56, this.project is the projection function passed into map:

and this.thisArg can be ignored for now. So line 56 is doing the following:

return source.subscribe(new MapSubscriber(subscriber, this.project, this.thisArg));

1. calling subscribe on source, which is the observable returned by of('World').
2. The observer ( next, error, complete, etc) which is passed into source.subscribe is going to be the Subscriber returned by MapSubscriber, which takes the current subscriber, and the project function passed into map as its arguments.

As a quick aside, this is a very common pattern for operators in RxJS. In fact, they all seem to follow the following template:

• export a public function, like map or filter or expand.
• export a class which implements Operator, such as MapOperator. This class implements Operator call method. It subscribes to the source observable, like
  return source.subscribe(new MapSubscriber(…));
  This links the observables into a subscriber/observer pipeline.
• A class which extends Subscriber. This class will implement methods such as _next.
  This is where the logic that makes each operator unique lives. For example, in map, the projection function will be invoked inside of MapSubscriber’s _next method. In filter the predicate function will be invoked inside of FilterSubscriber’s _next method, and so on.

I’ll provide an example of how to write your own operator in a future article (although it’s usually easier to just pipe together existing operators). In the meantime, the RxJS sources provide a nice guide here, and Nicholas Jamieson has a great example in this article.

Anyways, back to the debugging session.

Eventually, once subscribe is called, MapSubscriber._next will be invoked.

Notice that the projection function, project, which was passed into map is invoked on line 81, and the results (in this case 'Hello World!' ) will be returned, and then passed into this.destination.next(result) on line 86.

stepping into this.project.call puts us in the lambda we passed into the call to map

This explains how map applies the projection function to each value emitted by the source observable when it is subscribed to. That’s really all there to this step. If there were another operator in the pipeline, the observable returned by map would be fed into it.

This is a good example of how data flows through a single operator. But how does it flow through multiple operators…

Pipe (again)

To answer that, I must dig into pipe. It’s being invoked on the observable which is returned from of('World').

pipeFromArray is called on line 331 with operations, which is an array of all operators passed into pipe. In this case, it’s just the lonely map operator:

operations could hold many, many operators

The function returned from the call to pipeFromArray(operations) is invoked with this, which is a reference to the observable returned from of('World').

Since there is only one operator in this case (map), line 29 returns it.

Line 33 is interesting. It’s where all of the operators passed into pipe are composed using Array.prototype.reduce. It’s not invoked in situations where it is passed only one operator (perhaps for performance reasons?).

Let’s look at a slightly more complex example, with multiple map operators.

Multiple maps

Now that I have an understanding of what map and pipe are doing, I’ll try a more complicated example. This time, I’ll use the map operator three times!

Hello World of RxJS

The only real difference is that pipe will use reduce this time:

The input variable is still the observable returned from of('World').

By stepping through each function in fns as it is called by reduce, I can see the string being built up as it passes through each one of the map operators. Eventually producing the string Hello World of RxJS

With an understanding of how data flows through a single operator, it’s not hard to extend that understanding to multiple operators.

A little map and a little filter

Just for fun, I want to throw filter in the mix. The goal here is to confirm that map isn’t unique. I want to see that all operators follow that similar pattern.

Will log values 3 and 4

In this example, of(1,2,3) will return an observable which, upon subscription, will emit three separate values, 1, 2, and 3, and will then complete. Each of these three values will be fed into the pipeline one at a time. map will add one to each, and then re-emit the new values one-by-one on the observable it returns. filter subscribes to the observable returned by map, and runs each value through its predicate function ( x => x > 2 ). It will return an observable which emits any value which is greater than 2. In this case, it will emit values 3 and 4.

If you want to see a more detailed explanation of the subscriber chain and how operators subscribe to one another,

Summary

• We’ve seen that operators like map and filter are functions which take in and return observables.
• Each operator exposes a public function like map or filter, which is what we import from 'rxjs/operators' and pass into pipe.
• Each operator has a *Operator class which implements the Operator interface, so that it can subscribe to other observables.
• Each operator has a *Subscriber class which contains the logic for that operator (invocation of the projection function for map, invocation of the predicate function for filter, etc).
• We’ve also seen how pipe is used to compose operators together. Internally, it’s taking the values emitted by the source observable, and reducing it over the list of operators.

In the next article, I’ll look at some more advanced maps, and see how higher order observables are implemented. 🗺

#javascript #RxJS #Map #Pipe