Queues elegantly solve common application scaling and performance challenges. Bull is a Node library that implements a robust queue system based on Redis.

When handling requests from API clients, you might run into a situation where a request initiates a CPU-intensive operation that could potentially block other requests. Instead of processing such tasks immediately and blocking other requests, you can defer it to be processed in the future by adding information about the task in a processor called a queue. A task consumer will then pick up the task from the queue and process it.

Queues are helpful for solving common application scaling and performance challenges in an elegant way. According to the NestJS documentation, examples of problems that queues can help solve include:

• Smoothing out processing peaks
• Breaking up monolithic tasks that may otherwise block the Node.js event loop
• Providing a reliable communication channel across various services

Bull is a Node library that implements a fast and robust queue system based on Redis. Although it is possible to implement queues directly using Redis commands, Bull is an abstraction/wrapper on top of Redis. It provides an API that takes care of all the low-level details and enriches Redis’ basic functionality so that more complex use cases can be handled easily.

Installation

Before we begin using Bull, we need to have Redis installed. Follow the guide on Redis Labs guide to install Redis, then install Bull using npm or yarn.

npm install bull --save

Or:

yarn add bull

Creating a queue

Create a queue by instantiating a new instance of Bull.

Syntax

Queue(queueName: string, url?: string, opts?: QueueOptions): Queue

The optional url parameter is used to specify the Redis connection string. If no url is specified, bull will try to connect to default Redis server running on localhost:6379

QueueOptions interface

interface QueueOptions {
  limiter?: RateLimiter;
  redis?: RedisOpts;
  prefix?: string = 'bull'; // prefix for all queue keys.
  defaultJobOptions?: JobOpts;
  settings?: AdvancedSettings;
}

RateLimiter

limiter:RateLimiter is an optional field in QueueOptions used to configure maximum number and duration of jobs that can be processed at a time. See RateLimiter for more information.

RedisOption

redis: RedisOpts is also an optional field in QueueOptions. It’s an alternative to Redis url string. See RedisOpts for more information.

AdvancedSettings

settings: AdvancedSettings is an advanced queue configuration settings. It is optional, and Bull warns that shouldn’t override the default advanced settings unless you have a good understanding of the internals of the queue. See AdvancedSettings for more information.

A basic queue would look like this:

const Queue = require(bull);

const videoQueue - new Queue('video');

Creating a queue with QueueOptions

// limit the queue to a maximum of 100 jobs per 10 seconds
const Queue = require(bull);

const videoQueue - new Queue('video', {
  limiter: {
  max: 100,
  duration: 10000
  }
});

Each queue instance can perform three different roles: job producer, job consumer, and/or events listener. Each queue can have one or many producers, consumers, and listeners.

#node #bull #javascript #redis #developer