Node.js - Setting Up & Using Google Cloud Video Intelligence API

Node.js - Setting Up & Using Google Cloud Video Intelligence API

​ Do you need to annotate the content of a video automatically? Let's say you have a service that allows users to upload videos and you want to know content of each videos. It would take a lot of time and efforts if the process is done manually by watching each videos. Fortunately there are some services that can annotate videos and extract metadata. By doing so, it becomes possible to make the videos searchable. ​ ​ ​ ​ If you need that kind of service, you can consider Google Cloud Video Intelligence. It works by labelling a video with multiple labels using their library of 20,000 labels. It has the capability to extract metadata for indexing your video content, so that you can easily organize and search video content. Other features include shot detection to distinguish scene changes and integration with Google Cloud Storage. ​ ​ ​ ## Preparation ​ ​ ​ ​ 1. Create or select a Google Cloud project ​ A Google Cloud project is required to use this service. Open [Google Cloud console](https://console.cloud.google.com/project "Google Cloud console"), then create a new project or select existing project ​ 2. Enable billing for the project ​ Like other cloud platforms, Google requires you to enable billing for your project. If you haven't set up billing, open [billing page](https://console.cloud.google.com/billing "billing page"). ​ 3. Enable Video Intelligence API ​ To use an API, you must enable it first. Open [this page](https://console.cloud.google.com/flows/enableapi?apiid=videointelligence.googleapis.com "this page") to enable Video Intelligence API. ​ 4. Set up service account for authentication ​ As for authentication, you need to create a new service account. Create a new one on the [service account management page](https://console.cloud.google.com/iam-admin/serviceaccounts "service account management page") and download the credentials, or you can use your already created service account. ​ In your .env file, you have to add a new variable because it's needed by the library we are going to use. ​ ``` GOOGLE_APPLICATION_CREDENTIALS=/path/to/the/credentials ​ ``` ​ In addition, you also need to add GOOGLE*CLOUD*PROJECT_ID to your .env as well. Do you need to annotate the content of a video automatically? Let’s say you have a service that allows users to upload videos and you want to know content of each videos. It would take a lot of time and efforts if the process is done manually by watching each videos. Fortunately there are some services that can annotate videos and extract metadata. By doing so, it becomes possible to make the videos searchable.

Do you need to annotate the content of a video automatically? Let's say you have a service that allows users to upload videos and you want to know content of each videos. It would take a lot of time and efforts if the process is done manually by watching each videos. Fortunately there are some services that can annotate videos and extract metadata. By doing so, it becomes possible to make the videos searchable.

If you need that kind of service, you can consider Google Cloud Video Intelligence. It works by labelling a video with multiple labels using their library of 20,000 labels. It has the capability to extract metadata for indexing your video content, so that you can easily organize and search video content. Other features include shot detection to distinguish scene changes and integration with Google Cloud Storage.

Preparation
  1. Create or select a Google Cloud project

A Google Cloud project is required to use this service. Open Google Cloud console, then create a new project or select existing project

  1. Enable billing for the project

Like other cloud platforms, Google requires you to enable billing for your project. If you haven't set up billing, open billing page.

  1. Enable Video Intelligence API

To use an API, you must enable it first. Open this page to enable Video Intelligence API.

  1. Set up service account for authentication

As for authentication, you need to create a new service account. Create a new one on the service account management page and download the credentials, or you can use your already created service account.

In your .env file, you have to add a new variable because it's needed by the library we are going to use.

GOOGLE_APPLICATION_CREDENTIALS=/path/to/the/credentials

In addition, you also need to add GOOGLECLOUDPROJECT_ID to your .env as well.

Dependencies

This tutorial uses @google-cloud/video-intelligence and also dotenv for loading environment. Add the following dependencies to your package.json and run npm install

 "@google-cloud/video-intelligence": "~1.5.0" "dotenv": "~4.0.0"

Code

Below is the code example of how to annotate video with Google Video Intelligence API. The video that will be analyzed needs to be uploaded to Google Cloud Storage first. You can read our tutorial about how to upload file to Google Cloud Storage using Node.js.

// Loads environment variables
require('dotenv').config();
// Imports the Google Cloud Video Intelligence library
const videoIntelligence = require('@google-cloud/video-intelligence');
// Creates a client
const client = new videoIntelligence.VideoIntelligenceServiceClient({
projectId: process.env.GOOGLE_CLOUD_PROJECT_ID,
});
// URI of the video you want to analyze
const gcsUri = 'gs://{YOUR_BUCKET_NAME}/{PATH_TO_FILE}';
// Request config
const request = {
inputUri: gcsUri,
features: ['LABEL_DETECTION'],
};
// Execute request
client
.annotateVideo(request)
.then(results => {
console.log('Waiting for service to analyze the video. This may take a few minutes.');
return results[0].promise();
})
.then(results => {
console.log(JSON.stringify(results, null, 2));
// Gets annotations for video
const annotations = results[0].annotationResults[0];
// Gets labels for video from its annotations
const labels = annotations.segmentLabelAnnotations;
labels.forEach(label => {
console.log(`Label ${label.entity.description} occurs at:`);
label.segments.forEach(segment => {
const _segment = segment.segment;
_segment.startTimeOffset.seconds = _segment.startTimeOffset.seconds || 0;
_segment.startTimeOffset.nanos = _segment.startTimeOffset.nanos || 0;
_segment.endTimeOffset.seconds = _segment.endTimeOffset.seconds || 0;
_segment.endTimeOffset.nanos = _segment.endTimeOffset.nanos || 0;
console.log(
`\tStart: ${_segment.startTimeOffset.seconds}` +
`.${(_segment.startTimeOffset.nanos / 1e6).toFixed(0)}s`
);
console.log(
`\tEnd: ${_segment.endTimeOffset.seconds}.` +
`${(_segment.endTimeOffset.nanos / 1e6).toFixed(0)}s`
);
console.log(`Confidence level: ${segment.confidence}`);
});
});
})
.catch(err => {
console.error(`ERROR: ${err}`);
});

It may take a few minutes to get the annotation results depending on video length. annotateVideo returns a promise of array which first element has promise(). So you need to wait until the process is done by calling results[0].promise(). Meanwhile, you can add a console.log to show that the annotation is in progress.

Below is the result format. It's an array of 3 objects. The first object contains the annotation results - this is what we need to parse in order to understand what is the video about. The second object contains progress percentage, execution start time, and the last time the progress is updated.

[
{
"annotationResults":[
{ }
]
},
{
"annotationProgress":[
{
"inputUri":"/{YOUR_BUCKET_NAME}/{PATH_TO_FILE}",
"progressPercent":100,
"startTime":{
"seconds":"1546439976",
"nanos":559663000
},
"updateTime":{
"seconds":"1546440001",
"nanos":104220000
}
}
]
},
{ }
]

annotationResults is an array whose elements look like this

{
"entity": {
"entityId": "/m/01350r",
"description": "performance art",
"languageCode": "en-US"
},
"categoryEntities": [
{
"entityId": "/m/02jjt",
"description": "entertainment",
"languageCode": "en-US"
}
],
"segments": [
{
"segment": {
"startTimeOffset": {},
"endTimeOffset": {
"seconds": "269",
"nanos": 720000000
}
},
"confidence": 0.666665256023407
}
]
},

Each object in annotationResults represent a label along with video segments that strengthen the reason why the label is given. There is also a value that shows how confidence the service gives a label to a segment.

That's how to use Google Cloud Intelligence in Node.js. If you want to analyze images, you can read the tutorial about how to use Google Cloud Vision in Node.js

Creating a RESTful Web API with Node.js and Express.js from scratch

Creating a RESTful Web API with Node.js and Express.js from scratch

In this article, I’ll show you step by step how to create a RESTful Web API with Node.js and Express.js by building a simple and useful Todo API. This article assumes you have basic javascript knowledge and terminal using capabilities.

In this article, I’ll show you step by step how to create a RESTful Web API with Node.js and Express.js by building a simple and useful Todo API. This article assumes you have basic javascript knowledge and terminal using capabilities.

You can also build a Web API in Node.js by using another framework except Express.js but Express.js is one of the most popular web framework for Node.js.

You can found the final source code of this Web API in this github repository.

Let’s start to create our mentioned Web API.

Before start

If you have never used Node.js or npm package manager you should install them.

To check whether the Node.js is already installed on your computer, open your terminal and run node -v command. If you see your Node.js version it's installed. Otherwise go to below link.

Click here to download and install Node.js (You can choose LTS version)

And if you don’t have any IDE or text editor for writing javascript I advice you Visual Studio Code.

Click here to download VS Code (Optional)

About express-generator

In fact we could use <a href="https://expressjs.com/en/starter/generator.html" target="_blank">express-generator</a> tool which designed to creating an Express Web API quickly but I want to create this API from scratch because of that tool puts some extra files and folder structures that we don't need them now. But you can use this useful tool next time on creating new Web API. I won't use it now due to keep article simple.

Creating Project

Go to your workspace root folder and create a new folder there named "todo-api".

Then create "package.json" and "server.js" files into "todo-api" folder like below.

package.json

{
    "name": "todo-api",
    "version": "1.0.0",
    "scripts": {
        "start": "node server.js"
    },
    "dependencies": {
        "express": "^4.16.4"
    }
}

server.js

const http = require('http');
const express = require('express');
const app = express();
app.use(express.json());
app.use('/', function(req, res) {
    res.send('todo api works');
});
const server = http.createServer(app);
const port = 3000;
server.listen(port);
console.debug('Server listening on port ' + port);

After creating above files open your terminal in the "todo-api" folder and run npm installcommand.

This command will be install your project dependencies which pointed at the "package.json" file.

After finished package download process, downloaded dependency files will be installed into"node_modules" folder at the root of the "todo-api" folder.

After finished package installing then run npm start to start our Web API.

Now our Web API listening. To see result open your web browser then write localhost:3000 to address bar and press enter.

As result you’ll see our request handler response in your browser: “todo api works”.

This is a dead simple Express.js Web API. And it needs the some development. For example we need to an api endpoint to get todo items. So let’s add a new API endpoint for this.

Create a new folder named "routes" in the root of the "todo-api" folder.

Then create a "items.js" file inside of "routes" folder and put following codes inside it.

Your final folder structure should be like below;

/todo-api
/node_modules
/routes
    items.js
package.json
server.js

items.js

const express = require('express');
const router = express.Router();
const data = [
    {id: 1, title: 'Finalize project', order: 1, completed: false, createdOn: new Date()},
    {id: 2, title: 'Book ticket to London', order: 2, completed: false, createdOn: new Date()},
    {id: 3, title: 'Finish last article', order: 3, completed: false, createdOn: new Date()},
    {id: 4, title: 'Get a new t-shirt', order: 4, completed: false, createdOn: new Date()},
    {id: 5, title: 'Create dinner reservation', order: 5, completed: false, createdOn: new Date()},
];
router.get('/', function (req, res) {
    res.status(200).json(data);
});
router.get('/:id', function (req, res) {
    let found = data.find(function (item) {
        return item.id === parseInt(req.params.id);
    });
    if (found) {
        res.status(200).json(found);
    } else {
        res.sendStatus(404);
    }
});
module.exports = router;

Initial code of "items.js" file contains two endpoints. First one gets all todo items and second one gets one item which matches given id parameter.

Before testing items routes we should register it in the "server.js" file.

Modify "server.js" file like below to register new item routes.

server.js

const http = require('http');
const express = require('express');
const itemsRouter = require('./routes/items');
const app = express();
app.use(express.json());
app.use('/items', itemsRouter);
app.use('/', function(req, res) {
    res.send('todo api works');
});
const server = http.createServer(app);
const port = 3000;
server.listen(port);
console.debug('Server listening on port ' + port);

Now run npm start to start our Web API.

Then open your web browser and write localhost:3000/items to address bar and press enter.

You’ll see todo items json array in the response body.

And write localhost:3000/items/3 to address bar and press enter.

You’ll see the todo item which has id 3 in the response body.

But not finished up yet.

CRUD Operations and HTTP methods

I think we’ll need CRUD operations to Create, Read, Update and Delete todo items.

We have already two endpoints for getting items. So we need Create, Update and Delete endpoints.

Let’s add also these endpoints into the items.js file.

Our final "items.js" file and endpoints should be like below.

const express = require('express');
const router = express.Router();

const data = [
  {id: 1, title: 'Finalize project',          order: 1, completed: false, createdOn: new Date()},
  {id: 2, title: 'Book ticket to London',     order: 2, completed: false, createdOn: new Date()},
  {id: 3, title: 'Finish last article',       order: 3, completed: false, createdOn: new Date()},
  {id: 4, title: 'Get a new t-shirt',         order: 4, completed: false, createdOn: new Date()},
  {id: 5, title: 'Create dinner reservation', order: 5, completed: false, createdOn: new Date()},
];

router.get('/', function (req, res) {
  res.status(200).json(data);
});

router.get('/:id', function (req, res) {
  let found = data.find(function (item) {
    return item.id === parseInt(req.params.id);
  });

  if (found) {
    res.status(200).json(found);
  } else {
    res.sendStatus(404);
  }
});

router.post('/', function (req, res) {
  let itemIds = data.map(item => item.id);
  let orderNums = data.map(item => item.order);

  let newId = itemIds.length > 0 ? Math.max.apply(Math, itemIds) + 1 : 1;
  let newOrderNum = orderNums.length > 0 ? Math.max.apply(Math, orderNums) + 1 : 1;

  let newItem = {
    id: newId,
    title: req.body.title,
    order: newOrderNum,
    completed: false,
    createdOn: new Date()
  };

  data.push(newItem);

  res.status(201).json(newItem);
});

router.put('/:id', function (req, res) {
  let found = data.find(function (item) {
    return item.id === parseInt(req.params.id);
  });

  if (found) {
    let updated = {
      id: found.id,
      title: req.body.title,
      order: req.body.order,
      completed: req.body.completed
    };

    let targetIndex = data.indexOf(found);

    data.splice(targetIndex, 1, updated);

    res.sendStatus(204);
  } else {
    res.sendStatus(404);
  }
});

router.delete('/:id', function (req, res) {
  let found = data.find(function (item) {
    return item.id === parseInt(req.params.id);
  });

  if (found) {
    let targetIndex = data.indexOf(found);

    data.splice(targetIndex, 1);
  }

  res.sendStatus(204);
});

module.exports = router;

Short Explanation

I wanna explain shortly some points of our last codes.

First of all you must have noticed that our api works on a static data and keeps it on memory. All of our GET, POST, PUT and DELETE http methods just manipulate a json array. The purpose of this is to keep article simple and draw attention to the Web API structure.

Due to this situation our POST method has some extra logic such as calculating next item ids and order numbers.

So you can modify logic and data structures in these http methods to use a database or whatever you want.

Testing API with Postman

We have tested the GET methods of our Web API in our web browser and seen responses. But we can’t test directly POST, PUT and DELETE http methods in web browser.

If you want to test also other http methods you should use Postman or another http utility.

Now I’ll show you how to test the Web API with Postman

Before we start click here and install Postman.

When you first launch Postman after installing you’ll see start window. Close this start window by clicking close button on top right corner. Then you must see following screen.

An empty Postman request

Sending GET Request

Before sending a request to API we should start it by running npm startcommand as we do before.

After start the Web API and seeing “Server listening on…” message write localhost:3000/itemsto address bar as seen below and click Send button. You'll see todo items array as API response like below.

Sending a GET request with Postman

You can try similarly by giving an item id in request url like this localhost:3000/items/3

Sending POST Request

To sending a POST request and create a new todo item write localhost:3000/items to address bar and change HTTP verb to POST by clicking arrow at front of the address bar as seen below.

Sending a POST request with Postman

Before sending the POST request you should add request data to body of the request by clicking body tab and selecting raw and JSON as seen below.

Attaching a JSON body to POST request in Postman

Now click Send button to send POST request to the Web API. Then you must get “201 Created” http response code and seeing created item in the response body.

To see the last status of todo items send a get request to localhost:3000/itemsaddress. You must see newly created item at the end of the list.

Sending PUT Request

Sending PUT request is very similar to sending POST request.

The most obvious difference is request url should be pointed specific item like this localhost:3000/items/3

And you should choose PUT as http verb instead of POST and send all of the required data in the request body unlike POST.

For example you could send a JSON body in the PUT request as below.

An example JSON body for PUT request

{
    "title": "New title of todo item",
    "order": 3,
    "completed": false
}

When you click Send button you must get “204 No Content” http response code. You can check item you updated by sending a get request.

Sending DELETE Request

To send a DELETE request, change the request url to address a specific item id like this localhost:3000/items/3

And select DELETE as http verb and click Send button.

You must get “204 No Content” http response code as result of the DELETE operation.

Send a get request and see the last status of list.

About the DELETE Http Request

I want to say a few words about DELETE http request. You must have noticed something in our delete code. DELETE request returns “204 No Content” every situation.

Http DELETE requests are idempotent. So what that mean? If you delete a resource on server by sending DELETE request, it’s removed from the collection. And every next DELETE request on the same resource won’t change outcome. So you won’t get “404 Not Found” in the second request. Each request returns same response whether succeed or not. That’s mean idempotent operation.

Conclusion

Finally we’ve tested all http methods of our Web API.

As you can see, it works just fine.

Thanks for reading ❤

If you liked this post, share it with all of your programming buddies!

Learn how to use NestJS, Node.js framework to build a secure API

Learn how to use NestJS, Node.js framework to build a secure API

Learn how to use NestJS, a Node.js framework powered by TypeScript, to build a secure API

In this tutorial, you'll learn how to build a secure API using NestJS, a module-based architecture framework for Node.js powered by TypeScript.

NestJS helps developers create highly scalable, modular, and maintainable server-side web applications. It leverages the Express framework to easily implement the MVC (Model-View-Controller) pattern and to provide you with extensibility, as you can use any of the third-party modules available for Express. However, the most outstanding feature of NestJS is its native support for TypeScript, which lets you access optional static type-checking along with strong tooling for large apps and the latest ECMAScript features.

What You Will Build

In this tutorial, you'll build a feature-complete API that lets clients perform data operations on resources that describe a restaurant menu.

You'll be using a production client called "WHATABYTE Dashboard" to consume, test, and even try to hack the API!

This dashboard is inspired by the sleek web player from Spotify.

API access will be constrained by the following business rules:

  • Anyone can read data: read menu items.

  • Only users with a menu-admin role are authorized to write data: create, update, or delete menu items.

For simplicity, you'll store data in-memory and not in an external database for this phase of the tutorial.

Getting Started with NestJS

NestJS requires Node.js and NPM to run. Check if these are installed by running the following commands in your terminal:

node -v && npm -v

If you need to install any of them, follow the instructions provide by the Node.js Foundation for your operating system. This tutorial was tested using Node.js v10.16.3 and NPM v6.9.0.

NestJS offers a powerful CLI tool to create and build your application. To generate a new project, use npx to run the NestJS CLI without installing it globally in your system:

npx @nestjs/cli new nest-restaurant-api

The npx command is available with npm v5.2.0 and higher.

The CLI will ask you to choose a package manager, npm or yarn, and proceed to install project dependencies using your selection. To follow this tutorial choose npm.

Once the installation is complete you'll get a directory called nest-restaurant-api. Navigate to this directory:

# move into the project directory
cd nest-restaurant-api

Cleaning Up the NestJS Starter Project

For simplicity, you won't be writing any tests in this tutorial. However, you should write solid tests for any production-ready application. As such, delete the test directory and the src/app.controller.spec.ts file from your project:

rm -rf test/
rm src/app.controller.spec.ts

Refer to the NestJS Testing documentation for details on how to perform automated tests.

After that, delete the files defining AppController and AppService:

rm src/app.controller.ts src/app.service.ts

Deleting these files breaks AppModule as it depends on AppController and AppService. To fix that, open your project in your preferred IDE and update src/app.module.ts as follows:

// src/app.module.ts

import { Module } from '@nestjs/common';

@Module({
  imports: [],
  controllers: [],
  providers: [],
})
export class AppModule {}

Using Environmental Variables

src/main.ts is the entry point of your application; however, this file has hard-coded configuration dependencies that make your application less flexible and adaptable to different deployment environments.

Open src/main.ts and notice that the app is configured to listen for incoming requests on a hard-coded port number, 3000:

await app.listen(3000);

To fix this configuration rigidity, you'll use environmental variables to provide your application with configuration values, such process.env.PORT, instead of hard-coded ones.

To start, install dotenv in your project:

npm i dotenv

dotenv is a zero-dependency module that loads environment variables from a .env file into the global variable process.env.

Create this hidden file under the root project directory as follows:

touch .env

Open .env and populate it with the following variable:

PORT=7000

The configuration variables held by .env can be attached to process.env by calling the dotenv.config() method. As such, update src/main.ts to call this method right below the module imports and replace the hard-coded port number with process.env.PORT:

// src/main.ts

import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';
import * as dotenv from 'dotenv';

dotenv.config();

async function bootstrap() {
  const app = await NestFactory.create(AppModule);
  await app.listen(process.env.PORT);
}
bootstrap();

Now anytime you run your application, it will be listening for requests on port 7000 — or whatever the value of process.env.PORT may be. This allows your application to be externally configured by deployment services such as AWS or Heroku.

Caution! Your .env file eventually may contain sensitive information, such as API keys or secrets. As such, it's critical that you add it to your .gitignore file to prevent it from being committed to version control, such as git.

Installing the NestJS CLI Locally

Now that the project is clean and configured, you'll use the NestJS CLI locally as a development dependency to easily generate the architectural elements of your app. Run the following command to install it:

npm install --save-dev @nestjs/cli

It's critical to pass the --save-dev flag to npm to ensure that the package is installed locally for development and testing purposes and it's not included in the production bundle of your app.

Checkpoint

Similar to video games, it's important to save your progress as you work through a software project. While IDEs and text editors may offer the ability to roll back different versions of a project file, it's a good idea to use version control as your "checkpoint".

Create an empty Git repository for your project as follows:

git init

Add your current project files to the repository, ignoring any file listed in .gitignore:

git add .

Finally, commit the file bundle as follows:

git commit -m "Set up foundation of NestJS app"

Learn Node.js - Node.js API Development for Beginners

Learn Node.js API Development from absolute scratch. This video is for complete beginners getting started guide!

In this video you will learn the core fundamentals of Node JS so that you can start building API using Node JS. You will learn Modern JavaScript, Node JS event loop, Asynchronous programming, using node modules, npm modules and creating your own modules, creating server, connect to database and sending json responses.


Learn More

Node.js With Passport Authentication | Full Project

Full Stack Developers: Everything You Need to Know

MEAN Stack Tutorial MongoDB, ExpressJS, AngularJS and NodeJS

How to Perform Web-Scraping using Node.js

Moving from NodeJS to Go

Authenticate a Node ES6 API with JSON Web Tokens

The Complete Node.js Developer Course (3rd Edition)

Angular & NodeJS - The MEAN Stack Guide

NodeJS - The Complete Guide (incl. MVC, REST APIs, GraphQL)

Node.js: The Complete Guide to Build RESTful APIs (2018)