If you a JS developer that was always amazed by things Data Scientist doing with Machine learning and Artifical intelligence. Check out this blog and learn how you can create your own image classifiers using Angular, Javascript and Tensorflow in less than a few minutes!

AI (Artificial intelligence) and ML are definitely the biggest buzzwords of the last few years.

It feels like everyone talks about AI all around. But feelings are not facts, so, I’ve checked google trends to witness the fact that those two terms are kept raising:

Somehow, it always feels for me like this topic reserved for only data scientists, python developers or anyone else or are much smarter than me. So, I’ve checked the related queries at google trends:

No frontend, no javascript, no Angular. Mostly Python.

I have to admit, I always was amazed by the thing people doing with AI and as a front end developer, I always thought to myself why can’t we do AI in JS?

What is AI?

The theory and development of computer systems able to perform tasks normally requiring human intelligence, such as visual perception, speech recognition, decision-making and translation between Languages

And in other words: everything that computers do, but, needs the intelligence of a human.

Examples?

Siri, Alexa, Tesla, even Netflix use AI-based concepts to recommend your next tv show.

How does it work?

Believe it or not, it’s based in biology. Our mind is full of Neurons, which getting the inputs from our senses and generating the output through the axon. The important fact is — that, this is what creates the intelligence part of our brain.

Now, let’s try to take this thing, and convert it into a mathematical model.
How can we implement a Neuron using code in the most simple way?
It’s probably a node which getting some inputs and generating them into one output.

In most of the cases, one Neuron wouldn’t be enough for creating the prediction model we want. So we actually converting it into a neuron network which looks like this:

Any input value getting weight on the nodes, and by running some activation function we getting the output on the second edge.
But, let’s say we know that the output we got is not the expected one, o we need to recalculate the weights on the nodes so the model will fit our example. We actually pump up the numbers again for the same example so we will get the excepted result on our next iteration. This process called the training, and that’s how our model is getting improving every-time we adding more data into it.

One of the most efficient ways to create those neuron networks is a technology called TensorFlow.

And almost over a year ago Google announces Tensorflow.js!
At first, I thought Tensorflow.js is sort of a binding to the Tensorflow which is written in C, you have to download and install it on your machine. But, the fact is TensorFlow.js is a complete re-write of Tensorflow, in…JS!

Is JS strong enough to calculate a complex model?

If you asking yourself how the browser can do such mathematical process in a reasonable time? Thanks to a technology called WebGL we can run our JS code using our graphics card memory, which is much stronger and faster than our RAM memory.

we actually can do machine learning in the browser, without any dependency!

Image classifiers

One of the most exciting concepts related to AI is image classifiers.

I was always amazed by the ability of a computer system to understand and classify images as humans do.

Let’s take facebook example — “Someone added a photo you might be in” feature was mind-blowing for me, how the heck they know?!

We just said we can do the same in the browser? Let’s do it!

We’re going to use a pre-trained model, called “mobilenet”. The reason it’s called mobile net is that this model is designed for mobile devices, it loads really fast compared to other prediction models.

We’re gonna create 2 examples:

• Upload an image, and let the browser classify wheat’s in it.
• Wire up our webcam, and let the browser classify what’s we see in the webcam.

Image upload classifier

Step 1: Install TensorFlow & Mobilenet

npm i @tensorflow/tfjs
npm I @tensorflow-models/mobilenet

Step 2: Import mobile net in your component

import * as mobilenet from '@tensorflow-models/mobilenet';

Step 3: Load the mobilenet model

Let’s load the model OnInit and also add loading indication:

@Component({
  selector: 'app-image-classfier-upload',
  templateUrl: './image-classfier-upload.component.html',
  styleUrls: ['./image-classfier-upload.component.scss']
})
export class ImageClassfierUploadComponent implements OnInit {

  model: any;
  loading: boolean;

  constructor() { }

  async ngOnInit() {
    this.loading = true;
    this.model = await mobilenet.load();
    this.loading = false;
  }


}

Step 4: Prepare the HTML

Now — let’s add our input file section, along with the loading indication to the template:

<div class="cont d-flex justify-content-center align-items-center flex-column">

  <div class="custom-file">
      <input type="file" class="custom-file-input" (change)="fileChange($event)">
      <label class="custom-file-label">Select File</label>
  </div>

  <div *ngIf="loading">
      <img src="./assets/loading.gif">
  </div>


</div>

Step 5: Implement the fileChange() function

I’m adding the imgSrc class property which I’ll get from the FileReader, to show the image uploaded preview.

async fileChange(event) {
    const file = event.target.files[0];
    if (file) {
      const reader = new FileReader();

      reader.readAsDataURL(file);

      reader.onload = (res: any) => {
        this.imgSrc = res.target.result;

      };
    }
  }

Step 6: Classify the uploaded image

I’ve got the imgSrc, now I can run the model.classify() method and get the model predictions:

     setTimeout(async () => {
          this.predictions = await this.model.classify(this.img.nativeElement);
        });Ï

Now, let’s also update the template to show the predictions:

<div class="list-group">
      <div class="list-group-item" *ngFor="let item of predictions">
          {{item.className}} - {{item.probability | percent}}
      </div>
  </div>

And the result:

Everything works! and we’ve just created our first image classifier with only JavaScript!

Webcam classifier

Mobilenet classification can perform not only on static images, we actually can classify and predict live video stream!

Let’s add an HTML5video tag to our template:

<video autoplay muted width="300px" height="300px" #video></video>

Now, In order to wire our webcam, we have to set the stream. Let’s first get a hold of the video element with @ViewChild:

@ViewChild('video') video: ElementRef;

And implement the AfterViewInit life cycle:

async ngAfterViewInit() {
    const vid = this.video.nativeElement;

    if (navigator.mediaDevices.getUserMedia) {
      navigator.mediaDevices.getUserMedia({ video: true })
        .then((stream) => {
          vid.srcObject = stream;

        })
        .catch((err0r) => {
          console.log('Something went wrong!');
        });
    }
  }

Few things to notice here:

1. Since we’re changing DOM elements we’ve better use the AfterViewInit life cycle.
2. We’re checking if the user got the navigator.mediaDevices.getUserMedia to make sure we support all browsers.
3. We got a promise out of the getUserMedia function which contains the stream, and we set the video nativeElement.srcObject to this stream, and with that, we can see ourselves:

The last thing - Let’s run the classify() method every 3 seconds on the webcam stream:

setInterval(async () => {
      this.predictions = await this.model.classify(this.video.nativeElement);
   }, 3000);

And the result:

I’ve checked a remote control, a wallet, and a cell phone. You can see that the classifier actually succeed to classify those items!

It’s not 100% accurate, but still very impressive!

Let’s summarise what we just learned:

• You don’t have to be a data scientist to do AI any more!
• TensorFlow.js is an independent package, you can run it in the browser with a matter of a 1 simple import
• Is the future of FE developers is taking part in building AI-based prediction models? I’ll put my money on that :)

If you enjoyed this article, please share it with others who may enjoy it as well.!

Thanks for reading!

This post was originally published here

#angular #javascript #tensorflow #machine-learning #artificial-intelligence