How to Image Classification with TensorFlow 2.0?

Easy Image Classification with TensorFlow 2.0

Easy Image Classification with TensorFlow 2.0

Easy Image Classification with TensorFlow 2.0 ... Eager execution is enabled by default, without sacrificing the performance optimizations of graph-based execution. APIs are ... Tighter Keras integration as the high-level API.

Image Classification is one of the fundamental supervised tasks in the world of machine learning. TensorFlow’s new 2.0 version provides a totally new development ecosystem with Eager Execution enabled by default. By me, I assume most TF developers had a little hard time with TF 2.0 as we were habituated to use tf.Session and tf.placeholder that we can’t imagine TensorFlow without.

Today, we start with simple image classification without using TF Keras, so that we can take a look at the new API changes in TensorFlow 2.0

You can take a look at the Colab notebook for this story.

Trending AI Articles:

1. Machine Learning In Node.js With TensorFlow.js

2.Linear Regression using TensorFlow 2.0

3.Deep Learning Models with Tensorflow 2.0

Let’s import the data. For simplicity, use TensorFlow Datasets.

Data pipelines could be frustating ( Sometimes! ).

We need to play around with the low-level TF APIs rather than input pipelines. So, we import a well-designed dataset from TensorFlow Datasets directly. We will use the Horses Or Humans dataset.

img_classify_tf2.py

import tensorflow_datasets as tfds

dataset_name = 'horses_or_humans'

dataset = tfds.load( name=dataset_name , split=tfds.Split.TRAIN )
dataset = dataset.shuffle( 1024 ).batch( batch_size )

We can get a number of datasets readily available with TF Datasets.

Defining the model and related ops.

Remember what we needed for a CNN in Keras. Conv2D, MaxPooling2D, Flatten and Dense layers, right? We need to create these layers using the tf.nn module.

img_classify_tf2_1.py

leaky_relu_alpha = 0.2
dropout_rate = 0.5

def conv2d( inputs , filters , stride_size ):
    out = tf.nn.conv2d( inputs , filters , strides=[ 1 , stride_size , stride_size , 1 ] , padding=padding ) 
    return tf.nn.leaky_relu( out , alpha=leaky_relu_alpha ) 

def maxpool( inputs , pool_size , stride_size ):
    return tf.nn.max_pool2d( inputs , ksize=[ 1 , pool_size , pool_size , 1 ] , padding='VALID' , strides=[ 1 , stride_size , stride_size , 1 ] )

def dense( inputs , weights ):
    x = tf.nn.leaky_relu( tf.matmul( inputs , weights ) , alpha=leaky_relu_alpha )
    return tf.nn.dropout( x , rate=dropout_rate )

Also, we would require some weights. The shapes for our kernels ( filters ) need to be calculated.

img_classify_tf2_2.py

initializer = tf.initializers.glorot_uniform()
def get_weight( shape , name ):
    return tf.Variable( initializer( shape ) , name=name , trainable=True , dtype=tf.float32 )

shapes = [
    [ 3 , 3 , 3 , 16 ] , 
    [ 3 , 3 , 16 , 16 ] , 
    [ 3 , 3 , 16 , 32 ] , 
    [ 3 , 3 , 32 , 32 ] ,
    [ 3 , 3 , 32 , 64 ] , 
    [ 3 , 3 , 64 , 64 ] ,
    [ 3 , 3 , 64 , 128 ] , 
    [ 3 , 3 , 128 , 128 ] ,
    [ 3 , 3 , 128 , 256 ] , 
    [ 3 , 3 , 256 , 256 ] ,
    [ 3 , 3 , 256 , 512 ] , 
    [ 3 , 3 , 512 , 512 ] ,
    [ 8192 , 3600 ] , 
    [ 3600 , 2400 ] ,
    [ 2400 , 1600 ] , 
    [ 1600 , 800 ] ,
    [ 800 , 64 ] ,
    [ 64 , output_classes ] ,
]

weights = []
for i in range( len( shapes ) ):
    weights.append( get_weight( shapes[ i ] , 'weight{}'.format( i ) ) )

Note the trainable=True argument becomes necessary with tf.Variable. If not mentioned then we may receive an error regarding the differentiation of variables. In simpler words, a trainable variable is differentiable too.

Each weight is a tf.Variable with the trainable=True parameter which is important. Also, in TF 2.0, we get the tf.initializers module which makes it easier to initialize weights for neural networks. We need to encapsulate our weights in a weights array. This weights array will be used with the tf.optimizer.Adam for optimization.

Now, we assemble all the ops together to have a Keras-like model.

img_classify_tf2_3.py

def model( x ) :
    x = tf.cast( x , dtype=tf.float32 )
    c1 = conv2d( x , weights[ 0 ] , stride_size=1 ) 
    c1 = conv2d( c1 , weights[ 1 ] , stride_size=1 ) 
    p1 = maxpool( c1 , pool_size=2 , stride_size=2 )
    
    c2 = conv2d( p1 , weights[ 2 ] , stride_size=1 )
    c2 = conv2d( c2 , weights[ 3 ] , stride_size=1 ) 
    p2 = maxpool( c2 , pool_size=2 , stride_size=2 )
    
    c3 = conv2d( p2 , weights[ 4 ] , stride_size=1 ) 
    c3 = conv2d( c3 , weights[ 5 ] , stride_size=1 ) 
    p3 = maxpool( c3 , pool_size=2 , stride_size=2 )
    
    c4 = conv2d( p3 , weights[ 6 ] , stride_size=1 )
    c4 = conv2d( c4 , weights[ 7 ] , stride_size=1 )
    p4 = maxpool( c4 , pool_size=2 , stride_size=2 )

    c5 = conv2d( p4 , weights[ 8 ] , stride_size=1 )
    c5 = conv2d( c5 , weights[ 9 ] , stride_size=1 )
    p5 = maxpool( c5 , pool_size=2 , stride_size=2 )

    c6 = conv2d( p5 , weights[ 10 ] , stride_size=1 )
    c6 = conv2d( c6 , weights[ 11 ] , stride_size=1 )
    p6 = maxpool( c6 , pool_size=2 , stride_size=2 )

    flatten = tf.reshape( p6 , shape=( tf.shape( p6 )[0] , -1 ))

    d1 = dense( flatten , weights[ 12 ] )
    d2 = dense( d1 , weights[ 13 ] )
    d3 = dense( d2 , weights[ 14 ] )
    d4 = dense( d3 , weights[ 15 ] )
    d5 = dense( d4 , weights[ 16 ] )
    logits = tf.matmul( d5 , weights[ 17 ] )

    return tf.nn.softmax( logits )

Q. Why are declaring the model as a function? Later on, we will pass a batch of data to this function and get the outputs. We do not use Session as Eager execution is enabled by default. See this guide.

The loss function is easy.

def loss( pred , target ):
    return tf.losses.categorical_crossentropy( target , pred )

Next, comes the most confusing part for a beginner ( for me too! ). We will use tf.GradientTape for optimizing the model.

img_classify_tf2_4.py

optimizer = tf.optimizers.Adam( learning_rate )

def train_step( model, inputs , outputs ):
    with tf.GradientTape() as tape:
        current_loss = loss( model( inputs ), outputs)
    grads = tape.gradient( current_loss , weights )
    optimizer.apply_gradients( zip( grads , weights ) )
    print( tf.reduce_mean( current_loss ) )
    
 num_epochs = 256

for e in range( num_epochs ):
    for features in dataset:
        image , label = features[ 'image' ] , features[ 'label' ]
        train_step( model , image , tf.one_hot( label , depth=3 ) )

What’s happening here?

  1. We declare tf.GradientTape and within its scope, we call the model() and loss() methods in it. Hence, all the functions in these methods will be differentiated during backpropagation.
  2. We obtain the gradients using tape.gradient method.
  3. We optimize all the ops using the optimizer.apply_gradients method ( Earlier we used optimizer.minimize which is still available )

Read more about it from here.

Machine Learning Tutorial - Image Processing using Python, OpenCV, Keras and TensorFlow - YouTube

Using Deep Learning to identify objects and fruits 🍎🍌

Using Deep Learning to identify objects and fruits 🍎🍌

Repo containing the code for this workshop: https://github.com/rmotr/ml-workshop-image-recognition

In this FREE workshop we introduced image processing using Python (with OpenCV and Pillow) and its applications to Machine Learning using Keras, Scikit Learn and TensorFlow.

Thanks for reading ❤

If you liked this post, share it with all of your programming buddies!

Follow me on Facebook | *Twitter

**Learn More

Machine Learning A-Z™: Hands-On Python & R In Data Science

Python for Data Science and Machine Learning Bootcamp

Machine Learning, Data Science and Deep Learning with Python

Complete Guide to TensorFlow for Deep Learning with Python

Tensorflow Bootcamp For Data Science in Python

A Complete Machine Learning Project Walk-Through in Python

Top 10 Algorithms for Machine Learning Newbies

Machine Learning: how to go from Zero to Hero

Python Tutorial: Image processing with Python (Using OpenCV)

Computer Vision Using OpenCV

OpenCV Python Tutorial - Computer Vision With OpenCV In Python

How to Create your own image classifier with Angular and Tensorflow

How to Create your own image classifier with Angular and Tensorflow

Learn how you can create your own image classifiers using Angular, Javascript and Tensorflow in less than a few minutes!

If you a JS developer that was always amazed by things Data Scientist doing with Machine learning and Artifical intelligence. Check out this blog and learn how you can create your own image classifiers using Angular, Javascript and Tensorflow in less than a few minutes!

AI (Artificial intelligence) and ML are definitely the biggest buzzwords of the last few years.

It feels like everyone talks about AI all around. But feelings are not facts, so, I’ve checked google trends to witness the fact that those two terms are kept raising:

Somehow, it always feels for me like this topic reserved for only data scientists, python developers or anyone else or are much smarter than me. So, I’ve checked the related queries at google trends:

No frontend, no javascript, no Angular. Mostly Python.

I have to admit, I always was amazed by the thing people doing with AI and as a front end developer, I always thought to myself why can’t we do AI in JS?

What is AI?

The theory and development of computer systems able to perform tasks normally requiring human intelligence, such as visual perception, speech recognition, decision-making and translation between Languages

And in other words: everything that computers do, but, needs the intelligence of a human.

Examples?

Siri, Alexa, Tesla, even Netflix use AI-based concepts to recommend your next tv show.

How does it work?

Believe it or not, it’s based in biology. Our mind is full of Neurons, which getting the inputs from our senses and generating the output through the axon. The important fact is — that, this is what creates the intelligence part of our brain.

Now, let’s try to take this thing, and convert it into a mathematical model.
How can we implement a Neuron using code in the most simple way?
It’s probably a node which getting some inputs and generating them into one output.

In most of the cases, one Neuron wouldn’t be enough for creating the prediction model we want. So we actually converting it into a neuron network which looks like this:

Any input value getting weight on the nodes, and by running some activation function we getting the output on the second edge.
But, let’s say we know that the output we got is not the expected one, o we need to recalculate the weights on the nodes so the model will fit our example. We actually pump up the numbers again for the same example so we will get the excepted result on our next iteration. This process called the training, and that’s how our model is getting improving every-time we adding more data into it.

One of the most efficient ways to create those neuron networks is a technology called TensorFlow.

And almost over a year ago Google announces Tensorflow.js!
At first, I thought Tensorflow.js is sort of a binding to the Tensorflow which is written in C, you have to download and install it on your machine. But, the fact is TensorFlow.js is a complete re-write of Tensorflow, in…JS!

Is JS strong enough to calculate a complex model?

If you asking yourself how the browser can do such mathematical process in a reasonable time? Thanks to a technology called WebGL we can run our JS code using our graphics card memory, which is much stronger and faster than our RAM memory.

we actually can do machine learning in the browser, without any dependency!

Image classifiers

One of the most exciting concepts related to AI is image classifiers.

I was always amazed by the ability of a computer system to understand and classify images as humans do.

Let’s take facebook example — “Someone added a photo you might be in” feature was mind-blowing for me, how the heck they know?!

We just said we can do the same in the browser? Let’s do it!

We’re going to use a pre-trained model, called “mobilenet”. The reason it’s called mobile net is that this model is designed for mobile devices, it loads really fast compared to other prediction models.

We’re gonna create 2 examples:

  • Upload an image, and let the browser classify wheat’s in it.
  • Wire up our webcam, and let the browser classify what’s we see in the webcam.
Image upload classifier Step 1: Install TensorFlow & Mobilenet
npm i @tensorflow/tfjs
npm I @tensorflow-models/mobilenet
Step 2: Import mobile net in your component
import * as mobilenet from '@tensorflow-models/mobilenet';
Step 3: Load the mobilenet model

Let’s load the model OnInit and also add loading indication:

@Component({
  selector: 'app-image-classfier-upload',
  templateUrl: './image-classfier-upload.component.html',
  styleUrls: ['./image-classfier-upload.component.scss']
})
export class ImageClassfierUploadComponent implements OnInit {

  model: any;
  loading: boolean;

  constructor() { }

  async ngOnInit() {
    this.loading = true;
    this.model = await mobilenet.load();
    this.loading = false;
  }


}
Step 4: Prepare the HTML

Now — let’s add our input file section, along with the loading indication to the template:

<div class="cont d-flex justify-content-center align-items-center flex-column">

  <div class="custom-file">
      <input type="file" class="custom-file-input" (change)="fileChange($event)">
      <label class="custom-file-label">Select File</label>
  </div>

  <div *ngIf="loading">
      <img src="./assets/loading.gif">
  </div>


</div>
Step 5: Implement the fileChange() function

I’m adding the imgSrc class property which I’ll get from the FileReader, to show the image uploaded preview.

async fileChange(event) {
    const file = event.target.files[0];
    if (file) {
      const reader = new FileReader();

      reader.readAsDataURL(file);

      reader.onload = (res: any) => {
        this.imgSrc = res.target.result;

      };
    }
  }
Step 6: Classify the uploaded image

I’ve got the imgSrc, now I can run the model.classify() method and get the model predictions:

     setTimeout(async () => {
          this.predictions = await this.model.classify(this.img.nativeElement);
        });Ï

Now, let’s also update the template to show the predictions:

<div class="list-group">
      <div class="list-group-item" *ngFor="let item of predictions">
          {{item.className}} - {{item.probability | percent}}
      </div>
  </div>

And the result:

Everything works! and we’ve just created our first image classifier with only JavaScript!

Webcam classifier

Mobilenet classification can perform not only on static images, we actually can classify and predict live video stream!

Let’s add an HTML5video tag to our template:

<video autoplay muted width="300px" height="300px" #video></video>

Now, In order to wire our webcam, we have to set the stream. Let’s first get a hold of the video element with @ViewChild:

@ViewChild('video') video: ElementRef;

And implement the AfterViewInit life cycle:

async ngAfterViewInit() {
    const vid = this.video.nativeElement;

    if (navigator.mediaDevices.getUserMedia) {
      navigator.mediaDevices.getUserMedia({ video: true })
        .then((stream) => {
          vid.srcObject = stream;

        })
        .catch((err0r) => {
          console.log('Something went wrong!');
        });
    }
  }

Few things to notice here:

  1. Since we’re changing DOM elements we’ve better use the AfterViewInit life cycle.
  2. We’re checking if the user got the navigator.mediaDevices.getUserMedia to make sure we support all browsers.
  3. We got a promise out of the getUserMedia function which contains the stream, and we set the video nativeElement.srcObject to this stream, and with that, we can see ourselves:

The last thing - Let’s run the classify() method every 3 seconds on the webcam stream:

setInterval(async () => {
      this.predictions = await this.model.classify(this.video.nativeElement);
   }, 3000);

And the result:

I’ve checked a remote control, a wallet, and a cell phone. You can see that the classifier actually succeed to classify those items!

It’s not 100% accurate, but still very impressive!

Let’s summarise what we just learned:
  • You don’t have to be a data scientist to do AI any more!
  • TensorFlow.js is an independent package, you can run it in the browser with a matter of a 1 simple import
  • Is the future of FE developers is taking part in building AI-based prediction models? I’ll put my money on that :)

If you enjoyed this article, please share it with others who may enjoy it as well.!

Thanks for reading!

This post was originally published here