Learn TensorFlow Fundamentals in 20 Minutes

Learn TensorFlow Fundamentals in 20 Minutes

TensorFlow is an open-source software library for dataflow programming across a range of tasks. It is a symbolic math library, and is also used for machine learning applications such as neural networks.

TensorFlow: a modern machine learning library

TensorFlow, open sourced to the public by Google in November 2015, is the result of years of lessons learned from creating and using its predecessor, DistBelief.

It was made to be flexible, efficient, extensible, and portable. Computers of any shape and size can run it, from smartphones all the way up to huge computing clusters. It comes with lightweight software that can instantly productionize your trained model, effectively eliminating the need to reimplement models.

TensorFlow embraces the innovation and community engagement of open source, but has the support, guidance, and stability of a large

corporation.

Because of its multitude of strengths, TensorFlow is appropriate for individuals and businesses ranging from startups to companies as large as, well, Google. Since its open source release in November 2015, TensorFlow has become one of the most exciting machine learning libraries available. It is being used more and more in research, production, and education.

The library has seen continual improvements, additions, and optimizations, and the TensorFlow community has grown dramatically.TensorFlow: What’s in a Name?

Tensors are the standard way of representing data in deep learning. Simply put, tensors are just multidimensional arrays, an extension of two-dimensional tables (matrices) to data with higher dimensionality.

A tensor, put simply, is an n-dimensional matrix.

In general, you can think about tensors the same way you would matrices, if you are more comfortable with matrix math!

In this article, we will discuss and briefly go through TensorFlows basics: Variables, Sessions, Placeholders, etc Also, we will see how to install the TenorFlow library in your System.

Installing TensorFlow

If you are using a clean Python installation (probably set up for the purpose of learning TensorFlow), you can get started with the simple pip installation:

pip install tensorflow

This approach does, however, have the drawback that TensorFlow will override existing packages and install specific versions to satisfy dependencies.

If you are using this Python installation for other purposes as well, this will not do. One common way around this is to install TensorFlow in a virtual environment, managed by a utility called virtualenv.

Depending on your setup, you may or may not need to install virtualenv on your machine. To install virtualenv, type:

pip install virtualenv

See http://virtualenv.pypa.io for further instructions.

In order to install TensorFlow in a virtual environment, you must first create the virtual environment — in this book we choose to place these in the ~/envs folder, but feel free to put them anywhere you prefer:

cd ~
mkdir envs
virtualenv ~/envs/tensorflow

This will create a virtual environment named tensorflow in ~/envs (which will manifest as the folder ~/envs/tensorflow). To activate the environment, use:

source ~/envs/tensorflow/bin/activate

The prompt should now change to indicate the activated environment:

(tensorflow)

At this point the pip install command:

(tensorflow) pip install tensorflow

will install TensorFlow into the virtual environment, without impacting other packages installed on your machine. Finally, in order to exit the virtual environment, you type:

(tensorflow) deactivate

at which point you should get back the regular prompt:

Up until recently TensorFlow had been notoriously difficult to use with Windows machines. As of TensorFlow 0.12, however, Windows

integration is here! It is as simple as:

pip install tensorflow

for the CPU version, or:

pip install tensorflow-gpu

for the GPU-enabled version (assuming you already have CUDA 8).

“HelloWorld” in TensorFlow

Now, we are done with installing and setting up our TensorFlow environment. Let’s make a simple TensorFlow program that will combine the words “Hello” and “World” and display the phrase — “HelloWorld”.

While simple and straightforward, this example introduces many of the core elements of TensorFlow and the ways in which it is different from a regular Python program.

First, we run a simple install and version check (if you used the virtualenvinstallation option, make sure to activate it before running TensorFlow code):

/** inst_check.py **/
import tensorflow as tf
print(tf.__version__)

The above prints the version of the tensorflow in the terminal. Run the following command to execute the script:

python inst_check.py

In your termianl,it will display the version of your tensorflow:

python inst_check.py
1.4.0

If correct, the output will be the version of TensorFlow you have installed on your system. Version mismatches are the most probable cause of issues down the line.

We are done with verifying the TensorFlow version. Let’s implement the HelloWorld example. Below is the full code:

/** helloworld.py **/
import tensorflow as tf
h = tf.constant("Hello")
w = tf.constant(" World!")
hw = h + w
with tf.Session() as sess:
   ans = sess.run(hw)
   print(ans)

We assume you are familiar with Python and imports, in which case the first line:

import tensorflow as tf

requires no explanation. Next, we define the constants “Hello” and “ World!”, and combine them:

import tensorflow as tf
h = tf.constant(“Hello”)
w = tf.constant(“ World!”)
hw = h + w

At this point, you might wonder how (if at all) this is different from the simple Python code for doing this:

ph = “Hello”
pw = “ World!”
phw = h + w

The key point here is what the variable hw contains in each case. We can check this using the print

command. In the pure Python case we get this:

>print phw
Hello World!

In the TensorFlow case, however, the output is completely different:

>print hw
Tensor(“add:0”, shape=(), dtype=string)

Probably not what you expected!

hw = h + w

The TensorFlow line of code does not compute the sum of h and w, but rather adds the summation operation to a graph of computations

to be done later.

Next, the Session object acts as an interface to the external TensorFlow computation mechanism, and allows us to run parts of the computation graph we have already defined. The line:

ans = sess.run(hw)

actually computes hw (as the sum of h and w, the way it was defined previously), following which the printing of ans displays the expected “Hello World!” message.

This completes the first TensorFlow example. Run the following command to execute the script:

python helloworld.py

Session

Following the above code, you have noticed the constant use of Session. It is time we get to know what Session is.

A Session object is the part of the TensorFlow API that communicates between Python objects and data on our end, and the actual computational system where memory is allocated for the objects we define, intermediate variables are stored, and finally results are fetched for us.

sess = tf.Session()

The execution itself is then done with the .run() method of the Session object. When called, this method completes one set of computations in our graph in the following manner: it starts at the requested output(s) and then works backward, computing nodes that must be executed according to the set of dependencies. Therefore, the part of the graph that will be computed depends on our output query.

In our example, we requested that node f be computed and got its value, 5, as output:

outs = sess.run(f)

When our computation task is completed, it is good practice to close the session using the sess.close() command, making sure the resources used by our session are freed up. This is an important practice to maintain even though we are not obligated to do so for things to work:

sess.close()

A note on constructors

The tf.<operator> function could be thought of as a constructor, but to be more precise, this is actually not a constructor at all, but rather a factory method that sometimes does quite a bit more than just creating the operator objects.

Introduction to Computation Graphs

TensorFlow allows us to implement machine learning algorithms by creating and computing operations that interact with one another. These interactions form what we call a “computation graph,” with which we can intuitively represent complicated functional architectures.

For those to whom this concept is new, a graph refers to a set of interconnected entities, commonly called nodes or vertices. These nodes are connected to each other via edges. In a dataflow graph, the edges allow data to “flow” from one node to another in a directed manner.

In TensorFlow, each of the graph’s nodes represents an operation, possibly applied to some input, and can generate an output that is passed on to other nodes.

Operations in the graph include all kinds of functions, from simple arithmetic ones such as subtraction and multiplication to more complex ones, as we will see later on. They also include more general operations like the creation of summaries, generating constant values, and more.

Let’s take a look at a bare-bones example.

In the above example, we see the graph for basic addition. The function, represented by a circle, takes in two inputs, represented as arrows pointing into the function. It outputs the result of adding 1 and 4 together: 5, which is shown as an arrow pointing out of the function. The result could then be passed along to another function, or it might simply be returned to the client. We can also look at this graph as a simple equation:

The above illustrates how the two fundamental building blocks of graphs, nodes and edges, are used when constructing a computation graph. Let’s go over their properties:

  • Nodes, typically drawn as circles, ovals, or boxes, represent some sort of
  • computation or action being done on or with data in the graph’s context. In the above example, the operation “add” is the sole node.
  • Edges are the actual values that get passed to and from Operations, and are typically drawn as arrows. In the “add” example, the inputs 1 and 2 are both edges leading into the node, while the output 3 is an edge leading out of the node. Conceptually, we can think of edges as the link between different Operations as they carry information from one node to the next.

Now, here’s a slightly more interesting example:

There’s a bit more going on in this graph! The data is traveling from left to right (as indicated by the direction of the arrows), so let’s break down the graph, starting from the left.

  • 1. At the very beginning, we can see two values flowing into the graph, 9 and 5. They may be coming from a different graph, being read in from a file, or entered directly by the client.
  • 2. Each of these initial values is passed to one of two explicit “input” nodes, labeled a and b in the graphic. The “input” nodes simply pass on values given to them- node a receives the value 9 and outputs that same number to nodes c and d, while node b performs the same action with the value 5.
  • 3. Node c is a multiplication operation. It takes in the values 9 and 5 from nodes a and b, respectively, and outputs its result of 45 to node e. Meanwhile, node d performs addition with the same input values and passes the computed value of 14 along to node e.
  • 4. Finally, node e, the final node in our graph, is another “add” node. It receives the values of 45 and 14, adds them together, and spits out 59 as the final result of our graph.

Here’s how the above graphical representation might look as a series of equations:

If we wanted to solve for e and a=5 and b = 3 , we can just work backwards from e and plug in!

With that, the computation is complete! There are concepts worth pointing out here:

  • The pattern of using “input” nodes is useful, as it allows us to relay a single input value to a huge amount of future nodes. If we didn’t do this, the client (or whoever passed in the initial values) would have to explicitly pass each input value to multiple nodes in our graph. This way, the client only has to worry about passing in the appropriate values once and any repeated use of those inputs is abstracted away. We’ll touch a little more on abstracting graphs shortly.
  • Pop quiz: which node will run first- the multiplication node c, or the addition node d?
  • The answer: you can’t tell. From just this graph, it’s impossible to know which of c and d will execute first. Some might read the graph from left-to-right and top-to bottom and simply assume that node c would run first, but it’s important to note that the graph could have easily been drawn with d on top of c. Others may think of these nodes as running concurrently, but that may not always be the case, due to various implementation details or hardware limitations. In reality, it’s best to think of them as running independently of one another. Because node c doesn’t rely on any information from node d, it doesn’t have to wait for node d to do anything in order to complete its operation. The converse is also true: node d doesn’t need any information from node c.

Building your first TensorFlow graph

We became pretty familiar with the following graph in the last section:

Here’s what it looks like in TensorFlow code:

import tensorflow as tf
a = tf.constant(9, name=”input_a”)
b = tf.constant(5, name=”input_b”)
c = tf.mul(a,b, name=”mul_c”)
d = tf.add(a,b, name=”add_d”)
e = tf.add(c,d, name=”add_e”)

Let’s break this code down line by line. First, you’ll notice this import statement:

import tensorflow as tf

This, unsurprisingly, imports the TensorFlow library and gives it an alias of tf. This is by convention, as it’s much easer to type “tf,” rather than “tensorflow” over and over as we use its various functions!

Next, let’s focus on our first two variable assignments:

a = tf.constant(9, name=”input_a”)
b = tf.constant(5, name=”input_b”)

Here, we’re defining our “input” nodes, a and b. These lines use our first TensorFlow Operation: tf.constant(). In TensorFlow, any computation node in the graph is called an Operation, or Op for short. Ops take in zero or more Tensor objects as input and output zero or more Tensor objects.

To create an Operation, you call its associated Python constructor- in this case, tf.constant() creates a “constant” Op. It takes in a single tensor value, and outputs that same value to nodes that are directly connected to it.

For convenience, the function automatically converts the scalar numbers 9 and 5 into Tensor objects for us. We also pass in an optional string name parameter, which we can use to give an identifier to the nodes we create.

c = tf.mul(a,b, name=”mul_c”)
d = tf.add(a,b, name=”add_d”)

Here, we are defining the next two nodes in our graph, and they both use the nodes we defined previously. Node c uses the tf.mul. Op, which takes in two inputs and outputs the result of multiplying them together.

Similarly, node d uses tf.add, an Operation that outputs the result of adding two inputs together. We again pass in a name to both of these Ops (it’s something you’ll be seeing a lot of).

Notice that we don’t have to define the edges of the graph separately from the node- when you create a node in TensorFlow, you include all of the inputs that the Operation needs to compute, and the software draws the connections for you.

e = tf.add(c,d, name=”add_e”)

This last line defines the final node in our graph. e uses tf.add in a similar fashion to node d. However, this time it takes nodes c and d as input- exactly as its described in the graph above. With that, our first, albeit small, graph has been fully defined! If you were to execute the above in a Python script or shell, it would run, but it wouldn’t actually do anything.

Remember- this is just the definition part of the process. To get a brief taste of what running a graph looks like, we could add the following two lines at the end to get our graph to output the final node:

sess = tf.Session()
sess.run(e)

If you ran this in an interactive environment, such as the python shell or the

Jupyter/iPython Notebook, you would see the correct output:

…
>>> sess = tf.Session()
>>> sess.run(e)

Data Types

Tensors have a data type. The basic units of data that pass through a graph are numerical, Boolean, or string elements. When we print out the Tensor object c from our last code example, we see that its data type is a floating-point number. Since we didn’t specify the type of data, TensorFlow inferred it automatically.

For example, 9 is regarded as an integer, while anything with a decimal point, like 9.1, is regarded as a floating-point number.

We can explicitly choose what data type we want to work with by specifying it when we create the Tensor object. We can see what type of data was set for a given Tensor object by using the attribute dtype:

/** data_types.py **/
c = tf.constant(9.0, dtype=tf.float64)
print(c)
print(c.dtype)
Out:
Tensor(“Const_10:0”, shape=(), dtype=float64)
<dtype: ‘float64’>

Constants

In TensorFlow, constants are created using the function constant, which has the signature constant(value, dtype=None, shape=None, name='Const', verify_shape=False), where value is an actual constant value which will be used in further computation, dtype is the data type parameter (e.g., float32/64, int8/16, etc.), shape is optional dimensions, name is an optional name for the tensor, and the last parameter is a boolean which indicates verification of the shape of values.

If you need constants with specific values inside your training model, then the constant object can be used as in following example:

z = tf.constant(5.2, name="x", dtype=tf.float32) 

Tensor shape

The shape of a tensor is the number of elements in each dimension. TensorFlow automatically infers shapes during graph construction.The shape of a tensor, describes both the number of dimensions in a tensor as well as the length of each dimension.

Tensor shapes can either be Python lists or tuples containing an ordered set of integers: there are as many numbers in the list as there are dimensions, and each number describes the length of its corresponding dimension. For example, the list [3, 4] describes the shape of a 3-D tensor of length 3 in its first dimension and length 4 in its second dimension. Note that either tuples (()) or lists ([]) can be used to define shapes.

Let’s take a look at more examples to illustrate this further:

/** tensor_shapes.py **/
# Shapes that specify a 0-D Tensor (scalar) 
# e.g. any single number: 7, 1, 3, 4, etc.
s_0_list = []
s_0_tuple = ()
# Shape that describes a vector of length 3
# e.g. [1, 2, 3]
s_1 = [3]
# Shape that describes a 3-by-2 matrix
# e.g [[1 ,2],
# [3, 4],
# [5, 6]]
s_2 = (3, 2)

We can assign a flexible length by passing in None as a dimension’s value. Passing None as a shape will tell TensorFlow to allow a tensor of any shape. That is, a tensor with any amount of dimensions and any length for each dimension:

# Shape for a vector of any length:
s_1_flex = [None]
# Shape for a matrix that is any amount of rows tall, and 3 columns wide:
s_2_flex = (None, 3)
# Shape of a 3-D Tensor with length 2 in its first dimension, and variable-
# length in its second and third dimensions:
s_3_flex = [2, None, None]
# Shape that could be any Tensor
s_any = None

The tf.shape Op can be used to find the shape of a tensor if any need to in your graph. It simply takes in the Tensor object you’d like to find the shape for, and returns it as an int32 vector:

import tensorflow as tf
# …create some sort of mystery tensor
# Find the shape of the mystery 
tensorshape = tf.shape(mystery_tensor, name=”mystery_shape”)
  • Tensors are just a superset of matrices!
  • tf.shape, like any other Operation, doesn’t run until it is executed inside of a Session.

Names

Tensor objects can be identified by a name. This name is an intrinsic string name. As with dtype, we can use the .name attribute to see the name of the object:

/** names.py **/
with tf.Graph().as_default():
c1 = tf.constant(4,dtype=tf.float64,name=’c’)
c2 = tf.constant(4,dtype=tf.int32,name=’c’)
print(c1.name)
print(c2.name)
Out:
c:0
c_1:0

The name of the Tensor object is simply the name of its corresponding operation (“c”; concatenated with a colon), followed by the index of that tensor in the outputs of the operation that produced it — it is possible to have more than one.

Name scopes

In TensorFlow, large, complex graph could be grouped together, so as to make it easier to manage. Nodes can be grouped by name. It is done by using tf.name_scope(“prefix”) Op together with the useful with clause.

/** name_scopes.py **/
with tf.Graph().as_default():
c1 = tf.constant(4,dtype=tf.float64,name=’c’)
with tf.name_scope(“prefix_name”):
c2 = tf.constant(4,dtype=tf.int32,name=’c’)
c3 = tf.constant(4,dtype=tf.float64,name=’c’)
print(c1.name)
print(c2.name)
print(c3.name)
Out:
c:0
prefix_name/c:0
prefix_name/c_1:0

In this example we’ve grouped objects contained in variables c2 and c3 under the scope prefix_name, which shows up as a prefix in their names.

Prefixes are especially useful when we would like to divide a graph into subgraphs with some semantic meaning.

Feed dictionary

Feed is used to temporarily replace the output of an operation with a tensor value. The parameter feed_dict is used to override Tensor values in the graph, and it expects a Python dictionary object as input. The keys in the dictionary are handles to Tensor objects that should be overridden, while the values can be numbers, strings, lists, or NumPy arrays (as described previously). feed_dict is also useful for specifying input values.

Note : The values must be of the same type (or able to be converted to the same type) as the Tensor key.

Let’s show how we can use feed_dict to overwrite the value of a in the previous graph:

/** feed_dict.py **/
import tensorflow as tf
# Create Operations, Tensors, etc (using the default graph)
a = tf.add(2, 5)
b = tf.mul(a, 3)
# Start up a `Session` using the default graph
sess = tf.Session()
# Define a dictionary that says to replace the value of `a` with 15
replace_dict = {a: 15}
# Run the session, passing in `replace_dict` as the value to `feed_dict`
sess.run(b, feed_dict=replace_dict) # returns 45
# Open Session
sess = tf.Session()
# Run the graph, write summary statistics, etc.
…
# Close the graph, release its resources
sess.close()

Variables

TensorFlow uses special objects called Variables. Unlike other Tensor objects that are “refilled” with data each time we run a session. They can maintain a fixed state in a graph.

Variables like other Tensors, can be used as input for other operations in the graph.

Using Variables is done in two stages.

  • First the tf.Variable() function is called in order to create a Variable and define what value it will be initialized with.
  • An initialization operation is perfomed by running the session with the tf.global_variables_initializer() method, which allocates the memory for the Variable and sets its initial values.

Like other Tensor objects, Variables are computed only when the model runs, as we can see in the following example:

/** variable.py **/
init_val = tf.random_normal((1,5),0,1)
var = tf.Variable(init_val, name=’var’)
print(“pre run: \n{}”.format(var))
init = tf.global_variables_initializer()
with tf.Session() as sess:
sess.run(init)
post_var = sess.run(var)
print(“\npost run: \n{}”.format(post_var))
Out:
pre run:
Tensor(“var/read:0”, shape=(1, 5), dtype=float32)
post run:
[[ 0.85962135 0.64885855 0.25370994 -0.37380791 0.63552463]]

Note that if we run the code again, we see that a new variable is created each time, as indicated by the automatic concatenation of _1 to its name:

pre run:
Tensor(“var_1/read:0”, shape=(1, 5), dtype=float32)
Note: To reuse the same variable, we can use the tf.get_variables() function instead of tf.Variable().

Placeholders

Placeholders are structures designated by TensorFlow for feeding input values. They can be also thought of as empty Variables that will be filled with data later on. They are used by first constructing our graph and only when it is executed feeding them with the input data.

Placeholders have an optional shape argument. If a shape is not fed or is passed as None, then the placeholder can be fed with data of any size:

ph = tf.placeholder(tf.float32,shape=(None,10)) 
Whenever a placeholder is defined, it must be fed with some input values or else an exception will be thrown.
/** placeholders.py **/
import tensorflow as tf
x = tf.placeholder("float", None)
y = x * 2

with tf.Session() as session:
result = session.run(y, feed_dict={x: [1, 2, 3]})
print(result)

First, we import tensorflow as normal. Then we create a placeholder called x, i.e. a place in memory where we will store value later on.

Then, we create a Tensor called, which is the operation of multiplying x by 2. Note that we haven’t defined any initial values for x yet.

We now have an operation (y) defined, and can now run it in a session. We create a session object, and then run just the y variable. Note that this means, that if we defined a much larger graph of operations, we can run just a small segment of the graph. This subgraph evaluation is actually a bit selling point of TensorFlow, and one that isn’t present in many other libraries that do similar things.

Running y requires knowledge about the values of x. We define these inside the feed_dictargument to run. We state here that the values of x are [1, 2, 3]. We run y, giving us the result of [2, 4, 6].

Conclusion

TensorFlow is a powerful framework that makes working with mathematical expressions and multi-dimensional arrays a breeze — something fundamentally necessary in machine learning.

We have covered the basics of TensorFlow, this will get us started on our journey into the TensorFlow land. In my subsequent tutorials to come, we will see how to leverage the TensorFlow library to solve optimization problems and making a predictive analysis equation. We will also train a model to solve the XOR problem using the Linear Regression equation and Logistic Regression. Thanks for reading!

Originally published on https://blog.bitsrc.io

Machine Learning in JavaScript with TensorFlow.js

Machine Learning in JavaScript with TensorFlow.js

Machine Learning in JavaScript with TensorFlow.js

Google releases and showcases TensorFlow.js at the TensorFlow Dev Summit 2018. Machine learning in the browser has never been better. Here is the presentation:

ML in JS with TensorFlow.js Video Transcript

All right, hi everyone thanks for coming today. My name is Daniel, my name is Nikhil. We're from the Google brain team and today we're delighted to talk about JavaScript. So, Python has been one of the mainstream languages for scientific computing and it's been like that for a while and there's a lot of tools and libraries around Python but it's that where it ends. We're here today to take the job to convince you that JavaScript and the browser have a lot to offer and the TensorFlow playground is a great example of that. Curious, how many people have seen TensorFlow playground before?

Oh wow, quite a few. I'm very glad. So, for those of you that haven't seen it you can check it out after our talk at https://playground.tensorflow.org - it is an in-browser visualization of a small
neural network and it shows in real time all the internals of the network as it's training and this was a lot of fun to make and had a huge educational success.

We've been getting emails from high schools and universities that have been using this to teach students about machine learning. After we launched playground we were wondering why was it so successful and we think one big reason was that it was in the browser and the browser is this unique platform where with the things you build you can share with anyone with just a link and those people that open your app, don't have to install any drivers or any software it just works.

Another thing is it's the browser is highly interactive and so the user is going to be engaged with whatever you're building. Another big thing is that browsers, we didn't take advantage of this in the playground, but browsers have access to sensors like the microphone and the camera and the accelerometer and all of these sensors are behind standardized APIs that work on all
browsers, and the last thing, the most important thing, is the data that comes from these sensors doesn't ever have to leave the client, you don't have to upload anything to the server, which preserves privacy.

Now the playground that we built is powered by a small neural network 300 lines of vanilla JavaScript that we wrote as a one-off library. It doesn't scale, is just simple for loops and it wasn't engineered to be reusable. But it was clear to us that if we were to open the door to for people to merge machine learning and the browser we had to build a library and we did it.

We released deeplearn.js, a JavaScript library that is GPU accelerated, it does that via WebGL, which is a standard in the browser that allows you to render 3D graphics. We utilize it to do linear algebra for us and deeplearn.js allows you to both run inference in the browser and training entirely in the browser. When we released it we had an incredible momentum the community took
the plunges and took existing models in Python and ported it to the browser and build interactive fun things with it.

So one example is the style transfer. Another person ported the character RNN and then built a novel interface that allows you to explore all the different possible endings of a sentence, all generated by the model in real time.

Another example is a font generative model. There was a post about this that the person that built it allowed users to explore the hidden dimensions the interesting dimensions in the embedding space and you can see how they relate to boldness and slanted nosov the font and there was even educational examples like teachable machine that build this fun little game that taught people how computer vision models work so people could interact directly with the webcam now all the examples I showed you point to the incredible momentum we have with deep lunges and building on that - we're very excited today to announce that deep learn Jas is joining the tensorflow family and with that we are releasing a new ecosystem of libraries and tools for machine learning with JavaScript called tensorflow Jas now before we get into the details I want to go over three main use cases of how you can use tensorflow Jas today with the tools and libraries that we're releasing so one use case is you can write models directly in the browser and this has huge educational implications think of the playground that I just showed a second use case is a major use case is you can take a pre-existing model pre-trained model in python use a script and you can import it into the browser to do inference and a related use case is the same model that you take to do inference you can retrain it potentially with private data that comes from those sensors of the browser in the browser itself now to give you more of a schematic view we have the browser that utilizes WebGL to do fast linear algebra on top of it tensorflow jeaious has two sets of api's the apps API which used to be deep learn Jas and we worked hard to align the API with tensorflow Python it is powered by an automatic differentiation library that is built analogous to eager mode and on top of that we have a high level API layers API that allow you to use best practices and high level building blocks to write models what I'm also very excited today to announce is that we're releasing tools that can take a existing Kerris model or tensorflow safe model and port it automatically free for execution in the browser now to show you an example of our API we're going to go over a small program that tries to learn the coefficients of a quadratic function so the coefficients we're trying to learn our a B and C from data so we have our import TF from tensorflow J's for those of you that don't know this is a standard yes six important JavaScript very common we have our three tensors a B and C we mark them as variables which means that they are mutable and the optimizer can change them we have our f of X function that does the polynomial computation you can see here familiar API like TF ad + TF square like tensorflow in addition to that API we also have a chaining API which allows you to call these math operations on tensors itself and this leads to better readable code that is closer to how we write math chaining is very popular in in JavaScript world so that's the feed-forward part of the model now for the training part we need a loss function so here is a loss function there is just a mean squared error from between the prediction and the label we have our optimizer and SJ edge the optimizer and we train the model we call optimizer that minimize for some number of epochs and here I want to emphasize for those of you that have used Thea figure before or the talk before us Alexes talk the API in tens of ojs is aligned with the eager API in Python alright so clearly that's not how most people write machine learning because because those a low-level linear algebra ops can be quite verbose and for that we have our layers API and to show you an example of that we're gonna we're going to build a recurrent neural network that learns to sum two numbers but the complicated part is that those numbers like the number 90 + 10 are being fed character by character and then the neural network has to maintain an internal state with an Alice TM cell that that state then gets passed into a decoder and then the decoder has to output 100 character by characters so it's a sequence to sequence model now this may sound a little complicated but with the layers API is not that much lines of code we have our import TF from tensorflow J's the four we have our sequential model which just means it's a stack of layers for those of you that are familiar with TF layers in Python or Kerris this API looks very familiar we have the first two layers of the encoder the last three layers are the decoder and that's our model we then compile it with a loss and optimizer and a metric we want a monitor like accuracy and we call model dot fit with our data now what I want to point out here is the await keyword so model dot fit is an asynchronous call which means because in practice that can take
about 30 or 40 seconds in a browser and in those 30 or 40 seconds you don't want the main UI thread of the browser to be locked and this is why you get a callback with a history object after that's done and in between the GPU is going to do the work now the code I showed you is when you are trying to write models directly when you want to write models directly in the browser but as I said before a major use case where even with deep learning is with people importing models that were pre trained and they just want to do inference in the browser and before we jump into the details of that I want to show you a fun little game that our friends at Google brand studio built that takes advantage of an automatically pre trained model poured it into the browser and the game is called emoji scavenger hunt and the way it works I'm going to show you here a real demo with the phone it's in the browser let me see I see here so you can see I have a Chrome browser open up on a pixel phone you can see the URL on the top and the game uses the webcam and shows me an emoji and then I have some time some number of seconds to find the real version item of that emoji before the time runs out before we play the kill here is going to help me identify the objects that this game asks you ready I'm ready all right let's go alright alright watch watch have a watch didn't I spot a velvet come on yay we got that let's see what's our next item is choo-choo you gotta help me out here buddy oh yeah we got the shoe alright what's next right nice a banana banana yes use any whenever but this guy's got a banana oh come over here alright yeah high score here beer yeah it's 10:30 in the morning Daniel let's get back to the talk alright alright so I'm gonna jump into some of the technical details of how we actually built that game so what we did was we trained a model in tensorflow to be an object recognizer for the scavenger hunt game we chose about 400 different classes that would be reasonable for a game like this you know watches and bananas and and and beer so what we did was we use the tensorflow for poets code lab and in that code lab what you essentially do is you take a pre trained mobile net model and if you don't know what mobile net is it's a state-of-the-art computer vision model for edge devices so what we effectively did was we took that model and we retrained it for these classes now we have an object detector in the Python world how do we actually now get this into the browser well we provide a set of tools today that help you actually do that once it's in you skin the game and you you know make the computer talk and all that kind of fun stuff let's jump into how we actually convert that model so as Daniel mentioned earlier today we actually support two types of models so we support tensorflow saved models we have a convertor for that and we also have a convertor for Karis saved models so you define your model and you save it with a saved model this is a standard way to do that similarly for similarly this is code that you would do that for Karis is our important the next piece is that we actually convert it to the web today we're releasing a pip package it's tensorflow Jess you can install that there there's a script in there that lets you point to your tensorflow save model and point to an output directory and that output directory will be where those static build artifacts will go for the web caris is the same exact flow point to your hdf5 input and you have an output directory where those Bulldog effects will go now you statically host those on your website somewhere you know just simple static hosting and on the JavaScript side we provide an API that lets you load that model so this is what it looks like for tensor flow and the tensor flow state model you'll notice that it's a frozen model we don't right now support continuing training of this model well in the Karis case we actually let you continue training and we're working hard to keep these api's aligned in the future okay so under the cover what are we actually doing so we're doing some graph optimization which essentially means that we prune out nodes that you don't need to make the prediction you don't need that in the web we optimize waits for browser Auto caching so we pack in shard in chunks of four megabytes which helps your browser be quick the next time your page reloads today we support about 90 of the most commonly used tensorflow ups and we're working very hard to support more like control flops and we support 32 of the most commonly as Kara Slayers today and as I mentioned we let you continue training for Karis models and we let you do evaluation as well as make predictions from that mole okay so obviously there's a lot you can do with just porting your models to the web for inference but since the beginning of deep learned s we've made it a high priority to make sure that you can train directly in the browser you know this opens up the door for education and interactive tools like we saw with a playground as well as let you train with data that never leaves your client so this is huge for privacy so to show off what you can do with something like this we built another little game so the goal of the game is to play pac-man with your webcam now Daniel is gonna be my helper here he is much much better at this game that I am for some reason to say hi so there are three phases of the game phase one we're going to collect frames from the webcam and we're going to associate those with up down left and right these classes now Daniel is gonna do you know move his head up down left and right and he's just simply gonna play the game like that and you'll notice as he's collecting frames he's kind of moving around a little bit this kind of helps the model see different angles for that class and generalize a little bit better so after he's done collecting these frames we're gonna go and train our model so we're not actually training from scratch here when we hit that train button we're taking a pre train mobile net again porting that to the web and doing a retraining phase with that data that's local and we're using the layers API to do that in the browser here you want to press that train button alright our loss is going down it looks like we're learning something that's great so as soon as we press that play button what's gonna happen is we're gonna make predictions from the webcam those are gonna get plugged into those controls and it's going to control the Plaquemine pac-man game ready all right so you can see in the bottom right it's highlighting the class that it thinks it is and down if he moves his head around you'll see it you'll see a change like class and he's off. So so all of this code is online and you can go for kit we invite you to do so and obviously this is just a game but

You can imagine other types of applications of this. Like make a browser extension that lets you control the page for accessibility purposes so again all this code is online please go for kit and play and make something else with it okay Daniel I know this is fine I know .all right

Introducing TensorFlow.js: Machine Learning in Javascript

Introducing TensorFlow.js: Machine Learning in Javascript

Introducing TensorFlow.js: Machine Learning in Javascript: In this post, we’ll give you a quick overview of TensorFlow.js, and getting started resources you can use to try it out.

We’re excited to introduce TensorFlow.js, an open-source library you can use to define, train, and run machine learning models entirely in the browser, using Javascript and a high-level layers API. If you’re a Javascript developer who’s new to ML, TensorFlow.js is a great way to begin learning. Or, if you’re a ML developer who’s new to Javascript, read on to learn more about new opportunities for in-browser ML. In this post, we’ll give you a quick overview of TensorFlow.js, and getting started resources you can use to try it out.

In-Browser ML

Running machine learning programs entirely client-side in the browser unlocks new opportunities, like interactive ML! If you’re watching the livestream for the TensorFlow Developer Summit, during the TensorFlow.js talk you’ll find a demo where @dsmilkov and @nsthorat train a model to control a PAC-MAN game using computer vision and a webcam, entirely in the browser. You can try it out yourself, too, with the link below — and find the source in the examples folder.

If you’d like to try another game, give the Emoji Scavenger Hunt a whirl — this time, from a browser on your mobile phone.

ML running in the browser means that from a user’s perspective, there’s no need to install any libraries or drivers. Just open a webpage, and your program is ready to run. In addition, it’s ready to run with GPU acceleration. TensorFlow.js automatically supports WebGL, and will accelerate your code behind the scenes when a GPU is available. Users may also open your webpage from a mobile device, in which case your model can take advantage of sensor data, say from a gyroscope or accelerometer. Finally, all data stays on the client, making TensorFlow.js useful for low-latency inference, as well as for privacy preserving applications.

What can you do with TensorFlow.js?

If you’re developing with TensorFlow.js, here are three workflows you can consider.

  • You can import an existing, pre-trained model for inference. If you have an existing TensorFlow or Keras model you’ve previously trained offline, you can convert into TensorFlow.js format, and load it into the browser for inference.
  • You can re-train an imported model. As in the Pac-Man demo above, you can use transfer learning to augment an existing model trained offline using a small amount of data collected in the browser using a technique called Image Retraining. This is one way to train an accurate model quickly, using only a small amount of data.
  • Author models directly in browser. You can also use TensorFlow.js to define, train, and run models entirely in the browser using Javascript and a high-level layers API. If you’re familiar with Keras, the high-level layers API should feel familiar.
Let’s see some code

If you like, you can head directly to the samples or tutorials to get started. These show how-to export a model defined in Python for inference in the browser, as well as how to define and train models entirely in Javascript. As a quick preview, here’s a snippet of code that defines a neural network to classify flowers, much like on the getting started guide on TensorFlow.org. Here, we’ll define a model using a stack of layers.

import * as tf from ‘@tensorflow/tfjs’;
const model = tf.sequential();
model.add(tf.layers.dense({inputShape: [4], units: 100}));
model.add(tf.layers.dense({units: 4}));
model.compile({loss: ‘categoricalCrossentropy’, optimizer: ‘sgd’});

The layers API we’re using here supports all of the Keras layers found in the examples directory (including Dense, CNN, LSTM, and so on). We can then train our model using the same Keras-compatible API with a method call:

await model.fit(
  xData, yData, {
    batchSize: batchSize,
    epochs: epochs
});

The model is now ready to use to make predictions:

// Get measurements for a new flower to generate a prediction
// The first argument is the data, and the second is the shape.
const inputData = tf.tensor2d([[4.8, 3.0, 1.4, 0.1]], [1, 4]);

// Get the highest confidence prediction from our model
const result = model.predict(inputData);
const winner = irisClasses[result.argMax().dataSync()[0]];

// Display the winner
console.log(winner);

TensorFlow.js also includes a low-level API (previously deeplearn.js) and support for Eager execution. You can learn more about these by watching the talk at the TensorFlow Developer Summit.

An overview of TensorFlow.js APIs. TensorFlow.js is powered by WebGL and provides a high-level layers API for defining models, and a low-level API for linear algebra and automatic differentiation. TensorFlow.js supports importing TensorFlow SavedModels and Keras models.

How does TensorFlow.js relate to deeplearn.js?

Good question! TensorFlow.js, an ecosystem of JavaScript tools for machine learning, is the successor to deeplearn.js which is now called TensorFlow.js Core. TensorFlow.js also includes a Layers API, which is a higher level library for building machine learning models that uses Core, as well as tools for automatically porting TensorFlow SavedModels and Keras hdf5 models. For answers to more questions like this, check out the FAQ.

Where’s the best place to learn more?

To learn more about TensorFlow.js, visit the project homepage, check out the tutorials, and try the examples. You can also watch the talk from the 2018 TensorFlow Developer Summit, and follow TensorFlow on Twitter.

Thanks for reading, and we’re excited to see what you’ll create with TensorFlow.js!

Learn TensorFlow.js - Deep Learning and Neural Networks with JavaScript

Learn TensorFlow.js - Deep Learning and Neural Networks with JavaScript

This full course introduces the concept of client-side artificial neural networks. We will learn how to deploy and run models along with full deep learning applications in the browser! To implement this cool capability, we’ll be using TensorFlow.js (TFJS), TensorFlow’s JavaScript library.

By the end of this video tutorial, you will have built and deployed a web application that runs a neural network in the browser to classify images! To get there, we'll learn about client-server deep learning architectures, converting Keras models to TFJS models, serving models with Node.js, tensor operations, and more!

⭐️Course Sections⭐️

⌨️ 0:00 - Intro to deep learning with client-side neural networks

⌨️ 6:06 - Convert Keras model to Layers API format

⌨️ 11:16 - Serve deep learning models with Node.js and Express

⌨️ 19:22 - Building UI for neural network web app

⌨️ 27:08 - Loading model into a neural network web app

⌨️ 36:55 - Explore tensor operations with VGG16 preprocessing

⌨️ 45:16 - Examining tensors with the debugger

⌨️ 1:00:37 - Broadcasting with tensors

⌨️ 1:11:30 - Running MobileNet in the browser