Welcome to the third part of this series on building deep learning library in JavaScript from scratch.

In the first part of the series, we talked about the basic building blocks of all deep learning libraries—automatic gradients. Additionally, we showed how to implement autograd calculation in JavaScript.

In the second part of the series, We implemented tensors, math operations (add and matmul), a linear layer, and ReLU and softmax activation functions—put together, this gave us a very simple but working neural network.

In this part, we’ll pick up where we left off and implement the following;

• Sequential model
• Cross entropy loss
• Optimization

Sequential Model

As a deep learning practitioner, I’ll again assume you’re familiar to some extent with Sequential models, both in Keras and TensorFlow. But for a quick reminder, you can check out the Keras sequential model here.

In the previous part of the series, we implemented a simple neural network:

let linear1 =    new Linear(2,3)
	let relu =    new ReLU()
	let linear2 =   new Linear(3,2)
	let softmax =    new Softmax()

	var x = new Tensor(1,2,require_grad=true)
	x.setFrom([2,3]);

	// forward pass
	input = linear1.forward(x)
	input = relu.forward(input)
	input =  linear2.forward(input)
	output = softmax.forward(input);

	console.log(output.out) // output = [ 0.5025667023096377, 0.4974332976903622 ]
view raw
NeuralNet.js hosted with ❤ by GitHub

Now, we’ll convert this model into a Sequential model that consists of different numbers of neural layers combined together to form a single model. The Sequential model has more to it than that—but we’ll be discussing that a bit later.

#heartbeat #tensorflow #machine-learning #deep-learning #data-science #deep learning