Building a word2vec model with our deep learning library in JavaScript

Welcome to the fifth part of out series, where we’ve been building a deep learning library in Javascript that mimics the core functions of popular frameworks like TensorFlow and PyTorch.

In the previous parts of the series, we’ve been implementing some basic functions needed to do this.

In the first part, we explored implementing an automatic gradient in JavaScript and also learned how to create some basic maths operations in JavaScript.

In the second part, we dove deep into implementing some of the core parts in building a neural network, such as tensors, linear layers, and ReLU and softmax activation functions.

In the part three, we discussed how to create a Sequential model, implement stochastic gradient optimization, and also implement cross-entropy loss.

And in the fourth part, we implemented visualization of our neural network graph.

In this part, we will be putting a lot of these pieces together to build a word2vec model using our library.

Goals

• Text pre-processing and cleaning (including skip-gram)
• Create a neural network for the word2vec model
• Create a function to obtain the word2vec of a word

#heartbeat #deep-learning #tensorflow #javascript #pytorch