Introduction:-

In this article we are going to discuss about very interesting topic of natural language processing(NLP) Neural Machine translation (NMT) using Attention model. Machine translation is nothing but automatic translation of text from one language to another.

Here we will learn how to use sequence to sequence architecture (seq2seq) with Bahdanau’s Attention mechanism for NMT.

Prerequisite:-

This article assumes that you understand following:-

• Recurrent Neural Network(RNN), Long Short Term Memory (LSTM), Bidirectional Network.
• Sequence to sequence architecture (Encoder-Decoder).
• Attention Mechanism.

Before going through code we will discuss Bidirectional LSTM and Attention mechanism in short.

Bidirectional:-

If you understand LSTM then Bidirectional is quite simple. In bidirectional network you can use simple RNN(Recurrent Neural Network), GRU (Gated Recurrent Unit) or LSTM(Long short Term Memory). I am going to use LSTM in this article.

• Forward layer is our regular LSTM layer but Backward LSTM is layer who’s flow is in backward direction.
• At each time step input is passed in both forward and backward layers.
• Output at each time step is combination of both cells output(forward and backward layer). Therefore for prediction model will have knowledge of next words too.

#machine-translation #nlp #machine-learning #deep-learning