Table of contents

Introduction

Different types of creating word vectors

Euclidean Distance

What Next

Remark

References

Introduction

I’m going to introduce you to the general idea behind vector space models. You’re going to see their advantages along with some of their applications in natural language processing.

Example 1)

So suppose you have two questions. The first one is, where are you heading? And the second one is, where are you from?These sentences have identical words, except for the last ones. However, they both have a different meaning.

On the other hand, say you have two more questions whose words are completely different but both sentences mean the same thing.

Vector space models will help you identify whether the first pair of questions or the second pair are similar in meaning even if they do not share the same words.

Example 2)

They can be used to identify similarity for a question answering, paraphrasing, and summarization.

Example3)

Vector space models will also allow you to capture dependencies between words. Consider this sentence. You eat cereal from a bowl. Here, you can see that the word cereal and the word bowl are related.

Now let’s look at this other sentence. You buy something and someone else sells it. So what it’s saying is that someone sells something because someone else buys it. The second half of the sentence is dependent on the first half.

With vector space models, you will be able to capture this and many other types of relationships among different sets of words.

Vector space models are used in information extraction to answer of questions in the style of who, what, where, how, and etc.

Different types of creating word vectors

i) Word by Word

ii) Word by Document

Word by Word

In this section, we shall learn how to construct vectors based off a co-occurrence matrix.

We will see how you can encode a word or a document as a vector. To get a vector space model using a word by word design, we’ll make a co-occurrence matrix and extract vector presentations for the words in given corpus.

You’ll be able to get a vector space model using a word by document design using a similar approach.

Finally, we’ll see how in a vector space one can find relationships between words and vectors, also known as their similarity. The co-occurrence of two different words is the number of times that they appear in given corpus together within a certain word distance k.

For instance, suppose that your corpus has the following two sentences.

The row of the co-occurrence matrix corresponding to the word ‘data’ with a k value equal to 2 would be populated with the shown values. For the column corresponding to the word simple, you’d get a value equal to 2 because ‘data’ and ‘simple’ co-occur in the first sentence within a distance of 1 word, and in the second sentence within a distance of 2 words.The row of the co-occurrence matrix corresponding to the word ‘data’ would look like as shown in above image if you consider the co-occurrence with the words simple, raw, like, and I. In this case, the vector representation of the word ‘data’ would be equal to 2, 1, 1, 0. With a word by word design, you can get a representation with n entries, with n between 1 and the size of your entire vocabulary.

#python3 #nlp #deep learning