Text Generation Using N-Gram Model

Text Generation Using N-Gram Model

have learned that machines are capable of generating the new text by using some simple statistical and probabilistic techniques. In this article I wanted to share this extremely simple and intuitive method of creating Text Generation Model.

My interest in Artificial Intelligence and in particular in Natural Language Processing (NLP) has sparked exactly when I have learned that machines are capable of generating the new text by using some simple statistical and probabilistic techniques. In this article I wanted to share this extremely simple and intuitive method of creating Text Generation Model.

Background

As we all know, language has a sequential nature, hence the order in which words appear in the text matters a lot. This feature allows us to understand the context of a sentence even if there are some words missing (or in case if stumble across a word which meaning is unknown). Consider example below:

“Mary was scared because of the terrifying noise emitted by Chupacabra.”

Without some previous context we have no idea what “Chupacabra” indeed is, but we probably can say that we would be not happy to encounter this creature in real life.

This dependency of words inside the sentence can give us some clues about the nature of the missing word and sometimes we do not even need to know the whole context. In the example above, by looking only at “_noise emitted by_” we on the intuitive level can say that the following word should be a noun and not some other part of speech.

This brings us up to the idea behind the N-Gram_s, where the formal definition is “a contiguous sequence of _n items from a given sample of text”. The main idea is that given any text, we can split it into a list of unigrams (1-gram), bigrams (2-gram), trigrams (3-gram) etc.

For example:

Text: “I went running”

Unigrams: [(I), (went), (running)]

Bigrams: [(I, went), (went, running)]

As you can notice word “went” appeared in 2 bigrams: (I, went) and (went, running).

The other, more visual, way to look at it

artificial-intelligence language-modeling ngrams naturallanguageprocessing

Bootstrap 5 Complete Course with Examples

Bootstrap 5 Tutorial - Bootstrap 5 Crash Course for Beginners

Nest.JS Tutorial for Beginners

Hello Vue 3: A First Look at Vue 3 and the Composition API

Building a simple Applications with Vue 3

Deno Crash Course: Explore Deno and Create a full REST API with Deno

How to Build a Real-time Chat App with Deno and WebSockets

Convert HTML to Markdown Online

HTML entity encoder decoder Online

AI Innovations in Artificial Intelligence

Innovations in Artificial Intelligence - Various sectors in which AI is remarkably used & has brought changes in humanity - Education, Healthcare,automobile

Bursting the top 7 common Myths about Artificial Intelligence by Rebecca Harrison

Artificial Intelligence has been the go-to technology for companies and enterprises in recent years. The adoption of AI by enterprises all around the world has grown by 270% in the last four years a...

10 Most Amazing Artificial Intelligence Milestones To Know

Top 10 Artificial Intelligence Milestones to learn AI evolution - Origin,ELIZA,XCON,Statistics Introduction, Chess & jeopardy winner,autonomous vehicles

How to Create an Artificial Intelligence (AI) Model

Lemonade is one of this year’s hottest IPOs and a key reason for this is the company’s heavy investments in AI (Artificial Intelligence). The company has used this technology to develop bots to handle the purchase of policies and the managing of claims. In this post, you'll see How to Create an Artificial Intelligence (AI) Model

Top 10 Artificial Intelligence (AI) Interview Questions & Answers

In this Artificial Intelligence Interview Questions blog, you will understand the most frequently asked AI questions and their solutions.