TextBlob Spelling Correction

What is TextBlob?

TextBlob is a Python library for processing textual data. It provides a consistent API for diving into common natural language processing (NLP) tasks such as part-of-speech tagging, noun phrase extraction, sentiment analysis, and more.

Why TextBlob?

NLU is a subset of NLP in which an unstructured data or sentence is being converted into its structured form for performing NLP in terms of handling end to end interactions. Relation extraction, semantic parsing, sentiment analysis, Noun phrase extraction are few examples of NLU which itself is a subset of NLP. Now to work in these areas, TextBlob plays a great role which is not that efficiently done by NLTK.

Spelling Correction with TextBlob

Photo by Romain Vignes on Unsplash

STEP: 1 → Installing TextBlob

Sometimes tweets, reviews, or any blog data may contain typo errors, hence first we need to correct that data to reducing multiple copies of the same words, which represents the same meaning.

Installing TextBlob on your computer is very simple. You simply need to install it using pip.

STEP: 2 → Load the Input for preprocessing

Just we have to feed the computer from the basics so that it can be very well trained in natural language understanding and processing.

Next, load the input text (as docx) for which you need the correct the spelling, and the text we are about to handle is Immigrants in Toronto.

#text-analytics #spell-check #textblob #nlp

What is GEEK

Buddha Community

TextBlob Spelling Correction

What is TextBlob?

TextBlob is a Python library for processing textual data. It provides a consistent API for diving into common natural language processing (NLP) tasks such as part-of-speech tagging, noun phrase extraction, sentiment analysis, and more.

Why TextBlob?

NLU is a subset of NLP in which an unstructured data or sentence is being converted into its structured form for performing NLP in terms of handling end to end interactions. Relation extraction, semantic parsing, sentiment analysis, Noun phrase extraction are few examples of NLU which itself is a subset of NLP. Now to work in these areas, TextBlob plays a great role which is not that efficiently done by NLTK.

Spelling Correction with TextBlob

Photo by Romain Vignes on Unsplash

STEP: 1 → Installing TextBlob

Sometimes tweets, reviews, or any blog data may contain typo errors, hence first we need to correct that data to reducing multiple copies of the same words, which represents the same meaning.

Installing TextBlob on your computer is very simple. You simply need to install it using pip.

STEP: 2 → Load the Input for preprocessing

Just we have to feed the computer from the basics so that it can be very well trained in natural language understanding and processing.

Next, load the input text (as docx) for which you need the correct the spelling, and the text we are about to handle is Immigrants in Toronto.

#text-analytics #spell-check #textblob #nlp

Spelling Rectification App using TextBlob & pyspellchecker

Type in “Stendford University” on your search engine and you will notice that it provides you with results for “Standford University”. You must have realized that almost every time the search engine will provide you with the correct search results that you were looking for irrespective of spelling mistakes in your search query.

How is this made possible?

It is possible because the search engines use “spelling correction” algorithms before sending your request to the server so that the server returns you with results you were looking for. Spelling correction is a must-have for any modern search engine.

This was just one example of how and where Natural Language Processing is used to correct spelling mistakes. Spelling rectification helps you produce quality content and can be used in emails, documents, social media, articles(even Medium) etc.

In this article, we will learn to create an offline Spelling Rectification Application in Python using modules like TextBlob**, pyspellchecker & **Flask.

Steps to be followed :

1. Create a spelling rectification program in python
2. Integrate the program in a Flask Application

Before diving into the coding part let us first know about these modules briefly. If you are already familiar with these modules, you can directly jump to the next section.

#programming #python #spell-check #textblob #flask

Spelling Correction in Python with TextBlob

Spelling mistakes are common, and most people are used to software indicating if a mistake was made. From autocorrect on our phones, to red underlining in text editors, spell checking is an essential feature for many different products.

The first program to implement spell checking was written in 1971 for the DEC PDP-10. Called SPELL, it was capable of performing only simple comparisons of words and detecting one or two letter differences. As hardware and software advanced, so have spell checkers. Modern spell checkers are capable of handling morphology and using statistics to improve suggestions.

Python offers many modules to use for this purpose, making writing a simple spell checker an easy 20-minute ordeal.

One of these libraries being TextBlob, which is used for natural language processing that provides an intuitive API to work with.

In this article we’ll take a look at how to implement spelling correction in Python with TextBlob.

#python #textblob #nlp #machine learning #artificial intelligence

Spelling Correction: How to make an accurate and fast corrector

Dirty data leads to bad model quality. In real-world NLP problems we often meet texts with a lot of typos. As the result, we are unable to reach the best score. As painful as it may be, data should be cleaned before fitting.

We need an automatic spelling corrector which can fix words with typos and, at the same time not break correct spellings.

But how can we achieve this?

Let start with a Norvig’s spelling corrector and iteratively increase its capabilities.

Norvig’s approach

Peter Norvig (director of research at Google) described the following approach to spelling correction.

Let’s take a word and brute force all possible edits, such as delete, insert, transpose, replace and split. Eg. for word abc possible candidates will be: ab ac bc bac cba acb a_bc ab_c aabc abbc acbc adbc aebc etc.

Every word is added to a candidate list. We repeat this procedure for every word for a second time to get candidates with bigger edit distance (for cases with two errors).

Each candidate is estimated with unigram language model. For each vocabulary word frequencies are pre-calculated, based on some big text collections. The candidate word with highest frequency is taken as an answer.

Adding some context

First improvement — adding n-gram language model (3-grams). Let’s pre-calculate not only single words, but word and a small context (3 nearest words). Let’s estimate probability of some fragment as a product of all n-grams of n-size:

To make everything simple let’s calculate probability of n-gram of size n as a product of probabilities of all lower order grams (actually there are some smoothing technics, like Kneser–Ney — they improve model’s accuracy, but let’s talk about it later, see “Improve Accuracy” paragraph below):

To get a probability of n-gram from appearance frequencies we need to normalize frequencies (eg. divide number of 3-grams by number of 2-grams, etc.):

Now we can use our extended language model to estimate candidates with context.

#spelling-correction #data-preprocessing #nlp #data-science #python

Correct Your Sitting Posture - How to Sit at Your Desk

Learn on correct your sitting posture or how to improve your posture so that you can not only correct your bad posture but also help your back stay healthy.

#how #correct your sitting posture #how to sit at your desk correctly #how to sit at your desk correctly