Sentiment Analysis and Product Recommendation


The Internet has revolutionized the way we buy products. In the retail e-commerce world of online marketplace, where experiencing products are not feasible. Also, in today’s retail marketing world, there are so many new products are emerging every day. Therefore, customers need to rely largely on product reviews to make up their minds for better decision making on purchase. However, searching and comparing text reviews can be frustrating for users. Hence we need better numerical ratings system based on the reviews which will make customers purchase decision with ease.During their decision making process, consumers want to find useful reviews as quickly as possible using rating system. Therefore, models able to predict the user rating from the text review are critically important. Getting an overall sense of a textual review could in turn improve consumer experience. Also, it can help businesses to increase sales, and improve the product by understanding customer’s needs.The amazon review dataset for electronics products were considered. The reviews and ratings given by the user to different products as well as reviews about user’s experience with the product(s) were also considered.

Image for post

Problem Statement

The goal is to develop a model to predict user rating, usefulness of review and recommend most similar items to users based on collaborative filtering.

Data Collection

The electronics dataset consists of reviews and product information from amazon were collected. This dataset includes reviews (ratings, text, helpfulness votes) and product metadata (descriptions, category information, price, brand, and image features).Product Complete Reviews dataThis dataset includes electronics product reviews such as ratings, text, helpfulness votes. This dataset was obtained from The original data was in json format. The json was imported and decoded to convert json format to csv format. The sample dataset is shown below:

Image for post

Sample product reviews dataset

Each row corresponds to a customer review and includes the following variables:

Image for post

Product MetadataThis dataset includes electronics product metadata such as descriptions, category information, price, brand, and image features. This dataset was obtained from The json was imported and decoded to convert json format to csv format. The sample product meta dataset is shown below:

Image for post

Sample product meta dataset

Each row corresponds to product and includes the following variables:

Image for post

#lemmatization #amazon #nlp #stop-word #data analysis

What is GEEK

Buddha Community

Sentiment Analysis and Product Recommendation
Sofia  Maggio

Sofia Maggio


Sentiment Analysis in Python using Machine Learning

Sentiment analysis or opinion mining is a simple task of understanding the emotions of the writer of a particular text. What was the intent of the writer when writing a certain thing?

We use various natural language processing (NLP) and text analysis tools to figure out what could be subjective information. We need to identify, extract and quantify such details from the text for easier classification and working with the data.

But why do we need sentiment analysis?

Sentiment analysis serves as a fundamental aspect of dealing with customers on online portals and websites for the companies. They do this all the time to classify a comment as a query, complaint, suggestion, opinion, or just love for a product. This way they can easily sort through the comments or questions and prioritize what they need to handle first and even order them in a way that looks better. Companies sometimes even try to delete content that has a negative sentiment attached to it.

It is an easy way to understand and analyze public reception and perception of different ideas and concepts, or a newly launched product, maybe an event or a government policy.

Emotion understanding and sentiment analysis play a huge role in collaborative filtering based recommendation systems. Grouping together people who have similar reactions to a certain product and showing them related products. Like recommending movies to people by grouping them with others that have similar perceptions for a certain show or movie.

Lastly, they are also used for spam filtering and removing unwanted content.

How does sentiment analysis work?

NLP or natural language processing is the basic concept on which sentiment analysis is built upon. Natural language processing is a superclass of sentiment analysis that deals with understanding all kinds of things from a piece of text.

NLP is the branch of AI dealing with texts, giving machines the ability to understand and derive from the text. For tasks such as virtual assistant, query solving, creating and maintaining human-like conversations, summarizing texts, spam detection, sentiment analysis, etc. it includes everything from counting the number of words to a machine writing a story, indistinguishable from human texts.

Sentiment analysis can be classified into various categories based on various criteria. Depending upon the scope it can be classified into document-level sentiment analysis, sentence level sentiment analysis, and sub sentence level or phrase level sentiment analysis.

Also, a very common classification is based on what needs to be done with the data or the reason for sentiment analysis. Examples of which are

  • Simple classification of text into positive, negative or neutral. It may also advance into fine grained answers like very positive or moderately positive.
  • Aspect-based sentiment analysis- where we figure out the sentiment along with a specific aspect it is related to. Like identifying sentiments regarding various aspects or parts of a car in user reviews, identifying what feature or aspect was appreciated or disliked.
  • The sentiment along with an action associated with it. Like mails written to customer support. Understanding if it is a query or complaint or suggestion etc

Based on what needs to be done and what kind of data we need to work with there are two major methods of tackling this problem.

  • Matching rules based sentiment analysis: There is a predefined list of words for each type of sentiment needed and then the text or document is matched with the lists. The algorithm then determines which type of words or which sentiment is more prevalent in it.
  • This type of rule based sentiment analysis is easy to implement, but lacks flexibility and does not account for context.
  • Automatic sentiment analysis: They are mostly based on supervised machine learning algorithms and are actually very useful in understanding complicated texts. Algorithms in this category include support vector machine, linear regression, rnn, and its types. This is what we are gonna explore and learn more about.

In this machine learning project, we will use recurrent neural network for sentiment analysis in python.

#machine learning tutorials #machine learning project #machine learning sentiment analysis #python sentiment analysis #sentiment analysis

studio52 dubai

studio52 dubai


How to find the best video production company in Dubai?

How to find the best video production company in Dubai?We are the best video production company in Dubai, UAE. We offer Corporate Video, event video, animation video, safety video and timelapse video in most engaging and creative ways.

#video production company #video production dubai #video production services #video production services dubai #video production #video production house

studio52 dubai

studio52 dubai


Top Video Production Companies in Dubai 2021

Looking for the top video production companies in Dubai in 2021? Choose the right video production company to enhance your product and service with the best video services.

#dubai video production company #video production company #video production house #top video production companies in dubai 2021 #video production #video production companies in dubai

Ajay Kapoor


Software Product Development Company, SaaS Development Services

PixelCrayons: Being a top software product development company, we are known for providing robust, secure, feature-packed, and scalable Software product engineering solutions as per the specific needs of businesses.

Get SaaS-based software product development services from initial strategy & planning to final deployment and after delivery support.

Being a trusted SaaS product development company, we cover the entire array of software product engineering services from consulting to development, testing and devops.

We turn your software product idea into reality by putting custom skillsets in place. With 16+ years of domain expertise, we have created 13800+ successful projects and garnered 6800+ happy customers from 38+ countries.

#software product development companies in india #software product development services #software product development company #software product development india #product development companies in india #software product development companies

Sentiment Analysis of a book through Unsupervised Learning

Getting Started

In this tutorial, I will show you how to apply sentiment analysis to the text contained into a book through an Unsupervised Learning (UL) technique, based on the AFINN lexicon. This tutorial exploits the afinn Python package, which is available only for English and Danish. If your text is written into a different language, you could translate it before in English and use the afinn package.

This notebook applies sentiment analysis the Saint Augustine Confessions, which can be downloaded from the Gutemberg Project Page. The masterpiece is split in 13 books (or chapters). We have stored each book into a different file, named number.text (e.g. 1.txt and 2.txt). Each line of every file contains just one sentence.

You can download the code from my Github repository:

First of all import the Afinn class from the afinn package.

from afinn import Afinn

Then create a new Afinn object, by specifying the used language.

afinn = Afinn(language=’en’)

Calculate the Sentiment

Use the score give by Afinn to calculate the sentiment

The afinn object contains a method, called score(), which receives a sentence as input and returns a score as output. The score may be either positive, negative or neutral. We calculate the score of a book, simply by summing all the scores of all the sentence of that book. We define three variables> pos, neg and neutral, which store respectively the sum of all the positive, negative and neutral scores of all the sentences of a book.

Firstly, we define three indexes, which will be used after.

pos_index = []
neg_index = []
neutral_index = []

We open the file corresponding to each book through the open() function, we read all the lines through the function file.readlines() and for each line, we calculate the score.

Then, we can define three indexes to calculate the sentiment of a book: the positive sentiment index (pi), the negative sentiment index (ni) and the neutral sentiment index (nui). The pi of a book corresponds to the number of positive sentences in a book divided per the total number of sentences of the book. Similarly, we can calculate the ni and nui of a book.

for book in range(1,14):
    file = open('sources/' + str(book) + '.txt')
    lines = file.readlines()
    pos = 0
    neg = 0
    neutral = 0

    for line in lines:
        score = int(afinn.score(line))

        if score > 0:
            pos += 1
        elif score < 0:
            neg += 1
            neutral += 1

    n = len(lines)
    pos_index.append(pos / n)
    neg_index.append(neg / n)
    neutral_index.append(neutral / n)

#unsupervised-learning #sentiment-analysis #book-analysis #text-analysis #data-science