Dylan  Iqbal

Dylan Iqbal

1560827802

How to make your own sentiment analyzer using Python and Google’s Natural Language API

Imagine you are a product owner who wants to know what people are saying about your product in social media. Maybe your company launched a new product and you want to know how people reacted to it. You might want to use a sentiment analyzer like MonkeyLearn or Talkwalker. But wouldn’t it be cool if we could make our own sentiment analyzer? Let’s make it then!

In this tutorial, we are going to make a Telegram Bot that will do the sentiment analysis of tweets related to the keyword that we define.

Getting started

1. Install the libraries

We are going to use tweepy to gather the tweet data. We will use nltk to help us clean the tweets. Google Natural Language API will do the sentiment analysis. python-telegram-bot will send the result through Telegram chat.

pip3 install tweepy nltk google-cloud-language python-telegram-bot

2. Get Twitter API Keys

To be able to gather the tweets from Twitter, we need to create a developer account to get the Twitter API Keys first.

Go to Twitter Developer website, and create an account if you don’t have one.

Open Apps page, click “Create an app”, fill out the form and click “Create”.

Click on “Keys and tokens” tab, copy the API Key and API Secret Key in the “Consumer API keys” section.

Click the “Create” button under “Access token & access token secret” section. Copy the Access Token and Access Token Secret that have been generated.

Great! Now you should have four keys — API Key, API Secret Key, Access Token, and Access Token Secret. Save those keys for later use.

3. Enable Google Natural Language API

We need to enable the Google Natural Language API first if we want to use the service.

Go to Google Developers Console and create a new project (or select the one you have).

In the project dashboard, click “ENABLE APIS AND SERVICES”, and search for Cloud Natural Language API.

Click “ENABLE” to enable the API.

4. Create service account key

If we want to use Google Cloud services like Google Natural Language, we need a service account key. This is like our credential to use Google’s services.

Go to Google Developers Console, click “Credentials” tab, choose “Create credentials” and click “Service account key”.

Choose “App Engine default service account” and JSON as key type, then click “Create”.

There is a .json file that will be automatically downloaded, name it creds.json.

Set the GOOGLE_APPLICATION_CREDENTIALS with the path of our creds.json file in the terminal.

export GOOGLE_APPLICATION_CREDENTIALS='[PATH_TO_CREDS.JSON]'

If everything is good, then it’s time to write our program.

Write the program

This program will gather all the tweets containing the defined keyword in the last 24 hours with a maximum of 50 tweets. Then it will analyze the tweets’ sentiments one by one. We will send the result (average sentiment score) through Telegram chat.

This is a simple workflow of our program.

connect to the Twitter API -> search tweets based on the keyword -> clean all of the tweets -> get tweet’s sentiment score -> send the result
Let’s make a single function to define each flow.

1. Connect to the Twitter API

The first thing that we need to do is gather the tweets’ data, so we have to connect to the Twitter API first.

Import the tweepy library.

import tweepy

Define the keys that we generated earlier.

ACC_TOKEN = 'YOUR_ACCESS_TOKEN'
ACC_SECRET = 'YOUR_ACCESS_TOKEN_SECRET'
CONS_KEY = 'YOUR_CONSUMER_API_KEY'
CONS_SECRET = 'YOUR_CONSUMER_API_SECRET_KEY'

Make a function called authentication to connect to the API, with four parameters which are all of the keys.

def authentication(cons_key, cons_secret, acc_token, acc_secret):
    auth = tweepy.OAuthHandler(cons_key, cons_secret)
    auth.set_access_token(acc_token, acc_secret)
    api = tweepy.API(auth)
    return api

2. Search the tweets

We can search the tweets with two criteria, based on time or quantity. If it’s based on time, we define the time interval and if it’s based on quantity, we define the total tweets that we want to gather. Since we want to gather the tweets from the last 24 hours with maximum tweets of 50, we will use both of the criteria.

Since we want to gather the tweets from the last 24 hours, let’s take yesterday’s date as our time parameter.

from datetime import datetime, timedelta

today_datetime = datetime.today().now()
yesterday_datetime = today_datetime - timedelta(days=1)
today_date = today_datetime.strftime('%Y-%m-%d')
yesterday_date = yesterday_datetime.strftime('%Y-%m-%d')

Connect to the Twitter API using a function we defined before.

api = authentication(CONS_KEY,CONS_SECRET,ACC_TOKEN,ACC_SECRET)

Define our search parameters. q is where we define our keyword, since is the start date for our search, result_type='recent' means we are going to take the newest tweets, lang='en' is going to take the English tweets only, and items(total_tweets) is where we define the maximum tweets that we are going to take.

search_result = tweepy.Cursor(api.search, 
                              q=keyword, 
                              since=yesterday_date,
                              result_type='recent', 
                              lang='en').items(total_tweets)

Wrap those codes in a function called search_tweets with keyword and total_tweets as the parameters.

def search_tweets(keyword, total_tweets):
    today_datetime = datetime.today().now()
    yesterday_datetime = today_datetime - timedelta(days=1)
    today_date = today_datetime.strftime('%Y-%m-%d')
    yesterday_date = yesterday_datetime.strftime('%Y-%m-%d')
    api = authentication(CONS_KEY,CONS_SECRET,ACC_TOKEN,ACC_SECRET)
    search_result = tweepy.Cursor(api.search, 
                                  q=keyword, 
                                  since=yesterday_date, 
                                  result_type='recent', 
                                  lang='en').items(total_tweets)
    return search_result

3. Clean the tweets

Before we analyze the tweets sentiment, we need to clean the tweets a little bit so the Google Natural Language API can identify them better.

We will use the nltk and regex libraries to help us in this process.

import re

from nltk.tokenize import WordPunctTokenizer

We remove the username in every tweet, so basically we can remove everything that begins with @ and we use regex to do it.

user_removed = re.sub(r'@[A-Za-z0-9]+','',tweet.decode('utf-8'))

We also remove links in every tweet.

link_removed = re.sub('https?://[A-Za-z0-9./]+','',user_removed)

Numbers are also deleted from all of the tweets.

number_removed = re.sub('[^a-zA-Z]',' ',link_removed)

The last, convert all of the characters into lower space, then remove every unnecessary space.

lower_case_tweet = number_removed.lower()
tok = WordPunctTokenizer()
words = tok.tokenize(lower_case_tweet)
clean_tweet = (' '.join(words)).strip()

Wrap those codes into a function called clean_tweets with tweet as our parameter.

def clean_tweets(tweet):
    user_removed = re.sub(r'@[A-Za-z0-9]+','',tweet.decode('utf-8'))
    link_removed = re.sub('https?://[A-Za-z0-9./]+','',user_removed)
    number_removed = re.sub('[^a-zA-Z]', ' ', link_removed)
    lower_case_tweet= number_removed.lower()
    tok = WordPunctTokenizer()
    words = tok.tokenize(lower_case_tweet)
    clean_tweet = (' '.join(words)).strip()
    return clean_tweet

4. Get tweet’s sentiment

To be able to get a tweet’s sentiment, we will use Google Natural Language API.

The API provides Sentiment Analysis, Entities Analysis, and Syntax Analysis. We will only use the Sentiment Analysis for this tutorial.

In Google’s Sentiment Analysis, there are score and magnitude. Score is the score of the sentiment ranges from -1.0 (very negative) to 1.0 (very positive). Magnitude is the strength of sentiment and ranges from 0 to infinity.

For the sake of simplicity of this tutorial, we will only consider the score. If you are thinking of doing deep NLP analysis, you should consider the magnitude too.

Import the Google Natural Language library.

from google.cloud import language
from google.cloud.language import enums
from google.cloud.language import types

Make a function called get_sentiment_score which takes tweet as the parameter, and returns the sentiment score.

def get_sentiment_score(tweet):
    client = language.LanguageServiceClient()
    document = types\
               .Document(content=tweet,
                         type=enums.Document.Type.PLAIN_TEXT)
    sentiment_score = client\
                      .analyze_sentiment(document=document)\
                      .document_sentiment\
                      .score
    return sentiment_score

5. Analyze the tweets

Let’s make a function that will loop the list of tweets we get from search_tweets function and get the sentiment’s score of every tweet using get_sentiment_score function. Then we’ll calculate the average. The average score will determine whether the given keyword has a positive, neutral, or negative sentiment.

Define score equals to 0 , then use search_tweets function to get the tweets related to the keyword that we define.

score = 0
tweets = search_tweets(keyword, total_tweets)

Loop through the list of tweets, and do the cleaning using clean_tweets function that we created before.

for tweet in tweets:
    cleaned_tweet = clean_tweets(tweet.text.encode('utf-8'))

Get the sentiment score using get_sentiment_score function, and increment the score by adding sentiment_score.

for tweet in tweets:
    cleaned_tweet = clean_tweets(tweet.text.encode('utf-8'))
    sentiment_score = get_sentiment_score(cleaned_tweet)
    score += sentiment_score

Let’s print out each tweet and its sentiment so we can see the progress detail in the terminal.

for tweet in tweets:
    cleaned_tweet = clean_tweets(tweet.text.encode('utf-8'))
    sentiment_score = get_sentiment_score(cleaned_tweet)
    score += sentiment_score
    print('Tweet: {}'.format(cleaned_tweet))
    print('Score: {}\n'.format(sentiment_score))

Calculate the average score and pass it to final_score variable. Wrap all of the codes into analyze_tweets function, with keyword and total_tweets as the parameters.

def analyze_tweets(keyword, total_tweets):
    score = 0
    tweets = search_tweets(keyword, total_tweets)
    for tweet in tweets:
        cleaned_tweet = clean_tweets(tweet.text.encode('utf-8'))
        sentiment_score = get_sentiment_score(cleaned_tweet)
        score += sentiment_score
        print('Tweet: {}'.format(cleaned_tweet))
        print('Score: {}\n'.format(sentiment_score))
    final_score = round((score / float(total_tweets)),2)
    return final_score

6. Send the tweet’s sentiment score

Let’s make the last function in the workflow. This function will takes user’s keyword and calculate the average sentiment’s score. Then we’ll send it through Telegram Bot.

Get the keyword from the user.

keyword = update.message.text

Use analyze_tweets function to get the final score, keyword as our parameter, and set the total_tweets = 50 since we want to gather 50 tweets.

final_score = analyze_tweets(keyword, 50)

We define whether a given score is considered negative, neutral, or positive using Google’s score range, as we see in the image below.

if final_score <= -0.25:
    status = 'NEGATIVE | ❌'
elif final_score <= 0.25:
    status = 'NEUTRAL | 🔶'
else:
    status = 'POSITIVE | ✅'

Lastly, send the final_score and the status through Telegram Bot.

bot.send_message(chat_id=update.message.chat_id,
                 text='Average score for '
                       + str(keyword) 
                       + ' is ' 
                       + str(final_score) 
                       + ' | ' + status)

Wrap the codes into a function called send_the_result.

def send_the_result(bot, update):
    keyword = update.message.text
    final_score = analyze_tweets(keyword, 50)
    if final_score <= -0.25:
        status = 'NEGATIVE | ❌'
    elif final_score <= 0.25:
        status = 'NEUTRAL | 🔶'
    else:
        status = 'POSITIVE | ✅'
    bot.send_message(chat_id=update.message.chat_id,
                     text='Average score for '
                           + str(keyword) 
                           + ' is ' 
                           + str(final_score) 
                           + ' | ' + status)

7. Main program

Lastly, create another function called main to run our program. Don’t forget to change YOUR_TOKEN to your bot’s token.

from telegram.ext import Updater, MessageHandler, Filters

def main():
    updater = Updater('YOUR_TOKEN')
    dp = updater.dispatcher
    dp.add_handler(MessageHandler(Filters.text, send_the_result))
    updater.start_polling()
    updater.idle()

if __name__ == '__main__':
    main()

In the end, your code should look like this

import tweepy
import re

from telegram.ext import Updater, MessageHandler, Filters
from google.cloud import language
from google.cloud.language import enums
from google.cloud.language import types
from datetime import datetime, timedelta
from nltk.tokenize import WordPunctTokenizer

ACC_TOKEN = 'YOUR_ACCESS_TOKEN'
ACC_SECRET = 'YOUR_ACCESS_TOKEN_SECRET'
CONS_KEY = 'YOUR_CONSUMER_API_KEY'
CONS_SECRET = 'YOUR_CONSUMER_API_SECRET_KEY'

def authentication(cons_key, cons_secret, acc_token, acc_secret):
    auth = tweepy.OAuthHandler(cons_key, cons_secret)
    auth.set_access_token(acc_token, acc_secret)
    api = tweepy.API(auth)
    return api

def search_tweets(keyword, total_tweets):
    today_datetime = datetime.today().now()
    yesterday_datetime = today_datetime - timedelta(days=1)
    today_date = today_datetime.strftime('%Y-%m-%d')
    yesterday_date = yesterday_datetime.strftime('%Y-%m-%d')
    api = authentication(CONS_KEY,CONS_SECRET,ACC_TOKEN,ACC_SECRET)
    search_result = tweepy.Cursor(api.search, 
                                  q=keyword, 
                                  since=yesterday_date, 
                                  result_type='recent', 
                                  lang='en').items(total_tweets)
    return search_result

def clean_tweets(tweet):
    user_removed = re.sub(r'@[A-Za-z0-9]+','',tweet.decode('utf-8'))
    link_removed = re.sub('https?://[A-Za-z0-9./]+','',user_removed)
    number_removed = re.sub('[^a-zA-Z]', ' ', link_removed)
    lower_case_tweet= number_removed.lower()
    tok = WordPunctTokenizer()
    words = tok.tokenize(lower_case_tweet)
    clean_tweet = (' '.join(words)).strip()
    return clean_tweet

def get_sentiment_score(tweet):
    client = language.LanguageServiceClient()
    document = types\
               .Document(content=tweet,
                         type=enums.Document.Type.PLAIN_TEXT)
    sentiment_score = client\
                      .analyze_sentiment(document=document)\
                      .document_sentiment\
                      .score
    return sentiment_score

def analyze_tweets(keyword, total_tweets):
    score = 0
    tweets = search_tweets(keyword,total_tweets)
    for tweet in tweets:
        cleaned_tweet = clean_tweets(tweet.text.encode('utf-8'))
        sentiment_score = get_sentiment_score(cleaned_tweet)
        score += sentiment_score
        print('Tweet: {}'.format(cleaned_tweet))
        print('Score: {}\n'.format(sentiment_score))
    final_score = round((score / float(total_tweets)),2)
    return final_score

def send_the_result(bot, update):
    keyword = update.message.text
    final_score = analyze_tweets(keyword, 50)
    if final_score <= -0.25:
        status = 'NEGATIVE | ❌'
    elif final_score <= 0.25:
        status = 'NEUTRAL | 🔶'
    else:
        status = 'POSITIVE | ✅'
    bot.send_message(chat_id=update.message.chat_id,
                     text='Average score for '
                           + str(keyword) 
                           + ' is ' 
                           + str(final_score) 
                           + ' | ' + status)

def main():
    updater = Updater('YOUR_TOKEN')
    dp = updater.dispatcher
    dp.add_handler(MessageHandler(Filters.text, send_the_result))
    updater.start_polling()
    updater.idle()

if __name__ == '__main__':
    main()

Save the file and name it main.py, then run the program.

python3 main.py

Go to your telegram bot by accessing this URL: [[https://telegram.me/YOUR_BOT_USERNAME](https://telegram.me/YOUR_BOT_USERNAME.)](https://telegram.me/YOUR_BOT_USERNAME](https://telegram.me/YOUR_BOT_USERNAME.) "https://telegram.me/YOUR_BOT_USERNAME](https://telegram.me/YOUR_BOT_USERNAME.)"). Type any product, person name, or whatever you want and send it to your bot. If everything runs, there should be a detailed sentiment score for each tweet in the terminal. The bot will reply with the average sentiment score.

The pictures below are an example if I type valentino rossi and send it to the bot.

If you managed to follow the steps until the end of this tutorial, that’s awesome! You have your sentiment analyzer now, how cool is that!?

You can also check out my GitHub to get the code. Please do not hesitate to connect and leave a message in my Linkedin profile if you want to ask about anything.

Please leave a comment if you think there are any errors in my code or writing.

Thank you and good luck! :)

#python

What is GEEK

Buddha Community

How to make your own sentiment analyzer using Python and Google’s Natural Language API
Ray  Patel

Ray Patel

1619518440

top 30 Python Tips and Tricks for Beginners

Welcome to my Blog , In this article, you are going to learn the top 10 python tips and tricks.

1) swap two numbers.

2) Reversing a string in Python.

3) Create a single string from all the elements in list.

4) Chaining Of Comparison Operators.

5) Print The File Path Of Imported Modules.

6) Return Multiple Values From Functions.

7) Find The Most Frequent Value In A List.

8) Check The Memory Usage Of An Object.

#python #python hacks tricks #python learning tips #python programming tricks #python tips #python tips and tricks #python tips and tricks advanced #python tips and tricks for beginners #python tips tricks and techniques #python tutorial #tips and tricks in python #tips to learn python #top 30 python tips and tricks for beginners

Ray  Patel

Ray Patel

1619510796

Lambda, Map, Filter functions in python

Welcome to my Blog, In this article, we will learn python lambda function, Map function, and filter function.

Lambda function in python: Lambda is a one line anonymous function and lambda takes any number of arguments but can only have one expression and python lambda syntax is

Syntax: x = lambda arguments : expression

Now i will show you some python lambda function examples:

#python #anonymous function python #filter function in python #lambda #lambda python 3 #map python #python filter #python filter lambda #python lambda #python lambda examples #python map

Shardul Bhatt

Shardul Bhatt

1626775355

Why use Python for Software Development

No programming language is pretty much as diverse as Python. It enables building cutting edge applications effortlessly. Developers are as yet investigating the full capability of end-to-end Python development services in various areas. 

By areas, we mean FinTech, HealthTech, InsureTech, Cybersecurity, and that's just the beginning. These are New Economy areas, and Python has the ability to serve every one of them. The vast majority of them require massive computational abilities. Python's code is dynamic and powerful - equipped for taking care of the heavy traffic and substantial algorithmic capacities. 

Programming advancement is multidimensional today. Endeavor programming requires an intelligent application with AI and ML capacities. Shopper based applications require information examination to convey a superior client experience. Netflix, Trello, and Amazon are genuine instances of such applications. Python assists with building them effortlessly. 

5 Reasons to Utilize Python for Programming Web Apps 

Python can do such numerous things that developers can't discover enough reasons to admire it. Python application development isn't restricted to web and enterprise applications. It is exceptionally adaptable and superb for a wide range of uses.

Robust frameworks 

Python is known for its tools and frameworks. There's a structure for everything. Django is helpful for building web applications, venture applications, logical applications, and mathematical processing. Flask is another web improvement framework with no conditions. 

Web2Py, CherryPy, and Falcon offer incredible capabilities to customize Python development services. A large portion of them are open-source frameworks that allow quick turn of events. 

Simple to read and compose 

Python has an improved sentence structure - one that is like the English language. New engineers for Python can undoubtedly understand where they stand in the development process. The simplicity of composing allows quick application building. 

The motivation behind building Python, as said by its maker Guido Van Rossum, was to empower even beginner engineers to comprehend the programming language. The simple coding likewise permits developers to roll out speedy improvements without getting confused by pointless subtleties. 

Utilized by the best 

Alright - Python isn't simply one more programming language. It should have something, which is the reason the business giants use it. Furthermore, that too for different purposes. Developers at Google use Python to assemble framework organization systems, parallel information pusher, code audit, testing and QA, and substantially more. Netflix utilizes Python web development services for its recommendation algorithm and media player. 

Massive community support 

Python has a steadily developing community that offers enormous help. From amateurs to specialists, there's everybody. There are a lot of instructional exercises, documentation, and guides accessible for Python web development solutions. 

Today, numerous universities start with Python, adding to the quantity of individuals in the community. Frequently, Python designers team up on various tasks and help each other with algorithmic, utilitarian, and application critical thinking. 

Progressive applications 

Python is the greatest supporter of data science, Machine Learning, and Artificial Intelligence at any enterprise software development company. Its utilization cases in cutting edge applications are the most compelling motivation for its prosperity. Python is the second most well known tool after R for data analytics.

The simplicity of getting sorted out, overseeing, and visualizing information through unique libraries makes it ideal for data based applications. TensorFlow for neural networks and OpenCV for computer vision are two of Python's most well known use cases for Machine learning applications.

Summary

Thinking about the advances in programming and innovation, Python is a YES for an assorted scope of utilizations. Game development, web application development services, GUI advancement, ML and AI improvement, Enterprise and customer applications - every one of them uses Python to its full potential. 

The disadvantages of Python web improvement arrangements are regularly disregarded by developers and organizations because of the advantages it gives. They focus on quality over speed and performance over blunders. That is the reason it's a good idea to utilize Python for building the applications of the future.

#python development services #python development company #python app development #python development #python in web development #python software development

Art  Lind

Art Lind

1602990000

Get Google Trends using Python

In this post, we will show how we can use Python to get data from Google Trends. Let’s have a look at the top trending searches for today in the US (14th of March, 2020). As we can see, the top search is about Coronavirus tips with more than 2M searches, and at the 7th position is Rick Pitino with around 100K searches.

Image for post

Python package for getting the Google Trends

We will use the pytrends package which is an unofficial API for Google Trends which allows a simple interface for automating downloading of reports from Google Trends. The main feature is to allow the script to login to Google on your behalf to enable a higher rate limit. At this point, I want to mention that I couldn’t use this package and I created a new anaconda environment installing the pandas 0.25 version.

You can install the pytrends package with pip:

pip install pytrends

#google-trends #how-to-use-google-trend #google #google-api #python

Sival Alethea

Sival Alethea

1624302000

APIs for Beginners - How to use an API (Full Course / Tutorial)

What is an API? Learn all about APIs (Application Programming Interfaces) in this full tutorial for beginners. You will learn what APIs do, why APIs exist, and the many benefits of APIs. APIs are used all the time in programming and web development so it is important to understand how to use them.

You will also get hands-on experience with a few popular web APIs. As long as you know the absolute basics of coding and the web, you’ll have no problem following along.
⭐️ Unit 1 - What is an API
⌨️ Video 1 - Welcome (0:00:00)
⌨️ Video 2 - Defining Interface (0:03:57)
⌨️ Video 3 - Defining API (0:07:51)
⌨️ Video 4 - Remote APIs (0:12:55)
⌨️ Video 5 - How the web works (0:17:04)
⌨️ Video 6 - RESTful API Constraint Scavenger Hunt (0:22:00)

⭐️ Unit 2 - Exploring APIs
⌨️ Video 1 - Exploring an API online (0:27:36)
⌨️ Video 2 - Using an API from the command line (0:44:30)
⌨️ Video 3 - Using Postman to explore APIs (0:53:56)
⌨️ Video 4 - Please please Mr. Postman (1:03:33)
⌨️ Video 5 - Using Helper Libraries (JavaScript) (1:14:41)
⌨️ Video 6 - Using Helper Libraries (Python) (1:24:40)

⭐️ Unit 3 - Using APIs
⌨️ Video 1 - Introducing the project (1:34:18)
⌨️ Video 2 - Flask app (1:36:07)
⌨️ Video 3 - Dealing with API Limits (1:50:00)
⌨️ Video 4 - JavaScript Single Page Application (1:54:27)
⌨️ Video 5 - Moar JavaScript and Recap (2:07:53)
⌨️ Video 6 - Review (2:18:03)
📺 The video in this post was made by freeCodeCamp.org
The origin of the article: https://www.youtube.com/watch?v=GZvSYJDk-us&list=PLWKjhJtqVAblfum5WiQblKPwIbqYXkDoC&index=5
🔥 If you’re a beginner. I believe the article below will be useful to you ☞ What You Should Know Before Investing in Cryptocurrency - For Beginner
⭐ ⭐ ⭐The project is of interest to the community. Join to Get free ‘GEEK coin’ (GEEKCASH coin)!
☞ **-----CLICK HERE-----**⭐ ⭐ ⭐
Thanks for visiting and watching! Please don’t forget to leave a like, comment and share!

#apis #apis for beginners #how to use an api #apis for beginners - how to use an api #application programming interfaces #learn all about apis