Twitter Text Analysis in R

Helping communication practitioners get actionable insights through open data sources like Twitter. This article will focus on how can we use Twitter through R programming to extract valuable insights and communicate these findings to the relevant stakeholders using Tableau.

Open data sources are one of the best gifts for data scientists or analysts as they allow them to draw valuable insights for free, without having to worry about the data licenses. Twitter is one of the most popular social media application in the world as it’s free, and also allow users to tweet on any topics that come to their mind. This article will focus on how can we use Twitter through R programming to extract valuable insights and communicate these findings to the relevant stakeholders using Tableau.

Problem Statement

“How might we help the communication practitioners to get actionable insights from Twitter so that they can create more effective communication that caters to the needs & concerns of general public”

The chosen target user of this problem statement is the military communication practitioner, who is keen to understand what are the concerns the general public is having with regards to the military (i.e. National Service) in Singapore.

Characteristic of Twitter data source

For every data source, we can do a simple representation with the four ‘V’s of big data, mainly Volume, Velocity, Variety, and Veracity for a general understanding of this data source.

1. *Volume *— Scale of data
2. *Velocity *— Analysis of streaming data
3. *Variety *— Different forms of data
4. *Veracity *— Uncertainty of data

Connection to Twitter API

We can use twitteR library package in R Programming to access the Twitter API. Note that we will need to sign up for a Twitter developer account to access the API, as each user will be provided with a unique set of consumer key, consumer secret key, access token, and access secret key. Once we have set up the connection to Twitter API, we will request the tweets we want from Twitter by stating the search term (i.e. “National Service_”), the maximum number of tweets (i.e. _n = 1000), latitude longitude of Singapore with the search radius in miles (i.e. geocode = ‘1.3521, 103.8198, 279mi_’) and the language (i.e. _lang= ‘en’).

## import necessary library for Twitter API, data manipulation and text cleaning
library("twitteR")
library("ROAuth")
library("dplyr")
library("tidytext")
## Set up Twitter Connection
consumer_key <- '' ## removed due to confidentiality
consumer_secret<- '' ## removed due to confidentiality
access_token <- '' ## removed due to confidentiality
access_secret <- '' ## removed due to confidentiality
setup_twitter_oauth(consumer_key ,consumer_secret,access_token ,access_secret)
## extract english tweets using 'National Service' tweeted in Singapore, with retweets removed
tweets <- strip_retweets(searchTwitter('National Service',n=1000, geocode='1.3521,103.8198,279mi',lang='en'))
## print length of tweets
length(tweets)

Note that for the free version of Twitter public API, we are only able to request the last 7 days’ worth of tweets. As a result, the number of tweets we got in our request was just 17.

twitter open-data data-science text-mining sentiment-analysis

Data Science and Analytics market evolves to adapt to the constantly changing economic and business environments. Our latest survey report suggests that as the overall Data Science and Analytics market evolves to adapt to the constantly changing economic and business environments, data scientists and AI practitioners should be aware of the skills and tools that the broader community is working on. A good grip in these skills will further help data science enthusiasts to get the best jobs that various industries in their data science functions are offering.

Online Data Science Training in Noida at CETPA, best institute in India for Data Science Online Course and Certification. Call now at 9911417779 to avail 50% discount.

Data science is omnipresent to advanced statistical and machine learning methods. For whatever length of time that there is data to analyse, the need to investigate is obvious.

Let’s talk about Open Data : According to the International Open Data Charter(1), it defines open data as those digital data that are made available with the technical.

Pattern is an open-source python library and performs different NLP tasks. It is mostly used for text processing due to various functionalities it provides. Text Processing mainly requires Natural Language Processing( NLP), which is processing the data in a useful way so that the machine can understand the Human Language with the help of an application or product. Using NLP we can derive some information from the textual data such as sentiment, polarity, etc.