Friday, 12 August 2016

Data Scraping web application with R shiny

Every time we tweet about certain topics of our own interest and appeal. Sometimes this tweets are liked, re-tweeted and even disliked depending on other people's view. However, when the tweets are about interesting topics such as politics, businesses, sports or even religion, it becomes trending meme and can be analysed to retrieve the sentiments. Sentiment is just a view of an attitude towards a situation or event.

Prior to the determination of any sentiments or sentimental analysis, one must scrape the tweets and import the data to the analytical tool of their choice. For me, I am an R programmer and therefore will show this in R shiny. I will be explaining how to set up the R shiny web app like bellow.
In order to grasp more on this article you can view the web app here https://oscardoe.shinyapps.io/TweetFisher/



Main topics of the web app

User interface (ui.r)
Loading the required packages.
This work requires specific packages and therefore they must be installed and then loaded before starting to code or copy paste the code.
library(shiny)
library(twitteR)
library(shinythemes)
library(shiny)
library(twitteR)
library(wordcloud)
library(tm)
library(stringr)
library(rsconnect)
library(dashboard)
library(shinydashboard)

Specifying the tweet search terms.
This program is using the searchTwitter ion and therefore specific search terms can be specified. The search terms are the term you want to find its tweets. This can be a hashtag, user or just any name.
Location of tweets.
Number of tweets
Date range of the hashtag
 shinyUI(dashboardPage(
  #creating the title of the web app
   dashboardHeader(title ="Tweets Fisher"),
 #developing the interface
        dashboardSidebar(
          # requesting for a term to be searched
        textInput("term", "Enter a term", "Kenya"),
        #inputing the number of tweets you want scraped
               sliderInput("cant", "Select a number of tweets",min=5,max=1500, value = 50),
        #selecting the language of the tweets
               radioButtons("lang", "Select the language", c(
                 "English"="en",
                 "Spanish"="es")),
        #selecting the location of the tweets
              selectizeInput("location","Enter your search area",choices=list(
                                                                              kenya=("-1.2920659,36.82196,45mi"),
                                                                              kuwait=("29.3454657,47.9969453,80mi"))),
textInput("date1","Enter start date","2016-07-24"),
               textInput("date2","Enter End date","2016-07-25"),
                               submitButton(text = "Run")),
             
dashboardBody(
    h4("Last 15 tweets on your entered term"),
    tableOutput("table")),
  tabPanel("ANALYSIS",
           mainPanel("Follow me http://doenyamanga.blogspot.co.ke/"))
))


reqURL<-"https://api.twitter.com/oauth/request_token"
access_token<-"856193844-4wjipqL5kiiC1xuq3MjbGcVC9XMjS0AFYvvYrl5y"
access_token_secret<-"hr0ZtyZvbiKfA4iTx4k4ujVDjW69g5v4rYHzlXEx8b6vi"
apiKey<-"PLTb0HBxqdOyDsMIzBolLlRtv"
apiSecret<-"ypCPljAfCmp3GmXcthrbAxXqx9PgEYz8p8OElJjgvEZhsJVM0n"
my_oauth<-setup_twitter_oauth(apiKey,apiSecret,access_token,access_token_secret)

Server side (server.r)

library(shiny)
shinyServer(function(input,output){
  rawData<-reactive({
    tweets<-searchTwitter(input$term,n=input$cant,lang=input$lang,since=input$date1,until=input$date2,geocode=input$location)
    twListToDF(tweets)
    })
   output$table<-renderTable({
    head(rawData()[1],n=15)
  })
   #removing twitter handlers
   #rawDataa<-reactive({str_replace_all(rawData(),"@\\w+","")})
   #removing emojis
    tweets1<-reactive({iconv(rawDataa(),'UTF-8','ASCII')})
    wordCorpus<-reactive({Corpus(VectorSource(tweets1()))})
  tdm<-reactive({as.matrix(tweets1())})
    output$wordcl<-renderPlot({
      wordcloud(tdm(),random.order=T,colors=brewer.pal(8,"Dark2"))
    })
   

})

No comments:

Post a Comment