O'Reilly logo

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Text Mining with R

Book Description

Much of the data available today is unstructured and text-heavy, making it challenging for analysts to apply their usual data wrangling and visualization tools. With this practical book, you’ll explore text-mining techniques with tidytext, a package that authors Julia Silge and David Robinson developed using the tidy principles behind R packages like ggraph and dplyr. You’ll learn how tidytext and other tidy tools in R can make text analysis easier and more effective.

The authors demonstrate how treating text as data frames enables you to manipulate, summarize, and visualize characteristics of text. You’ll also learn how to integrate natural language processing (NLP) into effective workflows. Practical code examples and data explorations will help you generate real insights from literature, news, and social media.

  • Learn how to apply the tidy text format to NLP
  • Use sentiment analysis to mine the emotional content of text
  • Identify a document’s most important terms with frequency measurements
  • Explore relationships and connections between words with the ggraph and widyr packages
  • Convert back and forth between R’s tidy and non-tidy text formats
  • Use topic modeling to classify document collections into natural groups
  • Examine case studies that compare Twitter archives, dig into NASA metadata, and analyze thousands of Usenet messages

Table of Contents

  1. Preface
    1. Outline
    2. Topics This Book Does Not Cover
    3. About This Book
    4. Conventions Used in This Book
    5. Using Code Examples
    6. O’Reilly Safari
    7. How to Contact Us
    8. Acknowledgements
  2. 1. The Tidy Text Format
    1. Contrasting Tidy Text with Other Data Structures
    2. The unnest_tokens Function
    3. Tidying the Works of Jane Austen
    4. The gutenbergr Package
    5. Word Frequencies
    6. Summary
  3. 2. Sentiment Analysis with Tidy Data
    1. The sentiments Dataset
    2. Sentiment Analysis with Inner Join
    3. Comparing the Three Sentiment Dictionaries
    4. Most Common Positive and Negative Words
    5. Wordclouds
    6. Looking at Units Beyond Just Words
    7. Summary
  4. 3. Analyzing Word and Document Frequency: tf-idf
    1. Term Frequency in Jane Austen’s Novels
    2. Zipf’s Law
    3. The bind_tf_idf Function
    4. A Corpus of Physics Texts
    5. Summary
  5. 4. Relationships Between Words: N-grams and Correlations
    1. Tokenizing by N-gram
      1. Counting and Filtering N-grams
      2. Analyzing Bigrams
      3. Using Bigrams to Provide Context in Sentiment Analysis
      4. Visualizing a Network of Bigrams with ggraph
      5. Visualizing Bigrams in Other Texts
    2. Counting and Correlating Pairs of Words with the widyr Package
      1. Counting and Correlating Among Sections
      2. Examining Pairwise Correlation
    3. Summary
  6. 5. Converting to and from Nontidy Formats
    1. Tidying a Document-Term Matrix
      1. Tidying DocumentTermMatrix Objects
      2. Tidying dfm Objects
    2. Casting Tidy Text Data into a Matrix
    3. Tidying Corpus Objects with Metadata
      1. Example: Mining Financial Articles
    4. Summary
  7. 6. Topic Modeling
    1. Latent Dirichlet Allocation
      1. Word-Topic Probabilities
      2. Document-Topic Probabilities
    2. Example: The Great Library Heist
      1. LDA on Chapters
      2. Per-Document Classification
      3. By-Word Assignments: augment
    3. Alternative LDA Implementations
    4. Summary
  8. 7. Case Study: Comparing Twitter Archives
    1. Getting the Data and Distribution of Tweets
    2. Word Frequencies
    3. Comparing Word Usage
    4. Changes in Word Use
    5. Favorites and Retweets
    6. Summary
  9. 8. Case Study: Mining NASA Metadata
    1. How Data Is Organized at NASA
      1. Wrangling and Tidying the Data
      2. Some Initial Simple Exploration
    2. Word Co-ocurrences and Correlations
      1. Networks of Description and Title Words
      2. Networks of Keywords
    3. Calculating tf-idf for the Description Fields
      1. What Is tf-idf for the Description Field Words?
      2. Connecting Description Fields to Keywords
    4. Topic Modeling
      1. Casting to a Document-Term Matrix
      2. Ready for Topic Modeling
      3. Interpreting the Topic Model
      4. Connecting Topic Modeling with Keywords
    5. Summary
  10. 9. Case Study: Analyzing Usenet Text
    1. Preprocessing
      1. Preprocessing Text
    2. Words in Newsgroups
      1. Finding tf-idf Within Newsgroups
      2. Topic Modeling
    3. Sentiment Analysis
      1. Sentiment Analysis by Word
      2. Sentiment Analysis by Message
      3. N-gram Analysis
    4. Summary
  11. Bibliography
  12. Index