Working with tidy text

For this, we will use the tidytext package. This package is built on the philosophy of tidy data, introduced by Hadley Wickham in his 2014 paper (https://www.jstatsoft.org/article/view/v059i10). A dataset is tidy if the following three conditions are satisfied:

  • Each variable is a column
  • Each observation is a row
  • Each type of observational unit is a table

The tidytext package helps us turn our text into tidy form, by putting one token per row. Let's start by loading dplyr and tidytext. If you don't have tidytext, install it first using install.packages("tidytext").

Load the packages and let's transform our text into a data frame:

library(tidytext)library(dplyr)text_df <- data_frame(line = 1:4, text = text)

The unnest_tokens ...

Get Deep Learning with R for Beginners now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.