Mastering Machine Learning with R - Second Edition

Word frequency and topic models

As we have everything set up in the document-term matrix, we can move on to exploring word frequencies by creating an object with the column sums, sorted in descending order. It is necessary to use as.matrix() in the code to sum the columns. The default order is ascending, so putting - in front of freq will change it to descending:

    > freq <- colSums(as.matrix(dtm))        > ord <- order(-freq)

We will examine the head and tail of the object with the following code:

    > freq[head(ord)]        new  america  people   jobs    now  years         193      174     168    163    157    148         > freq[tail(ord)]        wright written yearold youngest youngstown zero              2       2       2        2          2    2

The most frequent word is new and, as you might expect, the president mentions america frequently. ...