O'Reilly logo

Practical Predictive Analytics by Ralph Winters

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Creating the test DTM

Now let's switch over to the test data set. First, take an identical sample size as was taken for the training data, and repeat the procedure starting with creating the term document matrix on the sample:

 OnlineRetail.test <- OnlineRetail.test[1:sample.size, ] dtMatrix.test <- create_matrix(OnlineRetail.test$Description, minDocFreq = 1,  removeNumbers = TRUE, minWordLength = 4, removeStopwords = TRUE, removePunctuation = TRUE,  stemWords = FALSE, weighting = weightTf)

As we did before, remove sparse terms from the matrix. Then, use the dim() function to see how many non-sparse terms remain:

dtMatrix.test <- removeSparseTerms(dtMatrix.test, 0.99) dim(dtMatrix.test) # reduced to 61 terms > [1] 10000 61

Take the first ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required