Java: Data Science Made Easy

Text classification

Text Classification is a problem where given a collection of texts and labels, it trains a model that can predict these labels for new unseen text. So the settings here are usual for supervised learning, except that now we have text data.

There are many possible classification problems, as follows:

• Spam detection: This predicts whether an e-mail is spam or not
• Sentiment analysis: This predicts whether the sentiment of the text is positive or negative
• Language detection: Given a text, this detects its language

The general workflow for text classification is similar in almost all cases:

• We tokenize and vectorize the text
• Then we fit a linear classifier treating each token as a feature

As we know that if we vectorize ...