Retrieving lemma, part-of-speech, and recognizing named entities from tokens using Stanford CoreNLP

Now that we know how to extract tokens or words from a given text, we will see how we can get different types of information from the tokens such as their lemmas, part-of-speech, and whether the token is a named entity.

The process of lemmatization group inflected forms of a word together so that they can be analyzed as a single text unit. This is similar to the process of stemming with a big difference that stemming does not consider context during its grouping. Therefore, lemmatization is particularly more useful for text data analysis than stemming but requires more computation power.

Part-of-speech tags of the tokens in an article or document ...

Get Java Data Science Cookbook now with the O’Reilly learning platform.

O’Reilly members experience live online training, plus books, videos, and digital content from nearly 200 publishers.