Hands-On Python Deep Learning for the Web

Stemming and lemmatization

Stemming and lemmatization are closely related terms in NLP, but with a slight but significant difference. The objective of both methods is to determine the root word that any given word originates from, such that any derivates of the root word can be matched to the root word in the dictionary.

Stemming is a rule-based process where the words are trimmed and sometimes appended with modifiers that indicate its root word. However, stemming might, at times, produce root words that don't exist in the human dictionary and so mean nothing to the human reader.

Lemmatization is the process of converting words to their lemma, or their root word, as given in the dictionary. So, the originally intended meaning of the word ...