April 2017
Intermediate to advanced
532 pages
12h 39m
English
A common step in text processing and tokenization is stemming. This is the conversion of whole words to a base form (called a word stem). For example, plurals might be converted to singular (dogs becomes dog), and forms such as walking and walker might become walk. Stemming can become quite complex and is typically handled with specialized NLP or search engine software (such as NLTK, OpenNLP, and Lucene, for example). We will ignore stemming for the purpose of our example here.
Read now
Unlock full access