June 2020
Intermediate to advanced
382 pages
11h 39m
English
Content preview from 40 Algorithms Every Programmer Should Know
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
Start your free trial
Stopwords
After word-level tokenization, we have a list of words that are used in the text. Some of these words are common words that are expected to appear in almost every document. These words do not provide any additional insight into the documents that they appear in. These words are called stopwords. They are usually removed in the data-processing phase. Some examples of stopwords are was, we, and the.