The most common limitation of count vectorizing is that the algorithm doesn't consider the whole corpus while considering the frequency of each token. The goal of vectorizing is normally preparing the data for a classifier; therefore it's necessary to avoid features that are present very often, because their information decreases when the number of global occurrences increases. For example, in a corpus about a sport, the word match could be present in a huge number of documents; therefore it's almost useless as a classification feature. To address this issue, we need a different approach. If we have a corpus C with n documents, we define term-frequency, the number of times a token occurs in a document, as:
We define