In this recipe, we will explore LSA utilizing a data dump of articles from Wikipedia. LSA translates into analyzing a corpus of documents to find hidden meaning or concepts in those documents.
In the first recipe of this chapter, we covered the basics of the TF (that is, term frequency) technique. In this recipe, we use HashingTF for calculating TF and use IDF to fit a model into the calculated TF. At its core, LSA uses singular value decomposition (SVD) on the term frequency document to reduce dimensionality and therefore extract the most important concepts. There are other cleanup steps that we need to do (for example, stop words and stemming) that will clean up the bag of ...
