Latent semantical analysis

Typically, latent semantical analysis (LSA) is applied to text after it has been processed by TfidfVectorizer or CountVectorizer. Compared to PCA, it applies SVD to the input dataset (which is usually a sparse matrix), producing semantic sets of words that are usually associated with the same concept. This is why LSA is used when the features are homogeneous (that is, all the words in the documents) and are present in large numbers.

An example of the same in Python with text and TfidfVectorizer is as follows. The output shows part of the content of a latent vector:

In: from sklearn.datasets import fetch_20newsgroups    categories = ['sci.med', 'sci.space'] twenty_sci_news = fetch_20newsgroups(categories=categories) ...

Get Python Data Science Essentials - Third Edition now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.