Scala and Spark for Big Data Analytics

Topic modeling

Topic modeling is a useful technique for detecting the topics or themes in a corpus of documents. This is an unsupervised algorithm, which can find themes in a set of documents. An example is to detect topics covered in a news article. Another example is to detect the ideas in a patent application.

The latent dirichlet allocation (LDA) is a popular clustering model using unsupervised algorithm, while latent semantic analysis (LSA) uses a probabilistic model on co-occurrence data.