The Handbook of NLP with Gensim

11

LDA Modeling

In Chapter 9, Understanding Discrete Distribution, and Chapter 10, Latent Dirichlet Allocation, we learned about the Dirichlet distribution and the structure of the LDA model, which equipped you with a sound theoretical background. In this chapter, we will go over the code to build an LDA model. I will touch upon the key decisions in building an LDA model, including text preprocessing, model hyperparameters, the determination of the number of topics, and how to use the model in production to score new documents. This is a special feature in this book that focuses on model implementation in production. In short, we will cover the following topics:

• Text preprocessing
• Experimenting with LDA modeling
• Building LDA models with a different ...