O'Reilly logo

Python Social Media Analytics by Michal Krystyanczuk, Siddhartha Chatterjee

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Applying LDA to forum conversations

In this section we will build a topic analysis class based on the lda library. As usual, we first import all the needed libraries:

import sysimport numpy as npimport ldaimport jsonimport pandas as pdfrom collections import Counter, OrderedDictimport nltkfrom nltk.corpus import stopwordsfrom itertools import *from sklearn.feature_extraction.text import CountVectorizer

Then, we create a topic_analysis class with 5 class properties: dataframe, vocab, model, vectorizer, and topics:

For datasets that exceed your RAM capacities, you should use a distributed computing approach.
class topic_analysis(object):  """  A class to extract topics and associate all the verbatims with a specific   topics Input: dataframe, ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required