Python Social Media Analytics

Creating the model

Now, we can use the dataset with labeled observations to create a Naive Bayes model:

### Create feature vectors 
 
vectorizer = TfidfVectorizer(min_df=5, 
                             max_df = 0.8, 
                             sublinear_tf=True, 
                             use_idf=True) 

Parameters:

• min_df = 5 discards words appearing in less than five documents
• max_df = 0.8 discards words appearing in more than 80% of the documents
• sublinear_tf = True uses sublinear weighting (scale the term frequency in logarithmic scale)
• use_idf = True enables the inverse document frequency

As a result, we obtain, train, and test vectors that can be directly used to train and validate models:

train_vectors = vectorizer.fit_transform(train_data)test_vectors = vectorizer.transform(test_data) ### Perform a logistic regression ...