Natural Language Processing and Computational Linguistics

Twitter for mining data

While Reddit is a great way to analyze a more structured form of data, we often turn to social media to analyze text – there seem to be more real-world implications, and for social scientists, it can serve as a treasure trove of textual data. Indeed, sentiment analysis for tweets is a very popular project for budding data scientists because it allows one to try their hand at both data collection and data analyzing.

In all our examples throughout the book, we have dealt with datasets that are usually loaded from the packages we use, such as the 20 Newsgroup dataset (from scikit-learn), the Lee news corpus (from Gensim), or the IMDB dataset (Keras). While it is important to know how to work with well-documented datasets ...