Mining the Social Web, 3rd Edition

Chapter 9. Twitter Cookbook

This cookbook is a collection of recipes for mining Twitter data. Each recipe is designed to solve a particular problem and to be as simple and atomic as possible so that multiple recipes can be composed into more complex recipes with minimal effort. Think of each recipe as being a building block that, while useful in its own right, is even more useful in concert with other building blocks that collectively constitute more complex units of analysis. Unlike the previous chapters, which contain a lot more prose than code, this one provides relatively little discussion and lets the code do more of the talking. The thought is that you’ll likely be manipulating and composing the code in various ways to achieve your own particular objectives.

While most recipes involve little more than issuing a parameterized API call and post-processing the response into a convenient format, some recipes are even simpler (involving just a few lines of code), and others are considerably more complex. This cookbook is designed to help you by presenting some common problems and their solutions. In some cases, it may not be common knowledge that the data you desire is really just a couple of lines of code away. The value proposition is in giving you code that you can trivially adapt to suit your own purposes.

One fundamental software dependency you’ll need for all of the recipes in this chapter is the twitter package, which you can install with pip per the rather predictable ...