Python Social Media Analytics

Rate Limits

After successful connection to the API, we have to prepare our scripts for data retrieval. As the API limits data access (Rate Limits), it is necessary to build an efficient workflow.

Twitter allows you to get up to 100 tweets per one call (thus, we set the parameter count to 100). If we want to retrieve more and we need to remember already downloaded tweets' IDs not to extract the same tweets during next calls. This procedure is commonly called paging.

The paging procedure is implemented in the script as follows:

pages_counter = 0 
number_of_pages = 100 while pages_counter < number_of_pages:    pages_counter += 1    res = requests.get(url, params = pms, auth=auth)    print("Connection status: %s" % res.reason)    tweets = res.json() ids ...