Python Social Media Analytics

Scraping time constraints

The main disadvantage of such an approach is the time needed to extract the information. If we take into consideration only the time spent waiting, we have to wait for 529 seconds to scroll a page 100 times. Then, we have to add processing time which will take another few seconds.

As a result, we will get up to 1,100 pins meaning we get around 11 pins per one page scroll (this value might differ for pins of different sizes). Even if we reduce the time dedicated to waiting it will still be difficult to scale this approach to get hundreds of thousands of pins. However, the purpose of this analysis is to get some targeted insight for personal use (a constraint of web scraping), and we'll still obtain a sufficient amount ...