So far, the examples in the book have been quite simple in the sense that we only scraped (mostly) a single page. When writing web scrapers, however, there are many occasions where you’ll wish to scrape multiple pages and even multiple websites. In this context, the name “web crawler” is oftentimes used, as it will “crawl” across a site or even the complete web. This chapter illustrates ...
6. From Web Scraping to Web Crawling
Get Practical Web Scraping for Data Science: Best Practices and Examples with Python now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.