© Seppe vanden Broucke and Bart Baesens 2018
Seppe vanden Broucke and Bart BaesensPractical Web Scraping for Data Sciencehttps://doi.org/10.1007/978-1-4842-3582-9_6

6. From Web Scraping to Web Crawling

Seppe vanden Broucke1  and Bart Baesens2
(1)
KU Leuven, Leuven, Belgium
(2)
Dept of Decision Sci & Info Managem, KU Leuven Dept of Decision Sci & Info Managem, Leuven, Belgium
 

So far, the examples in the book have been quite simple in the sense that we only scraped (mostly) a single page. When writing web scrapers, however, there are many occasions where you’ll wish to scrape multiple pages and even multiple websites. In this context, the name “web crawler” is oftentimes used, as it will “crawl” across a site or even the complete web. This chapter illustrates ...

Get Practical Web Scraping for Data Science: Best Practices and Examples with Python now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.