Example 2 – scraping book information

In this example, we will automate the browser to process the category and pagination link from the main URL provided. We are interested in extracting details from the Food and Drink category across multiple pages from http://books.toscrape.com/index.html.

An individual page from the category contains listings of products (Books), with certain information listed as follows:

  • title: Title of the book listed
  • titleLarge: Title of the book listed (complete title, found as a value to the title attribute)
  • price: Listed book price
  • stock: Stock information relating to the listed book
  • image: URL of book image
  • starRating: Rating (number of stars found)
  • url: URL of each listed book.

A similar example was also ...

Get Hands-On Web Scraping with Python now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.