In example 1, we tried the simple XPath-based technique for a URL with a limited number of results on a single page. In this case, we will be targeting a food and drink category, that is, http://books.toscrape.com/catalogue/category/books/food-and-drink_33/index.html, which has its content across pages. An XPath-based looping operation will be used in this example, which supports a more effective collection of data.
As we will be dealing with multiple pages, it's good practice to check for a few individual page URLs that can be found in the browser while moving through the listed pages. Most of the time, it might contain some patterns that can solve the puzzle easily, as ...