In this section, we will build a web crawler to demonstrate the real content-based scraping, targeting web content.
We will be scraping quotes from http://toscrape.com/ and targeting quotes from authors found at http://quotes.toscrape.com/. The crawler will collect the quote and author information from the first five listing pages and write the data into a CSV file. We will also explore the individual author page and extract information about the authors.
To begin with the basic planning and identification of the fields that we are willing to collect information from, please refer to Chapter 3, Using LXML, XPath, and CSS Selectors, Using web browser developer tools for accessing web content section:
'''Listing Quotes ...
