Using search methods to scrape information from a web page
In Chapter 2, Creating a BeautifulSoup Object, we discussed the fetching of information related to all books published by Packt Publishing available at the link http://www.packtpub.com/books. In the following exercise, we will use Beautiful Soup searching methods to scrape all books from the preceding URL. We will scrape the name of the book, published date, and price from the preceding URL.
The first step involved in this exercise is to analyze the HTML document and to understand its logical structure. This will help us to understand how the required information is stored within the HTML document.
We can use the Google Chrome developer tools to understand the logical structure of the page. ...
Get Getting Started with Beautiful Soup now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
