Parsing

To begin our parsing, we will need to use the library we mentioned earlier called BeautifulSoup. We imported it earlier, so now we just need to pass the page source into BeautifulSoup. We do that by means of the following code:

soup = BeautifulSoup(browser.page_source, "html5lib") 

Notice that the browser object contains a page_source attribute. That is all the HTML we retrieved with our get request earlier. The other parameter passed into BeautifulSoup is the parsing library it will use. Here, we will stick with html5lib.

Now, once the content of the page has been passed to BeautifulSoup, we want to start to extract the elements of interest. That's where the div elements with the info-container class come in. We are going to retrieve ...

Get Python Machine Learning Blueprints - Second Edition now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.