Parsing

To begin our parsing, we will need to use the library we mentioned earlier called BeautifulSoup. We imported it earlier, so now we just need to pass the page source into BeautifulSoup. We do that by means of the following code:

soup = BeautifulSoup(browser.page_source, "html5lib") 

Notice that the browser object contains a page_source attribute. That is all the HTML we retrieved with our get request earlier. The other parameter passed into BeautifulSoup is the parsing library it will use. Here, we will stick with html5lib.

Now, once the content of the page has been passed to BeautifulSoup, we want to start to extract the elements of interest. That's where the div elements with the info-container class come in. We are going to retrieve ...

Get Python Machine Learning Blueprints - Second Edition now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.