
110 Text Mining and Visualization: Case Studies Using Open-Source Tools
Algorithm Looping through pillreports.net webpages.
Require: target = positive integer (number of pages to collect)
Require: top index = positive integer (starting index of website
Require: connection to MongoDB database; collection created (reports)
Ensure: MongoDB Server Running
for count < target do
Pass the index to web scrape() for downloading the page
if number of table not 3 then
return false
else
return soup
end if
if soup not false then
Pass (soup,index, db connection, db collection) to parse tables()
Require: pill reports dict - empty python dictionary
for row in table do
col1=ro ...