How the Search-Ranking Webbot Works
Our search-ranking webbot uses the process detailed in Figure 11-4 to determine the ranking of a website using a specific search term. These are the steps:
Initialize variables for use, including the search criteria and the subject web page.
Fetch the subject web page from the search engine using the search term.
Parse the organic search results from the advertisement and navigation text.
Determine whether or not the desired website appears in this page's search results.
If the desired website is not found, keep looking deeper into the search results until the desired web page is found or the maximum number of attempts has been used.
If the desired website is found, record the ranking.
Report the results.
Figure 11-4. Search-ranking ...
Get Webbots, Spiders, and Screen Scrapers now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
