Scrape Customer Advice
Screen scraping can give you access to community features not yet implemented through the API—like customer buying advice.
Customer buying advice isn’t available through Amazon’s Web Services, so if you’d like to include this information on a remote site, you’ll have to get it from Amazon’s site through scraping. The first step to this hack is knowing where to find all of the customer advice on one page. The following URL links directly to the advice page for a given ASIN:
This Perl script, get_advice.pl , splits the advice page into two variables based on the headings “in addition to” and “instead of.” It then loops through those sections, using regular expressions to match the products’ information. The script then formats and prints the information.
#!/usr/bin/perl # get_advice.pl # A script to scrape Amazon to retrieve customer buying advice # Usage: perl get_advice.pl <asin> #Take the asin from the command-line my $asin =shift @ARGV or die "Usage:perl get_advice.pl <asin>\n"; #Assemble the URL my $url = "http://amazon.com/o/tg/detail/-/" . $asin . "/?vi=advice"; #Set up unescape-HTML rules my %unescape = ('"'=>'"', '&'=>'&', ' '=>' '); my $unescape_re = join '|' => keys %unescape; use strict; use LWP::Simple; #Request the URL my $content = get($url); die "Could not retrieve $url" unless $content; my($inAddition) = (join '', $content) =~ m!in addition to(.*?)<tr>[RETURN] <td colspan=3><br></td></tr>!mis; ...