Amazon Hacks

Loop Around the 10-Result Limit

When 10 products just aren’t enough, it’s time to get loopy.

Amazon returns 10 results per request, but there are times you want much more data than that. Along with each response, you’ll find the TotalResults and TotalPages values. By using this information and the page variable with subsequent requests, you can make several queries and combine the results.

The Code

This code builds on the previous XML/HTTP Perl example [Hack #80] but sets the URL assembly, request, and response in a loop. The page value is incremented each time through the loop until the value in TotalPages is met. Create the file amazon_http_loopy.pl with this code:

#!/usr/bin/perl
# amazon_http_loopy.pl
# A simple XML request/parse script that loops through all pages of
# a keyword search.
# Usage: perl amazon_http_loopy.pl <keyword> 

#Your Amazon developer's token
my $dev_key='insert developer token';

#Your Amazon affiliate code
my $af_tag='insert associate tag';

#Take the query from the command-line
my $keyword =shift @ARGV or die "Usage:perl amazon_http_loopy.pl [RETURN]
<keyword>\n";

use strict;

#Use the XML::Parser Perl module
use XML::Simple;
use LWP::Simple;

my $totalPages = 1;

#The loop starts here
for (my $thisPage = 1; $thisPage <= $totalPages; $thisPage++) {

    #Assemble the URL
    my $url = "http://xml.amazon.com/onca/xml3?t=" . $af_tag . 
        "&dev-t=" . $dev_key .
        "&type=lite&f=xml&mode=books&" .
        "KeywordSearch=" . $keyword . 
        "&page=" . $thisPage; my $content = get($url); die ...