Chapter 4. Project 1: Competitive Analysis

If you sell products, you’re almost certainly interested in watching the market for similar products. Consider one example of this: someone who monitors the book market, keeping track of the performance of a set of titles over time. This chapter shows how to use web services to pull together reports on product information from three different sources—Amazon, Google, and eBay. You’ll build a simple web application that will let an analyst enter product information and have a report emailed that contains the relevant information.

Application Features

Here’s the basic feature set:

  • User enters and saves search information for a book (the title and an ISBN).

  • User can create, view, delete, and run saved searches.

The application tracks the following data:

  • Amazon.com prices, availability, release date, and sales rank based on the supplied ISBN

  • Google rankings and top five search results for the book title

  • eBay search results for the book title, including number of matching listings, highest price, and number of bids

A few JSP pages can present the information and accept input from the user. These JSP pages rely on a simple Java class, Search. The Search class deals with connections to web services via a set of supporting classes: AmazonConnection, EbayConnection, and GoogleConnection. This relationship is shown in Figure 4-1.

Interaction between JSPs and Java support and connection classes
Figure 4-1. Interaction between JSPs and Java support and connection classes

First, the following list details the JSP pages and presentation objects used in the application.

list.jsp

A list of the previously saved searches

new.jsp

Create a new search

view.jsp

View the results of a search

delete.jsp

Delete a saved search

update.jsp

Request an existing search to be updated

default.css

Rudimentary stylesheet

Listing Searches

The first JSP, list.jsp , is shown in Figure 4-2. This is the main page for the web application. Clicking View takes users to a report page showing the data retrieved from the various web services. Delete removes the selected search from the web application, and Update causes the web application to refresh the data from the various web services.

Listing searches
Figure 4-2. Listing searches

The code for list.jsp is shown in Example 4-1. It can conceptually be broken into three main sections. First, there are a few initialization lines, the normal JSP and HTML headers. Next, a JSP scriptlet retrieves data from our supporting Java classes and performs some housekeeping to more cleanly present the data in the JSP. Finally, the actual HTML formatting and JSP display variables appear.

Example 4-1. Main search results JSP
<%@ page contentType="text/html; charset=iso-8859-1" language="java"
 import="com.cascadetg.ch04.*" errorPage="" %>
<%

    boolean isUpdating = false;

    java.util.Hashtable mySearches 
        = com.cascadetg.ch04.Search.getAllSearches( );
    if(mySearches != null)
    {
        java.util.Enumeration mySearchEnum = mySearches.elements( );

        while(mySearchEnum.hasMoreElements( ))
        {
            Search current = 
                (com.cascadetg.ch04.Search)mySearchEnum.nextElement( );
            if(current.isUpdating( ))
                isUpdating = true;
        }
    }

%>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>Chapter 4: Competitive Analysis, List</title>
<meta http-equiv="Content-Type" 
    content="text/html; charset=iso-8859-1" />
<% if (isUpdating) { %>
<meta http-equiv="refresh" content="15;url=list.jsp">
<% } %>
<link href="default.css" rel="stylesheet" type="text/css" />
</head>

<body>
<% if (isUpdating) { %>
<p align="right">Page will automatic refresh every 15 seconds 
while an update is in progress.</p>
<% } %>

<% if (mySearches == null) { %>
<p>No searches defined.</p>
<% } else { %>
<table width="100%"  border="0" cellspacing="3" cellpadding="3">
  <tr>
    <td><strong>Search</strong></td>
    <td width="200" nowrap="nowrap">&nbsp;</td>
  </tr>
<%
    java.util.Enumeration mySearchListing = mySearches.elements( );
    while(mySearchListing.hasMoreElements( )) 
    { 
        Search mySearch = (Search)mySearchListing.nextElement( );
        String productID = mySearch.getProductID( );
        %>
  <tr>
    <td class="list">
        <%= mySearch.getProductTitle( ) %> (<%= productID %>)</td>
    <td width="200" align="center" nowrap="nowrap" class="list">
    <a href="view.jsp?productID=<%=productID %>">View</a> - 
    <a href="delete.jsp?productID=<%=productID %>">Delete</a> - 
    <a href="update.jsp?productID=<%=productID %>">Update</a></td>
  </tr>
<% 
    }
%>
</table>
<% } %>
<p><a href="new.jsp">New Search</a></p>
</body>
</html>

Adding Searches

A user can click on the New Search link (as shown in Figure 4-2) to be brought to a form to add a new search. An example of this form is shown in Figure 4-3.

Adding a new search
Figure 4-3. Adding a new search

The book title field is self-explanatory. The ISBN field is provided to allow entry of an International Standard Book Number (ISBN), a worldwide standard for providing a unique identifying number for any published book.

The ISBN number can typically be found on the back cover of a book. You can also find ISBN numbers for books by browsing the Amazon web site or consulting with the publisher. If you just want to try the application without finding a new ISBN, you can click on the provided link.

Note

Another, different standard from ISBN is the Universal Product Code (UPC). Often, both the creator and distributors of a product introduce additional identifying codes for goods and services. Which codes to use is usually a business-driven question, but you’ll want to be aware of what numbering systems are in place.

In this example, we’re using the ISBN as the unique identifier for the books we’re searching for, but in the “real world,” you may want to use your own internal tracking system and then map that to the other systems. For example, if you carry both books (ISBN) and pickles (UPC), you’re using two different numbering systems.

The code for new.jsp file is fairly straightforward, as shown in Example 4-2. If the creation of a new search is successful, users are returned to list.jsp.

Example 4-2. New search JSP
<%@ page contentType="text/html; charset=iso-8859-1" 
language="java" errorPage="" %>
<%
    boolean success = false;
    boolean submitted = (request.getParameter("Submit") != null);
    if(submitted)
    {
        String productID = request.getParameter("productID");
        String productTitle = request.getParameter("productTitle");
        
        success = 
              com.cascadetg.ch04.Search.addSearch(productID, productTitle);
        if(success)
        {
            response.sendRedirect("list.jsp");
            return;
        }
    }
%>
<head>
<title>Chapter 4: Competitive Analysis</title>
<meta http-equiv="Content-Type" 
    content="text/html; charset=iso-8859-1" />
<link href="default.css" rel="stylesheet" type="text/css" />
</head>
<body>
<% if(submitted) { %>
<p>Error submitting. Try again.</p>
<% } %>
<form name="new_item" id="new_item" method="post" action="new.jsp">
  <table width="60%"  border="0" cellspacing="0" cellpadding="0">
    <tr>
      <td width="50%">Book ISBN :</td>
      <td><input name="productID" type="text" /></td>
    </tr>
    <tr>
      <td width="50%">Book Title :</td>
      <td><input name="productTitle" type="text" /></td>
    </tr>
    <tr>
      <td width="50%">&nbsp;</td>
      <td><input type="submit" name="Submit" value="Submit" /></td>
    </tr>
  </table>
  <p><a href="new.jsp?productID=0596004001&productTitle=
                       Mac%20OS%20X%20for%20Java%20Geeks&Submit=Submit">
  Add Mac OS X for Java Geeks</a> </p>
</form>
</body>
</html>

Viewing a Search

Figure 4-4 shows an example of the results of a search. The user can click the “Return to list” link in the upper-right corner to return to the search list.

Viewing search results
Figure 4-4. Viewing search results

As you can see from Example 4-3, view.jsp retrieves the results from the Search object. The results are then formatted into a HTML table. The heavy lifting is performed by the Search.update( ) method, as described later in this chapter.

Example 4-3. Search results JSP
<%@ page contentType="text/html; charset=iso-8859-1" 
language="java" errorPage="" %>
<%
    String productID = request.getParameter("productID");
    com.cascadetg.ch04.Search mySearch 
        = com.cascadetg.ch04.Search.getSearch(productID);
    Object[] attributeKeys = null;
    if(mySearch != null)
    {    
        java.util.Map myAttributes = mySearch.getAttributes( );
        if(myAttributes != null)
        {
            attributeKeys = myAttributes.keySet( ).toArray( );
        }
    };
%>
<head>
<title>Chapter 4: Competitive Analysis</title>
<meta http-equiv="Content-Type" 
    content="text/html; charset=iso-8859-1" />
<link href="default.css" rel="stylesheet" type="text/css" />
</head>

<body>
<p align="right"><a href="list.jsp">Return to list</a></p>
<% if(mySearch == null)
    { %>
    <p>Unable to find this product ID (<%=productID%>)</p>
<%    } else { %>
    
<h2>Book Title: <%= mySearch.getProductTitle( ) %>
    (ISBN: <%= mySearch.getProductID( ) %>)</h2>
<p>Is Updating: <%= mySearch.isUpdating( ) %></p>
<table width="100%"  border="0" cellspacing="3" cellpadding="3">
  <tr>
    <td nowrap="nowrap">Attribute</td>
    <td>Value</td>
  </tr>
  <%
  if(attributeKeys != null)
      for(int i = 0; i < attributeKeys.length; i++)
    {
        String key = (String)attributeKeys[i];
  %>
  <tr>
    <td nowrap="nowrap" class="list"><%= key %></td>
    <td class="list"><%= mySearch.getAttribute(key) %></td>
  </tr>
  <% } %>
</table>
<% } %>
<p><a href="list.jsp">Return to list</a></p>
<p><a href="new.jsp">New search</a></p>
</body>
</html>

Removing a Search

If we allow people to add searches, we also should allow them to remove them. When a user clicks delete from the main JSP (as shown in Figure 4-2), a simple form (as shown in Figure 4-5) prompts the user to confirm the deletion before processing the request.

Delete confirmation web page
Figure 4-5. Delete confirmation web page

The code for this delete confirmation dialog is shown in Example 4-4.

Example 4-4. Delete confirmation JSP code
<%@ page contentType="text/html; charset=iso-8859-1" 
language="java" errorPage="" %>
<%
    boolean delete = false;
    String productID = request.getParameter("productID");
    if(request.getParameter("Delete") != null)
        delete = true;
    
    boolean deleted = false;
    if(delete)
    {        
        deleted = 
            com.cascadetg.ch04.Search.removeSearch(productID);
    };
%>
<head>
<title>Chapter 4: Competitive Analysis</title>
<meta http-equiv="Content-Type" 
    content="text/html; charset=iso-8859-1" />
<link href="default.css" 
    rel="stylesheet" type="text/css" />
</head>
<body><% 
if (delete)
{ 
    if(deleted)
    {
%>
        <p>The item has been deleted.</p>
<%    } else {
%>        
        <p>Item not found.</p>
<%    }
} else{ %>
        <p>Are you sure you want to delete the 
            item "<%= productID %>"?</p>        
        <form name="deleteForm" id="deleteForm" 
            method="post" action="delete.jsp">
          <input type="hidden" name="productID" 
            value="<%= productID %>"/>
          <input name="Delete" type="submit" 
            id="Delete" value="Delete" />
         </form>
        <%
} 
%>
<p><a href="list.jsp">Return to list.</a></p>
</body>
</html>

Updating a Search

The Search object caches the data returned from the various web services in memory, but a user may wish to force the system to immediately refresh the data. Clicking on the update link on the main page (Figure 4-2) allows a user to do just that, after a confirmation web page, as shown in Figure 4-6.

Confirming an update request
Figure 4-6. Confirming an update request

The code for the update confirmation page is shown in Example 4-5.

Example 4-5. Update confirmation JSP
<%@ page contentType="text/html; charset=iso-8859-1" 
language="java" import="com.cascadetg.ch04.*" errorPage="" %>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" 
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<%
    boolean updateProduct = false;
    String productID = request.getParameter("productID");
    if(productID != null)
    {
        updateProduct = true;
        Search.getSearch(productID).update( );
    }

%>
<title>Chapter 4: Competitive Analysis</title>
<meta http-equiv="Content-Type" 
content="text/html; charset=iso-8859-1" />
<link href="default.css" rel="stylesheet" type="text/css" />
</head>

<body>
<% if (updateProduct) { %>
<p>Product updated.</p>
<% } else { %>

<p>Unknown product.</p>
<% } %>
<p><a href="list.jsp">Return to list.</a> </p>
</body>
</html>

Gathering Web Service Data

The various user interface elements, as shown so far, all rely on a single Search class for the actual web service data retrieval; the JSP pages don’t contain any code that actually communicates with web services. Instead, a single Search object relies on supporting classes, as shown in Figure 4-7, to retrieve the data.

Web service Search classes
Figure 4-7. Web service Search classes

The four static methods, addSearch( ), getAllSearches( ), getSearch( ), and removeSearch( ), are the main points of interest to the JSP pages. All data about the web services results is contained in a set of simple name/value pairs, contained in the java.util.Hashtable variable attributes. When a search is run, the Search object is passed to the three supporting web service connection classes, and the appropriate data is set in attributes.

The code shown in Example 4-6 shows the code for the Search class. Pay particular interest to the update( ) method.

Example 4-6. Core Search class
package com.cascadetg.ch04;

import java.util.Hashtable;

public class Search
{
    // Static search Hashtable used for persistence.
    // This means that the data stored by this application is lost
    // every time the server is restarted. Adding support for
    // persistence is left as an exercise for the reader - the most
    // likely, of course, being saving the data to a database in some
    // fashion.

    // If you are interested in adding persistence, you'll want to
    // intercept the addSearch( ), removeSearch( ), and update( ) methods
    // in this class.

    private static Hashtable searches = new Hashtable( );

    public static boolean addSearch(
        String productID,
        String productTitle)
    {
        if (productID == null)
            return false;
        if (productID.length( ) < 1)
            return false;
        if (productTitle == null)
            return false;
        if (productID.length( ) < 1)
            return false;

        if (searches.get(productID) != null)
            searches.remove(productID);

        Search mySearch = new Search( );
        mySearch.setProductID(productID);
        mySearch.setProductTitle(productTitle);
        searches.put(productID, mySearch);
        mySearch.update( );
        return true;
    }

    public static boolean removeSearch(String productID)
    {
        if (searches == null)
        {
            return true;
        }
        if (searches.get(productID) == null)
            return false;
        searches.remove(productID);
        return true;
    }

    public static Hashtable getAllSearches( )
    {
        return searches;
    }

    public static Search getSearch(String inProductID)
    {
        return (Search)searches.get(inProductID);
    }

    // Simple object data. Note the use of a Hashtable to store
    // a variable set of attributes

    private String productID;
    private String productTitle;
    private Hashtable attributes = new Hashtable( );

    public void setAttribute(String name, String value)
    {
        attributes.put(name, value);
    }

    public String getAttribute(String name)
    {
        return (String)attributes.get(name);
    }

    public java.util.Map getAttributes( )
    {

        java.util.TreeMap map =
            new java.util.TreeMap(java.text.Collator.getInstance( ));
        map.putAll(attributes);

        return map;
    }

    /** Returns the product ID (specifically, a book ISBN). */
    public String getProductID( )
    {
        return productID;
    }

    /** Set the product ID (specificially, a book ISBN). */
    public void setProductID(String productID)
    {
        this.productID = productID;
    }

    public String getProductTitle( )
    {
        return productTitle;
    }

    public void setProductTitle(String productTitle)
    {
        this.productTitle = productTitle;
    }

    // Updating data logic

    // Note that this updating logic isn't as useful as it should be,
    // as the update( ) ought to kick off a new thread.
    private boolean is_updating = false;
    public boolean isUpdating( )
    {
        return is_updating;
    }

    public void update( )
    {
        // Don't allow the user to set up dozens of hits on the site at
        // once. Ideally, you should be more conservative about kicking
        // off requests, perhaps using an aggregation approach (as
        // described in Chapter 8, News Aggregator
        if (is_updating)
            return;
        is_updating = true;

        // Create a new Search and set the values to the existing
        // search
        Search mySearch = new Search( );
        mySearch.setProductID(this.productID);
        mySearch.setProductTitle(this.productTitle);

        // Kick off the requests to the various web services
        new AmazonConnection( ).addAttributes(mySearch);
        new EbayConnection( ).addAttributes(mySearch);
        new GoogleConnection( ).addAttributes(mySearch);

        // Replace the old attributes with the newly discovered values
        this.attributes = mySearch.attributes;

        is_updating = false;
    }
}

It’s easy to imagine enhancements: the underlying system might implement a Connection interface, with the potential to add new web services dynamically.

One other thing you may have noticed: the getAttributes() method returns a java.util.Map, instead of the underlying Hashtable. As shown in Example 4-7, the code creates a TreeMap that keeps the attributes sorted by the system’s default locale interpretation of an “alphabetical” sorting routine.

Example 4-7. Sorting code
java.util.TreeMap map =
new java.util.TreeMap(java.text.Collator.getInstance( ));
    map.putAll(attributes);

Now, on to the real meat of the application—the web services connectivity.

Connecting to Amazon

The Amazon web services package is implemented according to the commonly regarded interpretation of web services—a series of SOAP methods, complete with a WSDL file describing their offerings. We will use Axis, as described in Chapter 3, to work with Amazon’s web services.

Before working with Amazon’s web services, you must register with Amazon’s developer program at http://www.amazon.com/webservices/, as shown in Chapter 4.

Signing up for Amazon web services
Figure 4-8. Signing up for Amazon web services

The instructions are straightforward: you must download the developer kit (a single file, kit.zip) and get a web services token. This token, a simple string, is used by Amazon to uniquely identify you when you are accessing web services. You should treat this token as you would any password.

As shown in Figure 4-9, you’ll find a number of examples for a variety of programming languages as well as the documentation.

Contents of the Amazon SDK
Figure 4-9. Contents of the Amazon SDK

Open the file kit/API Guide/index.html in your web browser to see the documentation, as shown in Figure 4-10.

Amazon SDK documentation
Figure 4-10. Amazon SDK documentation

In our case, we’ll use Axis to generate access code from the Amazon-supplied WSDL file. The Amazon WSDL file, as described in the READMEFIRST.txt file, can be found at http://soap.amazon.com/schemas3/AmazonWebServices.wsdl. This WSDL file is used by the Axis WSDL2Java tool to generate a set of Java bindings. The easiest way to do this is to open a command prompt and navigate to the Axis library installation directory. Then, execute the following command, all as one line, as shown in Example 4-8. Note that you need an Internet connection for this to work.

Example 4-8. Generating Amazon Java bindings from WSDL
C:\devenv\axis-1_1\lib>java -classpath commons-logging.jar;
    log4j-1.2.8.jar;wsdl4j.jar;axis.jar;commons-discovery.jar;
    jaxrpc.jar;saaj.jar 
  org.apache.axis.wsdl.WSDL2Java   
    http://soap.amazon.com/schemas3/AmazonWebServices.wsdl

If successful, there will be no visible output on the command line, but you will now have a set of Java classes corresponding to the Amazon SDK in the Axis directory. As shown in Figure 4-11, these classes are neatly sorted into a com.amazon.soap package. Although you are given the source to these classes by this tool, you won’t want to edit them manually. You will likely want to regenerate these files any time Amazon adds new functionality.

Generated Amazon Java Axis source
Figure 4-11. Generated Amazon Java Axis source

Copy these files out of this directory and into your Java source tree.

The bulk of the generated classes essentially serve as data holders for Amazon web service operations. To access the web services, use the AmazonSearchServiceLocator class to retrieve a service and an AmazonSearchPort to handle web service requests. The AmazonSearchServiceLocator class hierarchy is shown in Figure 4-12.

Amazon service locator class
Figure 4-12. Amazon service locator class

After retrieving a service, you use a search port object to perform requests. The generated class hierarchy is shown in Figure 4-13. Note that the methods of the AmazonSearchPort class take instances of other classes generated by the Axis toolkit as parameters.

Amazon port request class
Figure 4-13. Amazon port request class

The code shown in Example 4-9 shows how to actually make a connection to the Amazon web services. In this case, an instance of the AsinRequest class is used. The call to myAmazonSearchPort.asinSearchRequest(myAsinRequest) actually performs the request. It transforms the underlying data to SOAP, sends the data over the network, retrieves the resulting data, and transforms that data back into Java object(s), in this case a ProductInfo object.

Example 4-9. Connecting to Amazon web services
package com.cascadetg.ch04;

import java.util.Date;
import com.amazon.soap.*;

public class AmazonConnection
{

    public void addAttributes(Search inSearch)
    {
        inSearch.setAttribute(
            "Amazon Last Check",
            new Date( ).toLocaleString( ));

        // Mac OS X for Java Geeks ISBN
        //String isbn = "0596004001";

        try
        {
            AmazonSearchService myAmazonSearchService =
                new AmazonSearchServiceLocator( );
            AmazonSearchPort myAmazonSearchPort =
                myAmazonSearchService.getAmazonSearchPort( );

            AsinRequest myAsinRequest = new AsinRequest( );

            // Use this to set your Amazon Associates ID
            // For more info on Amazon Associates, see...
            // http://www.amazon.com/associates
            myAsinRequest.setTag(DeveloperTokens.amazon_associates);
            myAsinRequest.setDevtag(DeveloperTokens.amazon_token);
            myAsinRequest.setAsin(inSearch.getProductID( ));
            myAsinRequest.setLocale("us");
            myAsinRequest.setType("heavy");

            ProductInfo myProductInfo =
                myAmazonSearchPort.asinSearchRequest(myAsinRequest);

            Details[] myDetailsArray = myProductInfo.getDetails( );
            Details myDetail = null;
            if (myDetailsArray != null)
            {
                myDetail = myDetailsArray[0];

                inSearch.setAttribute(
                    "Amazon Product Name",
                    myDetail.getProductName( ));

                inSearch.setAttribute(
                    "Amazon Release Date",
                    myDetail.getReleaseDate( ));

                inSearch.setAttribute(
                    "Amazon Actual Price",
                    myDetail.getOurPrice( ));

                inSearch.setAttribute(
                    "Amazon List Price",
                    myDetail.getListPrice( ));

                inSearch.setAttribute(
                    "Amazon Used Price",
                    myDetail.getUsedPrice( ));

                inSearch.setAttribute(
                    "Amazon Sales Rank",
                    myDetail.getSalesRank( ));

                inSearch.setAttribute(
                    "Amazon Availability",
                    myDetail.getAvailability( ));

                //myDetail.getImageUrlSmall( );
                //myDetail.getImageUrlMedium( );
                //myDetail.getImageUrlLarge( );
            }
        } catch (Exception e)
        {
            e.printStackTrace( );

        }
    }
}

The code shown in Example 4-9 is as close to a standard RPC system as a web service is likely to be. None of the actual connectivity details are in the code: all of the URLs, ports, and other connectivity information are in the WSDL and then embedded in the code generated by the WSDL2Java tool.

Connecting to eBay

eBay’s web service implementation provides access to their systems via XML over HTTPS. A simple HTTPS connection provides in-transit security, and XML data is sent and received to perform operations. While establishing an HTTPS connection to the eBay servers is a relatively trivial matter, working with the XML documents is more time-consuming.

eBay in Transition

In mid-2004, eBay announced support for SOAP, initially limited to few core APIs. Depending on your needs, you may be able to use the SOAP interfaces provided by eBay in lieu of the XML-via-HTTPS mechanism described here.

As of this writing, both the eBay and PayPal platforms are undergoing significant changes, including the transition to SOAP, more complex user authorization systems, and integration of XML descriptions of complex user interface elements for working with more robust item metadata. Visit http://developer.ebay.com/ frequently for the latest information on the eBay platform.

As of mid-2004, there are several tiers of access offered by eBay, based more or less on the quantity of web service requests you require. The free, individual developer version allows for only 50 calls a day—not very many, but likely enough to build and do some basic testing of your application. The next step up, called Basic, offers 30,000 calls per month for $500 per year.

By default, you have access to the eBay sandbox server, a test environment set up for developers to test their application without incurring service fees. To access the production server, you must be certified by eBay (as of this writing, eBay charges $100 for certification). If you’re looking at building a production application that accesses eBay services, you need to include these access and certification fees in your budget. Don’t forget that it can take several days to get your application certified.

The eBay sandbox, http://sandbox.ebay.com/, shown in Figure 4-14, is a good development resource. It’s a recreation of the eBay web site (http://www.ebay.com/) and is intended for developers to test their application. There are several portions of the web site that aren’t implemented or work slightly differently; for example, you can’t actually enter a credit card and authenticate a user to sell items on the test server. Instead, you use eBay web services to do this programmatically using the web services API.

eBay sandbox site
Figure 4-14. eBay sandbox site

The eBay developer registration system is a bit more complex than other offerings in order to support more complex deployment and usage models. First, you need to register for eBay itself and then go through the six-step registration process (http://developer.ebay.com/ → Membership → Join). When you’re done registering, you can generate a set of three security tokens, a Developer ID (devID), Application ID (appID), and Certification ID (certID). Make sure that your pop-up blocking software is disabled when you generate your security tokens.

Warning

Don’t lose your eBay security tokens! You can generate them for the first time online, but if you lose them, you’ll have to go through eBay support. It can take several days to verify your information and receive new tokens.

eBay API wrapper

In order to make it easier to connect to eBay in this chapter (and others), the sample applications use a standard eBay connectivity class, as shown in Figure 4-15, as a reusable component for connecting to eBay’s XML-via-HTTPS web services.

eBay web service utility class
Figure 4-15. eBay web service utility class

The code for the eBay class is shown in Example 4-10. All the details of opening a connection to eBay, sending the XML, and then retrieving the results are wrapped by this class. The code is designed to make a call as simple as possible: create a new EbayAPISimpleCall object, set the eBay “verb” you wish to call, and add the arguments you need with EbayAPISimpleCall.setArgument().

Example 4-10. eBay API simple call
package com.cascadetg.ch04;

import java.net.*;
import java.io.*;
import java.util.Hashtable;

/**
* Based on the eBay sample code, APISimpleCallJava.java
 */

public class EbayAPISimpleCall
{

    // The eBay API function
    private String apiVerb = "GeteBayOfficialTime";

    /** API Verb used to get the official eBay time */
    public static final String GetOfficialTime = "GeteBayOfficialTime";
    /** API Verb used to search on eBay */
    public static final String GetSearchResults = "GetSearchResults";

    // The version threshold of compatibility for the eBay API call
    private String compatibilityLevel = "331";

    // The error level for the API call
    private String errorLevel = "1";

    // Set siteId to correspond to the US Site
    private String siteId = "0";

    // "0" is the only supported detail level for GetebayOfficialTime
    private String detailLevel = "0";

    public EbayAPISimpleCall( )
    {
    }

    boolean isProduction = false;

    public org.jdom.Document executeCall( )
    {
        // Supply a URL to the desired API server, for example:
        // Sandbox: https://api.sandbox.ebay.com/ws/api.dll
        // Production: https://api.ebay.com/ws/api.dll
        String targetURL;
        if (!isProduction)
        {
            targetURL = "https://api.sandbox.ebay.com/ws/api.dll";
        } else
        {
            targetURL = "https://api.ebay.com/ws/api.dll";
        }

        // This string will contain the assembled XML for the API call
        String eBayXML;

        try
        {

            // Create a new URL object using the specified targetURL
            URL website = new URL(targetURL);

            // Create a connection to the URL
            Object foo = website.openConnection( );

            // Note that this is actually an SSL connection, even
            // though cast to an HttpURLConnection. Different JDK
            // versions use a different SSL connection class, but they
            // all use HttpURLConnection as the base class (which has
            // the methods we need).
            HttpURLConnection connection =
                (HttpURLConnection)website.openConnection( );

            // Specify that the connection will be used for input and
            // output
            connection.setDoInput(true);
            connection.setDoOutput(true);

            // Specify the method for the URL request
            connection.setRequestMethod("POST");

            // Add the eBay specific headers needed for an eBay API
            // call
            // see: "Making the HTTP request" in the eBay API
            // documentation
            addeBayAPIHeaders(connection);

            // Build the XML call String for eBayGetOfficialTime
            eBayXML =
                buildBaseXML(
                    DeveloperTokens.eBay_userid,
                    DeveloperTokens.ebay_userPassword)
                    + buildVerbCall( );

            // Create the Output and Print Streams
            OutputStream output = connection.getOutputStream( );
            PrintStream pout = new PrintStream(output);

            // 'Upload' the eBayXML String
            pout.print(eBayXML);
            pout.close( );

            // Create the Input Streams
            InputStream input = connection.getInputStream( );
            BufferedInputStream bufIn = new BufferedInputStream(input);

            org.jdom.Document result =
                new org.jdom.input.SAXBuilder( ).build(bufIn);
            connection.disconnect( );
            return result;

        } catch (MalformedURLException ex)
        {
            ex.printStackTrace( );
        } catch (IOException ioException)
        {
            ioException.printStackTrace( );
        } catch (org.jdom.JDOMException JDOMprob)
        {
            JDOMprob.printStackTrace( );
        }

        return null;
    }

    // A test routine to verify that everything is working.
    public static void main(String[] args) throws IOException
    {
        EbayAPISimpleCall myCall = new EbayAPISimpleCall( );
        myCall.setApiVerb(GetOfficialTime);
        org.jdom.Document myDocument = myCall.executeCall( );
        new org.jdom.output.XMLOutputter( ).output(
            myDocument,
            System.out);
        myCall.setApiVerb(GetSearchResults);
        myCall.setArgument("Query", "test");
        myDocument = myCall.executeCall( );
        new org.jdom.output.XMLOutputter( ).output(
            myDocument,
            System.out);

        String result =
            myDocument.getRootElement( ).getChild("eBayTime").getText( );
        System.out.println(result);

        result =
            myDocument
                .getRootElement( )
                .getChild("Search")
                .getChild("GrandTotal")
                .getText( );
        System.out.println(result);
    }

    private void addeBayAPIHeaders(HttpURLConnection connection)
    {
        // Generate and add the Session Certificate Header
        connection.addRequestProperty(
            "X-EBAY-API-SESSION-CERTIFICATE",
            DeveloperTokens.ebay_devId
                + ";"
                + DeveloperTokens.appId
                + ";"
                + DeveloperTokens.certId);

        // Add the Compatibility Level Header
        connection.addRequestProperty(
            "X-EBAY-API-COMPATIBILITY-LEVEL",
            compatibilityLevel);

        // Add the Developer Name, Application Name, and Certification
        // Name headers
        connection.addRequestProperty(
            "X-EBAY-API-DEV-NAME",
            DeveloperTokens.ebay_devId);
        connection.addRequestProperty(
            "X-EBAY-API-APP-NAME",
            DeveloperTokens.appId);
        connection.addRequestProperty(
            "X-EBAY-API-CERT-NAME",
            DeveloperTokens.certId);

        // Add the API verb Header
        connection.addRequestProperty("X-EBAY-API-CALL-NAME", apiVerb);

        // Add the Site Id Header
        connection.addRequestProperty("X-EBAY-API-SITEID", siteId);

        // Add the Detail Level Header
        connection.addRequestProperty(
            "X-EBAY-API-DETAIL-LEVEL",
            detailLevel);

        // Add the Content-Type Header
        connection.addRequestProperty("Content-Type", "text/xml");

        // NOTE: eBay recommends setting the Content-Length header
        // see: "Making the HTTP request" in the eBay API documentation
    }

    private String buildBaseXML(String userId, String userPassword)
    {
        return "<?xml version='1.0' encoding='utf-8'?>\r\n"
            + "<request>\r\n"
            + "<RequestUserId>"
            + userId
            + "</RequestUserId>\r\n"
            + "<RequestPassword>"
            + userPassword
            + "</RequestPassword>\r\n";
    }

    Hashtable arguments = new Hashtable( );
    public void clearArguments( )
    {
        arguments = new Hashtable( );
    }

    public Hashtable getArguments( )
    {
        return arguments;
    }
    public void setArgument(String argument, String value)
    {
        arguments.put(argument, value);
    }
    public String getArgument(String argument)
    {
        return (String)arguments.get(argument);
    }
    public void setArguments(Hashtable arguments)
    {
        this.arguments = arguments;
    }

    private String buildVerbCall( )
    {
        StringBuffer verbCall = new StringBuffer( );
        verbCall.append("<ErrorLevel>");
        verbCall.append(errorLevel);
        verbCall.append("</ErrorLevel>\r\n");
        verbCall.append("<Verb>");
        verbCall.append(apiVerb);
        verbCall.append("</Verb>\r\n");
        verbCall.append("<DetailLevel>");
        verbCall.append(detailLevel);
        verbCall.append("</DetailLevel>\r\n");
        verbCall.append("<SiteId>");
        verbCall.append(siteId);
        verbCall.append("</SiteId>\r\n");
        if (arguments.size( ) > 0)
        {
            java.util.Enumeration keys = arguments.keys( );
            while (keys.hasMoreElements( ))
            {
                String current = (String)keys.nextElement( );
                verbCall.append("<");
                verbCall.append(current);
                verbCall.append(">");
                verbCall.append(arguments.get(current));
                verbCall.append("</");
                verbCall.append(current);
                verbCall.append(">");
            }

        }

        verbCall.append("</request>");

        return verbCall.toString( );
    }

    /**
     * @return Returns the detailLevel.
     */
    public String getDetailLevel( )
    {
        return detailLevel;
    }

    /**
     * @param detailLevel
     *            The detailLevel to set.
     */
    public void setDetailLevel(String detailLevel)
    {
        this.detailLevel = detailLevel;
    }

    /**
     * @return Returns the errorLevel.
     */
    public String getErrorLevel( )
    {
        return errorLevel;
    }

    /**
     * @param errorLevel
     *            The errorLevel to set.
     */
    public void setErrorLevel(String errorLevel)
    {
        this.errorLevel = errorLevel;
    }

    /**
     * @return Returns the siteId.
     */
    public String getSiteId( )
    {
        return siteId;
    }

    /**
     * @param siteId
     *            The siteId to set.
     */
    public void setSiteId(String siteId)
    {
        this.siteId = siteId;
    }

    /**
     * @return Returns the compatibilityLevel.
     */
    public String getCompatibilityLevel( )
    {
        return compatibilityLevel;
    }

    /**
     * @param compatibilityLevel
     *            The compatibilityLevel to set. This controls the
     *            versioning of the XML interface for the API. This is
     *            how eBay allows multiple versions of their API to be
     *            supported simultaneously.
     */
    public void setCompatibilityLevel(String compatibilityLevel)
    {
        this.compatibilityLevel = compatibilityLevel;
    }

    /**
     * @return Returns the isProduction.
     */
    public boolean isProduction( )
    {
        return isProduction;
    }

    /**
     * @param isProduction
     *            By default (isProduction = false), the server will
     *            contact the eBay test environment, called the
     *            "sandbox." If you wish to use the production
     *            environment, you'll need to set this to true.
     */
    public void setProduction(boolean isProduction)
    {
        this.isProduction = isProduction;
    }

    public String getApiVerb( )
    {
        return apiVerb;
    }

    /**
     * Two standard verbs are provided with this class,
     * GeteBayOfficialTime and GetSearchResults. Others can be added
     * from the eBay documentation.
     */
    public void setApiVerb(String apiVerb)
    {
        this.apiVerb = apiVerb;
    }
}

For eBay, a verb is the term that describes the name of a web service method. Each verb describes a particular action, with a set of one or more possible parameters. These verbs generally have self-describing names, such as GeteBayOfficialTime or GetSearchResults. eBay’s documentation lists many verbs; using our EbayAPISimpleCall. setApiVerb() method, it’s easy to call any of them. Two static String objects are defined in this class; they correspond to the official eBay verbs. An enterprising developer might imagine developing a custom class hierarchy corresponding to the eBay framework. However, keep in mind that this class hierarchy quickly looks like the bindings generated by a SOAP framework!

When you’re ready, call EbayAPISimpleCall.executeCall( ), and you’ll receive a result via a JDOM object (which holds the XML returned by eBay). If the result is an eBay error message, that is the contents of the XML; otherwise, it should be the data you need.

The code to make the call into eBay is only support infrastructure; it doesn’t actually help you write your application. It’s analogous to the code generated by the WSDL2Java code in the previous example (talking to Amazon), but in this case, you have to maintain it yourself.

What Is JDOM?

JDOM is a library designed to make working with XML feel more “natural” to a Java developer. It can be downloaded from http://www.jdom.org/. There are a few JAR files included with the distribution; you need these to build this project. As of this writing, I’m using JDOM Beta 9. Downloading the ZIP binary archive, find several JAR files in the lib directory, including ant.jar, jaxen-core.jar, jaxen-jdom.jar, saxpath.jar, xalan.jar, xerces.jar, and xml-apis.jar. Add these to your class path and you’re ready to start working with JDOM.

eBay connectivity

Now that you have a reusable wrapper for accessing eBay web services, it’s time to look at the code to actually get useful data back from the eBay service.

As mentioned earlier, you must create and validate a user on the sandbox in order to perform certain activities, such as searches. The main( ) method of the code shown in Example 4-11 can be used to authenticate a user as a command-line utility if needed. The addAttributes() method is called by the Search object to retrieve data from eBay—a straightforward operation, given the wrapper shown in Example 4-10.

Example 4-11. Retrieving data from eBay
package com.cascadetg.ch04;

import java.util.Date;
import org.jdom.output.XMLOutputter;

public class EbayConnection
{

    public org.jdom.Document validateSandboxUser(
        String username,
        String password)
    {
        EbayAPISimpleCall myCall = new EbayAPISimpleCall( );
        // Validate this user on the sandbox server only
        myCall.setProduction(false);
        myCall.setApiVerb("ValidateTestUserRegistration");
        myCall.setArgument("RequestUserId", username);
        myCall.setArgument("RequestPassword", password);
        return myCall.executeCall( );
    }

    /**
     * Call this class, passing in the username and password you would
     * like to register on the eBay test server.
     * 
     * @param args
     */
    static public void main(String[] args)
    {
        if (args == null)
        {
            System.out.println("Need two arguments (name & password).");
            return;
        }
        if (args.length < 2)
        {
            System.out.println("Need two arguments (name & password).");
            return;
        }
        try
        {
            new XMLOutputter( ).output(
                new EbayConnection( ).validateSandboxUser(
                    args[0],
                    args[1]),
                System.out);
        } catch (Exception e)
        {
            System.out.println("Unable to register user.");
            e.printStackTrace( );
        }
    }

    public void addAttributes(Search inSearch)
    {
        inSearch.setAttribute(
            "eBay Last Check",
            new Date( ).toLocaleString( ));

        EbayAPISimpleCall myCall = new EbayAPISimpleCall( );
        myCall.setApiVerb(EbayAPISimpleCall.GetSearchResults);
        myCall.setArgument("Query", inSearch.getProductTitle( ));
        myCall.setArgument("Order", "MetaHighestPriceSort");
        org.jdom.Document myResults = myCall.executeCall( );

        org.jdom.Element root = myResults.getRootElement( );
        long count =
            Long.parseLong(
                root
                    .getChild("Search")
                    .getChild("GrandTotal")
                    .getText( ));
        inSearch.setAttribute(
            "eBay Total Matching Listings",
            Long.toString(count));

        if (count > 0)
        {
            org.jdom.Element item =
                root.getChild("Search").getChild("Items").getChild(
                    "Item");

            inSearch.setAttribute(
                "eBay Highest Bid Item Price",
                item.getChildText("LocalizedCurrentPrice"));

            inSearch.setAttribute(
                "eBay Highest Bid Item Bids",
                item.getChildText("BidCount"));

            inSearch.setAttribute(
                "eBay Highest Bid Item Link",
                item.getChildText("Link"));
        }
    }
}
Authorizing a Sandbox User Account

Let’s say that you want to test the system’s search capability by adding a new item (for example, you’d like to add my book, Mac OS X for Java Geeks, so it will show up in a search). Or, you’d like to create a seller and a couple of buyers on the sandbox to test the full flow of an auction.

To do either activity, you must mark a user account on the sandbox server as an authorized seller. Start by using the web interface on http://sandbox.ebay.com/ to create a user. Note that you can’t go through the seller authentication mechanism via your web browser: it will fail at the end with a warning.

Instead, after you’ve created a user account, use the code in EbayConnection.java to authorize the account as a seller. Assuming everything is compiled and working properly, just execute this command:

java -classpath .;..\lib\jdom.jar com.cascadetg.ch04
                .EbayConnection username password

This registers the user as having been validated on the eBay server.

Lets take a closer look at the connectivity code, as shown in Example 4-12. It starts by creating a object to represent the call to eBay, EbayAPISimpleCall. It then sets the eBay verb to use. In the sample, a search is being performed, but you can find documentation on other supported verbs at http://developer.ebay.com/DevZone/docs/API_Doc/index.asp.

Next, the arguments are set. For a search query, the search terms must be specified—in this case, the title of the book. The application also needs the highest value auction result, so a second argument, Order, is added with the string MetaHighestPriceSort (as specified by the eBay documentation for the search verb). The call is then issued, and an XML document containing the results of the query is returned. From there, it’s just a question of using the JDOM interfaces to walk through the returned XML.

Example 4-12. eBay connection code.
   EbayAPISimpleCall myCall = new EbayAPISimpleCall( );
   myCall.setApiVerb(EbayAPISimpleCall.GetSearchResults);
   myCall.setArgument("Query", inSearch.getProductTitle( ));
   myCall.setArgument("Order", "MetaHighestPriceSort");
   org.jdom.Document myResults = myCall.executeCall( );

To more fully support this, you may wish to perform more robust error handling (indeed, you’ll be required to do this to pass eBay’s certification), but it shows how even with a nonstandard API, a bit of work can decouple the details of establishing the network connection from the rest of your application logic.

Connecting to Google

After the previous examples, it may surprise you to see how easy it is to connect to Google’s web service offerings. Visiting http://www.google.com/apis/, shown in Figure 4-16, it’s pretty straightforward to start building applications using the Google web service interface. Make sure you download the developer kit as shown on the front page; you will need the provided libraries.

Google API home page
Figure 4-16. Google API home page

Google does offer APIs via SOAP and WSDL, but for Java developers, there’s a pre-built library that does all the work for you. It’s as if they took the time to run Axis WSDL2Java for you, and then just provided you with a library to use (in fact, that’s very close to what they actually did). It’s entirely possible to use Axis to generate the Java code from their WSDL file (posted at http://api.google.com/GoogleSearch.wsdl) just as you did for Amazon. Instead, you should rely on the Java library they provided, called googleapi.jar, as shown in Figure 4-17. You must add this library to your class path. For more information on the Google API, open the APIs_Reference.html file.

Contents of the Google SDK
Figure 4-17. Contents of the Google SDK

The code as shown in Example 4-13 could hardly be simpler. More lines are spent looping through the results and adding them to the Search object’s attributes than it takes to get data back from the Google server itself. The code converts the GoogleSearchResult objects to Strings and add them to the results given, but it’s easy to imagine using the various methods on the GoogleSearchResult object to get more detail and provide even better formatting.

Example 4-13. Google connectivity
package com.cascadetg.ch04;
import java.util.Date;
import com.google.soap.search.*;

public class GoogleConnection
{
    public void addAttributes(Search inSearch)
    {
        inSearch.setAttribute(
            "Google Last Check",
            new Date( ).toLocaleString( ));

        GoogleSearch search = new GoogleSearch( );

        // Set mandatory attributes
        search.setKey(DeveloperTokens.googleKey);
        search.setQueryString(inSearch.getProductTitle( ));

        // Set optional attributes
        search.setSafeSearch(true);
        // Invoke the actual search
        GoogleSearchResult result = null;
        try
        {
            result = search.doSearch( );
        } catch (GoogleSearchFault e)
        {
            e.printStackTrace( );
        }
        // process the result

        if (result != null)
        {
            inSearch.setAttribute(
                "Google Number of Hits",
                Integer.toString(
                    result.getEstimatedTotalResultsCount( )));

            GoogleSearchResultElement[] mySearchElements =
                result.getResultElements( );

            for (int i = 0; i < mySearchElements.length; i++)
            {
                inSearch.setAttribute(
                    "Google Result " + i,
                    mySearchElements[i].toString( ));
                if (i > 4)
                {
                    i = mySearchElements.length;
                }
            }
        }
    }
}

Developer Tokens

You may have noticed references to a DeveloperTokens class (e.g. DeveloperTokens.amazon_associates, or DeveloperTokens.amazon_token). This class is a single static class with a set of Strings containing the various developer tokens used throughout this chapter. The class shown in Example 4-14 contains only placeholder strings of the approximate correct length.

Example 4-14. Private developer tokens
package com.cascadetg.ch04;

public class DeveloperTokens
{

    // Amazon userID token
    public static String amazon_token = "12345678901234";
    // Amazon assoicates ID
    public static String amazon_associates = "123456789-20";

    // eBay userID (the "visible" user identification, i.e. the buyer &
    // seller user.
    public static String eBay_userid = "1234567890";
    public static String ebay_userPassword = "1234567890";

    // devId, appId, certId as specified by the eBay Developer's
    // Program      
    public static String ebay_devId = "123456789012345678901234567890";
    public static String appId =      "123456789012345678901234567890";
    public static String certId =     "123456789012345678901234567890";

    // Google web services ID key
    public static String googleKey = "12345678901234567890123456789012";
}

As you’ve seen, actually making the connection to a provider isn’t that hard (especially after creating bindings or utility classes to make it easier to access). Different vendors have different registration systems, pricing models, and different expectations in terms of security, but at least all offer free access to their system for developers to start building and testing applications, and all have published documentation and interfaces.

Get Real World Web Services now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial