Learning Java

Talking to CGI Programs and Servlets

CGI stands for Common Gateway Interface; it is an API for writing applications (often scripts) that can be run by a web server to service a particular range of URLs. Servlets are an implementation very similar to CGI using a component-ized framework in Java. CGI programs and servlets can perform dynamic activities like automatically generating web documents. More important, they can accept data sent from the browser; they are most frequently used to process forms.

The name/value pairs of HTML form fields are encoded by the client web browser in a special format and sent to the application using one of two methods. The first method, using the HTTP command GET , encodes the user’s input into the URL and requests the corresponding document. The server recognizes that the first part of the URL refers to a program and invokes it, passing along the information encoded in the URL as a parameter. The second method uses the HTTP command POST to ask the server to accept the encoded data and pass it to the CGI program as a stream.

In Java, we can create a URL that refers to a CGI program and send it data using either the GET or POST methods. Why would we want to talk to a CGI? Well, CGI remains a widely used technique for building web applications. Other techniques such as opening sockets or talking via RMI are coming on strong, but CGI has been in widespread use for several years. Another important reason for using CGI is that many firewalls block socket connections entirely. But all firewalls that allow web access have to let us use GET and POST to talk to CGIs. So CGI programs can be used as a last resort communications mechanism between applets and servers.

In this section, we’ll talk about writing the client side of these applications. Later in this chapter we’ll talk about writing servlets for the server side of the application. We’ll present the two data sending techniques GET and POST lightly here and in more detail when we revisit them in Section 12.5.

Using the GET Method

Using the GET method of encoding data in a URL is pretty easy. All we have to do is create a URL pointing to a server program and use a simple convention to tack on the encoded name/value pairs that make up our data. For example, the following code snippet opens a URL to a CGI program called login.cgi on the server myhost and passes it two name/value pairs. It then prints whatever text the CGI sends back:

URL url = new URL( 
    // this string should be URL-encoded as well
    "http://myhost/cgi-bin/login.cgi?Name=Pat&Password=foobar");

BufferedReader bin = new BufferedReader ( 
  new InputStreamReader( url.openStream( ) ));

String line; 
while ( (line = bin.readLine( )) != null )
    System.out.println( line );

To form the new URL, we start with the URL of login.cgi; we add a question mark (?), which marks the beginning of the form data, followed by the first name/value pair. We can add as many pairs as we want, separated by ampersand (&) characters. The rest of our code simply opens the stream and reads back the response from the server. Remember that creating a URL doesn’t actually open the connection. In this case, the URL connection was made implicitly when we called openStream( ). Although we are assuming here that our CGI sends back text, it could send anything. (In theory of course we could use the getContentType( ) method of the URL to check the MIME type of any returned data, and try to retrieve the data as an object using getContent( ) as well).

It’s important to point out that we have skipped a step here. This example works because our name/value pairs happen to be simple text. If any “non-printable” or special characters (including ? or &) are in the pairs, they have to be encoded first. The java.net.URLEncoder class provides a utility for encoding the data. We’ll show how to use it in the next example.

Another important thing to note is that although this example sends a password field, you should never do so using this simplistic approach. All of the data we’re sending goes in clear text across the network (it is not encrypted). And in this case the password field would appear anywhere the URL is printed as well (e.g., server logs). We’ll talk about secure web communications later in this chapter.

Using the POST Method

Next we’ll create a small application that acts like an HTML form. It gathers data from two text fields—name and password—and posts the data to a specified URL using the HTTP POST method. If you look ahead to Chapter 15, which covers the Swing GUI text components, you will see that that writing an application that displays actual HTML text and can post using forms just like a web browser is simple. So why would we want to do things the hard way?

There are many reasons that an application (or applet) might want to communicate with a CGI or servlet. For example, compatability with another web-based application might be important, or you might need to gain access to a server through a firewall where direct socket connections (and hence normal RMI) are not available. HTTP has become the lingua franca of the Net and despite its limitations (or more likely because of its simplicity) it has rapidly become one of the most widely supported protocols in the world. All of the other reasons that one would write a client GUI application (as opposed to a pure web/HTML-based application) also present themselves: a client-side GUI can do sophisticated presentation and field validation while, with the technique presented here, still use web-enabled services over the network.

Here’s the code:

//file: Post.java
import java.net.*;
import java.io.*;
import java.awt.*;
import java.awt.event.*;
import javax.swing.*;

public class Post extends JPanel implements ActionListener {
  JTextField nameField, passwordField;
  String postURL;

  GridBagConstraints constraints = new GridBagConstraints( );
  void addGB( Component component, int x, int y ) {
    constraints.gridx = x;  constraints.gridy = y;
    add ( component, constraints );
  }

  public Post( String postURL ) {
    this.postURL = postURL;
    JButton postButton = new JButton("Post");
    postButton.addActionListener( this );
    setLayout( new GridBagLayout( ) );
    addGB( new JLabel("Name:"), 0,0 );
    addGB( nameField = new JTextField(20), 1,0 );
    addGB( new JLabel("Password:"), 0,1 );
    addGB( passwordField = new JPasswordField(20),1,1 );
    constraints.gridwidth = 2;
    addGB( postButton, 0,2 );
  }

  public void actionPerformed(ActionEvent e) {
    postData( );
  }

  protected void postData( ) {
    StringBuffer sb = new StringBuffer( );
    sb.append( URLEncoder.encode("Name") + "=" );
    sb.append( URLEncoder.encode(nameField.getText( )) );
    sb.append( "&" + URLEncoder.encode("Password") + "=" );
    sb.append( URLEncoder.encode(passwordField.getText( )) );
    String formData = sb.toString( );

    try {
      URL url = new URL( postURL );
      HttpURLConnection urlcon = 
          (HttpURLConnection) url.openConnection( );
      urlcon.setRequestMethod("POST");
      urlcon.setRequestProperty("Content-type", 
          "application/x-www-form-urlencoded");
      urlcon.setDoOutput(true);
      urlcon.setDoInput(true);
      PrintWriter pout = new PrintWriter( new OutputStreamWriter(
          urlcon.getOutputStream( ), "8859_1"), true );
      pout.print( formData );
      pout.flush( );

      // read results...
      if ( urlcon.getResponseCode( ) != HttpURLConnection.HTTP_OK )
        System.out.println("Posted ok!");
      else {
        System.out.println("Bad post...");
        return;
      }
      //InputStream in = urlcon.getInputStream( );
      // ...

    } catch (MalformedURLException e) {
      System.out.println(e);     // bad postURL
    } catch (IOException e2) {
      System.out.println(e2);    // I/O error
    }
  }

  public static void main( String [] args ) {
    JFrame frame = new JFrame("SimplePost");
    frame.getContentPane( ).add( new Post( args[0] ), "Center" );
    frame.pack( );
    frame.setVisible(true);

  }
}

When you run this application, you must specify the URL of the server program on the command line. For example:

% java Post http://www.myserver.example/cgi-bin/login.cgi

The beginning of the application creates the form; there’s nothing here that won’t be obvious after you’ve read the chapters on Swing. All the magic happens in the protected postData( ) method. First we create a StringBuffer and load it with name/value pairs, separated by ampersands. (We don’t need the initial question mark when we’re using the POST method, because we’re not appending to a URL string.) Each pair is first encoded using the static URLEncoder.encode( ) method. We ran the name fields through the encoder as well as the value fields, even though we know that they contain no special characters.

Next we set up the connection to the CGI program. In our previous example, we didn’t have to do anything special to send the data, because the request was made by the web browser for us. Here, we have to carry some of the weight of talking to the remote web server. Fortunately, the HttpURLConnection object does most of the work for us; we just have to tell it that we want to do a POST to the URL and the type of data we are sending. We ask for the URLConnection object using the URL’s openConnection( ) method. We know that we are using the HTTP protocol, so we should be able to cast it safely to an HttpURLConnection type, which has the support we need.

Next we use setRequestMethod( ) to tell the connection we want to do a POST operation. We also use setRequestProperty( ) to set the “Content-Type” field of our HTTP request to the appropriate type—in this case, the proper MIME type for encoded form data. (This helps the server sort out what we’re sending.) Finally, we use the setDoOutput( ) and setDoInput( ) methods to tell the connection that we want to both send and receive stream data. The URL connection infers from this combination that we are going to do a POST operation. Next we get an output stream from the connection with getOutputStream( ) and create a PrintWriter so we can easily write our encoded data.

After we post the data, our application calls getResponseCode( ) to see whether the HTTP response code from the server indicates that the POST was successful. Other response codes (defined as constants in HttpURLConnection) indicate various failures. At the end, we indicate where we could have read back the text of the response. For this application, we’ll assume that simply knowing the post was successful was sufficient.

Although form-encoded data (as indicated by the MIME type we specified for the Content-Type field) is the most common, other types of communications are possible. We could have used the input and output streams to exchange arbitrary data types with the CGI program (provided that the CGI program was capable of listening for a connection from us). One great feature of servlets, which we’ll discuss momentarily, is that they can easily exchange arbitrary data with the client.

If you are writing a server application that needs to decode form data, you can use the java.net.URLDecoder to undo the operation of the URLEncoder.

SSL and Secure Web Communications

The previous examples sent a field called Password to the server. However, standard HTTP doesn’t provide encryption to hide our data. Fortunately, adding security for GET and POST operations is easy. Where available you simply have to use a secure form of the HTTP protocol—HTTPS.

HTTPS is a version of the standard HTTP protocol run over SSL (Secure Socket Layer) sockets, which use public-key encryption techniques to encrypt the data sent. Most web browsers currently come with built-in support for HTTPS (or raw SSL sockets). Therefore, if your web server supports HTTPS, you can use a browser to send and receive secure data simply by specifying the https protocol in your URLs. This is not something your code has to deal with directly. Applets written using the Java plug-in have access to the HTTPS protocol handler. For other applications you will have to make sure that your environments have supplied an HTTPS (SSL) protocol handler, or set up the data connection yourself using other secure means.