Perl & LWP

Chapter 5. Forms

Much of the interesting data of the Web is accessible only through HTML forms. This chapter shows you how to write programs to submit form data and get the resulting page. In covering this unavoidably complex topic, we consider packing form data into GET and POST requests, how each type of HTML form element produces form data, and how to automate the process of submitting form data and processing the responses.

The basic model for the Web is that the typical item is a “document” with a known URL, and when you want to access it (whether it’s the Rhoda episode guide, or the front page of today’s Boston Globe), you just get it, no questions asked. Even when there are cookies or HTTP authentication involved, these are basically just addenda to the process of requesting the known URL from the appropriate server. But some web resources require parameters beyond just their URL, parameters that are generally fed in by the user through HTML forms, and that the browser then sends either as dynamic parts of a URL (in the case of a GET request) or as content of a POST request.

A program on the receiving end of form data may simply use it as a query for searching other data, such as scanning all the RFCs and listing the ones by specific authors. Or a program may store the data, as with taking the user’s data and saving it as a new post to a message base. Or a program may do grander things with the user-provided data, such as debiting the credit card number provided, logging the ...