Java Network Programming, Second Edition

Parsing HTML

Sometimes you want to read HTML, looking for information without actually displaying it on the screen. For instance, more than one author I know has written a “book ticker” program to track the hour-by-hour progress of her book in the Amazon.com bestseller list. The hardest part of this program isn’t retrieving the raw HTML. It’s reading through the raw HTML to find the one line that contains the book’s ranking. As another example, consider a Web Whacker-style program that downloads a web site or part thereof to a local PC with all links intact. Downloading the files once you have the URLs is easy. But reading through the document to find the URLs of the linked pages is considerably more complex.

Both of these examples are parsing problems. While parsing a clearly defined language that doesn’t allow syntax errors, such as Java or XML, is relatively straightforward, parsing a flexible language that attempts to recover from errors, like HTML, is extremely difficult. It’s easier to write in HTML than it is to write in a strict language like XML, but it’s much harder to read such a language. Ease of use for the page author has been favored at the cost of ease of development for the programmer.

Fortunately, the javax.swing.text.html and javax.swing.text.html.parser packages include classes that do most of the hard work for you. They’re primarily intended for the internal use of HotJava and the JEditorPane class discussed in the last section. Consequently, they can be a ...