Better, Faster, Lighter Java

Chapter 9. Simple Spider

I once had the pleasure of building a house with a carpenter who was from Mexico; I learned a great deal in the process. Despite my broken Spanish (I once twisted the language so badly that I told him that I was going home to marry my sister), he was able to impart an incredible amount of wisdom. My observations were, shall we say... less wise. I told him that I was proud of my hammer. It had a sleek and heavy head, and a composite graphite handle that made it easy to swing. The head was waffled so it would grip the nails. He simply chuckled. At the end of the day, he had used his 10-year-old, plain-Jane hammer to drive in four times the nails as I did, with more accuracy and precision. He told me, “No esta el martillo:” it’s not the hammer.

Luckily, I build code better than I build houses. In this chapter, I continue to put the five basic principles into action. I create a simple service with a light user interface to drive it. The service is an open source web indexer, primarily used to provide site search behavior for a single web site. It is called Simple Spider and is available at http://www.relevancellc.com/halloway/SimpleSpider.html.

I’ll give you an insider’s view of the real client requirements that spawned the application in the first place. The requirements were minimal and straightforward, but there was still a lot of work to do to understand the problem space. Notice large functionality areas that could be built for this application or reused ...