O'Reilly logo

Instant Web Scraping with Java by Ryan Mitchell

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Writing more complicated scraper (Intermediate)

This recipe isn't a huge technical leap over the last one, but it will introduce you to automated scraping (hitting the go button and letting your scraper traverse the Web by itself) and scraping multiple pages.

Getting ready

Before beginning this recipe, you'll need a copy of the project we created in the previous recipe.

How to do it...

  1. Although our code is still fairly simple, it's a good idea to think about organizing it into packages within our project. Java packages are good for bundling up sets of functionalities within a project, and reducing namespace problems when you have lots of code lying around.
  2. To create a new package within the Scraper project, go to File | New | Package in the Eclipse ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required