O'Reilly logo

Instant Web Scraping with Java by Ryan Mitchell

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Writing robust, scalable code (Advanced)

You didn't start learning Java so that you could write one-page scripts, and our ability to crawl the Web is going to be limited very quickly if that's all we're using it for. In this recipe, we will write a multi class scraper that scrapes across multiple websites.

Getting ready

We'll build on our work in the earlier recipe Writing a simple scraper (Simple), but will create two new packages that are able to import, create instances of, and communicate between each other.

How to do it...

  1. Create a new package within the Scraper project named com.packtpub.JavaScraping.RobustScraper.WikiScraper, and create an additional project called com.packtpub.JavaScraping.RobustScraper.Tools.
  2. Create a class in the WikiScraper ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required