O'Reilly logo

Instant Web Scraping with Java by Ryan Mitchell

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Faster scraping through threading (Intermediate)

Although processors were originally developed with only one core, modern multicore processors (processors with multiple separate and independent CPUs embedded on a single chip) run most efficiently when they are executing multiple threads (separate, independent sets of instructions) at once.

If you have a powerful quad-core processor completely dedicated to executing your program, but your program is not using threading, you are essentially limited to using 25 percent of your processor's capabilities. In this recipe, we will learn how to take advantage of your machine (especially important when running on a dedicated server) and speed up your scraping.

Getting ready

Writing software that uses threading ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required