O'Reilly logo

Instant Web Scraping with Java by Ryan Mitchell

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Scraping Ajax Pages (Advanced)

One of the fundamental (and frustrating) differences between a web scraper and a web browser is the ability to execute client-side code, such as JavaScript. This isn't usually a problem, but JavaScript can sometimes be used to encode or hide data that makes it difficult to get at for a web scraper. You'll never take your browser's JavaScript execution capabilities for granted again, as we try to recreate the same functionality in Java.

Getting ready

It is sometimes possible to get around needing to execute JavaScript on a page by scraping the JavaScript itself (sometimes obfuscated and annoying, sometimes pre-packaged in convenient Google Maps address arrays—JavaScript parsing is always an adventure). But because of ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required