O'Reilly logo

Wicked Cool Ruby Scripts by Steve Pugh

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Web Scraper

Web Scraper

webScraper.rb

This version of the web scraper has the same functionality as the one found in Chapter 7 (see "#44 Link Scrape" on Link Scrape, "#45 Image Scrape" on How It Works, and "#46 Scraper" on Hacking the Script). The difference between the script in Chapter 7 and the one below is the addition of GetoptLong and RDoc. This version has the advantage of a standard help message, along with command-line arguments for the specific functions.

The Code

 # == Synopsis
  #
  #  webScraper.rb: scrape specific information from websites
  #
  #
  # == Usage
  #
  # webScraper.rb [OPTIONS] ... URL # # -h, --help # show help # # --links , -l # scrape ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required