O'Reilly logo

Linux Pocket Guide, 2nd Edition by Daniel J. Barrett

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Name

wget — stdin  stdout  - file  -- opt  --help  --version

Synopsis

wget [options] URL

The wget command hits a URL and downloads the data to a file or standard output. It’s great for capturing individual web pages, downloading files, or duplicating entire web site hierarchies to arbitrary depth. For example, let’s capture the Yahoo home page:

$ wget http://www.yahoo.com
23:19:51 (220.84 KB/s) - `index.html' saved [31434]

which is saved to a file index.html in the current directory. wget has the added ability to resume a download if it gets interrupted in the middle, say, due to a network failure: just run wget -c with the same URL and it picks up where it left off.

Perhaps the most useful feature of wget is its ability to download files without needing a web browser:

$ wget http://www.example.com/files/manual.pdf

This is great for large files like videos and ISO images. You can even write shell scripts to download sets of files if you know their names:

$ for i in 1 2 3; do wget http://example.com/$i.mpeg; done

Another similar command is curl, which writes to standard output by default—unlike wget, which duplicates the original page and file names by default.

$ curl http://www.yahoo.com > mypage.html

wget has over 70 options, so we’ll cover just a few important ones. (curl has a different set of options; see its manpage.)

Useful options

-i filename

Read URLs from the given file and retrieve them in turn.

-O filename

Write all the captured HTML to the given file, one page appended after the ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required