Mac OS X Hacks

Downloading Files from the Command Line

Few Mac users know of the utility named curl, shipped with every 10.2 Macintosh, or of the easily installed wget. Both allow you to download from the command line — and with a little magic to boot.

There are hundreds of ways to download files located on the Net — FTP, HTTP, NNTP, Gnutella, Hotline, Carracho, the list of possible options goes on and on. There is, however, an odd man out in these protocols, and that’s HTTP. Most web browsers are designed to view web pages (as you’d expect); they’re not designed to download mass amounts of files from a public web directory. This often leaves users with a few meager choices: should they manually and slowly download each file themselves or go out and find some software that could do it for them?

With OS X, your answer comes in the form of free software allowing you to download from the command line [Hack #48] — one installed by default, and one obtainable through Fink (http://fink.sf.net/) [Hack #58]. Investigating the preinstalled utility makes it sounds innocent enough:

curl is a client to get documents/files from or send docu-
ments to a server, using any of the supported protocols
(HTTP, HTTPS, FTP, GOPHER, DICT, TELNET, LDAP or FILE).
The command is designed to work without user interaction
or any kind of interactivity.

Further reading through its manual (accessible by entering man curl as a shell command or a slightly longer version with curl --manual) shows a wide range of features, including ...