Webbots, Spiders, and Screen Scrapers, 2nd Edition

Chapter 13. FTP Webbots

File transfer protocol (FTP) is among the oldest Internet protocols.^[41] It dates from the Internet’s predecessor ARPANET, which was originally funded by the Eisenhower administration.^[42] Research centers started using FTP to exchange large files in the early 1970s, and FTP became the de facto transport protocol for email, a status it maintained until the early 1990s. Today, system administrators commonly use FTP to give web developers access to files on remote webservers. Though it’s an older protocol, FTP still allows computers with dissimilar technologies to share files, independent of file structure and operating system.

Example FTP Webbot

To better understand how an FTP-capable webbot may be useful, consider this scenario. ...