References
"Distributed High-performance Web Crawlers: A Survey of the State of the Art." 2003. http://www.cs.ucsd.edu/~dboswell/PastWork/WebCrawlingSurvey.pdf.
"The Viuva Negra crawler: An experience report." Software—Practice & Experience, Volume 38 (2), pp. 161–188, 2006.
and ."Mercator: A Scalable, Extensible Web Crawler." Compaq Systems Research Center, 1999.
and ."Focused crawling: a new approach to topic-specific Web resource discovery." WWW8 International World Wide Web Conference, vol. 31, pp. 1623–1640. Toronto, 1999. http://www8.org/w8-papers/5a-search-query/crawling/index.html.
, and ."Design and Implementation of a High-Performance ...
and .Get Algorithms of the Intelligent Web now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.