I was at a data warehousing conference and talking with a top executive from a leading bank about Hadoop. As I was telling him about the technology, he interjected, “But does it have any use for us? We don’t have any Internet usage to analyze!” Well, he was just voicing a common misconception. Hadoop is not a technology meant for analyzing web usage or log files only; it has a genuine use in the world of petabytes (of 1,000 terabytes apiece). It is a super-clever technology that can help you manage very large volumes of data efficiently and quickly—without spending a fortune on hardware.
Hadoop may have started in ...