The Big Data Market

Chapter 1. The Big Data Market

THIS REPORT IS A DATA-DRIVEN STUDY of the complete big data market. It is derived from live data triangulated across the entire business world—websites, meetups, hiring patterns, business relationships, blogs, press, forums, SEC filings...everything—using data crawlers and proprietary natural-language parsing technology developed by Spiderbook. This bottoms-up data methodology is in sharp contrast to traditional approaches dependent on anecdotal evidence derived from small-sample user and analyst client surveys.

But as revolutionary as big data is, our analysis of more than 500,000 of the largest companies in the world reveals that a very small percentage of them have embraced big data methodologies in reality. One could argue that big data is still very much in the early adoption phase of the technology adoption model.

Note

One could argue that big data is still very much in the early adoption phase of the technology adoption model.

The numbers, perhaps studied for the first time looking at actual data, suggest that there remains a lot of room for growth in the big data market, and newer technologies like Spark are overtaking Hadoop’s MapReduce, the current reigning patriarch of the open source big data movement.

This report first covers the flagship of the big data technologies, Hadoop. Next we look into the use cases for big data technologies, highlighting some surprising results about budgets and spending on big data and data science. We follow ...