Planning for Big Data
In an age where everything is measurable, understanding big data is an essential.
The actinomycetes (source: Internet Archive Book Images)
Introduction
In February 2011, over 1,300 people came together for the inaugural O’Reilly Strata Conference in Santa Clara, California. Though representing diverse fields, from insurance to media and high-tech to healthcare, attendees buzzed with a new-found common identity: they were data scientists. Entrepreneurial and resourceful, combining programming skills with math, data scientists have emerged as a new profession leading the march towards data-driven business.
This new profession rides on the wave of big data. Our businesses are creating ever more data, and as consumers we are sources of massive streams of information, thanks to social networks and smartphones. In this raw material lies much of value: insight about businesses and markets, and the scope to create new kinds of hyper-personalized products and services.
Learn faster. Dig deeper. See farther.
Five years ago, only big business could afford to profit from big data: Walmart and Google, specialized financial traders. Today, thanks to an open source project called Hadoop, commodity Linux hardware and cloud computing, this power is in reach for everyone. A data revolution is sweeping business, government and science, with consequences as far reaching and long lasting as the web itself.
Every revolution has to start somewhere, and the question for many is “how can data science and big data help my organization?” After years of data processing choices being straightforward, there’s now a diverse landscape to negotiate. What’s more, to become data-driven, you must grapple with changes that are cultural as well as technological.
The aim of this book is to help you understand what big data is, why it matters, and where to get started. If you’re already working with big data, hand this book to your colleagues or executives to help them better appreciate the issues and possibilities.
I am grateful to my fellow O’Reilly Radar authors for contributing articles in addition to myself: Alistair Croll, Julie Steele and Mike Loukides.
Edd Dumbill
Program Chair, O’Reilly Strata Conference
February 2012