Planning for Big Data

Planning for Big Data

by Edd Wilder-James
Released March 2012
Publisher(s): O'Reilly Media, Inc.
ISBN: 9781449329648

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

In an age where everything is measurable, understanding big data is an essential. From creating new data-driven products through to increasing operational efficiency, big data has the potential to make your organization both more competitive and more innovative.

As this emerging field transitions from the bleeding edge to enterprise infrastructure, it's vital to understand not only the technologies involved, but the organizational and cultural demands of being data-driven.

Written by O'Reilly Radar's experts on big data, this anthology describes:

  • The broad industry changes heralded by the big data era
  • What big data is, what it means to your business, and how to start solving data problems
  • The software that makes up the Hadoop big data stack, and the major enterprise vendors' Hadoop solutions
  • The landscape of NoSQL databases and their relative merits
  • How visualization plays an important part in data work

Publisher resources

View/Submit Errata

Table of contents

  1. Planning for Big Data
  2. Introduction
  3. 1. The Feedback Economy
    2. Data-Obese, Digital-Fast
    3. The Big Data Supply Chain
      1. Data collection
      2. Ingesting and cleaning
      3. Hardware
      4. Platforms
      5. Machine learning
      6. Human exploration
      7. Storage
      8. Sharing and acting
      9. Measuring and collecting feedback
    4. Replacing Everything with Data
    5. A Feedback Economy
  4. 2. What Is Big Data?
    2. What Does Big Data Look Like?
      1. Volume
      2. Velocity
      3. Variety
    3. In Practice
      1. Cloud or in-house?
      2. Big data is big
      3. Big data is messy
      4. Culture
      5. Know where you want to go
  5. 3. Apache Hadoop
    2. The Core of Hadoop: MapReduce
    3. Hadoop’s Lower Levels: HDFS and MapReduce
    4. Improving Programmability: Pig and Hive
    5. Improving Data Access: HBase, Sqoop, and Flume
      1. Getting data in and out
    6. Coordination and Workflow: Zookeeper and Oozie
    7. Management and Deployment: Ambari and Whirr
    8. Machine Learning: Mahout
    9. Using Hadoop
  6. 4. Big Data Market Survey
    2. Just Hadoop?
    3. Integrated Hadoop Systems
      1. EMC Greenplum
      2. IBM
      3. Microsoft
      4. Oracle
      5. Availability
    4. Analytical Databases with Hadoop Connectivity
      1. Quick facts
    5. Hadoop-Centered Companies
      1. Cloudera
      2. Hortonworks
      3. An overview of Hadoop distributions (part 1)
      4. An overview of Hadoop distributions (part 2)
    6. Notes
  7. 5. Microsoft’s Plan for Big Data
    2. Microsoft’s Hadoop Distribution
    3. Developers, Developers, Developers
    4. Streaming Data and NoSQL
    5. Toward an Integrated Environment
    6. The Data Marketplace
    7. Summary
  8. 6. Big Data in the Cloud
    2. IaaS and Private Clouds
    3. Platform solutions
      1. Amazon Web Services
        1. Elastic Map Reduce
        2. DynamoDB
      2. Google
        1. BigQuery
        2. Prediction API
      3. Microsoft
    4. Big data cloud platforms compared
    5. Conclusion
    6. Notes
  9. 7. Data Marketplaces
    2. What Do Marketplaces Do?
    3. Infochimps
    4. Factual
    5. Windows Azure Data Marketplace
    6. DataMarket
    7. Data Markets Compared
    8. Other Data Suppliers
  10. 8. The NoSQL Movement
    2. Size, Response, Availability
    3. Changing Data and Cheap Lunches
    4. The Sacred Cows
    5. Other features
    6. In the End
  11. 9. Why Visualization Matters
    1. A Picture Is Worth 1000 Rows
    2. Types of Visualization
      1. Explaining and exploring
    3. Your Customers Make Decisions, Too
    4. Do Yourself a Favor and Hire a Designer
  12. 10. The Future of Big Data
    2. More Powerful and Expressive Tools for Analysis
    3. Streaming Data Processing
    4. Rise of Data Marketplaces
    5. Development of Data Science Workflows and Tools
    6. Increased Understanding of and Demand for Visualization
  13. About the Author
  14. Copyright

Product information

  • Title: Planning for Big Data
  • Author(s): Edd Wilder-James
  • Release date: March 2012
  • Publisher(s): O'Reilly Media, Inc.
  • ISBN: 9781449329648