Data Preparation in the Big Data Era

Chapter 1. Data Preparation in the Era of Big Data

Introduction

Preparing and cleaning data for any kind of analysis is notoriously costly, time­ consuming, and prone to error, with conventional estimates holding that 80% of the total time spent on analysis is spent on data preparation.¹ Accelerating investment in big data analytics—$44 billion total in 2014 alone, according to Gartner—elevates the stakes for successfully preparing data from many sources for use in data analysis.

Substantial ROI gains can be realized by modernizing the techniques and tools enterprises employ in cleaning, combining, and transforming data. This report introduces the business benefits of big data and analyzes the issues that organizations face today with traditional data preparation and integration. It also introduces the need for a new approach that scales, known as data curation, and discusses how to deliver the results.

This report will cover the following key topics:

• Starting with the business question
• Understanding your data
• Selecting the data to use
• Analyzing your current data strategy
• Assessing alternative ETL and data curation products
• Delivering the results

Starting with the Business Question

What are you aiming and analyzing for, exactly?

We are currently living in the big data era, with huge business opportunities and challenges for every industry. Data is growing at an exponential rate worldwide: in 2016, global Internet traffic will reach 90 exabytes per month, according to a recent ...