Chapter 2. Preprocessing Data

Building real world data analytic solutions requires accurate data. In this chapter, we discuss how to collect, clean, normalize, and transform raw data into a standard format such as Comma-Separated Values (CSV) format or JavaScript Object Notation (JSON), using a tool to process a messy data called OpenRefine.

In this chapter, we will cover the following:

  • Data sources
  • Data scrubbing
  • Data reduction methods
  • Data formats
  • Getting started with OpenRefine

Data sources

Data source is a term for all the technology related to the extraction and storage of data. A data source can be anything from a simple text file to a big database. The raw data can come from observation logs, sensors, transactions, or user behavior.

A dataset

Get Practical Data Analysis - Second Edition now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.