Practical Data Wrangling

Table of contents

1. Preface
   1. What this book covers
   2. What you need for this book
   3. Who this book is for
   4. Conventions
   5. Reader feedback
   6. Customer support
      1. Downloading the example code
      2. Downloading the color images of this book
      3. Errata
      4. Piracy
      5. Questions
2. Programming with Data
   1. Understanding data wrangling
      1. Getting and reading data
      2. Cleaning data
      3. Shaping and structuring data
      4. Storing data
   2. The tools for data wrangling
      1. Python
      2. R
   3. Summary
3. Introduction to Programming in Python
   1. External resources
   2. Logistical overview
      1. Installation requirements
      2. Using other learning resources
      3. Python 2 versus Python 3
   3. Running programs in python
      1. Using text editors to write and manage programs
         1. Writing the hello world program
      2. Using the terminal to run programs
         1. Running the Hello World program
      3. What if it didn't work?
   4. Data types, variables, and the Python shell
      1. Numbers - integers and floats
         1. Why integers? 
      2. Strings
      3. Booleans
      4. The print function
      5. Variables
         1. Adding to a variable
         2. Subtracting from a variable
         3. Multiplication
         4. Division
         5. Naming variables
      6. Arrays (lists, if you ask Python)
      7. Dictionaries 
   5. Compound statements
      1. Compound statement syntax and indentation level
      2. For statements and iterables
      3. If statements
         1. Else and elif clauses
      4. Functions
         1. Passing arguments to a function
         2. Returning values from a function
   6. Making annotations within programs
   7. A programmer's resources
      1. Documentation
      2. Online forums and mailing lists
   8.  Summary
4. Reading, Exploring, and Modifying Data - Part I
   1. External resources
   2. Logistical overview
      1. Installation requirements
      2. Data
      3. File system setup
   3. Introducing a basic data wrangling work flow
   4. Introducing the JSON file format
   5. Opening and closing a file in Python using file I/O
      1. The open function and file objects
      2. File structure - best practices to store your data
      3. Opening a file
   6. Reading the contents of a file
      1. Modules in Python
      2. Parsing a JSON file using the json module
   7. Exploring the contents of a data file
      1. Extracting the core content of the data
      2. Listing out all of the variables in the data
   8. Modifying a dataset
      1. Extracting data variables from the original dataset
         1. Using a for loop to iterate over the data
         2. Using a nested for loop to iterate over the data variables
   9. Outputting the modified data to a new file
   10. Specifying input and output file names in the Terminal
       1. Specifying the filenames from the Terminal
   11. Summary
5. Reading, Exploring, and Modifying Data - Part II
   1. Logistical overview
      1. File system setup
      2. Data
      3. Installing pandas
   2. Understanding the CSV format
   3. Introducing the CSV module
   4. Using the CSV module to read CSV data
   5. Using the CSV module to write CSV data
   6. Using the pandas module to read and process data
      1. Counting the total road length in 2011 revisited
   7. Handling non-standard CSV encoding and dialect
   8. Understanding XML
      1. XML versus JSON
   9. Using the XML module to parse XML data
      1. XPath
   10. Summary
6. Manipulating Text Data - An Introduction to Regular Expressions
   1. Logistical overview
      1. Data
      2. File structure setup
   2. Understanding the need for pattern recognition
   3. Introducting regular expressions
      1. Writing and using a regular expression
      2. Special characters
      3. Matching whitespace
      4. Matching the start of string
      5. Matching the end of a string
      6. Matching a range of characters
      7. Matching any one of several patterns
      8. Matching a sequence instead of just one character
      9. Putting patterns together
      10. Extracting a pattern from a string
      11. The regex split() function
      12. Python regex documentation
   4. Looking for patterns
   5. Quantifying the existence of patterns
      1. Creating a regular expression to match the street address
      2. Counting the number of matches
      3. Verifying the correctness of the matches
   6. Extracting patterns
      1. Outputting the data to a new file
   7. Summary
7. Cleaning Numerical Data - An Introduction to R and RStudio
   1. Logistical overview
      1. Data
      2. Directory structure
      3. Installing R and RStudio
   2. Introducing R and RStudio
   3. Familiarizing yourself with RStudio
      1. Running R commands
      2. Setting the working directory
      3. Reading data
      4. The R dataframe
      5. R vectors
      6. Indexing R dataframes
      7. Finding the 2011 total in R
   4. Conducting basic outlier detection and removal
   5. Handling NA values
      1. Deleting missing values
      2. Replacing missing values with a constant
      3. Imputation of missing values
   6. Variable names and contents
   7. Summary
8. Simplifying Data Manipulation with dplyr
   1. Logistical overview
      1. Data
      2. File system setup
      3. Installing the dplyr and tibble packages
   2. Introducing dplyr
   3. Getting started with dplyr
   4. Chaining operations together
   5. Filtering the rows of a dataframe
   6. Summarizing data by category
   7. Rewriting code using dplyr
   8. Summary
9. Getting Data from the Web
   1. Logistical overview
      1. Filesystem setup
      2. Installing the requests module
      3. Internet connection
   2. Introducing APIs
   3. Using Python to retrieve data from APIs
   4. Using URL parameters to filter the results
   5. Summary
10. Working with Large Datasets
    1. Logistical overview 
       1. System requirements
       2. Data
       3. File system setup
       4. Installing MongoDB
       5. Planning out your time
       6. Cleaning up
    2. Understanding computer memory
    3. Understanding databases
    4. Introducing MongoDB
    5. Interfacing with MongoDB from Python
    6. Summary