Data Visualization Toolkit: Using JavaScript, Rails™, and Postgres to Present Data and Geospatial Information

Data Visualization Toolkit: Using JavaScript, Rails™, and Postgres to Present Data and Geospatial Information

by Barrett Clark
Released August 2016
Publisher(s): Addison-Wesley Professional
ISBN: 9780134465159

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Create Beautiful Visualizations that Free Your Data to Tell Powerful Truths

“The depth of Barrett Clark’s knowledge shines through in his writing: clear, concise, and confident. Barrett has been practicing all of this stuff in his day job for many years–Postgres, D3, GIS, all of it. The knowledge in this book is real-world and hard-earned!” –From the Foreword by Obie Fernandez

Data Visualization Toolkit is your hands-on, practical, and holistic guide to the art of visualizing data. You’ll learn how to use Rails, jQuery, D3, Leaflet, PostgreSQL, and PostGIS together, creating beautiful visualizations and maps that give your data a voice and to make it “dance.”

Barrett Clark teaches through real-world problems and examples developed specifically to illuminate every technique you need to generate stunningly effective visualizations. You’ll move from the absolute basics toward deep dives, mastering diverse visualizations and discovering when to use each. Along the way, you’ll build three start-to-finish visualization applications, using actual real estate, weather, and travel datasets.

Clark addresses every component of data visualization: your data, database, application server, visualization libraries, and more. He explains data transformations; presents expert techniques in JavaScript, Ruby, and SQL; and illuminates key concepts associated with both descriptive statistics and geospatial data. Throughout, everything is aimed at one goal: to help you cut through the clutter and let your data tell all it can.

This guide will help you

  • Explore and understand the data visualization technology stack

  • Master the thought process and steps involved in importing data

  • Extract, transform, and load data in usable, reliable form

  • Handle spotty data, or data that doesn’t line up with what your chart expects

  • Use D3 to build pie and bar charts, scatter and box plots, and more

  • Work effectively with time-series data

  • Tweak Ruby and SQL to optimize performance with large datasets

  • Use raw SQL in Rails: window functions, subqueries, and common table expressions

  • Build chord diagrams and time-series aggregates

  • Use separate databases or schema for reporting databases

  • Integrate geographical data via geospatial SQL queries

  • Construct maps with Leaflet and Rails

  • Query geospatial data the “Rails way” and the “raw SQL way”

    •

    Table of contents

    1. About This E-Book
    2. Title Page
    3. Copyright Page
    4. Dedication Page
    5. Contents
    6. Foreword
    7. Preface
    8. Acknowledgments
    9. About the Author
    10. Part I: ActiveRecord and D3
      1. Chapter 1. D3 and Rails
        1. Your Toolbox—A Three-Ring Circus
          1. Database
          2. Application Server
          3. Graphing Library
        2. Maryland Residential Sales App
          1. Evaluating Data
          2. Data Fields
          3. Simple Pie Chart
        3. Summary
      2. Chapter 2. Transforming Data with ActiveRecord and D3
        1. Pie Chart Revisited
          1. Legible Labels
          2. Mouseover Effects
          3. You Can Function
        2. Bar Chart
          1. New Views, New Routes
          2. Bar Chart Controller Actions
          3. Bar Chart JavaScript
        3. Scatter Plot
          1. Scatter Plot?
          2. Scatter Plot Controller Actions
          3. Scatter Plot Views and Routes
          4. Scatter Plot JavaScript
        4. Scatter Plot Revisited
        5. Box Plot
          1. Quartiles
          2. Boxes, Whiskers, Circles, What?!
          3. Box Plot Data and Views
          4. Box Plot JavaScript
        6. Summary
      3. Chapter 3. Working with Time Series Data
        1. Historic Daily Weather Data
        2. Weather Rails App
          1. Weather Readings Model
          2. Weather Readings Import
          3. Weather Stations Model
          4. Weather Stations Import
        3. Simple Line Graph
          1. Weather Controller
          2. Fetch the Data
          3. The View Files
          4. Draw the Line Graph
        4. Tweak 1: Simple Multiline Graph
        5. Tweak 2: Add Circle to Highlight the Maximum Temperature
        6. Tweak 3: Add Circle to Highlight the Minimum Temperature
        7. Tweak 4: Add Text to Display the Temperature Change
        8. Tweak 5: Add a Line Between the Focus Circles
        9. Summary
      4. Chapter 4. Working with Large Datasets
        1. Git and Large Files
        2. The Cloud
        3. Hotlinking
        4. Benchmarking
          1. Benchmark and Compare
          2. Benchmark All the Things
        5. Querying “Big Data”
          1. Using Scopes in the Model
          2. Adding Indices
        6. When Benchmarks and Statistics Lie
        7. Summary
    11. Part II: Using SQL in Rails
      1. Chapter 5. Window Functions, Subqueries, and Common Table Expression
        1. Why Use SQL?
          1. Database Portability Is a Lie
          2. Tripping Over ActiveRecord
        2. User-Defined Functions
          1. Why?
          2. Heresy!
          3. How?
        3. How to Use SQL in Rails
        4. Scatter Plot with Mortgage Payment
        5. Window Functions
          1. Window Functions Greatest Hits
          2. Lead and Lag
          3. Partitions
          4. First Value and Last Value
          5. Row Number
        6. Using Subqueries
        7. Common Table Expression
        8. CTE and the Heatmap
          1. The Query
          2. The Controller and View
          3. The JavaScript
        9. Summary
      2. Chapter 6. The Chord Diagram
        1. The Matrix Is the Truth
        2. Flight Departures Data
        3. Departures App
          1. Airports
          2. Carriers
          3. Departures
        4. Transforming the Data
          1. Fetching the Data
          2. Generating the Matrix
          3. Finalizing the Matrix
        5. Create the Views
          1. Departures Controller and Routes
          2. Departures View
          3. Departures Style
        6. Draw the Chord Diagram
        7. Disjointed City Pairs
          1. Using the Lead Window Function to Find Empty Leg Flights
          2. Optimizing Slow Queries with the Materialized View
          3. Draw the Disjointed City Pairs Chord Diagram
        8. Summary
      3. Chapter 7. Time Series Aggregates in Postgres
        1. Finding Flight Segments
          1. Creating a Series of Time
          2. Turning Data into Time Series Data
        2. Graphing the Timeline
          1. Basic Timeline
          2. Fancy Timeline
        3. Summary
      4. Chapter 8. Using a Separate Reporting Database
        1. Transactional versus Reporting Databases
          1. Worker Processes
          2. Postgres Schemas
        2. Working with Multiple Schemas in Rails
          1. Defining the Schema Connection
          2. Creating a New Schema
        3. Creating Objects in the Reporting Schema
          1. Materialized View in the Reporting Schema
          2. Tables in the Reporting Schema
        4. Summary
    12. Part III: Geospatial Rails
      1. Chapter 9. Working with Geospatial Data in Rails
        1. GIS Primer
          1. It’s (Longitude, Latitude) Not (Latitude, Longitude)
          2. Decimal Degrees
          3. Degrees, Minutes, Seconds (DMS)
          4. Datum
          5. Map Projection
          6. Spatial Reference System Identifier (SRID)
          7. Three Feature Types
        2. PostGIS
          1. Postgres Contrib Modules
          2. Installing PostGIS
          3. PostGIS Functions
        3. ActiveRecord and PostGIS
          1. ActiveRecord PostGIS Adapter
          2. Rails PostGIS Configuration
          3. PostGIS Hosting Considerations
        4. Using Geospatial Data in Rails
          1. Creating Geospatial Table Fields
          2. Latitude and Longitude
          3. Simple GIS Calculation
        5. Working with Shapefiles
          1. Shapefile Import Schema
          2. Importing from a Shapefile
          3. Shapefile ETL
          4. Update Missing lonlat Data
        6. Summary
      2. Chapter 10. Making Maps with Leaflet and Rails
        1. Leaflet
          1. Map Tiles
          2. Map Layers
        2. Incorporating Leaflet into Rails to Visualize Weather Stations
          1. Using a Separate Rails Layout for the Map
          2. Map Controller
          3. Map Index
          4. Map Data GeoJSON View
          5. Mapping the Weather Stations
        3. Visualizing Airports
          1. Markers
          2. Marker Cluster
          3. Drawing Flight Paths
        4. Visualizing Zip Codes
          1. Updating the Maryland Residential Sales App for PostGIS
          2. Zip Code Geographies
          3. Importing the Zip Code Shapefile
          4. Mapping Zip Codes
          5. Choropleth
        5. Summary
      3. Chapter 11. Querying Geospatial Data
        1. Finding Items within a Bounding Box
          1. What Is a Bounding Box?
        2. Writing a Bounding Box Query
          1. Writing a Bounding Box Query Using SQL and PostGIS
          2. Writing a Bounding Box Query Using ActiveRecord
          3. Finding Items within the Bounding Box
        3. Finding Items Near a Point
          1. Writing the Query
          2. Using ActiveRecord
        4. Calculating Distance
        5. Summary
    13. Afterword
    14. Appendix A. Ruby and Rails Setup
      1. Install Ruby
        1. Create the App
        2. More Gems
        3. Config Files
      2. Finalize the Setup
    15. Appendix B. Brief Postgres Overview
      1. Installing Postgres
        1. From Source
        2. Package Manager
        3. Postgres.app
      2. SQL Tools
        1. Command Line
        2. GUI Tool
      3. Bulk Importing Data
        1. COPY SQL Statement
        2. \copy PSQL Command
        3. pg_restore
      4. The Query Plan
    16. Appendix C. SQL Join Overview
      1. Join Example Database Setup
      2. Inner Join
      3. Left Outer Join
      4. Right Outer Join
      5. Full Outer Join
      6. Cross Join
      7. Self Join
    17. Index
    18. Code Snippets

    Product information

    • Title: Data Visualization Toolkit: Using JavaScript, Rails™, and Postgres to Present Data and Geospatial Information
    • Author(s): Barrett Clark
    • Release date: August 2016
    • Publisher(s): Addison-Wesley Professional
    • ISBN: 9780134465159