Practical Linear Algebra for Data Science

Practical Linear Algebra for Data Science

by Mike X Cohen
Released September 2022
Publisher(s): O'Reilly Media, Inc.
ISBN: 9781098120610

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Buy on Amazon Buy on ebooks.com
Start your free trial

Book description

If you want to work in any computational or technical field, you need to understand linear algebra. As the study of matrices and operations acting upon them, linear algebra is the mathematical basis of nearly all algorithms and analyses implemented in computers. But the way it's presented in decades-old textbooks is much different from how professionals use linear algebra today to solve real-world modern applications.

This practical guide from Mike X Cohen teaches the core concepts of linear algebra as implemented in Python, including how they're used in data science, machine learning, deep learning, computational simulations, and biomedical data processing applications. Armed with knowledge from this book, you'll be able to understand, implement, and adapt myriad modern analysis methods and algorithms.

Ideal for practitioners and students using computer technology and algorithms, this book introduces you to:

  • The interpretations and applications of vectors and matrices
  • Matrix arithmetic (various multiplications and transformations)
  • Independence, rank, and inverses
  • Important decompositions used in applied linear algebra (including LU and QR)
  • Eigendecomposition and singular value decomposition
  • Applications including least-squares model fitting and principal components analysis

Publisher resources

View/Submit Errata

Table of contents

  1. Preface
    1. Conventions Used in This Book
    2. Using Code Examples
    3. O’Reilly Online Learning
    4. How to Contact Us
    5. Acknowledgments
  2. 1. Introduction
    1. What Is Linear Algebra and Why Learn It?
    2. About This Book
    3. Prerequisites
      1. Math
      2. Attitude
      3. Coding
    4. Mathematical Proofs Versus Intuition from Coding
    5. Code, Printed in the Book and Downloadable Online
    6. Code Exercises
    7. How to Use This Book (for Teachers and Self Learners)
  3. 2. Vectors, Part 1
    1. Creating and Visualizing Vectors in NumPy
      1. Geometry of Vectors
    2. Operations on Vectors
      1. Adding Two Vectors
      2. Geometry of Vector Addition and Subtraction
      3. Vector-Scalar Multiplication
      4. Scalar-Vector Addition
      5. Transpose
      6. Vector Broadcasting in Python
    3. Vector Magnitude and Unit Vectors
    4. The Vector Dot Product
      1. The Dot Product Is Distributive
      2. Geometry of the Dot Product
    5. Other Vector Multiplications
      1. Hadamard Multiplication
      2. Outer Product
      3. Cross and Triple Products
    6. Orthogonal Vector Decomposition
    7. Summary
    8. Code Exercises
  4. 3. Vectors, Part 2
    1. Vector Sets
    2. Linear Weighted Combination
    3. Linear Independence
      1. The Math of Linear Independence
      2. Independence and the Zeros Vector
    4. Subspace and Span
    5. Basis
      1. Definition of Basis
    6. Summary
    7. Code Exercises
  5. 4. Vector Applications
    1. Correlation and Cosine Similarity
    2. Time Series Filtering and Feature Detection
    3. k-Means Clustering
    4. Code Exercises
      1. Correlation Exercises
      2. Filtering and Feature Detection Exercises
      3. k-Means Exercises
  6. 5. Matrices, Part 1
    1. Creating and Visualizing Matrices in NumPy
      1. Visualizing, Indexing, and Slicing Matrices
      2. Special Matrices
    2. Matrix Math: Addition, Scalar Multiplication, Hadamard Multiplication
      1. Addition and Subtraction
      2. “Shifting” a Matrix
      3. Scalar and Hadamard Multiplications
    3. Standard Matrix Multiplication
      1. Rules for Matrix Multiplication Validity
      2. Matrix Multiplication
      3. Matrix-Vector Multiplication
    4. Matrix Operations: Transpose
      1. Dot and Outer Product Notation
    5. Matrix Operations: LIVE EVIL (Order of Operations)
    6. Symmetric Matrices
      1. Creating Symmetric Matrices from Nonsymmetric Matrices
    7. Summary
    8. Code Exercises
  7. 6. Matrices, Part 2
    1. Matrix Norms
      1. Matrix Trace and Frobenius Norm
    2. Matrix Spaces (Column, Row, Nulls)
      1. Column Space
      2. Row Space
      3. Null Spaces
    3. Rank
      1. Ranks of Special Matrices
      2. Rank of Added and Multiplied Matrices
      3. Rank of Shifted Matrices
      4. Theory and Practice
    4. Rank Applications
      1. In the Column Space?
      2. Linear Independence of a Vector Set
    5. Determinant
      1. Computing the Determinant
      2. Determinant with Linear Dependencies
      3. The Characteristic Polynomial
    6. Summary
    7. Code Exercises
  8. 7. Matrix Applications
    1. Multivariate Data Covariance Matrices
    2. Geometric Transformations via Matrix-Vector Multiplication
    3. Image Feature Detection
    4. Summary
    5. Code Exercises
      1. Covariance and Correlation Matrices Exercises
      2. Geometric Transformations Exercises
      3. Image Feature Detection Exercises
  9. 8. Matrix Inverse
    1. The Matrix Inverse
    2. Types of Inverses and Conditions for Invertibility
    3. Computing the Inverse
      1. Inverse of a 2 × 2 Matrix
      2. Inverse of a Diagonal Matrix
      3. Inverting Any Square Full-Rank Matrix
      4. One-Sided Inverses
    4. The Inverse Is Unique
    5. Moore-Penrose Pseudoinverse
    6. Numerical Stability of the Inverse
    7. Geometric Interpretation of the Inverse
    8. Summary
    9. Code Exercises
  10. 9. Orthogonal Matrices and QR Decomposition
    1. Orthogonal Matrices
    2. Gram-Schmidt
    3. QR Decomposition
      1. Sizes of Q and R
      2. QR and Inverses
    4. Summary
    5. Code Exercises
  11. 10. Row Reduction and LU Decomposition
    1. Systems of Equations
      1. Converting Equations into Matrices
      2. Working with Matrix Equations
    2. Row Reduction
      1. Gaussian Elimination
      2. Gauss-Jordan Elimination
      3. Matrix Inverse via Gauss-Jordan Elimination
    3. LU Decomposition
      1. Row Swaps via Permutation Matrices
    4. Summary
    5. Code Exercises
  12. 11. General Linear Models and Least Squares
    1. General Linear Models
      1. Terminology
      2. Setting Up a General Linear Model
    2. Solving GLMs
      1. Is the Solution Exact?
      2. A Geometric Perspective on Least Squares
      3. Why Does Least Squares Work?
    3. GLM in a Simple Example
    4. Least Squares via QR
    5. Summary
    6. Code Exercises
  13. 12. Least Squares Applications
    1. Predicting Bike Rentals Based on Weather
      1. Regression Table Using statsmodels
      2. Multicollinearity
      3. Regularization
    2. Polynomial Regression
    3. Grid Search to Find Model Parameters
    4. Summary
    5. Code Exercises
      1. Bike Rental Exercises
      2. Multicollinearity Exercise
      3. Regularization Exercise
      4. Polynomial Regression Exercise
      5. Grid Search Exercises
  14. 13. Eigendecomposition
    1. Interpretations of Eigenvalues and Eigenvectors
      1. Geometry
      2. Statistics (Principal Components Analysis)
      3. Noise Reduction
      4. Dimension Reduction (Data Compression)
    2. Finding Eigenvalues
    3. Finding Eigenvectors
      1. Sign and Scale Indeterminacy of Eigenvectors
    4. Diagonalizing a Square Matrix
    5. The Special Awesomeness of Symmetric Matrices
      1. Orthogonal Eigenvectors
      2. Real-Valued Eigenvalues
    6. Eigendecomposition of Singular Matrices
    7. Quadratic Form, Definiteness, and Eigenvalues
      1. The Quadratic Form of a Matrix
      2. Definiteness
      3. 𝐀 T 𝐀 Is Positive (Semi)definite
    8. Generalized Eigendecomposition
    9. Summary
    10. Code Exercises
  15. 14. Singular Value Decomposition
    1. The Big Picture of the SVD
      1. Singular Values and Matrix Rank
    2. SVD in Python
    3. SVD and Rank-1 “Layers” of a Matrix
    4. SVD from EIG
      1. SVD of 𝐀 T 𝐀
      2. Converting Singular Values to Variance, Explained
      3. Condition Number
    5. SVD and the MP Pseudoinverse
    6. Summary
    7. Code Exercises
  16. 15. Eigendecomposition and SVD Applications
    1. PCA Using Eigendecomposition and SVD
      1. The Math of PCA
      2. The Steps to Perform a PCA
      3. PCA via SVD
    2. Linear Discriminant Analysis
    3. Low-Rank Approximations via SVD
      1. SVD for Denoising
    4. Summary
    5. Exercises
      1. PCA
      2. Linear Discriminant Analyses
      3. SVD for Low-Rank Approximations
      4. SVD for Image Denoising
  17. 16. Python Tutorial
    1. Why Python, and What Are the Alternatives?
    2. IDEs (Interactive Development Environments)
    3. Using Python Locally and Online
      1. Working with Code Files in Google Colab
    4. Variables
      1. Data Types
      2. Indexing
    5. Functions
      1. Methods as Functions
      2. Writing Your Own Functions
      3. Libraries
      4. NumPy
      5. Indexing and Slicing in NumPy
    6. Visualization
    7. Translating Formulas to Code
    8. Print Formatting and F-Strings
    9. Control Flow
      1. Comparators
      2. If Statements
      3. For Loops
      4. Nested Control Statements
    10. Measuring Computation Time
    11. Getting Help and Learning More
      1. What to Do When Things Go Awry
    12. Summary
  18. Index
  19. About the Author

Product information

  • Title: Practical Linear Algebra for Data Science
  • Author(s): Mike X Cohen
  • Release date: September 2022
  • Publisher(s): O'Reilly Media, Inc.
  • ISBN: 9781098120610