Math for Programmers

Table of contents

1. Math for Programmers
2. Copyright
3. dedication
4. contents
5. front matter
   1. preface
      1. How this book was designed
      2. Mathematical ideas we cover
   2. acknowledgments
   3. about this book
      1. Who should read this book?
      2. How this book is organized
      3. About the code
      4. liveBook discussion forum
   4. about the author
   5. about the cover illustration
6. 1 Learning math with code
   1. 1.1 Solving lucrative problems with math and software
      1. 1.1.1 Predicting financial market movements
      2. 1.1.2 Finding a good deal
      3. 1.1.3 Building 3D graphics and animations
      4. 1.1.4 Modeling the physical world
   2. 1.2 How not to learn math
      1. 1.2.1 Jane wants to learn some math
      2. 1.2.2 Slogging through math textbooks
   3. 1.3 Using your well-trained left brain
      1. 1.3.1 Using a formal language
      2. 1.3.2 Build your own calculator
      3. 1.3.3 Building abstractions with functions
   4. Summary
7. Part 1. Vectors and graphics
8. 2 Drawing with 2D vectors
   1. 2.1 Picturing 2D vectors
      1. 2.1.1 Representing 2D vectors
      2. 2.1.2 2D drawing in Python
      3. 2.1.3 Exercises
   2. 2.2 Plane vector arithmetic
      1. 2.2.1 Vector components and lengths
      2. 2.2.2 Multiplying vectors by numbers
      3. 2.2.3 Subtraction, displacement, and distance
      4. 2.2.4 Exercises
   3. 2.3 Angles and trigonometry in the plane
      1. 2.3.1 From angles to components
      2. 2.3.2 Radians and trigonometry in Python
      3. 2.3.3 From components back to angles
      4. 2.3.4 Exercises
   4. 2.4 Transforming collections of vectors
      1. 2.4.1 Combining vector transformations
      2. 2.4.2 Exercises
   5. 2.5 Drawing with Matplotlib
   6. Summary
9. 3 Ascending to the 3D world
   1. 3.1 Picturing vectors in 3D space
      1. 3.1.1 Representing 3D vectors with coordinates
      2. 3.1.2 3D drawing with Python
      3. 3.1.3 Exercises
   2. 3.2 Vector arithmetic in 3D
      1. 3.2.1 Adding 3D vectors
      2. 3.2.2 Scalar multiplication in 3D
      3. 3.2.3 Subtracting 3D vectors
      4. 3.2.4 Computing lengths and distances
      5. 3.2.5 Computing angles and directions
      6. 3.2.6 Exercises
   3. 3.3 The dot product: Measuring vector alignment
      1. 3.3.1 Picturing the dot product
      2. 3.3.2 Computing the dot product
      3. 3.3.3 Dot products by example
      4. 3.3.4 Measuring angles with the dot product
      5. 3.3.5 Exercises
   4. 3.4 The cross product: Measuring oriented area
      1. 3.4.1 Orienting ourselves in 3D
      2. 3.4.2 Finding the direction of the cross product
      3. 3.4.3 Finding the length of the cross product
      4. 3.4.4 Computing the cross product of 3D vectors
      5. 3.4.5 Exercises
   5. 3.5 Rendering a 3D object in 2D
      1. 3.5.1 Defining a 3D object with vectors
      2. 3.5.2 Projecting to 2D
      3. 3.5.3 Orienting faces and shading
      4. 3.5.4 Exercises
   6. Summary
10. 4 Transforming vectors and graphics
    1. 4.1 Transforming 3D objects
       1. 4.1.1 Drawing a transformed object
       2. 4.1.2 Composing vector transformations
       3. 4.1.3 Rotating an object about an axis
       4. 4.1.4 Inventing your own geometric transformations
       5. 4.1.5 Exercises
    2. 4.2 Linear transformations
       1. 4.2.1 Preserving vector arithmetic
       2. 4.2.2 Picturing linear transformations
       3. 4.2.3 Why linear transformations?
       4. 4.2.4 Computing linear transformations
       5. 4.2.5 Exercises
    3. Summary
11. 5 Computing transformations with matrices
    1. 5.1 Representing linear transformations with matrices
       1. 5.1.1 Writing vectors and linear transformations as matrices
       2. 5.1.2 Multiplying a matrix with a vector
       3. 5.1.3 Composing linear transformations by matrix multiplication
       4. 5.1.4 Implementing matrix multiplication
       5. 5.1.5 3D animation with matrix transformations
       6. 5.1.6 Exercises
    2. 5.2 Interpreting matrices of different shapes
       1. 5.2.1 Column vectors as matrices
       2. 5.2.2 What pairs of matrices can be multiplied?
       3. 5.2.3 Viewing square and non-square matrices as vector functions
       4. 5.2.4 Projection as a linear map from 3D to 2D
       5. 5.2.5 Composing linear maps
       6. 5.2.6 Exercises
    3. 5.3 Translating vectors with matrices
       1. 5.3.1 Making plane translations linear
       2. 5.3.2 Finding a 3D matrix for a 2D translation
       3. 5.3.3 Combining translation with other linear transformations
       4. 5.3.4 Translating 3D objects in a 4D world
       5. 5.3.5 Exercises
    4. Summary
12. 6 Generalizing to higher dimensions
    1. 6.1 Generalizing our definition of vectors
       1. 6.1.1 Creating a class for 2D coordinate vectors
       2. 6.1.2 Improving the Vec2 class
       3. 6.1.3 Repeating the process with 3D vectors
       4. 6.1.4 Building a vector base class
       5. 6.1.5 Defining vector spaces
       6. 6.1.6 Unit testing vector space classes
       7. 6.1.7 Exercises
    2. 6.2 Exploring different vector spaces
       1. 6.2.1 Enumerating all coordinate vector spaces
       2. 6.2.2 Identifying vector spaces in the wild
       3. 6.2.3 Treating functions as vectors
       4. 6.2.4 Treating matrices as vectors
       5. 6.2.5 Manipulating images with vector operations
       6. 6.2.6 Exercises
    3. 6.3 Looking for smaller vector spaces
       1. 6.3.1 Identifying subspaces
       2. 6.3.2 Starting with a single vector
       3. 6.3.3 Spanning a bigger space
       4. 6.3.4 Defining the word dimension
       5. 6.3.5 Finding subspaces of the vector space of functions
       6. 6.3.6 Subspaces of images
       7. 6.3.7 Exercises
    4. Summary
13. 7 Solving systems of linear equations
    1. 7.1 Designing an arcade game
       1. 7.1.1 Modeling the game
       2. 7.1.2 Rendering the game
       3. 7.1.3 Shooting the laser
       4. 7.1.4 Exercises
    2. 7.2 Finding intersection points of lines
       1. 7.2.1 Choosing the right formula for a line
       2. 7.2.2 Finding the standard form equation for a line
       3. 7.2.3 Linear equations in matrix notation
       4. 7.2.4 Solving linear equations with NumPy
       5. 7.2.5 Deciding whether the laser hits an asteroid
       6. 7.2.6 Identifying unsolvable systems
       7. 7.2.7 Exercises
    3. 7.3 Generalizing linear equations to higher dimensions
       1. 7.3.1 Representing planes in 3D
       2. 7.3.2 Solving linear equations in 3D
       3. 7.3.3 Studying hyperplanes algebraically
       4. 7.3.4 Counting dimensions, equations, and solutions
       5. 7.3.5 Exercises
    4. 7.4 Changing basis by solving linear equations
       1. 7.4.1 Solving a 3D example
       2. 7.4.2 Exercises
    5. Summary
14. Part 2. Calculus and physical simulation
15. 8 Understanding rates of change
    1. 8.1 Calculating average flow rate from volume
       1. 8.1.1 Implementing an average_flow_rate function
       2. 8.1.2 Picturing the average flow rate with a secant line
       3. 8.1.3 Negative rates of change
       4. 8.1.4 Exercises
    2. 8.2 Plotting the average flow rate over time
       1. 8.2.1 Finding the average flow rate in different time intervals
       2. 8.2.2 Plotting the interval flow rates
       3. 8.2.3 Exercises
    3. 8.3 Approximating instantaneous flow rates
       1. 8.3.1 Finding the slope of small secant lines
       2. 8.3.2 Building the instantaneous flow rate function
       3. 8.3.3 Currying and plotting the instantaneous flow rate function
       4. 8.3.4 Exercises
    4. 8.4 Approximating the change in volume
       1. 8.4.1 Finding the change in volume for a short time interval
       2. 8.4.2 Breaking up time into smaller intervals
       3. 8.4.3 Picturing the volume change on the flow rate graph
       4. 8.4.4 Exercises
    5. 8.5 Plotting the volume over time
       1. 8.5.1 Finding the volume over time
       2. 8.5.2 Picturing Riemann sums for the volume function
       3. 8.5.3 Improving the approximation
       4. 8.5.4 Definite and indefinite integrals
    6. Summary
16. 9 Simulating moving objects
    1. 9.1 Simulating a constant velocity motion
       1. 9.1.1 Adding velocities to the asteroids
       2. 9.1.2 Updating the game engine to move the asteroids
       3. 9.1.3 Keeping the asteroids on the screen
       4. 9.1.4 Exercises
    2. 9.2 Simulating acceleration
       1. 9.2.1 Accelerating the spaceship
    3. 9.3 Digging deeper into Euler’s method
       1. 9.3.1 Carrying out Euler’s method by hand
       2. 9.3.2 Implementing the algorithm in Python
    4. 9.4 Running Euler’s method with smaller time steps
       1. 9.4.1 Exercises
    5. Summary
17. 10 Working with symbolic expressions
    1. 10.1 Finding an exact derivative with a computer algebra system
       1. 10.1.1 Doing symbolic algebra in Python
    2. 10.2 Modeling algebraic expressions
       1. 10.2.1 Breaking an expression into pieces
       2. 10.2.2 Building an expression tree
       3. 10.2.3 Translating the expression tree to Python
       4. 10.2.4 Exercises
    3. 10.3 Putting a symbolic expression to work
       1. 10.3.1 Finding all the variables in an expression
       2. 10.3.2 Evaluating an expression
       3. 10.3.3 Expanding an expression
       4. 10.3.4 Exercises
    4. 10.4 Finding the derivative of a function
       1. 10.4.1 Derivatives of powers
       2. 10.4.2 Derivatives of transformed functions
       3. 10.4.3 Derivatives of some special functions
       4. 10.4.4 Derivatives of products and compositions
       5. 10.4.5 Exercises
    5. 10.5 Taking derivatives automatically
       1. 10.5.1 Implementing a derivative method for expressions
       2. 10.5.2 Implementing the product rule and chain rule
       3. 10.5.3 Implementing the power rule
       4. 10.5.4 Exercises
    6. 10.6 Integrating functions symbolically
       1. 10.6.1 Integrals as antiderivatives
       2. 10.6.2 Introducing the SymPy library
       3. 10.6.3 Exercises
    7. Summary
18. 11 Simulating force fields
    1. 11.1 Modeling gravity with a vector field
       1. 11.1.1 Modeling gravity with a potential energy function
    2. 11.2 Modeling gravitational fields
       1. 11.2.1 Defining a vector field
       2. 11.2.2 Defining a simple force field
    3. 11.3 Adding gravity to the asteroid game
       1. 11.3.1 Making game objects feel gravity
       2. 11.3.2 Exercises
    4. 11.4 Introducing potential energy
       1. 11.4.1 Defining a potential energy scalar field
       2. 11.4.2 Plotting a scalar field as a heatmap
       3. 11.4.3 Plotting a scalar field as a contour map
    5. 11.5 Connecting energy and forces with the gradient
       1. 11.5.1 Measuring steepness with cross sections
       2. 11.5.2 Calculating partial derivatives
       3. 11.5.3 Finding the steepness of a graph with the gradient
       4. 11.5.4 Calculating force fields from potential energy with the gradient
       5. 11.5.5 Exercises
    6. Summary
19. 12 Optimizing a physical system
    1. 12.1 Testing a projectile simulation
       1. 12.1.1 Building a simulation with Euler’s method
       2. 12.1.2 Measuring properties of the trajectory
       3. 12.1.3 Exploring different launch angles
       4. 12.1.4 Exercises
    2. 12.2 Calculating the optimal range
       1. 12.2.1 Finding the projectile range as a function of the launch angle
       2. 12.2.2 Solving for the maximum range
       3. 12.2.3 Identifying maxima and minima
       4. 12.2.4 Exercises
    3. 12.3 Enhancing our simulation
       1. 12.3.1 Adding another dimension
       2. 12.3.2 Modeling terrain around the cannon
       3. 12.3.3 Solving for the range of the projectile in 3D
       4. 12.3.4 Exercises
    4. 12.4 Optimizing range using gradient ascent
       1. 12.4.1 Plotting range versus launch parameters
       2. 12.4.2 The gradient of the range function
       3. 12.4.3 Finding the uphill direction with the gradient
       4. 12.4.4 Implementing gradient ascent
       5. 12.4.5 Exercises
    5. Summary
20. 13 Analyzing sound waves with a Fourier series
    1. 13.1 Combining sound waves and decomposing them
    2. 13.2 Playing sound waves in Python
       1. 13.2.1 Producing our first sound
       2. 13.2.2 Playing a musical note
       3. 13.2.3 Exercises
    3. 13.3 Turning a sinusoidal wave into a sound
       1. 13.3.1 Making audio from sinusoidal functions
       2. 13.3.2 Changing the frequency of a sinusoid
       3. 13.3.3 Sampling and playing the sound wave
       4. 13.3.4 Exercises
    4. 13.4 Combining sound waves to make new ones
       1. 13.4.1 Adding sampled sound waves to build a chord
       2. 13.4.2 Picturing the sum of two sound waves
       3. 13.4.3 Building a linear combination of sinusoids
       4. 13.4.4 Building a familiar function with sinusoids
       5. 13.4.5 Exercises
    5. 13.5 Decomposing a sound wave into its Fourier series
       1. 13.5.1 Finding vector components with an inner product
       2. 13.5.2 Defining an inner product for periodic functions
       3. 13.5.3 Writing a function to find Fourier coefficients
       4. 13.5.4 Finding the Fourier coefficients for the square wave
       5. 13.5.5 Fourier coefficients for other waveforms
       6. 13.5.6 Exercises
    6. Summary
21. Part 3. Machine learning applications
22. 14 Fitting functions to data
    1. 14.1 Measuring the quality of fit for a function
       1. 14.1.1 Measuring distance from a function
       2. 14.1.2 Summing the squares of the errors
       3. 14.1.3 Calculating cost for car price functions
       4. 14.1.4 Exercises
    2. 14.2 Exploring spaces of functions
       1. 14.2.1 Picturing cost for lines through the origin
       2. 14.2.2 The space of all linear functions
       3. 14.2.3 Exercises
    3. 14.3 Finding the line of best fit using gradient descent
       1. 14.3.1 Rescaling the data
       2. 14.3.2 Finding and plotting the line of best fit
       3. 14.3.3 Exercises
    4. 14.4 Fitting a nonlinear function
       1. 14.4.1 Understanding the behavior of exponential functions
       2. 14.4.2 Finding the exponential function of best fit
       3. 14.4.3 Exercises
    5. Summary
23. 15 Classifying data with logistic regression
    1. 15.1 Testing a classification function on real data
       1. 15.1.1 Loading the car data
       2. 15.1.2 Testing the classification function
       3. 15.1.3 Exercises
    2. 15.2 Picturing a decision boundary
       1. 15.2.1 Picturing the space of cars
       2. 15.2.2 Drawing a better decision boundary
       3. 15.2.3 Implementing the classification function
       4. 15.2.4 Exercises
    3. 15.3 Framing classification as a regression problem
       1. 15.3.1 Scaling the raw car data
       2. 15.3.2 Measuring the “BMWness” of a car
       3. 15.3.3 Introducing the sigmoid function
       4. 15.3.4 Composing the sigmoid function with other functions
       5. 15.3.5 Exercises
    4. 15.4 Exploring possible logistic functions
       1. 15.4.1 Parameterizing logistic functions
       2. 15.4.2 Measuring the quality of fit for a logistic function
       3. 15.4.3 Testing different logistic functions
       4. 15.4.4 Exercises
    5. 15.5 Finding the best logistic function
       1. 15.5.1 Gradient descent in three dimensions
       2. 15.5.2 Using gradient descent to find the best fit
       3. 15.5.3 Testing and understanding the best logistic classifier
       4. 15.5.4 Exercises
    6. Summary
24. 16 Training neural networks
    1. 16.1 Classifying data with neural networks
    2. 16.2 Classifying images of handwritten digits
       1. 16.2.1 Building the 64-dimensional image vectors
       2. 16.2.2 Building a random digit classifier
       3. 16.2.3 Measuring performance of the digit classifier
       4. 16.2.4 Exercises
    3. 16.3 Designing a neural network
       1. 16.3.1 Organizing neurons and connections
       2. 16.3.2 Data flow through a neural network
       3. 16.3.3 Calculating activations
       4. 16.3.4 Calculating activations in matrix notation
       5. 16.3.5 Exercises
    4. 16.4 Building a neural network in Python
       1. 16.4.1 Implementing an MLP class in Python
       2. 16.4.2 Evaluating the MLP
       3. 16.4.3 Testing the classification performance of an MLP
       4. 16.4.4 Exercises
    5. 16.5 Training a neural network using gradient descent
       1. 16.5.1 Framing training as a minimization problem
       2. 16.5.2 Calculating gradients with backpropagation
       3. 16.5.3 Automatic training with scikit-learn
       4. 16.5.4 Exercises
    6. 16.6 Calculating gradients with backpropagation
       1. 16.6.1 Finding the cost in terms of the last layer weights
       2. 16.6.2 Calculating the partial derivatives for the last layer weights using the chain rule
       3. 16.6.3 Exercises
    7. Summary
25. Appendix A. Getting set up with Python
    1. A.1 Checking for an existing Python installation
    2. A.2 Downloading and installing Anaconda
    3. A.3 Using Python in interactive mode
       1. A.3.1 Creating and running a Python script file
       2. A.3.2 Using Jupyter notebooks
26. Appendix B. Python tips and tricks
    1. B.1 Python numbers and math
       1. B.1.1 The math module
       2. B.1.2 Random numbers
    2. B.2 Collections of data in Python
       1. B.2.1 Lists
       2. B.2.2 Other iterables
       3. B.2.3 Generators
       4. B.2.4 Tuples
       5. B.2.5 Sets
       6. B.2.6 NumPy arrays
       7. B.2.7 Dictionaries
       8. B.2.8 Useful collection functions
    3. B.3 Working with functions
       1. B.3.1 Giving functions more inputs
       2. B.3.2 Keyword arguments
       3. B.3.3 Functions as data
       4. B.3.4 Lambdas: Anonymous functions
       5. B.3.5 Applying functions to NumPy arrays
    4. B.4 Plotting data with Matplotlib
       1. B.4.1 Making a scatter plot
       2. B.4.2 Making a line chart
       3. B.4.3. More plot customizations
    5. B.5 Object-oriented programming in Python
       1. B.5.1 Defining classes
       2. B.5.2 Defining methods
       3. B.5.3 Special methods
       4. B.5.4 Operator overloading
       5. B.5.5 Class methods
       6. B.5.6 Inheritance and abstract classes
27. Appendix C. Loading and rendering 3D Models with OpenGL and PyGame
    1. C.1 Recreating the octahedron from chapter 3
    2. C.2 Changing our perspective
    3. C.3 Loading and rendering the Utah teapot
    4. C.4 Exercises
28. index