book

Math for Programmers

Name: Math for Programmers
Author: Paul Orland
ISBN: 9781617295355

by Paul Orland

January 2021

Intermediate to advanced

688 pages

19h 22m

English

Manning Publications

Audiobook available

Read now

Unlock full access

Math for Programmers
Copyright
dedication
contents
front matter
prefaceHow this book was designedMathematical ideas we coveracknowledgmentsabout this bookWho should read this book?How this book is organizedAbout the codeliveBook discussion forumabout the authorabout the cover illustration
1 Learning math with code
1.1 Solving lucrative problems with math and software1.1.1 Predicting financial market movements1.1.2 Finding a good deal1.1.3 Building 3D graphics and animations1.1.4 Modeling the physical world1.2 How not to learn math1.2.1 Jane wants to learn some math1.2.2 Slogging through math textbooks1.3 Using your well-trained left brain1.3.1 Using a formal language1.3.2 Build your own calculator1.3.3 Building abstractions with functionsSummary
Part 1. Vectors and graphics
2 Drawing with 2D vectors
2.1 Picturing 2D vectors2.1.1 Representing 2D vectors2.1.2 2D drawing in Python2.1.3 Exercises2.2 Plane vector arithmetic2.2.1 Vector components and lengths2.2.2 Multiplying vectors by numbers2.2.3 Subtraction, displacement, and distance2.2.4 Exercises2.3 Angles and trigonometry in the plane2.3.1 From angles to components2.3.2 Radians and trigonometry in Python2.3.3 From components back to angles2.3.4 Exercises2.4 Transforming collections of vectors2.4.1 Combining vector transformations2.4.2 Exercises2.5 Drawing with MatplotlibSummary
3 Ascending to the 3D world
3.1 Picturing vectors in 3D space3.1.1 Representing 3D vectors with coordinates3.1.2 3D drawing with Python3.1.3 Exercises3.2 Vector arithmetic in 3D3.2.1 Adding 3D vectors3.2.2 Scalar multiplication in 3D3.2.3 Subtracting 3D vectors3.2.4 Computing lengths and distances3.2.5 Computing angles and directions3.2.6 Exercises3.3 The dot product: Measuring vector alignment3.3.1 Picturing the dot product3.3.2 Computing the dot product3.3.3 Dot products by example3.3.4 Measuring angles with the dot product3.3.5 Exercises3.4 The cross product: Measuring oriented area3.4.1 Orienting ourselves in 3D3.4.2 Finding the direction of the cross product3.4.3 Finding the length of the cross product3.4.4 Computing the cross product of 3D vectors3.4.5 Exercises3.5 Rendering a 3D object in 2D3.5.1 Defining a 3D object with vectors3.5.2 Projecting to 2D3.5.3 Orienting faces and shading3.5.4 ExercisesSummary
4 Transforming vectors and graphics
4.1 Transforming 3D objects4.1.1 Drawing a transformed object4.1.2 Composing vector transformations4.1.3 Rotating an object about an axis4.1.4 Inventing your own geometric transformations4.1.5 Exercises4.2 Linear transformations4.2.1 Preserving vector arithmetic4.2.2 Picturing linear transformations4.2.3 Why linear transformations?4.2.4 Computing linear transformations4.2.5 ExercisesSummary

5 Computing transformations with matrices
5.1 Representing linear transformations with matrices5.1.1 Writing vectors and linear transformations as matrices5.1.2 Multiplying a matrix with a vector5.1.3 Composing linear transformations by matrix multiplication5.1.4 Implementing matrix multiplication5.1.5 3D animation with matrix transformations5.1.6 Exercises5.2 Interpreting matrices of different shapes5.2.1 Column vectors as matrices5.2.2 What pairs of matrices can be multiplied?5.2.3 Viewing square and non-square matrices as vector functions5.2.4 Projection as a linear map from 3D to 2D5.2.5 Composing linear maps5.2.6 Exercises5.3 Translating vectors with matrices5.3.1 Making plane translations linear5.3.2 Finding a 3D matrix for a 2D translation5.3.3 Combining translation with other linear transformations5.3.4 Translating 3D objects in a 4D world5.3.5 ExercisesSummary
6 Generalizing to higher dimensions
6.1 Generalizing our definition of vectors6.1.1 Creating a class for 2D coordinate vectors6.1.2 Improving the Vec2 class6.1.3 Repeating the process with 3D vectors6.1.4 Building a vector base class6.1.5 Defining vector spaces6.1.6 Unit testing vector space classes6.1.7 Exercises6.2 Exploring different vector spaces6.2.1 Enumerating all coordinate vector spaces6.2.2 Identifying vector spaces in the wild6.2.3 Treating functions as vectors6.2.4 Treating matrices as vectors6.2.5 Manipulating images with vector operations6.2.6 Exercises6.3 Looking for smaller vector spaces6.3.1 Identifying subspaces6.3.2 Starting with a single vector6.3.3 Spanning a bigger space6.3.4 Defining the word dimension6.3.5 Finding subspaces of the vector space of functions6.3.6 Subspaces of images6.3.7 ExercisesSummary
7 Solving systems of linear equations
7.1 Designing an arcade game7.1.1 Modeling the game7.1.2 Rendering the game7.1.3 Shooting the laser7.1.4 Exercises7.2 Finding intersection points of lines7.2.1 Choosing the right formula for a line7.2.2 Finding the standard form equation for a line7.2.3 Linear equations in matrix notation7.2.4 Solving linear equations with NumPy7.2.5 Deciding whether the laser hits an asteroid7.2.6 Identifying unsolvable systems7.2.7 Exercises7.3 Generalizing linear equations to higher dimensions7.3.1 Representing planes in 3D7.3.2 Solving linear equations in 3D7.3.3 Studying hyperplanes algebraically7.3.4 Counting dimensions, equations, and solutions7.3.5 Exercises7.4 Changing basis by solving linear equations7.4.1 Solving a 3D example7.4.2 ExercisesSummary
Part 2. Calculus and physical simulation
8 Understanding rates of change
8.1 Calculating average flow rate from volume8.1.1 Implementing an average_flow_rate function8.1.2 Picturing the average flow rate with a secant line8.1.3 Negative rates of change8.1.4 Exercises8.2 Plotting the average flow rate over time8.2.1 Finding the average flow rate in different time intervals8.2.2 Plotting the interval flow rates8.2.3 Exercises8.3 Approximating instantaneous flow rates8.3.1 Finding the slope of small secant lines8.3.2 Building the instantaneous flow rate function8.3.3 Currying and plotting the instantaneous flow rate function8.3.4 Exercises8.4 Approximating the change in volume8.4.1 Finding the change in volume for a short time interval8.4.2 Breaking up time into smaller intervals8.4.3 Picturing the volume change on the flow rate graph8.4.4 Exercises8.5 Plotting the volume over time8.5.1 Finding the volume over time8.5.2 Picturing Riemann sums for the volume function8.5.3 Improving the approximation8.5.4 Definite and indefinite integralsSummary
9 Simulating moving objects
9.1 Simulating a constant velocity motion9.1.1 Adding velocities to the asteroids9.1.2 Updating the game engine to move the asteroids9.1.3 Keeping the asteroids on the screen9.1.4 Exercises9.2 Simulating acceleration9.2.1 Accelerating the spaceship9.3 Digging deeper into Euler’s method9.3.1 Carrying out Euler’s method by hand9.3.2 Implementing the algorithm in Python9.4 Running Euler’s method with smaller time steps9.4.1 ExercisesSummary
10 Working with symbolic expressions
10.1 Finding an exact derivative with a computer algebra system10.1.1 Doing symbolic algebra in Python10.2 Modeling algebraic expressions10.2.1 Breaking an expression into pieces10.2.2 Building an expression tree10.2.3 Translating the expression tree to Python10.2.4 Exercises10.3 Putting a symbolic expression to work10.3.1 Finding all the variables in an expression10.3.2 Evaluating an expression10.3.3 Expanding an expression10.3.4 Exercises10.4 Finding the derivative of a function10.4.1 Derivatives of powers10.4.2 Derivatives of transformed functions10.4.3 Derivatives of some special functions10.4.4 Derivatives of products and compositions10.4.5 Exercises10.5 Taking derivatives automatically10.5.1 Implementing a derivative method for expressions10.5.2 Implementing the product rule and chain rule10.5.3 Implementing the power rule10.5.4 Exercises10.6 Integrating functions symbolically10.6.1 Integrals as antiderivatives10.6.2 Introducing the SymPy library10.6.3 ExercisesSummary
11 Simulating force fields
11.1 Modeling gravity with a vector field11.1.1 Modeling gravity with a potential energy function11.2 Modeling gravitational fields11.2.1 Defining a vector field11.2.2 Defining a simple force field11.3 Adding gravity to the asteroid game11.3.1 Making game objects feel gravity11.3.2 Exercises11.4 Introducing potential energy11.4.1 Defining a potential energy scalar field11.4.2 Plotting a scalar field as a heatmap11.4.3 Plotting a scalar field as a contour map11.5 Connecting energy and forces with the gradient11.5.1 Measuring steepness with cross sections11.5.2 Calculating partial derivatives11.5.3 Finding the steepness of a graph with the gradient11.5.4 Calculating force fields from potential energy with the gradient11.5.5 ExercisesSummary
12 Optimizing a physical system
12.1 Testing a projectile simulation12.1.1 Building a simulation with Euler’s method12.1.2 Measuring properties of the trajectory12.1.3 Exploring different launch angles12.1.4 Exercises12.2 Calculating the optimal range12.2.1 Finding the projectile range as a function of the launch angle12.2.2 Solving for the maximum range12.2.3 Identifying maxima and minima12.2.4 Exercises12.3 Enhancing our simulation12.3.1 Adding another dimension12.3.2 Modeling terrain around the cannon12.3.3 Solving for the range of the projectile in 3D12.3.4 Exercises12.4 Optimizing range using gradient ascent12.4.1 Plotting range versus launch parameters12.4.2 The gradient of the range function12.4.3 Finding the uphill direction with the gradient12.4.4 Implementing gradient ascent12.4.5 ExercisesSummary
13 Analyzing sound waves with a Fourier series
13.1 Combining sound waves and decomposing them13.2 Playing sound waves in Python13.2.1 Producing our first sound13.2.2 Playing a musical note13.2.3 Exercises13.3 Turning a sinusoidal wave into a sound13.3.1 Making audio from sinusoidal functions13.3.2 Changing the frequency of a sinusoid13.3.3 Sampling and playing the sound wave13.3.4 Exercises13.4 Combining sound waves to make new ones13.4.1 Adding sampled sound waves to build a chord13.4.2 Picturing the sum of two sound waves13.4.3 Building a linear combination of sinusoids13.4.4 Building a familiar function with sinusoids13.4.5 Exercises13.5 Decomposing a sound wave into its Fourier series13.5.1 Finding vector components with an inner product13.5.2 Defining an inner product for periodic functions13.5.3 Writing a function to find Fourier coefficients13.5.4 Finding the Fourier coefficients for the square wave13.5.5 Fourier coefficients for other waveforms13.5.6 ExercisesSummary
Part 3. Machine learning applications
14 Fitting functions to data
14.1 Measuring the quality of fit for a function14.1.1 Measuring distance from a function14.1.2 Summing the squares of the errors14.1.3 Calculating cost for car price functions14.1.4 Exercises14.2 Exploring spaces of functions14.2.1 Picturing cost for lines through the origin14.2.2 The space of all linear functions14.2.3 Exercises14.3 Finding the line of best fit using gradient descent14.3.1 Rescaling the data14.3.2 Finding and plotting the line of best fit14.3.3 Exercises14.4 Fitting a nonlinear function14.4.1 Understanding the behavior of exponential functions14.4.2 Finding the exponential function of best fit14.4.3 ExercisesSummary
15 Classifying data with logistic regression
15.1 Testing a classification function on real data15.1.1 Loading the car data15.1.2 Testing the classification function15.1.3 Exercises15.2 Picturing a decision boundary15.2.1 Picturing the space of cars15.2.2 Drawing a better decision boundary15.2.3 Implementing the classification function15.2.4 Exercises15.3 Framing classification as a regression problem15.3.1 Scaling the raw car data15.3.2 Measuring the “BMWness” of a car15.3.3 Introducing the sigmoid function15.3.4 Composing the sigmoid function with other functions15.3.5 Exercises15.4 Exploring possible logistic functions15.4.1 Parameterizing logistic functions15.4.2 Measuring the quality of fit for a logistic function15.4.3 Testing different logistic functions15.4.4 Exercises15.5 Finding the best logistic function15.5.1 Gradient descent in three dimensions15.5.2 Using gradient descent to find the best fit15.5.3 Testing and understanding the best logistic classifier15.5.4 ExercisesSummary
16 Training neural networks
16.1 Classifying data with neural networks16.2 Classifying images of handwritten digits16.2.1 Building the 64-dimensional image vectors16.2.2 Building a random digit classifier16.2.3 Measuring performance of the digit classifier16.2.4 Exercises16.3 Designing a neural network16.3.1 Organizing neurons and connections16.3.2 Data flow through a neural network16.3.3 Calculating activations16.3.4 Calculating activations in matrix notation16.3.5 Exercises16.4 Building a neural network in Python16.4.1 Implementing an MLP class in Python16.4.2 Evaluating the MLP16.4.3 Testing the classification performance of an MLP16.4.4 Exercises16.5 Training a neural network using gradient descent16.5.1 Framing training as a minimization problem16.5.2 Calculating gradients with backpropagation16.5.3 Automatic training with scikit-learn16.5.4 Exercises16.6 Calculating gradients with backpropagation16.6.1 Finding the cost in terms of the last layer weights16.6.2 Calculating the partial derivatives for the last layer weights using the chain rule16.6.3 ExercisesSummary
Appendix A. Getting set up with Python
A.1 Checking for an existing Python installationA.2 Downloading and installing AnacondaA.3 Using Python in interactive modeA.3.1 Creating and running a Python script fileA.3.2 Using Jupyter notebooks
Appendix B. Python tips and tricks
B.1 Python numbers and mathB.1.1 The math moduleB.1.2 Random numbersB.2 Collections of data in PythonB.2.1 ListsB.2.2 Other iterablesB.2.3 GeneratorsB.2.4 TuplesB.2.5 SetsB.2.6 NumPy arraysB.2.7 DictionariesB.2.8 Useful collection functionsB.3 Working with functionsB.3.1 Giving functions more inputsB.3.2 Keyword argumentsB.3.3 Functions as dataB.3.4 Lambdas: Anonymous functionsB.3.5 Applying functions to NumPy arraysB.4 Plotting data with MatplotlibB.4.1 Making a scatter plotB.4.2 Making a line chartB.4.3. More plot customizationsB.5 Object-oriented programming in PythonB.5.1 Defining classesB.5.2 Defining methodsB.5.3 Special methodsB.5.4 Operator overloadingB.5.5 Class methodsB.5.6 Inheritance and abstract classes
Appendix C. Loading and rendering 3D Models with OpenGL and PyGame
C.1 Recreating the octahedron from chapter 3C.2 Changing our perspectiveC.3 Loading and rendering the Utah teapotC.4 Exercises
index

Content preview from Math for Programmers

16 Training neural networks

This chapter covers

Classifying images of handwritten digits as vector data
Designing a type of neural network called a multilayer perceptron
Evaluating a neural network as a vector transformation
Fitting a neural network to data with a cost function and gradient descent
Calculating partial derivatives for neural networks in backpropagation

In the final chapter of this book, we combine almost everything you’ve learned so far to introduce one of the most famous machine learning tools used today: artificial neural networks. Artificial neural networks, or neural networks for short, are mathematical functions whose structure is loosely based on the structure of the human brain. These are called artificial to distinguish ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781617295355Publisher Support Publisher Website Errata Page

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Math for Programmers

by Paul Orland

16 Training neural networks

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.