book

15 Math Concepts Every Data Scientist Should Know

Name: 15 Math Concepts Every Data Scientist Should Know
Author: David Hoyle
ISBN: 9781837634187

by David Hoyle

August 2024

Intermediate to advanced

510 pages

17h 15m

English

Packt Publishing

Read now

Unlock full access

15 Math Concepts Every Data Scientist Should Know
Contributors
About the author
About the reviewer
Preface
Who this book is forWhat this book coversTo get the most out of this bookDownload the example code filesConventions usedGet in touchShare your thoughtsDownload a free PDF copy of this book
Part 1: Essential Concepts
Chapter 1: Recap of Mathematical Notation and Terminology
Technical requirementsNumber systemsNotation for numbers and fieldsComplex numbersWhat we learnedLinear algebraVectorsMatricesWhat we learnedSums, products, and logarithmsSums and the 𝚺 notationProducts and the notationLogarithmsWhat we learnedDifferential and integral calculusDifferentiationFinding maxima and minimaIntegrationWhat we learnedAnalysisLimitsOrder notationTaylor series expansionsWhat we learnedCombinatoricsBinomial coefficientsWhat we learnedSummaryNotes and further reading
Chapter 2: Random Variables and Probability Distributions
Technical requirementsAll data is randomA little exampleSystematic variation can be learned – random variation can’tRandom variation is not just measurement errorWhat are the consequences of data being random?What we learnedRandom variables and probability distributionsA new concept – random variablesSummarizing probability distributionsContinuous distributionsTransforming and combining random variablesNamed distributionsWhat we learnedSampling from distributionsHow datasets relate to random variables and probability distributionsHow big is the population from which a dataset is sampled?How to sampleGenerating your own random numbers code exampleSampling from numpy distributions code exampleWhat we learnedUnderstanding statistical estimatorsConsistency, bias, and efficiencyThe empirical distribution functionWhat we learnedThe Central Limit TheoremSums of random variablesCLT code exampleCLT example with discrete variablesComputational estimation of a PDF from dataKDE code exampleWhat we learnedSummaryExercises
Chapter 3: Matrices and Linear Algebra
Technical requirementsInner and outer products of vectorsInner product of two vectorsOuter product of two vectorsWhat we learnedMatrices as transformationsMatrix multiplicationThe identity matrixThe inverse matrixMore examples of matrices as transformationsMatrix transformation code exampleWhat we learnedMatrix decompositionsEigen-decompositionsEigenvector and eigenvaluesEigen-decomposition of a square matrixEigen-decomposition code exampleSingular value decompositionThe SVD of a complex matrixWhat we learnedMatrix propertiesTraceDeterminantWhat we learnedMatrix factorization and dimensionality reductionDimensionality reductionPrincipal component analysisNon-negative matrix factorizationWhat we learnedSummaryExercisesNotes and further reading
Chapter 4: Loss Functions and Optimization
Technical requirementsLoss functions – what are they?Risk functionsThere are many loss functionsDifferent loss functions = different end resultsLoss functions for anythingA loss function by any other nameWhat we learnedLeast SquaresThe squared-loss functionOLS regressionOLS, outliers, and robust regressionWhat we learnedLinear modelsPractical issuesThe model residualsOLS regression code exampleWhat we learnedGradient descentLocating the minimum of a simple risk functionGradient descent code exampleGradient descent is a general techniqueBeyond simple gradient descentWhat we learnedSummaryExercises
Chapter 5: Probabilistic Modeling
Technical requirementsLikelihoodA simple probabilistic modelLog likelihoodMaximum likelihood estimationWhat we have learnedBayes’ theoremConditional probability and Bayes’ theoremPriorsThe posteriorWhat we have learnedBayesian modelingBayesian model averagingMAP estimationAs becomes large the prior becomes irrelevantLeast squares as an approximation to Bayesian modelingWhat we have learnedBayesian modeling in practiceAnalytic approximation of the posteriorComputational samplingMCMC code exampleProbabilistic programming languagesWhat we have learnedSummaryExercises

Part 2: Intermediate Concepts
Chapter 6: Time Series and Forecasting
Technical requirementsWhat is time series data?What does auto-correlation mean for modeling time series data?The auto-correlation function (ACF)The partial auto-correlation function (PACF)Other data science implications of time series dataWhat we have learnedARIMA modelsIntegratedAuto-regressionMoving averageCombining the AR(p), I(d), and MA(q) into an ARIMA modelVariants of ARIMA modelingWhat we have learnedARIMA modeling in practiceUnit root testingInterpreting ACF and PACF plotsauto.arimaWhat we have learnedMachine learning approaches to time series analysisRoutine application of machine learning to time series analysisDeep learning approaches to time series analysisAutoML approaches to time series analysisWhat we have learnedSummaryExercisesNotes and further reading
Chapter 7: Hypothesis Testing
Technical requirementsWhat is a hypothesis test?ExampleThe general form of a hypothesis testThe p-valueThe effect of increasing sample sizeThe effect of decreasing noiseOne-tailed and two-tailed testsUsing samples variances in the test statistic – the t-testComputationally intensive methods for p-value estimationParametric versus non-parametric hypothesis testsWhat we learnedConfidence intervalsWhat does a confidence interval really represent?Confidence intervals for any parameterA confidence interval code exampleWhat we learnedType I and Type II errors, and powerWhat we learnedSummaryExercisesNotes and further reading
Chapter 8: Model Complexity
Technical requirementsGeneralization, overfitting, and the role of model complexityOverfittingWhy overfitting is badOverfitting increases the variability of predictionsUnderfitting is also a problemMeasuring prediction errorWhat we learnedThe bias-variance trade-offProof of the bias-variance trade-off formulaDouble descent – a modern twist on the generalization error diagramWhat we learnedModel complexity measures for model selectionSelecting between classes of modelsAkaike Information CriterionBayesian Information CriterionWhat we learnedSummaryNotes and further reading
Chapter 9: Function Decomposition
Technical requirementsWhy do we want to decompose a function?What is a decomposition of a function?Example 1 – decomposing a one-dimensional function into symmetric and anti-symmetric partsExample 2 – decomposing a time series into its seasonal and non-seasonal componentsWhat we’ve learnedExpanding a function in terms of basis functionsWhat we’ve learnedFourier seriesWhat we’ve learnedFourier transformsThe multi-dimensional Fourier transformWhat we’ve learnedThe discrete Fourier transformDFT code exampleUses of the DFTWhat is the difference between the DFT, Fourier series, and the Fourier transform?What we’ve learnedSummaryExercises
Chapter 10: Network Analysis
Technical requirementsGraphs and network dataNetwork data is about relationshipsExample 1 – substituting goods in a supermarketExample 2 – international tradeWhat is a graph?What we’ve learnedBasic characteristics of graphsUndirected and directed edgesThe adjacency matrixIn-degree and out-degreeCentralityWhat we’ve learnedDifferent types of graphsFully connected graphsDisconnected graphsDirected acyclic graphsSmall-world networksScale-free networksWhat we’ve learnedCommunity detection and decomposing graphsWhat is a community?How to do community detectionCommunity detection algorithmsCommunity detection code exampleWhat we’ve learnedSummaryExercisesNotes and further reading
Part 3: Selected Advanced Concepts
Chapter 11: Dynamical Systems
Technical requirementsWhat is a dynamical system and what is an evolution equation?Time can be discrete or continuousTime does not have to mean chronological timeEvolution equationsWhat we learnedFirst-order discrete Markov processesVariations of first-order Markov processesA Markov process is a probabilistic modelThe transition probability matrixProperties of the transition probability matrixEpidemic modeling with a first-order discrete Markov processThe transition probability matrix is a networkUsing the transition matrix to generate state trajectoriesEvolution of the state probability distributionStationary distributions and limiting distributionsFirst-order discrete Markov processes are memorylessLikelihood of the state sequenceWhat we learnedHigher-order discrete Markov processesSecond-order discrete Markov processesEvolution of the state probability distribution in higher-order modelsA higher-order discrete Markov process is a first-order discrete Markov process in disguiseHigher-order discrete Markov processes are still memorylessWhat we learnedHidden Markov ModelsEmission probabilitiesMaking inferences with an HMMWhat we learnedSummaryExercisesNotes and further reading
Chapter 12: Kernel Methods
Technical requirementsThe role of inner products in common learning algorithmsSometimes we need new features in our inner productsWhat we learnedThe kernel trickWhat is a kernel?Commonly used kernelsKernel functions for other mathematical objectsCombining kernelsPositive semi-definite kernelsMercer’s theorem and the kernel trickKernelized algorithmsWhat we learnedAn example of a kernelized learning algorithmkFDA code exampleWhat we learnedSummaryExercises
Chapter 13: Information Theory
Technical requirementsWhat is information and why is it useful?The concept of informationThe mathematical definition of informationInformation theory applies to continuous distributions as wellWhy we measure information on a logarithmic scaleWhy is quantifying information useful?What we’ve learnedEntropy as expected informationEntropyWhat we’ve learnedMutual informationConditional entropyMutual information for continuous variablesMutual information as a measure of correlationMutual information code exampleWhat we’ve learnedThe Kullback-Leibler divergenceRelative entropyKL-divergence for continuous variablesUsing the KL-divergence for approximationVariational inferenceWhat we’ve learnedSummaryExercisesNotes and further reading
Chapter 14: Non-Parametric Bayesian Methods
Technical requirementsWhat are non-parametric Bayesian methods?We still have parametersThe different types of non-parametric Bayesian methodsThe pros and cons of non-parametric Bayesian methodsWhat we learnedGaussian processesThe kernel functionFitting GPR modelsPrediction using GPR modelsGPR code exampleWhat we learnedDirichlet processesHow do DPs differ from GPs?The DP notationSampling a function from a DPGenerating a sample of data from a DPBayesian non-parametric inference using a DPWhat we learnedSummaryExercises
Chapter 15: Random Matrices
Technical requirementsWhat is a random matrix?What we learnedUsing random matrices to represent interactions in large-scale systemsWhat we learnedUniversal behavior of large random matricesThe Wigner semicircle lawWhat does RMT study?Universal is universalThe classical Gaussian matrix ensemblesWhat we learnedRandom matrices and high-dimensional covariance matricesThe Marčenko-Pastur distribution is a bulk distributionUniversality in the singular values of The Marčenko-Pastur distribution and neural networksWhat we learnedSummaryExercisesNotes and further reading
Index
Why subscribe?
Other Books You May EnjoyPackt is searching for authors like youShare your thoughtsDownload a free PDF copy of this book

Overview

Unlock new levels of insight into data science with '15 Math Concepts Every Data Scientist Should Know' by David Hoyle. From foundational concepts to advanced applications, this book demonstrates the power of mathematics in understanding and designing data science algorithms using Python.

What this Book will help me do

Master random variables and probability distributions to analyze why data varies.
Leverage matrices and linear algebra to handle and transform data effectively.
Apply advanced data science techniques such as kernel-based learning and information theory.
Develop predictive models using mathematical frameworks and Python integration.
Utilize Bayesian non-parametric methods for proficient probabilistic modeling.

Author(s)

David Hoyle brings over three decades of pivotal experience in statistical modeling and data science, delivering actionable insights for major industry leaders. An advocate for practical learning, David specializes in demystifying challenging concepts through relatable examples and actionable knowledge.

Who is it for?

This book is perfect for data scientists, analysts, or machine learning practitioners with some prior exposure to data science who seek deeper comprehension of the math underlying their tools. If you're eager to reinforce your foundational knowledge or explore advanced concepts for professional growth, this is the ideal resource.

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781837634187

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills