book

Essential Math for Data Science

by Thomas Nield

May 2022

Intermediate to advanced

352 pages

9h 15m

English

O'Reilly Media, Inc.

Read now

Unlock full access

Includes

Has Sandbox

Conventions Used in This BookUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgments
Number TheoryOrder of OperationsVariablesFunctionsSummationsExponentsLogarithmsEuler’s Number and Natural LogarithmsEuler’s NumberNatural LogarithmsLimitsDerivativesPartial DerivativesThe Chain RuleIntegralsConclusionExercises
Understanding ProbabilityProbability Versus StatisticsProbability MathJoint ProbabilitiesUnion ProbabilitiesConditional Probability and Bayes’ TheoremJoint and Union Conditional ProbabilitiesBinomial DistributionBeta DistributionConclusionExercises
What Is Data?Descriptive Versus Inferential StatisticsPopulations, Samples, and BiasDescriptive StatisticsMean and Weighted MeanMedianModeVariance and Standard DeviationThe Normal DistributionThe Inverse CDFZ-ScoresInferential StatisticsThe Central Limit TheoremConfidence IntervalsUnderstanding P-ValuesHypothesis TestingThe T-Distribution: Dealing with Small SamplesBig Data Considerations and the Texas Sharpshooter FallacyConclusionExercises
What Is a Vector?Adding and Combining VectorsScaling VectorsSpan and Linear DependenceLinear TransformationsBasis VectorsMatrix Vector MultiplicationMatrix MultiplicationDeterminantsSpecial Types of MatricesSquare MatrixIdentity MatrixInverse MatrixDiagonal MatrixTriangular MatrixSparse MatrixSystems of Equations and Inverse MatricesEigenvectors and EigenvaluesConclusionExercises
A Basic Linear RegressionResiduals and Squared ErrorsFinding the Best Fit LineClosed Form EquationInverse Matrix TechniquesMatrix DecompositionGradient DescentOverfitting and VarianceStochastic Gradient DescentThe Correlation CoefficientStatistical SignificanceCoefficient of DeterminationStandard Error of the EstimatePrediction IntervalsTrain/Test SplitsMultiple Linear RegressionConclusionExercises
Understanding Logistic RegressionPerforming a Logistic RegressionLogistic FunctionFitting the Logistic CurveMultivariable Logistic RegressionUnderstanding the Log-OddsR-SquaredP-ValuesTrain/Test SplitsConfusion MatricesBayes’ Theorem and ClassificationReceiver Operator Characteristics/Area Under CurveClass ImbalanceConclusionExercises
When to Use Neural Networks and Deep LearningA Simple Neural NetworkActivation FunctionsForward PropagationBackpropagationCalculating the Weight and Bias DerivativesStochastic Gradient DescentUsing scikit-learnLimitations of Neural Networks and Deep LearningConclusionExercise
Redefining Data ScienceA Brief History of Data ScienceFinding Your EdgeSQL ProficiencyProgramming ProficiencyData VisualizationKnowing Your IndustryProductive LearningPractitioner Versus AdvisorWhat to Watch Out For in Data Science JobsRole DefinitionOrganizational Focus and Buy-InAdequate ResourcesReasonable ObjectivesCompeting with Existing SystemsA Role Is Not What You ExpectedDoes Your Dream Job Not Exist?Where Do I Go Now?Conclusion
Using LaTeX Rendering with SymPyBinomial Distribution from ScratchBeta Distribution from ScratchDeriving Bayes’ TheoremCDF and Inverse CDF from ScratchUse e to Predict Event Probability Over TimeHill Climbing and Linear RegressionHill Climbing and Logistic RegressionA Brief Intro to Linear ProgrammingMNIST Classifier Using scikit-learn

Chapter 1Chapter 2Chapter 3Chapter 4Chapter 5Chapter 6Chapter 7

Content preview from Essential Math for Data Science

Preface

In the past 10 years or so, there has been a growing interest in applying math and statistics to our everyday work and lives. Why is that? Does it have to do with the accelerated interest in “data science,” which Harvard Business Review called “the Sexiest Job of the 21st Century”? Or is it the promise of machine learning and “artificial intelligence” changing our lives? Is it because news headlines are inundated with studies, polls, and research findings but unsure how to scrutinize such claims? Or is it the promise of “self-driving” cars and robots automating jobs in the near future?

I will make the argument that the disciplines of math and statistics have captured mainstream interest because of the growing availability of data, and we need math, statistics, and machine learning to make sense of it. Yes, we do have scientific tools, machine learning, and other automations that call to us like sirens. We blindly trust these “black boxes,” devices, and softwares; we do not understand them but we use them anyway.

While it is easy to believe computers are smarter than we are (and this idea is frequently marketed), the reality cannot be more the opposite. This disconnect can be precarious on so many levels. Do you really want an algorithm or AI performing criminal sentencing or driving a vehicle, but nobody including the developer can explain why it came to a specific decision? Explainability is the next frontier of statistical computing and AI. This can begin only when we ...