book

Practical Statistics for Data Scientists, 2nd Edition

by Peter Bruce, Andrew Bruce, Peter Gedeck

May 2020

Beginner

360 pages

9h 16m

English

O'Reilly Media, Inc.

Book available

Read now

Unlock full access

Conventions Used in This BookUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgments
Elements of Structured DataFurther ReadingRectangular DataData Frames and IndexesNonrectangular Data StructuresFurther ReadingEstimates of LocationMeanMedian and Robust EstimatesExample: Location Estimates of Population and Murder RatesFurther ReadingEstimates of VariabilityStandard Deviation and Related EstimatesEstimates Based on PercentilesExample: Variability Estimates of State PopulationFurther ReadingExploring the Data DistributionPercentiles and BoxplotsFrequency Tables and HistogramsDensity Plots and EstimatesFurther ReadingExploring Binary and Categorical DataModeExpected ValueProbabilityFurther ReadingCorrelationScatterplotsFurther ReadingExploring Two or More VariablesHexagonal Binning and Contours (Plotting Numeric Versus Numeric Data)Two Categorical VariablesCategorical and Numeric DataVisualizing Multiple VariablesFurther ReadingSummary
Random Sampling and Sample BiasBiasRandom SelectionSize Versus Quality: When Does Size Matter?Sample Mean Versus Population MeanFurther ReadingSelection BiasRegression to the MeanFurther ReadingSampling Distribution of a StatisticCentral Limit TheoremStandard ErrorFurther ReadingThe BootstrapResampling Versus BootstrappingFurther ReadingConfidence IntervalsFurther ReadingNormal DistributionStandard Normal and QQ-PlotsLong-Tailed DistributionsFurther ReadingStudent’s t-DistributionFurther ReadingBinomial DistributionFurther ReadingChi-Square DistributionFurther ReadingF-DistributionFurther ReadingPoisson and Related DistributionsPoisson DistributionsExponential DistributionEstimating the Failure RateWeibull DistributionFurther ReadingSummary
A/B TestingWhy Have a Control Group?Why Just A/B? Why Not C, D,…?Further ReadingHypothesis TestsThe Null HypothesisAlternative HypothesisOne-Way Versus Two-Way Hypothesis TestsFurther ReadingResamplingPermutation TestExample: Web StickinessExhaustive and Bootstrap Permutation TestsPermutation Tests: The Bottom Line for Data ScienceFurther ReadingStatistical Significance and p-Valuesp-ValueAlphaType 1 and Type 2 ErrorsData Science and p-ValuesFurther Readingt-TestsFurther ReadingMultiple TestingFurther ReadingDegrees of FreedomFurther ReadingANOVAF-StatisticTwo-Way ANOVAFurther ReadingChi-Square TestChi-Square Test: A Resampling ApproachChi-Square Test: Statistical TheoryFisher’s Exact TestRelevance for Data ScienceFurther ReadingMulti-Arm Bandit AlgorithmFurther ReadingPower and Sample SizeSample SizeFurther ReadingSummary
Simple Linear RegressionThe Regression EquationFitted Values and ResidualsLeast SquaresPrediction Versus Explanation (Profiling)Further ReadingMultiple Linear RegressionExample: King County Housing DataAssessing the ModelCross-ValidationModel Selection and Stepwise RegressionWeighted RegressionFurther ReadingPrediction Using RegressionThe Dangers of ExtrapolationConfidence and Prediction IntervalsFactor Variables in RegressionDummy Variables RepresentationFactor Variables with Many LevelsOrdered Factor VariablesInterpreting the Regression EquationCorrelated PredictorsMulticollinearityConfounding VariablesInteractions and Main EffectsRegression DiagnosticsOutliersInfluential ValuesHeteroskedasticity, Non-Normality, and Correlated ErrorsPartial Residual Plots and NonlinearityPolynomial and Spline RegressionPolynomialSplinesGeneralized Additive ModelsFurther ReadingSummary
Naive BayesWhy Exact Bayesian Classification Is ImpracticalThe Naive SolutionNumeric Predictor VariablesFurther ReadingDiscriminant AnalysisCovariance MatrixFisher’s Linear DiscriminantA Simple ExampleFurther ReadingLogistic RegressionLogistic Response Function and LogitLogistic Regression and the GLMGeneralized Linear ModelsPredicted Values from Logistic RegressionInterpreting the Coefficients and Odds RatiosLinear and Logistic Regression: Similarities and DifferencesAssessing the ModelFurther ReadingEvaluating Classification ModelsConfusion MatrixThe Rare Class ProblemPrecision, Recall, and SpecificityROC CurveAUCLiftFurther ReadingStrategies for Imbalanced DataUndersamplingOversampling and Up/Down WeightingData GenerationCost-Based ClassificationExploring the PredictionsFurther ReadingSummary
K-Nearest NeighborsA Small Example: Predicting Loan DefaultDistance MetricsOne Hot EncoderStandardization (Normalization, z-Scores)Choosing KKNN as a Feature EngineTree ModelsA Simple ExampleThe Recursive Partitioning AlgorithmMeasuring Homogeneity or ImpurityStopping the Tree from GrowingPredicting a Continuous ValueHow Trees Are UsedFurther ReadingBagging and the Random ForestBaggingRandom ForestVariable ImportanceHyperparametersBoostingThe Boosting AlgorithmXGBoostRegularization: Avoiding OverfittingHyperparameters and Cross-ValidationSummary
Principal Components AnalysisA Simple ExampleComputing the Principal ComponentsInterpreting Principal ComponentsCorrespondence AnalysisFurther ReadingK-Means ClusteringA Simple ExampleK-Means AlgorithmInterpreting the ClustersSelecting the Number of ClustersHierarchical ClusteringA Simple ExampleThe DendrogramThe Agglomerative AlgorithmMeasures of DissimilarityModel-Based ClusteringMultivariate Normal DistributionMixtures of NormalsSelecting the Number of ClustersFurther ReadingScaling and Categorical VariablesScaling the VariablesDominant VariablesCategorical Data and Gower’s DistanceProblems with Clustering Mixed DataSummary

Content preview from Practical Statistics for Data Scientists, 2nd Edition

Bibliography

[Baumer-2017] Baumer, Benjamin, Daniel Kaplan, and Nicholas Horton. Modern Data Science with R. Boca Raton, Fla.: Chapman & Hall/CRC Press, 2017.
[bokeh] Bokeh Development Team. “Bokeh: Python library for interactive visualization” (2014). https://bokeh.pydata.org.
[Deng-Wickham-2011] Deng, Henry, and Hadley Wickham. “Density Estimation in R.” September 2011. https://oreil.ly/-Ny_6.
[Donoho-2015] Donoho, David. “50 Years of Data Science.” September 18, 2015. https://oreil.ly/kqFb0.
[Duong-2001] Duong, Tarn. “An Introduction to Kernel Density Estimation.” 2001. https://oreil.ly/Z5A7W.
[Few-2007] Few, Stephen. “Save the Pies for Dessert.” Visual Business Intelligence Newsletter. Perceptual Edge. August 2007. https://oreil.ly/_iGAL.
[Freedman-2007] Freedman, David, Robert Pisani, and Roger Purves. Statistics. 4th ed. New York: W. W. Norton, 2007.
[Hintze-Nelson-1998] Hintze, Jerry L., and Ray D. Nelson. “Violin Plots: A Box Plot–Density Trace Synergism.” The American Statistician 52, no. 2 (May 1998): 181–84.
[Galton-1886] Galton, Francis. “Regression Towards Mediocrity in Hereditary Stature.” The Journal of the Anthropological Institute of Great Britain and Ireland 15 (1886): 246–63. https://oreil.ly/DqoAk.
[ggplot2] Wickham, Hadley. ggplot2: Elegant Graphics for Data Analysis. New York: Springer-Verlag New York, 2009. https://oreil.ly/O92vC.
[Hyndman-Fan-1996] Hyndman, Rob J., and Yanan Fan. “Sample Quantiles in Statistical Packages.” American ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Start your free trial

Practical Statistics for Data Scientists

Peter Bruce, Andrew Bruce

Fundamentals of Data Engineering

Joe Reis, Matt Housley

Fundamentals of Data Engineering

Joe Reis, Matt Housley

Deep Learning Illustrated: A Visual, Interactive Guide to Artificial Intelligence

Jon Krohn, Grant Beyleveld, Aglaé Bassens

Publisher Resources

ISBN: 9781492072935Errata Page Supplemental Content

Bibliography

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,and much more.

You might also like

Practical Statistics for Data Scientists

Fundamentals of Data Engineering

Fundamentals of Data Engineering

Deep Learning Illustrated: A Visual, Interactive Guide to Artificial Intelligence

Publisher Resources

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.