book

Dive Into Data Science

by Bradford Tuckfield

July 2023

Beginner

288 pages

8h 11m

English

No Starch Press

Read now

Unlock full access

Who Is This Book For?About This BookSetting Up the EnvironmentWindowsmacOSLinuxInstalling Packages with PythonOther ToolsSummary
Your First Day as CEOFinding Patterns in DatasetsUsing .csv Files to Review and Store DataDisplaying Data with PythonCalculating Summary StatisticsAnalyzing Subsets of DataNighttime DataSeasonal DataVisualizing Data with MatplotlibDrawing and Displaying a Simple PlotClarifying Plots with Titles and LabelsPlotting Subsets of DataTesting Different Plot TypesExploring CorrelationsCalculating CorrelationsUnderstanding Strong vs. Weak CorrelationsFinding Correlations Between VariablesCreating Heat MapsExploring FurtherSummary
Predicting Customer DemandCleaning Erroneous DataPlotting Data to Find TrendsPerforming Linear RegressionApplying Algebra to the Regression LineCalculating Error MeasurementsUsing Regression to Forecast Future TrendsTrying More Regression ModelsMultivariate Linear Regression to Predict SalesTrigonometry to Capture VariationsChoosing the Best Regression to Use for ForecastingExploring FurtherSummary
Reading Population DataSummary StatisticsRandom SamplesDifferences Between Sample DataPerforming Hypothesis TestingThe t-TestNuances of Hypothesis TestingComparing Groups in a Practical ContextSummary
The Need for ExperimentationRunning Experiments to Test New HypothesesUnderstanding the Math of A/B TestingTranslating the Math into PracticeOptimizing with the Champion/Challenger FrameworkPreventing Mistakes with Twyman’s Law and A/A TestingUnderstanding Effect SizesCalculating the Significance of DataApplications and Advanced ConsiderationsThe Ethics of A/B TestingSummary

Minimizing Customer AttritionUsing Linear Probability Models to Find High-Risk CustomersPlotting Attrition RiskConfirming Relationships with Linear RegressionPredicting the FutureMaking Business RecommendationsMeasuring Prediction AccuracyUsing Multivariate LPMsCreating New MetricsConsidering the Weaknesses of LPMsPredicting Binary Outcomes with Logistic RegressionDrawing Logistic CurvesFitting the Logistic Function to Our DataApplications of Binary ClassificationSummary
Predicting Website TrafficReading and Plotting News Article DataUsing Linear Regression as a Prediction MethodUnderstanding Supervised Learningk-Nearest NeighborsImplementing k-NNPerforming k-NN with Python’s sklearnUsing Other Supervised Learning AlgorithmsDecision TreesRandom ForestsNeural NetworksMeasuring Prediction AccuracyWorking with Multivariate ModelsUsing Classification Instead of RegressionSummary
Unsupervised Learning vs. Supervised LearningGenerating and Exploring DataRolling the DiceUsing Another Kind of DieThe Origin of Observations with ClusteringClustering in Business ApplicationsAnalyzing Multiple DimensionsE-M ClusteringThe Guessing StepThe Expectation StepThe Maximization StepThe Convergence StepOther Clustering MethodsOther Unsupervised Learning MethodsSummary
Understanding How Websites WorkCreating Your First Web ScraperParsing HTML CodeScraping an Email AddressSearching for Addresses DirectlyPerforming Searches with Regular ExpressionsUsing Metacharacters for Flexible SearchesFine-Tuning Searches with Escape SequencesCombining Metacharacters for Advanced SearchesUsing Regular Expressions to Search for Email AddressesConverting Results to Usable DataUsing Beautiful SoupParsing HTML Label ElementsScraping and Parsing HTML TablesAdvanced ScrapingSummary
Popularity-Based RecommendationsItem-Based Collaborative FilteringMeasuring Vector SimilarityCalculating Cosine SimilarityImplementing Item-Based Collaborative FilteringUser-Based Collaborative FilteringCase Study: Music RecommendationsGenerating Recommendations with Advanced SystemsSummary
Using NLP to Detect PlagiarismUnderstanding the word2vec NLP ModelQuantifying Similarities Between WordsCreating a System of EquationsAnalyzing Numeric Vectors in word2vecManipulating Vectors with Mathematical CalculationsDetecting Plagiarism with word2vecUsing Skip-ThoughtsTopic ModelingOther Applications of NLPSummary
Winning Soccer Games with SQLReading and Analyzing DataGetting Familiar with SQLSetting Up a SQL DatabaseRunning SQL QueriesCombining Data by Joining TablesWinning Soccer Games with RGetting Familiar with RApplying Linear Regression in RUsing R to Plot DataGaining Other Valuable SkillsSummary

Content preview from Dive Into Data Science

2 Forecasting

Let’s look at some data science tools that can help you predict the future. In this chapter, we’ll introduce a simple business scenario in which a company needs to forecast customer demand. We’ll then talk about how tools from data science can be applied to make an accurate forecast and how that forecast can lead to better business decision-making.

We’ll use linear regression for our forecasting, and we’ll discuss both univariate and multivariate linear regression. Finally, we’ll look at extrapolation of regression lines and how to evaluate various regression models to choose the best one.