book

Building Data Science Solutions with Anaconda

Name: Building Data Science Solutions with Anaconda
Author: Dan Meador
ISBN: 9781800568785

by Dan Meador

May 2022

Beginner to intermediate

330 pages

7h 19m

English

Packt Publishing

Read now

Unlock full access

Building Data Science Solutions with Anaconda
ForewordContributorsAbout the authorAbout the reviewers
Preface
Who this book is forWhat this book coversTo get the most out of this bookDownload the example code filesDownload the color imagesConventions usedGet in touchShare Your Thoughts
Part 1: The Data Science Landscape – Open Source to the Rescue
Chapter 1: Understanding the AI/ML landscape
Introducing Artificial Intelligence (AI)Defining AIDefining a data scientistUnderstanding the current state of AI and MLKnowing the difference between AI and MLUnderstanding the massive generation of new dataEvaluating how AI delivers business valueUnderstanding the main types of ML modelsSupervised learningUnsupervised learningReinforcement learningEvaluating the problem typeDealing with out-of-date modelsDifference between online and batch learningHow models become stale: model driftInstalling packages with AnacondaHow to use Anaconda Individual Edition to download packagesHow to handle dependencies with condaCreating separate work areas with Anaconda environmentsSummary
Chapter 2: Analyzing Open Source Software
Technical requirementsUnderstanding open sourceForking an OSS repository with Git and GitHubDefining open source softwareAdvantages of OSSUnderstanding the top four OSS licensesCopyleft versus permissive licensesHow to find out what license a library usesEvaluating a new tool or libraryGitHub starsAgeHow long since it's been updatedNumber of maintainersAge of open issues/PRsNumber of external dependenciesImporting packages with Anaconda and conda-forgeUpdating to the latest conda versionCreating a conda virtual environmentThe differences between modules, packages, and librariesEvaluating and using scikit-learnEvaluation metricsGetting up and running with scikit-learnSummary
Chapter 3: Using the Anaconda Distribution to Manage Packages
Technical requirementsLearning how dependency resolution worksHow pip and conda are differentDiscovering what conda environments are and how to use them Creating environments in condaCreating environments in NavigatorInstalling packages via NavigatorInstalling packages via condaExporting environments to Anconda.orgManaging channels with Anaconda Navigator and condaUnderstanding what a channel isSetting channel priorityUsing advanced conda info and settingsUsing conda info to see configuration informationSetting up your conda settings fileConda cheat sheetConda general commandsConda environment commandsSummary
Chapter 4: Working with Jupyter Notebooks and NumPy
Technical requirementsWorking with Jupyter notebooksCreating a new Jupyter notebookWorking with Jupyter notebook cellsLine and cell magic in Jupyter cellsAccessing the system command lineUsing NumPy to perform calculations quicklyCreating and manipulating NumPy arraysUnderstanding why NumPy's ndarrays are fastSummary
Part 2: Data Is the New Oil, Models Are the New Refineries
Chapter 5: Cleaning and Visualizing Data
Technical requirementsCleaning data with pandasInstalling pandas in your conda environmentWorking with CSVsAnalyzing and cleaning dataDealing with missing dataCreating a deep copy of a Data FrameVisualization with MatplotlibPreparing data for plottingPlotting dataCustomizing the plotShowing the plotPlotting a scatter plot and polynomial regression line Summary
Chapter 6: Overcoming Bias in AI/ML
Technical requirementsDefining bias versus discriminationBias in AI/MLDiscrimination in AI/MLOvercoming proxy biasExamples of proxy biasHow to prevent proxy biasOvercoming sample biasExamples of sample biasRacial/gender biasHow to prevent sample biasOvercoming exclusion biasExamples of exclusion biasHow to prevent exclusion biasOvercoming measurement biasExamples of measurement biasHow to prevent measurement biasOvercoming societal AI biasExamples of societal biasFinding bias in an exampleSummary

Chapter 7: Choosing the Best AI Algorithm
Technical requirementsDefining your problemModel problem typesAlgorithms by problem typeUnderstanding regression problems with examplesLinear regressionRandom forestSupport vector machinesArtificial neural networksClassificationClassification algorithmsClassification exampleLogistic regressionDecision trees/random forestK-nearest neighborsAnomaly detectionOne-class SVMIsolation forestsClustering problemsDBScanK-means clusteringSummary
Chapter 8: Dealing with Common Data Problems
Technical requirementsDealing with too much dataChecking feature correlationDetecting NaN valuesDealing with valid NaN valuesDealing with invalid NaN valuesFinding and correcting data entriesRetrieving specific pandas items by conditionWorking with categorical values with one-hot encodingOne-hot encoding with pandasOrdinal encodingFeature scalingCreating a histogram with pandasUsing the R2 score to evaluate a modelUsing the MSE score to evaluate a modelUsing the MAE score to evaluate a modelOvercoming the limits of capped valuesRecovering the raw datasetWorking with date formatsSummary
Part 3: Practical Examples and Applications
Chapter 9: Building a Regression Model with scikit-learn
Technical requirementsWalking through the data science workflowSetting up and understanding the problem spaceSetting up your workspaceCombining two CSV filesExploring and cleaning the dataChecking for missing valuesChecking for redundant featuresFocusing on the key featuresCreating and evaluating regression algorithmsComparing regression and classificationPreparing the data for trainingEvaluating potential models using MSE and R2 scoresTraining your modelsAnalyzing model results with MSE and R2 scoreR2 scoreTraining a KNN modelLinear regressionMaking use of our resultsSummary
Chapter 10: Explainable AI - Using LIME and SHAP
Technical requirementsUnderstanding the value of interpretationKnowing the difference between interpreting and explainingLooking at legal reasons for interpretabilityLooking at moral reasons for interpretabilityLooking at business reasons for interpretabilityLooking at model improvement reasons for interpretabilityUnderstanding models that are interpretable by designInterpreting decision treesGraphing a decision treeExplaining a model's outcome with LIMECreating a LIME exampleWeighing the drawbacks of LIMEExplaining a model's outcome with SHAP Avoid confusion with Shapley valuesCreating a SHAP exampleLooking at the SHAP resultWeighing the drawbacks of SHAPThinking through shortcomings of interpretation and XAISummary
Chapter 11: Tuning Hyperparameters and Versioning Your Model
Technical requirements Creating a scikit-learn pipelinescikit-learn estimators and transformersCreating a scikit-learn pipelineTesting out various algorithm methodsFeeding live production data into pipelinesFinding optimal hyperparameters with GridSearchCVDefining the difference between hyperparameters and parametersUsing a grid search on a random forest pipelineVersioning and storing your modelPickling a modelLoading your pickled modelStoring your model with joblibSummaryClose
Why subscribe?
Other Books You May EnjoyPackt is searching for authors like youShare Your Thoughts

Content preview from Building Data Science Solutions with Anaconda

Chapter 11: Tuning Hyperparameters and Versioning Your Model

The journey of a data scientist is always an iterative one. Understanding how to create a process that is scalable and repeatable ensures that you can smoothly move through all the phases of data cleaning and model discovery.

In this chapter, we will cover how to create a pipeline that will combine a lot of the small steps we have learned throughout the book into an easier flow. We will then see how you can use a grid search to uncover the best hyperparameters to ensure you are creating the best possible model. We will then show you how you can create saved and versioned models to let you easily return to a previous model at any point in time. All these skills will allow for much greater ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781800568785

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Building Data Science Solutions with Anaconda

by Dan Meador

Chapter 11: Tuning Hyperparameters and Versioning Your Model

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.