book

Hands-On Explainable AI (XAI) with Python

Name: Hands-On Explainable AI (XAI) with Python
Author: Denis Rothman
ISBN: 9781800208131

by Denis Rothman

July 2020

Beginner to intermediate

454 pages

9h 20m

English

Packt Publishing

Read now

Unlock full access

Preface
Who this book is forWhat this book coversTo get the most out of this bookGet in touch
Explaining Artificial Intelligence with Python
Defining explainable AIGoing from black box models to XAI white box modelsExplaining and interpretingDesigning and extractingThe XAI executive functionThe XAI medical diagnosis timelineThe standard AI program used by a general practitionerDefinition of a KNN algorithmA KNN in PythonWest Nile virus – a case of life or deathHow can a lethal mosquito bite go unnoticed?What is the West Nile virus?How did the West Nile virus get to Chicago?XAI can save lives using Google Location HistoryDownloading Google Location HistoryGoogle's Location History extraction toolReading and displaying Google Location History dataInstallation of the basemap packagesThe import instructionsImporting the dataProcessing the data for XAI and basemapSetting up the plotting options to display the mapEnhancing the AI diagnosis with XAIEnhanced KNNXAI applied to the medical diagnosis experimental programDisplaying the KNN plotNatural language explanationsDisplaying the Location History mapShowing mosquito detection data and natural language explanationsA critical diagnosis is reached with XAISummaryQuestionsReferencesFurther reading
White Box XAI for AI Bias and Ethics
Moral AI bias in self-driving carsLife and death autopilot decision makingThe trolley problemThe MIT Moral Machine experimentReal life and death situationsExplaining the moral limits of ethical AIStandard explanation of autopilot decision treesThe SDC autopilot dilemmaImporting the modulesRetrieving the datasetReading and splitting the dataTheoretical description of decision tree classifiersCreating the default decision tree classifierTraining, measuring, and saving the modelDisplaying a decision treeXAI applied to an autopilot decision treeStructure of a decision treeThe default output of the default structure of a decision treeThe customized output of a customized structure of a decision treeThe output of a customized structure of a decision treeUsing XAI and ethics to control a decision treeLoading the modelAccuracy measurementsSimulating real-time casesIntroducing ML bias due to noiseIntroducing ML ethics and lawsCase 1 – not overriding traffic regulations to save four pedestriansCase 2 – overriding traffic regulationsCase 3 – introducing emotional intelligence in the autopilotSummaryQuestionsReferencesFurther reading
Explaining Machine Learning with Facets
Getting started with FacetsInstalling Facets on Google ColaboratoryRetrieving the datasetsReading the data filesFacets OverviewCreating feature statistics for the datasetsImplementing the feature statistics codeImplementing the HTML code to display feature statisticsSorting the Facets statistics overviewSorting data by feature orderXAI motivation for sorting featuresSorting by non-uniformitySorting by alphabetical orderSorting by amount missing/zeroSorting by distribution distanceFacets DiveBuilding the Facets Dive display codeDefining the labels of the data pointsDefining the color of the data pointsDefining the binning of the x axis and y axisDefining the scatter plot of the x axis and the y axisSummaryQuestionsReferencesFurther reading
Microsoft Azure Machine Learning Model Interpretability with SHAP
Introduction to SHAPKey SHAP principlesSymmetryNull playerAdditivityA mathematical expression of the Shapley valueSentiment analysis exampleShapley value for the first feature, "good"Shapley value for the second feature, "excellent"Verifying the Shapley valuesGetting started with SHAPInstalling SHAPImporting the modulesImporting the dataIntercepting the datasetVectorizing the datasetsLinear models and logistic regressionCreating, training, and visualizing the output of a linear modelDefining a linear modelAgnostic model explaining with SHAPCreating the linear model explainerCreating the plot functionExplaining the output of the model's predictionExplaining intercepted dataset reviews with SHAPExplaining the original IMDb reviews with SHAPSummaryQuestionsReferencesFurther readingAdditional publications
Building an Explainable AI Solution from Scratch
Moral, ethical, and legal perspectivesThe U.S. census data problemUsing pandas to display the dataMoral and ethical perspectivesThe moral perspectiveThe ethical perspectiveThe legal perspectiveThe machine learning perspectiveDisplaying the training data with Facets DiveAnalyzing the training data with FacetsVerifying the anticipated outputsUsing KMC to verify the anticipated resultsAnalyzing the output of the KMC algorithmConclusion of the analysisTransforming the input dataWIT applied to a transformed datasetSummaryQuestionsReferencesFurther reading
AI Fairness with Google's What-If Tool (WIT)
Interpretability and explainability from an ethical AI perspectiveThe ethical perspectiveThe legal perspectiveExplaining and interpretingPreparing an ethical datasetGetting started with WITImporting the datasetPreprocessing the dataCreating data structures to train and test the modelCreating a DNN modelTraining the modelCreating a SHAP explainerThe plot of Shapley valuesModel outputs and SHAP valuesThe WIT datapoint explorer and editorCreating WITThe datapoint editorFeaturesPerformance and fairnessGround truthCost ratioSlicingFairnessThe ROC curve and AUCThe PR curveThe confusion matrixSummaryQuestionsReferencesFurther reading
A Python Client for Explainable AI Chatbots
The Python client for DialogflowInstalling the Python client for Google DialogflowCreating a Google Dialogflow agentEnabling APIs and servicesThe Google Dialogflow Python clientEnhancing the Google Dialogflow Python clientCreating a dialog functionThe constraints of an XAI implementation on DialogflowCreating an intent in DialogflowThe training phrases of the intentThe response of an intentDefining a follow-up intent for an intentThe XAI Python clientInserting interactions in the MDPInteracting with Dialogflow with the Python clientA CUI XAI dialog using Google DialogflowDialogflow integration for a websiteA Jupyter Notebook XAI agent managerGoogle AssistantSummaryQuestionsFurther reading
Local Interpretable Model-Agnostic Explanations (LIME)
Introducing LIMEA mathematical representation of LIMEGetting started with LIMEInstalling LIME on Google ColaboratoryRetrieving the datasets and vectorizing the datasetAn experimental AutoML moduleCreating an agnostic AutoML templateBagging classifiersGradient boosting classifiersDecision tree classifiersExtra trees classifiersInterpreting the scoresTraining the model and making predictionsThe interactive choice of classifierFinalizing the prediction processInterception functionsThe LIME explainerCreating the LIME explainerInterpreting LIME explanationsExplaining the predictions as a listExplaining with a plotConclusions of the LIME explanation processSummaryQuestionsReferencesFurther reading
The Counterfactual Explanations Method
The counterfactual explanations methodDataset and motivationsVisualizing counterfactual distances in WITExploring data point distances with the default viewThe logic of counterfactual explanationsBeliefTruthJustificationSensitivityThe choice of distance functionsThe L1 normThe L2 normCustom distance functionsThe architecture of the deep learning modelInvoking WITThe custom prediction function for WITLoading a Keras modelRetrieving the dataset and modelSummaryQuestionsReferencesFurther reading

Contrastive XAI
The contrastive explanations methodGetting started with the CEM applied to MNISTInstalling Alibi and importing the modulesImporting the modules and the datasetImporting the modulesImporting the datasetPreparing the dataDefining and training the CNN modelCreating the CNN modelTraining the CNN modelLoading and testing the accuracy of the modelDefining and training the autoencoderCreating the autoencoderTraining and saving the autoencoderComparing the original images with the decoded imagesPertinent negativesCEM parametersInitializing the CEM explainerPertinent negative explanationsSummaryQuestionsReferencesFurther reading
Anchors XAI
Anchors AI explanationsPredicting incomeClassifying newsgroup discussionsAnchor explanations for ImageNetInstalling Alibi and importing the modulesLoading an InceptionV3 modelDownloading an imageProcessing the image and making predictionsBuilding the anchor image explainerExplaining other categoriesOther images and difficultiesSummaryQuestionsReferencesFurther reading
Cognitive XAI
Cognitive rule-based explanationsFrom XAI tools to XAI conceptsDefining cognitive XAI explanationsA cognitive XAI methodImporting the modules and the dataThe dictionariesThe global parametersThe cognitive explanation functionThe marginal contribution of a featureA mathematical perspectiveThe Python marginal cognitive contribution functionA cognitive approach to vectorizersExplaining the vectorizer for LIMEExplaining the IMDb vectorizer for SHAPHuman cognitive input for the CEMRule-based perspectivesSummaryQuestionsFurther reading
Answers to the Questions
Chapter 1, Explaining Artificial Intelligence with PythonChapter 2, White Box XAI for AI Bias and EthicsChapter 3, Explaining Machine Learning with FacetsChapter 4, Microsoft Azure Machine Learning Model Interpretability with SHAPChapter 5, Building an Explainable AI Solution from ScratchChapter 6, AI Fairness with Google's What-If Tool (WIT)Chapter 7, A Python Client for Explainable AI ChatbotsChapter 8, Local Interpretable Model-Agnostic Explanations (LIME)Chapter 9, The Counterfactual Explanations MethodChapter 10, Contrastive XAIChapter 11, Anchors XAIChapter 12, Cognitive XAI
Other Books You May Enjoy
Index

Overview

Delve into the fascinating world of explainable AI (XAI) with this hands-on guide. Using Python and a variety of XAI tools, you'll master techniques to interpret, visualize, and explain AI model behavior. Equip yourself with the knowledge to integrate AI solutions into applications, ensuring they are fair, secure, and trustworthy.

What this Book will help me do

Master the integration of explainable AI tools using Python, TensorFlow, and other systems.
Develop the ability to detect and handle bias and ethics issues in machine learning projects.
Gain practical experience in creating XAI visualizations for clear communication of AI results.
Understand the strengths and limitations of popular XAI methods through detailed hands-on examples.
Implement fair and transparent AI systems that add value and trustworthiness to business applications.

Author(s)

Denis Rothman, a highly experienced AI professional, brings a wealth of knowledge in machine learning, deep learning, and XAI. Known for his approachable teaching style, Denis blends complex concepts with practical applications. His work focuses on enabling readers to build ethical and understandable AI systems through well-structured and actionable examples.

Who is it for?

This book is tailored for machine learning practitioners and Python users seeking to enhance their understanding of explainable AI. Ideal for data scientists looking to improve model transparency, professionals aiming to incorporate XAI in business solutions, and project managers navigating the ethical landscape of AI. This book assumes a basic knowledge of Python and machine learning fundamentals for the best learning experience.

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Deep Learning with Python, Second Edition

Publisher Resources

ISBN: 9781800208131

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills