book

Production-Ready Applied Deep Learning

Name: Production-Ready Applied Deep Learning
ISBN: 9781803243665

by Tomasz Palczewski, Jaejun (Brandon) Lee, Lenin Mookiah

August 2022

Intermediate to advanced

322 pages

7h 50m

English

Packt Publishing

Read now

Unlock full access

Production-Ready Applied Deep Learning
ContributorsAbout the authorsAbout the reviewers
Preface
Who this book is forWhat this book coversTo get the most out of this book Download the example code filesDownload the color imagesConventions usedGet in touchShare Your Thoughts
Part 1 – Building a Minimum Viable Product
Chapter 1: Effective Planning of Deep Learning-Driven Projects
Technical requirementsWhat is DL?Understanding the role of DL in our daily livesOverview of DL projectsProject planningBuilding minimum viable productsBuilding fully featured productsDeployment and maintenanceProject evaluationPlanning a DL projectDefining goal and evaluation metricsStakeholder identificationTask organizationResource allocationDefining a timelineManaging a projectSummaryFurther reading
Chapter 2: Data Preparation for Deep Learning Projects
Technical requirementsSetting up notebook environmentsSetting up a Python environmentInstalling AnacondaSetting up a DL project using AnacondaData collection, data cleaning, and data preprocessingCollecting dataCleaning dataData preprocessingExtracting features from dataConverting text using bag-of-wordsApplying term frequency-inverse document frequency (TF-IDF) transformationCreating one-hot encoding (one-of-k)Creating ordinal encodingConverting a colored image into a grayscale imagePerforming dimensionality reductionApplying fuzzy matching to handle similarity between stringsPerforming data visualizationPerforming basic visualizations using MatplotlibDrawing statistical graphs using SeabornIntroduction to DockerIntroduction to DockerfilesBuilding a custom Docker imageSummary
Chapter 3: Developing a Powerful Deep Learning Model
Technical requirementsGoing through the basic theory of DLHow does DL work?DL model training Components of DL frameworksThe data loading logicThe model definitionModel training logicImplementing and training a model in PyTorchPyTorch data loading logicPyTorch model definitionPyTorch model trainingImplementing and training a model in TFTF data loading logicTF model definitionTF model trainingDecomposing a complex, state-of-the-art model implementationStyleGANImplementation in PyTorchImplementation in TFSummary
Chapter 4: Experiment Tracking, Model Management, and Dataset Versioning
Technical requirementsOverview of DL project trackingComponents of DL project trackingTools for DL project trackingDL project tracking with Weights & BiasesSetting up W&BDL project tracking with MLflow and DVCSetting up MLflowSetting up MLflow with DVCDataset versioning – beyond Weights & Biases, MLflow, and DVCSummary
Part 2 – Building a Fully Featured Product
Chapter 5: Data Preparation in the Cloud
Technical requirementsData processing in the cloudIntroduction to ETLData processing system architectureIntroduction to Apache SparkResilient distributed datasets and DataFramesLoading dataProcessing data using Spark operationsProcessing data using user-defined functionsExporting dataSetting up a single-node EC2 instance for ETLSetting up an EMR cluster for ETLCreating a Glue job for ETLCreating a Glue Data CatalogSetting up a Glue contextReading dataDefining the data processing logicWriting dataUtilizing SageMaker for ETLCreating a SageMaker notebookRunning a Spark job through a SageMaker notebookRunning a job from a custom container through a SageMaker notebookComparing the ETL solutions in AWSSummary
Chapter 6: Efficient Model Training
Technical requirementsTraining a model on a single machineUtilizing multiple devices for training in TensorFlowUtilizing multiple devices for training in PyTorchTraining a model on a clusterModel parallelismData parallelismTraining a model using SageMakerSetting up model training for SageMakerTraining a TensorFlow model using SageMakerTraining a PyTorch model using SageMakerTraining a model in a distributed fashion using SageMakerSageMaker with HorovodTraining a model using HorovodSetting up a Horovod clusterConfiguring a TensorFlow training script for HorovodConfiguring a PyTorch training script for HorovodTraining a DL model on a Horovod clusterTraining a model using RaySetting up a Ray clusterTraining a model in a distributed fashion using RayTraining a model using KubeflowIntroducing KubernetesSetting up model training for KubeflowTraining a TensorFlow model in a distributed fashion using KubeflowTraining a PyTorch model in a distributed fashion using KubeflowSummary

Chapter 7: Revealing the Secret of Deep Learning Models
Technical requirementsObtaining the best performing model using hyperparameter tuningHyperparameter tuning techniquesHyperparameter tuning toolsUnderstanding the behavior of the model with Explainable AIPermutation Feature ImportanceFeature ImportanceSHapley Additive exPlanations (SHAP)Local Interpretable Model-agnostic Explanations (LIME)Summary
Part 3 – Deployment and Maintenance
Chapter 8: Simplifying Deep Learning Model Deployment
Technical requirementsIntroduction to ONNX Running inference using ONNX RuntimeConversion between TensorFlow and ONNXConverting a TensorFlow model into an ONNX modelConverting an ONNX model into a TensorFlow modelConversion between PyTorch and ONNXConverting a PyTorch model into an ONNX modelConverting an ONNX model into a PyTorch modelSummary
Chapter 9: Scaling a Deep Learning Pipeline
Technical requirementsInferencing using Elastic Kubernetes ServicePreparing an EKS clusterConfiguring EKSCreating an inference endpoint using the TensorFlow model on EKSCreating an inference endpoint using a PyTorch model on EKSCommunicating with an endpoint on EKSImproving EKS endpoint performance using Amazon Elastic InferenceResizing EKS cluster dynamically using autoscalingInferencing using SageMakerSetting up an inference endpoint using the Model classSetting up a TensorFlow inference endpointSetting up a PyTorch inference endpointSetting up an inference endpoint from an ONNX modelHandling prediction requests in batches using Batch TransformImproving SageMaker endpoint performance using AWS SageMaker NeoImproving SageMaker endpoint performance using Amazon Elastic InferenceResizing SageMaker endpoints dynamically using autoscalingHosting multiple models on a single SageMaker inference endpointSummary
Chapter 10: Improving Inference Efficiency
Technical requirementsNetwork quantization – reducing the number of bits used for model parametersPerforming post-training quantizationPerforming quantization-aware trainingWeight sharing – reducing the number of distinct weight valuesPerforming weight sharing in TensorFlowPerforming weight sharing in PyTorchNetwork pruning – eliminating unnecessary connections within the networkNetwork pruning in TensorFlowNetwork pruning in PyTorchKnowledge distillation – obtaining a smaller network by mimicking the predictionNetwork Architecture Search – finding the most efficient network architectureSummary
Chapter 11: Deep Learning on Mobile Devices
Preparing DL models for mobile devicesGenerating a TF Lite modelGenerating a TorchScript modelCreating iOS apps with a DL modelRunning TF Lite model inference on iOSRunning TorchScript model inference on iOSCreating Android apps with a DL modelRunning TF Lite model inference on AndroidRunning TorchScript model inference on AndroidSummary
Chapter 12: Monitoring Deep Learning Endpoints in Production
Technical requirementsIntroduction to DL endpoint monitoring in productionExploring tools for monitoringExploring tools for alertingMonitoring using CloudWatchMonitoring a SageMaker endpoint using CloudWatchMonitoring a model throughout the training process in SageMakerMonitoring a live inference endpoint from SageMakerMonitoring an EKS endpoint using CloudWatchSummary
Chapter 13: Reviewing the Completed Deep Learning Project
Reviewing a DL projectConducting a post-implementation reviewUnderstanding the true value of the projectGathering the reusable knowledge, concepts, and artifacts for future projectsSummary
Index
Why subscribe?
Other Books You May EnjoyPackt is searching for authors like youShare Your Thoughts

Content preview from Production-Ready Applied Deep Learning

12 Monitoring Deep Learning Endpoints in Production

Due to the difference in development and production settings, it is difficult to assure the performance of deep learning (DL) models once they are deployed. If any difference exists in model behavior, it must be captured within a reasonable time; otherwise, it can affect downstream applications in negative ways.

In this chapter, our goal is to explain existing solutions for monitoring DL model behavior in production. We will start by clearly describing the benefit of monitoring and what it takes to keep the overall system running in a stable manner. Then, we will discuss popular tools for monitoring DL models and alerting. Out of the various tools we introduce, we will get our hands dirty ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781803243665

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Production-Ready Applied Deep Learning

by Tomasz Palczewski, Jaejun (Brandon) Lee, Lenin Mookiah

12

Monitoring Deep Learning Endpoints in Production

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.