book

Practical Deep Learning for Cloud, Mobile, and Edge

by Anirudh Koul, Siddha Ganju, Meher Kasam

October 2019

Intermediate to advanced

583 pages

14h 36m

English

O'Reilly Media, Inc.

Read now

Unlock full access

Preface
To the Backend/Frontend/Mobile Software DeveloperTo the Data ScientistTo the StudentTo the TeacherTo the Robotics EnthusiastWhat to Expect in Each ChapterConventions Used in This BookUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgmentsGroup AcknowledgmentsPersonal Acknowledgments
1. Exploring the Landscape of Artificial Intelligence
An ApologyThe Real IntroductionWhat Is AI?Motivating ExamplesA Brief History of AIExciting BeginningsThe Cold and Dark DaysA Glimmer of HopeHow Deep Learning Became a ThingRecipe for the Perfect Deep Learning SolutionDatasetsModel ArchitectureFrameworksHardwareResponsible AIBiasAccountability and ExplainabilityReproducibilityRobustnessPrivacySummaryFrequently Asked Questions
2. What’s in the Picture: Image Classification with Keras
Introducing KerasPredicting an Image’s CategoryInvestigating the ModelImageNet DatasetModel ZoosClass Activation MapsSummary
3. Cats Versus Dogs: Transfer Learning in 30 Lines with Keras
Adapting Pretrained Models to New TasksA Shallow Dive into Convolutional Neural NetworksTransfer LearningFine TuningHow Much to Fine TuneBuilding a Custom Classifier in Keras with Transfer LearningOrganize the DataBuild the Data PipelineNumber of ClassesBatch SizeData AugmentationModel DefinitionTrain the ModelSet Training ParametersStart TrainingTest the ModelAnalyzing the ResultsFurther ReadingSummary
4. Building a Reverse Image Search Engine: Understanding Embeddings
Image SimilarityFeature ExtractionSimilarity SearchVisualizing Image Clusters with t-SNEImproving the Speed of Similarity SearchLength of Feature VectorsReducing Feature-Length with PCAScaling Similarity Search with Approximate Nearest NeighborsApproximate Nearest-Neighbor BenchmarkWhich Library Should I Use?Creating a Synthetic DatasetBrute ForceAnnoyNGTFaissImproving Accuracy with Fine TuningFine Tuning Without Fully Connected LayersSiamese Networks for One-Shot Face VerificationCase StudiesFlickrPinterestCelebrity DoppelgangersSpotifyImage CaptioningSummary
5. From Novice to Master Predictor: Maximizing Convolutional Neural Network Accuracy
Tools of the TradeTensorFlow DatasetsTensorBoardWhat-If Tooltf-explainCommon Techniques for Machine Learning ExperimentationData InspectionBreaking the Data: Train, Validation, TestEarly StoppingReproducible ExperimentsEnd-to-End Deep Learning Example PipelineBasic Transfer Learning PipelineBasic Custom Network PipelineHow Hyperparameters Affect AccuracyTransfer Learning Versus Training from ScratchEffect of Number of Layers Fine-Tuned in Transfer LearningEffect of Data Size on Transfer LearningEffect of Learning RateEffect of OptimizersEffect of Batch SizeEffect of ResizingEffect of Change in Aspect Ratio on Transfer LearningTools to Automate Tuning for Maximum AccuracyKeras TunerAutoAugmentAutoKerasSummary
6. Maximizing Speed and Performance of TensorFlow: A Handy Checklist
GPU Starvationnvidia-smiTensorFlow Profiler + TensorBoardHow to Use This ChecklistPerformance ChecklistData PreparationData ReadingData AugmentationTrainingInferenceData PreparationStore as TFRecordsReduce Size of Input DataUse TensorFlow DatasetsData ReadingUse tf.dataPrefetch DataParallelize CPU ProcessingParallelize I/O and ProcessingEnable Nondeterministic OrderingCache DataTurn on Experimental OptimizationsAutotune Parameter ValuesData AugmentationUse GPU for AugmentationTrainingUse Automatic Mixed PrecisionUse Larger Batch SizeUse Multiples of EightFind the Optimal Learning RateUse tf.functionOvertrain, and Then GeneralizeInstall an Optimized Stack for the HardwareOptimize the Number of Parallel CPU ThreadsUse Better HardwareDistribute TrainingExamine Industry BenchmarksInferenceUse an Efficient ModelQuantize the ModelPrune the ModelUse Fused OperationsEnable GPU PersistenceSummary
7. Practical Tools, Tips, and Tricks
InstallationTrainingModelDataPrivacyEducation and ExplorationOne Last Question
8. Cloud APIs for Computer Vision: Up and Running in 15 Minutes
The Landscape of Visual Recognition APIsClarifaiMicrosoft Cognitive ServicesGoogle Cloud VisionAmazon RekognitionIBM Watson Visual RecognitionAlgorithmiaComparing Visual Recognition APIsService OfferingsCostAccuracyBiasGetting Up and Running with Cloud APIsTraining Our Own Custom ClassifierTop Reasons Why Our Classifier Does Not Work SatisfactorilyComparing Custom Classification APIsPerformance Tuning for Cloud APIsEffect of Resizing on Image Labeling APIsEffect of Compression on Image Labeling APIsEffect of Compression on OCR APIsEffect of Resizing on OCR APIsCase StudiesThe New York TimesUberGiphyOmniEarthPhotobucketStaplesInDro RoboticsSummary
9. Scalable Inference Serving on Cloud with TensorFlow Serving and KubeFlow
Landscape of Serving AI PredictionsFlask: Build Your Own ServerMaking a REST API with FlaskDeploying a Keras Model to FlaskPros of Using FlaskCons of Using FlaskDesirable Qualities in a Production-Level Serving SystemHigh AvailabilityScalabilityLow LatencyGeographic AvailabilityFailure HandlingMonitoringModel VersioningA/B TestingSupport for Multiple Machine Learning LibrariesGoogle Cloud ML Engine: A Managed Cloud AI Serving StackPros of Using Cloud ML EngineCons of Using Cloud ML EngineBuilding a Classification APITensorFlow ServingInstallationKubeFlowPipelinesFairingInstallationPrice Versus Performance ConsiderationsCost Analysis of Inference-as-a-ServiceCost Analysis of Building Your Own StackSummary

10. AI in the Browser with TensorFlow.js and ml5.js
JavaScript-Based Machine Learning Libraries: A Brief HistoryConvNetJSKeras.jsONNX.jsTensorFlow.jsTensorFlow.js ArchitectureRunning Pretrained Models Using TensorFlow.jsModel Conversion for the BrowserTraining in the BrowserFeature ExtractionData CollectionTrainingGPU Utilizationml5.jsPoseNetpix2pixBenchmarking and Practical ConsiderationsModel SizeInference TimeCase StudiesSemi-ConductorTensorSpaceMetacarAirbnb’s Photo ClassificationGAN LabSummary
11. Real-Time Object Classification on iOS with Core ML
The Development Life Cycle for Artificial Intelligence on MobileA Brief History of Core MLAlternatives to Core MLTensorFlow LiteML KitFritzApple’s Machine Learning ArchitectureDomain-Based FrameworksML FrameworkML Performance PrimitivesBuilding a Real-Time Object Recognition AppConversion to Core MLConversion from KerasConversion from TensorFlowDynamic Model DeploymentOn-Device TrainingFederated LearningPerformance AnalysisBenchmarking Models on iPhonesMeasuring Energy ImpactBenchmarking LoadReducing App SizeAvoid Bundling the ModelUse QuantizationUse Create MLCase StudiesMagic SudokuSeeing AIHomeCourtInstaSaber + YoPuppetSummary
12. Not Hotdog on iOS with Core ML and Create ML
Collecting DataApproach 1: Find or Collect a DatasetApproach 2: Fatkun Chrome Browser ExtensionApproach 3: Web Scraper Using Bing Image Search APITraining Our ModelApproach 1: Use Web UI-based ToolsApproach 2: Use Create MLApproach 3: Fine Tuning Using KerasModel Conversion Using Core ML ToolsBuilding the iOS AppFurther ExplorationSummary
13. Shazam for Food: Developing Android Apps with TensorFlow Lite and ML Kit
The Life Cycle of a Food Classifier AppAn Overview of TensorFlow LiteTensorFlow Lite ArchitectureModel Conversion to TensorFlow LiteBuilding a Real-Time Object Recognition AppML Kit + FirebaseObject Classification in ML KitCustom Models in ML KitHosted ModelsA/B Testing Hosted ModelsUsing the Experiment in CodeTensorFlow Lite on iOSPerformance OptimizationsQuantizing with TensorFlow Lite ConverterTensorFlow Model Optimization ToolkitFritzA Holistic Look at the Mobile AI App Development CycleHow Do I Collect Initial Data?How Do I Label My Data?How Do I Train My Model?How Do I Convert the Model to a Mobile-Friendly Format?How Do I Make my Model Performant?How Do I Build a Great UX for My Users?How Do I Make the Model Available to My Users?How Do I Measure the Success of My Model?How Do I Improve My Model?How Do I Update the Model on My Users’ Phones?The Self-Evolving ModelCase StudiesLose It!Portrait Mode on Pixel 3 PhonesSpeaker Recognition by AlibabaFace Contours in ML KitReal-Time Video Segmentation in YouTube StoriesSummary
14. Building the Purrfect Cat Locator App with TensorFlow Object Detection API
Types of Computer-Vision TasksClassificationLocalizationDetectionSegmentationApproaches to Object DetectionInvoking Prebuilt Cloud-Based Object Detection APIsReusing a Pretrained ModelObtaining the ModelTest Driving Our ModelDeploying to a DeviceBuilding a Custom Detector Without Any CodeThe Evolution of Object DetectionPerformance ConsiderationsKey Terms in Object DetectionIntersection over UnionMean Average PrecisionNon-Maximum SuppressionUsing the TensorFlow Object Detection API to Build Custom ModelsData CollectionLabeling the DataPreprocessing the DataInspecting the ModelTrainingModel ConversionImage SegmentationCase StudiesSmart RefrigeratorCrowd CountingFace Detection in Seeing AIAutonomous CarsSummary
15. Becoming a Maker: Exploring Embedded AI at the Edge
Exploring the Landscape of Embedded AI DevicesRaspberry PiIntel Movidius Neural Compute StickGoogle Coral USB AcceleratorNVIDIA Jetson NanoFPGA + PYNQArduinoA Qualitative Comparison of Embedded AI DevicesHands-On with the Raspberry PiSpeeding Up with the Google Coral USB AcceleratorPort to NVIDIA Jetson NanoComparing the Performance of Edge DevicesCase StudiesJetBotSquatting for Metro TicketsCucumber SorterFurther ExplorationSummary
16. Building an Autonomous Car in Under an Hour: Reinforcement Learning with AWS DeepRacer
A Brief Introduction to Reinforcement LearningWhy Learn Reinforcement Learning with an Autonomous Car?Practical Deep Reinforcement Learning with DeepRacerBuilding Our First Reinforcement LearningStep 1: Create ModelStep 2: Configure TrainingStep 3: Model TrainingStep 4: Evaluating the Performance of the ModelReinforcement Learning in ActionHow Does a Reinforcement Learning System Learn?Reinforcement Learning TheoryReinforcement Learning Algorithm in AWS DeepRacerDeep Reinforcement Learning Summary with DeepRacer as an ExampleStep 5: Improving Reinforcement Learning ModelsRacing the AWS DeepRacer CarBuilding the TrackAWS DeepRacer Single-Turn Track TemplateRunning the Model on AWS DeepRacerDriving the AWS DeepRacer Vehicle AutonomouslyFurther ExplorationDeepRacer LeagueAdvanced AWS DeepRacerAI Driving OlympicsDIY RobocarsRoboraceSummary
A Crash Course in Convolutional Neural Networks
Machine LearningPerceptronActivation FunctionsNeural NetworksBackpropagationShortcoming of Neural NetworksDesired Properties of an Image ClassifierConvolutionPoolingStructure of a CNNFurther Exploration
Index
About the Authors

Content preview from Practical Deep Learning for Cloud, Mobile, and Edge

Chapter 12. Not Hotdog on iOS with Core ML and Create ML

“I’m a rich,” said Jian-Yang, a newly minted millionaire in an interview with Bloomberg (Figure 12-1). What did he do? He created the Not Hotdog app (Figure 12-2) and made the world “a better place.”

Jian Yang being interviewed by Bloomberg News after Periscope acquires his “Not Hotdog” technology (source: HBO’s Silicon Valley)

To the few of us who may be confused (including a third of the authors of this book), we are making a reference to HBO’s Silicon Valley, a show in which one of the characters is tasked with making SeeFood—the “Shazam for food.” It was meant to classify pictures of food and give recipes and nutritional information. Hilariously, the app ends up being good only for recognizing hot dogs. Anything else would be classified as “Not Hotdog.”

There are a few reasons we chose to reference this fictitious app. It’s very much a part of popular culture and something many people can easily relate to. It’s an exemplar: easy enough to build, yet powerful enough to see the magic of deep learning in a real-world application. It is also very trivially generalizable to recognize more than one class of items.

The Not Hotdog app in action (image source: Apple App Store listing for the Not Hotdog app)

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Practical Simulations for Machine Learning

Publisher Resources

ISBN: 9781492034858Errata Page

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Practical Deep Learning for Cloud, Mobile, and Edge

by Anirudh Koul, Siddha Ganju, Meher Kasam

Chapter 12. Not Hotdog on iOS with Core ML and Create ML

Figure 12-1. Jian-Yang being interviewed by Bloomberg News after Periscope acquires his “Not Hotdog” technology (image source: From HBO’s Silicon Valley)

Figure 12-2. The Not Hotdog app in action (image source: Apple App Store listing for the Not ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.