book

Practical Deep Learning for Cloud, Mobile, and Edge

by Anirudh Koul, Siddha Ganju, Meher Kasam

October 2019

Intermediate to advanced

583 pages

14h 36m

English

O'Reilly Media, Inc.

Read now

Unlock full access

To the Backend/Frontend/Mobile Software DeveloperTo the Data ScientistTo the StudentTo the TeacherTo the Robotics EnthusiastWhat to Expect in Each ChapterConventions Used in This BookUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgmentsGroup AcknowledgmentsPersonal Acknowledgments
An ApologyThe Real IntroductionWhat Is AI?Motivating ExamplesA Brief History of AIExciting BeginningsThe Cold and Dark DaysA Glimmer of HopeHow Deep Learning Became a ThingRecipe for the Perfect Deep Learning SolutionDatasetsModel ArchitectureFrameworksHardwareResponsible AIBiasAccountability and ExplainabilityReproducibilityRobustnessPrivacySummaryFrequently Asked Questions
Introducing KerasPredicting an Image’s CategoryInvestigating the ModelImageNet DatasetModel ZoosClass Activation MapsSummary
Adapting Pretrained Models to New TasksA Shallow Dive into Convolutional Neural NetworksTransfer LearningFine TuningHow Much to Fine TuneBuilding a Custom Classifier in Keras with Transfer LearningOrganize the DataBuild the Data PipelineNumber of ClassesBatch SizeData AugmentationModel DefinitionTrain the ModelSet Training ParametersStart TrainingTest the ModelAnalyzing the ResultsFurther ReadingSummary
Image SimilarityFeature ExtractionSimilarity SearchVisualizing Image Clusters with t-SNEImproving the Speed of Similarity SearchLength of Feature VectorsReducing Feature-Length with PCAScaling Similarity Search with Approximate Nearest NeighborsApproximate Nearest-Neighbor BenchmarkWhich Library Should I Use?Creating a Synthetic DatasetBrute ForceAnnoyNGTFaissImproving Accuracy with Fine TuningFine Tuning Without Fully Connected LayersSiamese Networks for One-Shot Face VerificationCase StudiesFlickrPinterestCelebrity DoppelgangersSpotifyImage CaptioningSummary
Tools of the TradeTensorFlow DatasetsTensorBoardWhat-If Tooltf-explainCommon Techniques for Machine Learning ExperimentationData InspectionBreaking the Data: Train, Validation, TestEarly StoppingReproducible ExperimentsEnd-to-End Deep Learning Example PipelineBasic Transfer Learning PipelineBasic Custom Network PipelineHow Hyperparameters Affect AccuracyTransfer Learning Versus Training from ScratchEffect of Number of Layers Fine-Tuned in Transfer LearningEffect of Data Size on Transfer LearningEffect of Learning RateEffect of OptimizersEffect of Batch SizeEffect of ResizingEffect of Change in Aspect Ratio on Transfer LearningTools to Automate Tuning for Maximum AccuracyKeras TunerAutoAugmentAutoKerasSummary
GPU Starvationnvidia-smiTensorFlow Profiler + TensorBoardHow to Use This ChecklistPerformance ChecklistData PreparationData ReadingData AugmentationTrainingInferenceData PreparationStore as TFRecordsReduce Size of Input DataUse TensorFlow DatasetsData ReadingUse tf.dataPrefetch DataParallelize CPU ProcessingParallelize I/O and ProcessingEnable Nondeterministic OrderingCache DataTurn on Experimental OptimizationsAutotune Parameter ValuesData AugmentationUse GPU for AugmentationTrainingUse Automatic Mixed PrecisionUse Larger Batch SizeUse Multiples of EightFind the Optimal Learning RateUse tf.functionOvertrain, and Then GeneralizeInstall an Optimized Stack for the HardwareOptimize the Number of Parallel CPU ThreadsUse Better HardwareDistribute TrainingExamine Industry BenchmarksInferenceUse an Efficient ModelQuantize the ModelPrune the ModelUse Fused OperationsEnable GPU PersistenceSummary
InstallationTrainingModelDataPrivacyEducation and ExplorationOne Last Question
The Landscape of Visual Recognition APIsClarifaiMicrosoft Cognitive ServicesGoogle Cloud VisionAmazon RekognitionIBM Watson Visual RecognitionAlgorithmiaComparing Visual Recognition APIsService OfferingsCostAccuracyBiasGetting Up and Running with Cloud APIsTraining Our Own Custom ClassifierTop Reasons Why Our Classifier Does Not Work SatisfactorilyComparing Custom Classification APIsPerformance Tuning for Cloud APIsEffect of Resizing on Image Labeling APIsEffect of Compression on Image Labeling APIsEffect of Compression on OCR APIsEffect of Resizing on OCR APIsCase StudiesThe New York TimesUberGiphyOmniEarthPhotobucketStaplesInDro RoboticsSummary
Landscape of Serving AI PredictionsFlask: Build Your Own ServerMaking a REST API with FlaskDeploying a Keras Model to FlaskPros of Using FlaskCons of Using FlaskDesirable Qualities in a Production-Level Serving SystemHigh AvailabilityScalabilityLow LatencyGeographic AvailabilityFailure HandlingMonitoringModel VersioningA/B TestingSupport for Multiple Machine Learning LibrariesGoogle Cloud ML Engine: A Managed Cloud AI Serving StackPros of Using Cloud ML EngineCons of Using Cloud ML EngineBuilding a Classification APITensorFlow ServingInstallationKubeFlowPipelinesFairingInstallationPrice Versus Performance ConsiderationsCost Analysis of Inference-as-a-ServiceCost Analysis of Building Your Own StackSummary

JavaScript-Based Machine Learning Libraries: A Brief HistoryConvNetJSKeras.jsONNX.jsTensorFlow.jsTensorFlow.js ArchitectureRunning Pretrained Models Using TensorFlow.jsModel Conversion for the BrowserTraining in the BrowserFeature ExtractionData CollectionTrainingGPU Utilizationml5.jsPoseNetpix2pixBenchmarking and Practical ConsiderationsModel SizeInference TimeCase StudiesSemi-ConductorTensorSpaceMetacarAirbnb’s Photo ClassificationGAN LabSummary
The Development Life Cycle for Artificial Intelligence on MobileA Brief History of Core MLAlternatives to Core MLTensorFlow LiteML KitFritzApple’s Machine Learning ArchitectureDomain-Based FrameworksML FrameworkML Performance PrimitivesBuilding a Real-Time Object Recognition AppConversion to Core MLConversion from KerasConversion from TensorFlowDynamic Model DeploymentOn-Device TrainingFederated LearningPerformance AnalysisBenchmarking Models on iPhonesMeasuring Energy ImpactBenchmarking LoadReducing App SizeAvoid Bundling the ModelUse QuantizationUse Create MLCase StudiesMagic SudokuSeeing AIHomeCourtInstaSaber + YoPuppetSummary
Collecting DataApproach 1: Find or Collect a DatasetApproach 2: Fatkun Chrome Browser ExtensionApproach 3: Web Scraper Using Bing Image Search APITraining Our ModelApproach 1: Use Web UI-based ToolsApproach 2: Use Create MLApproach 3: Fine Tuning Using KerasModel Conversion Using Core ML ToolsBuilding the iOS AppFurther ExplorationSummary
The Life Cycle of a Food Classifier AppAn Overview of TensorFlow LiteTensorFlow Lite ArchitectureModel Conversion to TensorFlow LiteBuilding a Real-Time Object Recognition AppML Kit + FirebaseObject Classification in ML KitCustom Models in ML KitHosted ModelsA/B Testing Hosted ModelsUsing the Experiment in CodeTensorFlow Lite on iOSPerformance OptimizationsQuantizing with TensorFlow Lite ConverterTensorFlow Model Optimization ToolkitFritzA Holistic Look at the Mobile AI App Development CycleHow Do I Collect Initial Data?How Do I Label My Data?How Do I Train My Model?How Do I Convert the Model to a Mobile-Friendly Format?How Do I Make my Model Performant?How Do I Build a Great UX for My Users?How Do I Make the Model Available to My Users?How Do I Measure the Success of My Model?How Do I Improve My Model?How Do I Update the Model on My Users’ Phones?The Self-Evolving ModelCase StudiesLose It!Portrait Mode on Pixel 3 PhonesSpeaker Recognition by AlibabaFace Contours in ML KitReal-Time Video Segmentation in YouTube StoriesSummary
Types of Computer-Vision TasksClassificationLocalizationDetectionSegmentationApproaches to Object DetectionInvoking Prebuilt Cloud-Based Object Detection APIsReusing a Pretrained ModelObtaining the ModelTest Driving Our ModelDeploying to a DeviceBuilding a Custom Detector Without Any CodeThe Evolution of Object DetectionPerformance ConsiderationsKey Terms in Object DetectionIntersection over UnionMean Average PrecisionNon-Maximum SuppressionUsing the TensorFlow Object Detection API to Build Custom ModelsData CollectionLabeling the DataPreprocessing the DataInspecting the ModelTrainingModel ConversionImage SegmentationCase StudiesSmart RefrigeratorCrowd CountingFace Detection in Seeing AIAutonomous CarsSummary
Exploring the Landscape of Embedded AI DevicesRaspberry PiIntel Movidius Neural Compute StickGoogle Coral USB AcceleratorNVIDIA Jetson NanoFPGA + PYNQArduinoA Qualitative Comparison of Embedded AI DevicesHands-On with the Raspberry PiSpeeding Up with the Google Coral USB AcceleratorPort to NVIDIA Jetson NanoComparing the Performance of Edge DevicesCase StudiesJetBotSquatting for Metro TicketsCucumber SorterFurther ExplorationSummary
A Brief Introduction to Reinforcement LearningWhy Learn Reinforcement Learning with an Autonomous Car?Practical Deep Reinforcement Learning with DeepRacerBuilding Our First Reinforcement LearningStep 1: Create ModelStep 2: Configure TrainingStep 3: Model TrainingStep 4: Evaluating the Performance of the ModelReinforcement Learning in ActionHow Does a Reinforcement Learning System Learn?Reinforcement Learning TheoryReinforcement Learning Algorithm in AWS DeepRacerDeep Reinforcement Learning Summary with DeepRacer as an ExampleStep 5: Improving Reinforcement Learning ModelsRacing the AWS DeepRacer CarBuilding the TrackAWS DeepRacer Single-Turn Track TemplateRunning the Model on AWS DeepRacerDriving the AWS DeepRacer Vehicle AutonomouslyFurther ExplorationDeepRacer LeagueAdvanced AWS DeepRacerAI Driving OlympicsDIY RobocarsRoboraceSummary
Machine LearningPerceptronActivation FunctionsNeural NetworksBackpropagationShortcoming of Neural NetworksDesired Properties of an Image ClassifierConvolutionPoolingStructure of a CNNFurther Exploration

Content preview from Practical Deep Learning for Cloud, Mobile, and Edge

Chapter 4. Building a Reverse Image Search Engine: Understanding Embeddings

Bob just bought a new home and is looking to fill it up with some fancy modern furniture. He’s flipping endlessly through furniture catalogs and visiting furniture showrooms, but hasn’t yet landed on something he likes. Then one day, he spots the sofa of his dreams—a unique L-shaped white contemporary sofa in an office reception. The good news is that he knows what he wants. The bad news is that he doesn’t know where to buy it from. The brand and model number is not written on the sofa. Asking the office manager doesn’t help either. So, he takes a few photos from different angles to ask around in local furniture shops, but tough luck: no one knows this particular brand. And searching on the internet with keywords like “white L-shaped,” “modern sofa” gives him thousands of results, but not the one he’s looking for.

Alice hears Bob’s frustration and asks, “Why don’t you try reverse image search?” Bob uploads his images on Google and Bing’s Reverse Image Search and quickly spots a similar-looking image on an online shopping website. Taking this more perfect image from the website, he does a few more reverse image searches and finds other websites offering the same sofa at cheaper prices. After a few minutes of being online, Bob has officially ordered his dream sofa!

Reverse image search (or as it is more technically known, instance retrieval) enables developers and researchers to build scenarios beyond simple ...