book

AI at the Edge

by Daniel Situnayake, Jenny Plunkett

January 2023

Intermediate to advanced

512 pages

14h 21m

English

O'Reilly Media, Inc.

Book available

Read now

Unlock full access

About This BookWhat to ExpectWhat You Need to Know AlreadyResponsible, Ethical, and Effective AIFurther ResourcesConventions Used in This BookUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgments
Defining Key TermsEmbeddedThe Edge (and the Internet of Things)Artificial IntelligenceMachine LearningEdge AIEmbedded Machine Learning and Tiny Machine LearningDigital Signal ProcessingWhy Do We Need Edge AI?To Understand the Benefits of Edge AI, Just BLERPEdge AI for GoodKey Differences Between Edge AI and Regular AISummary
Common Use Cases for Edge AIGreenfield and Brownfield ProjectsReal-World ProductsTypes of ApplicationsKeeping Track of ObjectsUnderstanding and Controlling SystemsUnderstanding People and Living ThingsTransforming SignalsBuilding Applications ResponsiblyResponsible Design and AI EthicsBlack Boxes and BiasTechnology That Harms, Not HelpsSummary
Sensors, Signals, and Sources of DataTypes of Sensors and SignalsAcoustic and VibrationVisual and SceneMotion and PositionForce and TactileOptical, Electromagnetic, and RadiationEnvironmental, Biological, and ChemicalOther SignalsProcessors for Edge AIEdge AI Hardware ArchitectureMicrocontrollers and Digital Signal ProcessorsSystem-on-ChipDeep Learning AcceleratorsFPGAs and ASICsEdge ServersMulti-Device ArchitecturesDevices and WorkloadsSummary
Feature EngineeringWorking with Data StreamsDigital Signal Processing AlgorithmsCombining Features and SensorsArtificial Intelligence AlgorithmsAlgorithm Types by FunctionalityAlgorithm Types by ImplementationOptimization for Edge DevicesOn-Device TrainingSummary
Building a Team for AI at the EdgeDomain ExpertiseDiversityStakeholdersRoles and ResponsibilitiesHiring for Edge AILearning Edge AI SkillsTools of the TradeSoftware EngineeringWorking with DataAlgorithm DevelopmentRunning Algorithms On-DeviceEmbedded Software Engineering and ElectronicsEnd-to-End Platforms for Edge AISummary
The Edge AI WorkflowResponsible AI in the Edge AI WorkflowDo I Need Edge AI?Describing a ProblemDo I Need to Deploy to the Edge?Do I Need Machine Learning?Practical ExerciseDetermining FeasibilityMoral FeasibilityBusiness FeasibilityDataset FeasibilityTechnological FeasibilityMaking a Final DecisionPlanning an Edge AI ProjectSummary
What Does a Dataset Look Like?The Ideal DatasetDatasets and Domain ExpertiseData, Ethics, and Responsible AIMinimizing UnknownsEnsuring Domain ExpertiseData-Centric Machine LearningEstimating Data RequirementsA Practical Workflow for Estimating Data RequirementsGetting Your Hands on DataThe Unique Challenges of Capturing Data at the EdgeStoring and Retrieving DataGetting Data into StoresCollecting MetadataEnsuring Data QualityEnsuring Representative DatasetsReviewing Data by SamplingLabel NoiseCommon Data ErrorsDrift and ShiftThe Uneven Distribution of ErrorsPreparing DataLabelingFormattingData CleaningFeature EngineeringSplitting Your DataData AugmentationData PipelinesBuilding a Dataset over TimeSummary
Product and Experience DesignDesign PrinciplesScoping a SolutionSetting Design GoalsArchitectural DesignHardware, Software, and ServicesBasic Application ArchitecturesComplex Application Architectures and Design PatternsWorking with Design PatternsAccounting for Choices in DesignDesign DeliverablesSummary

An Iterative Workflow for Edge AI DevelopmentExplorationGoal SettingBootstrappingTest and IterateDeploymentSupportSummary
Evaluating Edge AI SystemsWays to Evaluate a SystemUseful MetricsTechniques for EvaluationEvaluation and Responsible AIDeploying Edge AI ApplicationsPredeployment TasksMid-Deployment TasksPostdeployment TasksSupporting Edge AI ApplicationsPostdeployment MonitoringImproving a Live ApplicationEthics and Long-Term SupportWhat Comes Next
Problem ExplorationSolution ExplorationGoal SettingSolution DesignWhat Solutions Already Exist?Solution Design ApproachesDesign ConsiderationsEnvironmental ImpactBootstrappingDefine Your Machine Learning ClassesDataset GatheringEdge ImpulseChoose Your Hardware and SensorsData CollectioniNaturalistDataset LimitationsDataset Licensing and Legal ObligationsCleaning Your DatasetUploading Data to Edge ImpulseDSP and Machine Learning WorkflowDigital Signal Processing BlockMachine Learning BlockTesting the ModelLive ClassificationModel TestingTest Your Model LocallyDeploymentCreate LibraryMobile Phone and ComputerPrebuilt Binary FlashingImpulse RunnerGitHub Source CodeIterate and Feedback LoopsAI for GoodRelated WorksDatasetsResearch
Problem ExplorationSolution ExplorationGoal SettingSolution DesignWhat Solutions Already Exist?Solution Design ApproachesDesign ConsiderationsEnvironmental and Social ImpactBootstrappingDefine Your Machine Learning ClassesDataset GatheringEdge ImpulseChoose Your Hardware and SensorsData CollectionData Ingestion FirmwareUploading Data to Edge ImpulseCleaning Your DatasetDataset Licensing and Legal ObligationsDSP and Machine Learning WorkflowDigital Signal Processing BlockMachine Learning BlockTesting the ModelLive ClassificationModel TestingDeploymentPrebuilt Binary FlashingGitHub Source CodeIterate and Feedback LoopsRelated WorksResearchNews and Other Articles
Problem ExplorationGoal SettingSolution DesignWhat Solutions Already Exist?Solution Design ApproachesDesign ConsiderationsEnvironmental and Social ImpactBootstrappingDefine Your Machine Learning ClassesDataset GatheringEdge ImpulseChoose Your Hardware and SensorsData CollectionData Ingestion FirmwareCleaning Your DatasetDataset Licensing and Legal ObligationsDSP and Machine Learning WorkflowDigital Signal Processing BlockMachine Learning BlocksTesting the ModelLive ClassificationModel TestingDeploymentPrebuilt Binary FlashingGitHub Source CodeIterate and Feedback LoopsRelated WorksResearchNews and Other Articles

Content preview from AI at the Edge

About the Authors

Daniel Situnayake is head of machine learning at Edge Impulse, where he leads embedded machine learning research and development. He is coauthor of the O’Reilly book TinyML: Machine Learning with TensorFlow Lite on Arduino and Ultra-Low Power Microcontrollers, the standard textbook on embedded machine learning, and has delivered guest lectures at Harvard, UC Berkeley, and UNIFEI. Dan previously worked on TensorFlow Lite at Google, and cofounded Tiny Farms, the first US company using automation to produce insect protein at industrial scale. He began his career lecturing in automatic identification and data capture at Birmingham City University.

Jenny Plunkett is a senior developer relations engineer at Edge Impulse, where she is a technical speaker, developer evangelist, and technical content creator. In addition to maintaining the Edge Impulse documentation, she has also created developer-facing resources for Arm Mbed OS and Pelion IoT. She has presented workshops and tech talks for major tech conferences such as the Grace Hopper Celebration, Edge AI Summit, Embedded Vision Summit, and more. Jenny previously worked as a software engineer and IoT consultant for Arm Mbed and Pelion. She graduated with a BS in electrical engineering from The University of Texas at Austin.