AI at the Edge

AI at the Edge

by Daniel Situnayake, Jenny Plunkett
Released January 2023
Publisher(s): O'Reilly Media, Inc.
ISBN: 9781098120207

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Buy on Amazon Buy on ebooks.com
Start your free trial

Book description

Edge AI is transforming the way computers interact with the real world, allowing IoT devices to make decisions using the 99% of sensor data that was previously discarded due to cost, bandwidth, or power limitations. With techniques like embedded machine learning, developers can capture human intuition and deploy it to any target--from ultra-low power microcontrollers to embedded Linux devices.

This practical guide gives engineering professionals, including product managers and technology leaders, an end-to-end framework for solving real-world industrial, commercial, and scientific problems with edge AI. You'll explore every stage of the process, from data collection to model optimization to tuning and testing, as you learn how to design and support edge AI and embedded ML products. Edge AI is destined to become a standard tool for systems engineers. This high-level road map helps you get started.

  • Develop your expertise in AI and ML for edge devices
  • Understand which projects are best solved with edge AI
  • Explore key design patterns for edge AI apps
  • Learn an iterative workflow for developing AI systems
  • Build a team with the skills to solve real-world problems
  • Follow a responsible AI process to create effective products

Table of contents

  1. Foreword
  2. Preface
    1. About This Book
    2. What to Expect
    3. What You Need to Know Already
    4. Responsible, Ethical, and Effective AI
    5. Further Resources
    6. Conventions Used in This Book
    7. Using Code Examples
    8. O’Reilly Online Learning
    9. How to Contact Us
    10. Acknowledgments
  3. 1. A Brief Introduction to Edge AI
    1. Defining Key Terms
      1. Embedded
      2. The Edge (and the Internet of Things)
      3. Artificial Intelligence
      4. Machine Learning
      5. Edge AI
      6. Embedded Machine Learning and Tiny Machine Learning
      7. Digital Signal Processing
    2. Why Do We Need Edge AI?
      1. To Understand the Benefits of Edge AI, Just BLERP
      2. Edge AI for Good
      3. Key Differences Between Edge AI and Regular AI
    3. Summary
  4. 2. Edge AI in the Real World
    1. Common Use Cases for Edge AI
      1. Greenfield and Brownfield Projects
      2. Real-World Products
    2. Types of Applications
      1. Keeping Track of Objects
      2. Understanding and Controlling Systems
      3. Understanding People and Living Things
      4. Transforming Signals
    3. Building Applications Responsibly
      1. Responsible Design and AI Ethics
      2. Black Boxes and Bias
      3. Technology That Harms, Not Helps
    4. Summary
  5. 3. The Hardware of Edge AI
    1. Sensors, Signals, and Sources of Data
      1. Types of Sensors and Signals
      2. Acoustic and Vibration
      3. Visual and Scene
      4. Motion and Position
      5. Force and Tactile
      6. Optical, Electromagnetic, and Radiation
      7. Environmental, Biological, and Chemical
      8. Other Signals
    2. Processors for Edge AI
      1. Edge AI Hardware Architecture
      2. Microcontrollers and Digital Signal Processors
      3. System-on-Chip
      4. Deep Learning Accelerators
      5. FPGAs and ASICs
      6. Edge Servers
      7. Multi-Device Architectures
      8. Devices and Workloads
    3. Summary
  6. 4. Algorithms for Edge AI
    1. Feature Engineering
      1. Working with Data Streams
      2. Digital Signal Processing Algorithms
      3. Combining Features and Sensors
    2. Artificial Intelligence Algorithms
      1. Algorithm Types by Functionality
      2. Algorithm Types by Implementation
      3. Optimization for Edge Devices
      4. On-Device Training
    3. Summary
  7. 5. Tools and Expertise
    1. Building a Team for AI at the Edge
      1. Domain Expertise
      2. Diversity
      3. Stakeholders
      4. Roles and Responsibilities
      5. Hiring for Edge AI
      6. Learning Edge AI Skills
    2. Tools of the Trade
      1. Software Engineering
      2. Working with Data
      3. Algorithm Development
      4. Running Algorithms On-Device
      5. Embedded Software Engineering and Electronics
      6. End-to-End Platforms for Edge AI
    3. Summary
  8. 6. Understanding and Framing Problems
    1. The Edge AI Workflow
      1. Responsible AI in the Edge AI Workflow
    2. Do I Need Edge AI?
      1. Describing a Problem
      2. Do I Need to Deploy to the Edge?
      3. Do I Need Machine Learning?
      4. Practical Exercise
    3. Determining Feasibility
      1. Moral Feasibility
      2. Business Feasibility
      3. Dataset Feasibility
      4. Technological Feasibility
      5. Making a Final Decision
      6. Planning an Edge AI Project
    4. Summary
  9. 7. How to Build a Dataset
    1. What Does a Dataset Look Like?
    2. The Ideal Dataset
    3. Datasets and Domain Expertise
    4. Data, Ethics, and Responsible AI
      1. Minimizing Unknowns
      2. Ensuring Domain Expertise
    5. Data-Centric Machine Learning
    6. Estimating Data Requirements
      1. A Practical Workflow for Estimating Data Requirements
    7. Getting Your Hands on Data
      1. The Unique Challenges of Capturing Data at the Edge
    8. Storing and Retrieving Data
      1. Getting Data into Stores
      2. Collecting Metadata
    9. Ensuring Data Quality
      1. Ensuring Representative Datasets
      2. Reviewing Data by Sampling
      3. Label Noise
      4. Common Data Errors
      5. Drift and Shift
      6. The Uneven Distribution of Errors
    10. Preparing Data
      1. Labeling
      2. Formatting
      3. Data Cleaning
      4. Feature Engineering
      5. Splitting Your Data
      6. Data Augmentation
      7. Data Pipelines
    11. Building a Dataset over Time
    12. Summary
  10. 8. Designing Edge AI Applications
    1. Product and Experience Design
      1. Design Principles
      2. Scoping a Solution
      3. Setting Design Goals
    2. Architectural Design
      1. Hardware, Software, and Services
      2. Basic Application Architectures
      3. Complex Application Architectures and Design Patterns
      4. Working with Design Patterns
    3. Accounting for Choices in Design
      1. Design Deliverables
    4. Summary
  11. 9. Developing Edge AI Applications
    1. An Iterative Workflow for Edge AI Development
      1. Exploration
      2. Goal Setting
      3. Bootstrapping
      4. Test and Iterate
      5. Deployment
      6. Support
    2. Summary
  12. 10. Evaluating, Deploying, and Supporting Edge AI Applications
    1. Evaluating Edge AI Systems
      1. Ways to Evaluate a System
      2. Useful Metrics
      3. Techniques for Evaluation
      4. Evaluation and Responsible AI
    2. Deploying Edge AI Applications
      1. Predeployment Tasks
      2. Mid-Deployment Tasks
      3. Postdeployment Tasks
    3. Supporting Edge AI Applications
      1. Postdeployment Monitoring
      2. Improving a Live Application
      3. Ethics and Long-Term Support
    4. What Comes Next
  13. 11. Use Case: Wildlife Monitoring
    1. Problem Exploration
    2. Solution Exploration
    3. Goal Setting
    4. Solution Design
      1. What Solutions Already Exist?
      2. Solution Design Approaches
      3. Design Considerations
      4. Environmental Impact
      5. Bootstrapping
      6. Define Your Machine Learning Classes
    5. Dataset Gathering
      1. Edge Impulse
      2. Choose Your Hardware and Sensors
      3. Data Collection
      4. iNaturalist
      5. Dataset Limitations
      6. Dataset Licensing and Legal Obligations
      7. Cleaning Your Dataset
      8. Uploading Data to Edge Impulse
    6. DSP and Machine Learning Workflow
      1. Digital Signal Processing Block
      2. Machine Learning Block
    7. Testing the Model
      1. Live Classification
      2. Model Testing
      3. Test Your Model Locally
    8. Deployment
      1. Create Library
      2. Mobile Phone and Computer
      3. Prebuilt Binary Flashing
      4. Impulse Runner
      5. GitHub Source Code
    9. Iterate and Feedback Loops
    10. AI for Good
    11. Related Works
      1. Datasets
      2. Research
  14. 12. Use Case: Food Quality Assurance
    1. Problem Exploration
    2. Solution Exploration
    3. Goal Setting
    4. Solution Design
      1. What Solutions Already Exist?
      2. Solution Design Approaches
      3. Design Considerations
      4. Environmental and Social Impact
      5. Bootstrapping
      6. Define Your Machine Learning Classes
    5. Dataset Gathering
      1. Edge Impulse
      2. Choose Your Hardware and Sensors
      3. Data Collection
      4. Data Ingestion Firmware
      5. Uploading Data to Edge Impulse
      6. Cleaning Your Dataset
      7. Dataset Licensing and Legal Obligations
    6. DSP and Machine Learning Workflow
      1. Digital Signal Processing Block
      2. Machine Learning Block
    7. Testing the Model
      1. Live Classification
      2. Model Testing
    8. Deployment
      1. Prebuilt Binary Flashing
      2. GitHub Source Code
    9. Iterate and Feedback Loops
    10. Related Works
      1. Research
      2. News and Other Articles
  15. 13. Use Case: Consumer Products
    1. Problem Exploration
    2. Goal Setting
    3. Solution Design
      1. What Solutions Already Exist?
      2. Solution Design Approaches
      3. Design Considerations
      4. Environmental and Social Impact
      5. Bootstrapping
      6. Define Your Machine Learning Classes
    4. Dataset Gathering
      1. Edge Impulse
      2. Choose Your Hardware and Sensors
      3. Data Collection
      4. Data Ingestion Firmware
      5. Cleaning Your Dataset
      6. Dataset Licensing and Legal Obligations
    5. DSP and Machine Learning Workflow
      1. Digital Signal Processing Block
      2. Machine Learning Blocks
    6. Testing the Model
      1. Live Classification
      2. Model Testing
    7. Deployment
      1. Prebuilt Binary Flashing
      2. GitHub Source Code
    8. Iterate and Feedback Loops
    9. Related Works
      1. Research
      2. News and Other Articles
  16. Index
  17. About the Authors

Product information

  • Title: AI at the Edge
  • Author(s): Daniel Situnayake, Jenny Plunkett
  • Release date: January 2023
  • Publisher(s): O'Reilly Media, Inc.
  • ISBN: 9781098120207