O'Reilly logo

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

TinyML

TinyML

by Daniel Situnayake, Pete Warden
Publisher: O'Reilly Media, Inc.
Release Date: December 2019
ISBN: 9781492052036
View table of contents
Start reading

Book Description

Neural networks are getting smaller. Much smaller. The OK Google team, for example, has run machine learning models that are just 14 kilobytes in size—small enough to work on the digital signal processor in an Android phone. With this practical book, you’ll learn about TensorFlow Lite for Microcontrollers, a miniscule machine learning library that allows you to run machine learning algorithms on tiny hardware.

Authors Pete Warden and Daniel Situnayake explain how you can train models that are small enough to fit into any environment, including small embedded devices that can run for a year or more on a single coin cell battery. Ideal for software and hardware developers who want to build embedded devices using machine learning, this guide shows you how to create a TinyML project step-by-step. No machine learning or microcontroller experience is necessary.

  • Learn practical machine learning applications on embedded devices, including simple uses such as speech recognition and gesture detection
  • Train models such as speech, accelerometer, and image recognition, you can deploy on Arduino and other embedded platforms
  • Understand how to work with Arduino and ultralow-power microcontrollers
  • Use techniques for optimizing latency, energy usage, and model and binary size

Table of Contents

  1. Preface
    1. Conventions Used in This Book
    2. Using Code Examples
    3. O’Reilly Online Learning
    4. How to Contact Us
    5. Acknowledgments
  2. 1. Introduction
    1. What Is TinyML?
      1. What platforms does TinyML include?
    2. Embedded devices
    3. Changing landscape
  3. 2. Getting Started
    1. Who Is This Book Aimed At?
    2. What Hardware Do You Need?
    3. What Software Do You Need?
    4. What Do We Hope You’ll Learn?
  4. 3. Get Up to Speed on Machine Learning
    1. What Machine Learning Actually Is
    2. The Deep Learning Workflow
      1. Decide on a Goal
      2. Collect a Dataset
      3. Design a Model Architecture
      4. Train the Model
      5. Convert the Model
      6. Run inference
      7. Evaluate and Troubleshoot
    3. Wrapping Up
  5. 4. The “hello world” Of TinyML: Building and Training a Model
    1. What We’re Building
    2. Meet Our Machine Learning Toolchain
      1. Python and Jupyter Notebooks
      2. Google Colaboratory
      3. TensorFlow and Keras
    3. Building our Model
      1. Importing Dependencies
      2. Generating Data
      3. Splitting the Data
      4. Defining a Basic Model
    4. Training our Model
      1. Training Metrics
      2. Graphing the history
      3. Improving our Model
      4. Testing
    5. Converting the Model for TensorFlow Lite
      1. Converting to a C File
    6. Wrapping Up
  6. 5. The “hello world” of TinyML: Building an Application
    1. Walking Through The Tests
      1. Including Dependencies
      2. Setting up the Test
      3. Getting Ready To Log Data
      4. Mapping our Model
      5. Creating an AllOpsResolver
      6. Defining a tensor arena
      7. Creating an Interpreter
      8. Inspect the Input Tensor
      9. Running Inference on an Input
      10. Reading the Output
      11. Running the Tests
    2. Project File Structure
    3. Walk Through the Source
      1. Starting with main_functions.cc
      2. Handling Output with output_handler.cc
      3. Wrapping Up main_functions.cc
      4. Understanding main.cc
      5. Running Our Application
    4. Wrapping Up
  7. 6. The “hello world” of TinyML: Deploying to Microcontrollers
    1. What Exactly Is a Microcontroller?
    2. Arduino
      1. Handling Output on Arduino
      2. Making Your Own Changes
    3. SparkFun Edge
      1. Handling Output on SparkFun Edge
      2. Running the Example
      3. Build the Binary
      4. Sign the Binary
      5. Flash the Binary
      6. Making Your Own Changes
    4. ST Microelectronics STM32F746G Discovery kit
      1. Handling Output on STM32F746G
      2. Running the Example
      3. Making your own changes
    5. Wrapping Up
  8. 7. Wake Word Detection: Building an Application
    1. What We’re Building
    2. Application Architecture
      1. Introducing Our Model
      2. All the Moving Parts
    3. Walking Through the Tests
      1. The Basic Flow
      2. The Audio Provider
      3. The Feature Provider
      4. Recognize Commands
      5. Command Responder
    4. Listening for Wake Words
      1. Running our application
    5. Deploying to Microcontrollers
      1. Arduino
      2. SparkFun Edge
      3. ST Microelectronics STM32F746G Discovery Kit
    6. Wrapping Up
  9. 8. Wake Word Detection: Training a Model
    1. Training Our New Model
      1. Training in Colab
    2. Using the Model in Our Project
      1. Replacing the Model
      2. Updating the Labels
      3. Updating command_responder.cc
      4. Other Ways to Run the Scripts
    3. How the Model Works
      1. Visualizing the Inputs
      2. How Does the Feature Generation Work?
      3. Understanding the Model Architecture
      4. Understanding the Model Output
    4. Training with Your Own Data
      1. The Speech Commands Dataset
      2. Training on Your Own Dataset
      3. How to Record Your Own Audio
      4. Data Augmentation
      5. Model Architectures
    5. Wrapping Up
  10. 9. Person Detection: Building an Application
    1. What We’re Building
    2. Application Architecture
      1. Introducing our Model
      2. All the Moving Parts
    3. Walking Through the Tests
      1. The Basic Flow
      2. The Image Provider
      3. The Detection Responder
    4. Detecting People
    5. Deploying to Microcontrollers
      1. Arduino
      2. SparkFun Edge
    6. Wrapping Up
  11. 10. Person Detection: Training a Model
    1. Picking a Machine
    2. Setting up a Google Cloud Instance
    3. Training Framework Choice
    4. Building the Dataset
    5. Training the Model
    6. TensorBoard
    7. Evaluating the Model
    8. Exporting the Model to TensorFlow Lite
      1. Exporting to a GraphDef Protobuf File
      2. Freezing the Weights
      3. Quantizing and Converting to TensorFlow Lite
      4. Converting into a C Source File
    9. Training for Other Categories
    10. Understanding the Architecture
    11. Wrapping Up
  12. 11. Magic Wand: Building an Application
    1. What We’re Building
    2. Application Architecture
      1. Introducing our Model
      2. All the Moving Parts
    3. Walking Through the Tests
      1. The Basic Flow
      2. The Accelerometer Handler
      3. The Gesture Predictor
      4. The Output Handler
    4. Detecting Gestures
    5. Deploying to Microcontrollers
      1. Arduino
      2. SparkFun Edge
    6. Wrapping Up
  13. 12. Magic Wand: Training A Model
    1. Training A Model
      1. Training in Colab
      2. Other Ways To Run The Scripts
    2. How the Model Works
      1. Visualizing the Input
      2. Understanding the Model Architecture
    3. Training with Your Own Data
      1. Capturing Data
      2. Modifying the Training Scripts
      3. Training
      4. Using the New Model In Our Project
    4. Wrapping Up
      1. Learning Machine Learning
      2. What’s Next
  14. 13. TensorFlow Lite for Microcontrollers
    1. What is TensorFlow Lite for Microcontrollers?
      1. TensorFlow
      2. TensorFlow Lite
      3. TensorFlow Lite for Microcontrollers
      4. Requirements
      5. Why Is the Model Interpreted?
      6. Project Generation
    2. Build Systems
      1. Specializing Code
      2. Makefiles
      3. Writing Tests
    3. Supporting a New Hardware Platform
      1. Figure Out How to Print to a Log
      2. Implement DebugLog()
      3. Run All the Targets
      4. Integrating with the Makefile Build
    4. Supporting a New IDE or Build System
    5. Integrating Code Changes Between Projects and Repositories
    6. Contributing Back to Open Source
    7. Supporting New Hardware Accelerators
    8. Understanding the File Format
      1. Flatbuffers
    9. Porting TensorFlow Lite Mobile Ops to Micro
      1. Separate Reference Code
      2. Create Micro Copy of Operator
      3. Port Test to Micro Framework
      4. Build a Bazel Test
      5. Add to AllOpsResolver
      6. Build a Makefile Test
    10. Wrapping Up
  15. 14. Designing Your Own TinyML Applications
    1. Design Process
    2. Do You Need a Microcontroller, or Will a Larger Device Work?
    3. Understanding What’s Possible
    4. Follow in Someone Else’s Footsteps
    5. Find Some Similar Models to Train
    6. Look at the Data
    7. Wizard of Oz-ing
    8. Get it Working on the Desktop First
  16. 15. Optimizing Latency
    1. First Make Sure it Matters
    2. Hardware Changes
    3. Model Improvements
      1. Estimating Model Latency
      2. How to Speed up Your Model
    4. Quantization
    5. Product Design
    6. Code Optimizations
      1. Performance Profiling
    7. Optimizing Operations
      1. Look for Implementations that Are Already Optimized
      2. Write Your Own Optimized Implementation
      3. Taking Advantage of Hardware Features
      4. Accelerators and Co-Processors
    8. Contributing Back to Open Source
    9. Wrapping Up
  17. 16. Optimizing Energy Usage
    1. Developing intuition
      1. Typical component power usage
      2. Hardware choice
    2. Measuring real power usage
    3. Estimating power usage for a model
    4. Improving power usage
      1. Duty cycling
      2. Cascading design
    5. Wrapping up
  18. 17. Optimizing Model and Binary Size
    1. Understanding your system’s limits
    2. Estimating memory usage
      1. Flash usage
      2. RAM usage
    3. Ballpark figures for model accuracy and size on different problems
      1. Speech wake-word model
      2. Accelerometer predictive maintenance model
      3. Person presence detection
    4. Model choice
    5. Reducing the size of your executable
      1. Measuring code size
      2. How much space is TensorFlow Lite for Microcontrollers taking?
      3. OpResolver
      4. Understanding the size of individual functions
      5. Framework constants
    6. Truly tiny models
    7. Wrapping up
  19. 18. Debugging
    1. Accuracy loss between training and deployment
      1. Preprocessing differences
      2. Debugging preprocessing
      3. On-device evaluation
    2. Numerical differences
      1. Are the differences a problem?
      2. Establish a metric
      3. Compare against a baseline
      4. Swap out implementations
    3. Mysterious crashes and hangs
      1. Desktop debugging
      2. Log tracing
      3. Shotgun debugging
      4. Memory corruption
    4. Wrapping up
  20. 19. Porting Models from TensorFlow to TensorFlow Lite
    1. Understand what ops are needed
    2. Look at existing op coverage in TensorFlow Lite
    3. Move preprocessing and postprocessing into application code
    4. Implement required ops if you have to
    5. Optimize ops
    6. Wrapping up
  21. 20. Privacy, Security, and Deployment
    1. Privacy
    2. Privacy Design Document
      1. Data Collection
      2. Data Usage
      3. Data Sharing and Storage
      4. Consent
    3. Using a Privacy Design Document
    4. Security
      1. Protecting models
    5. Deployment
      1. Moving from a development board to a product
    6. Wrapping up
  22. 21. Learning more
    1. TinyML Foundation
    2. SIG Micro
    3. TensorFlow website
    4. Other frameworks
    5. Twitter
    6. Friends of TinyML
    7. Wrapping up
  23. A. Using and generating an Arduino library zip
  24. B. Capturing audio on Arduino