book

TinyML Cookbook - Second Edition

Name: TinyML Cookbook - Second Edition
Author: Gian Marco Iodice
ISBN: 9781837637362

by Gian Marco Iodice

November 2023

Intermediate to advanced

664 pages

14h 14m

English

Packt Publishing

Read now

Unlock full access

Preface
Who this book is forWhat this book coversTo get the most out of this bookGet in touch
Getting Ready to Unlock ML on Microcontrollers
Technical requirementsIntroduction to tinyMLWhat is tinyML?Why ML on microcontrollers?Why run ML on-device?The opportunities and challenges for tinyMLDeployment environments for tinyMLJoin the tinyML community!Overview of deep learningDeep neural networksConvolutional neural networksModel quantizationLearning the difference between power and energyVoltage versus currentPower versus energyProgramming microcontrollersMemory architecturePeripheralsGeneral-purpose input/output (GPIO or IO)Analog/digital convertersSerial communicationTimersIntroduction to the development platformsArduino Nano 33 BLE SenseRaspberry Pi PicoSparkFun RedBoard Artemis NanoSetting up the software development environmentGetting ready with Arduino IDEGetting ready with TensorFlowGetting ready with Edge ImpulseDeploying a sketch on microcontrollersGetting readyHow to do it…There’s more…with the SparkFun Artemis Nano!SummaryLearn more on Discord
Unleashing Your Creativity with Microcontrollers
Technical requirementsTransmitting data over serial communicationGetting readyHow to do it…There’s more…Reading serial data and uploading files to Google Drive with PythonGetting readyReading data from the serial port with pySerialEnabling the Google Drive APIHow to do it…There’s more…Implementing an LED status indicator on the breadboardGetting readyPrototyping on a breadboardHow to do it…There’s more…with the SparkFun Artemis Nano!Controlling an external LED with the GPIOGetting readyUnderstanding the LED functionalityIntroducing the GPIO peripheralUsing the mbed::DigitalOut functionHow to do it…Connecting the LED to the GPIO pinProgramming the GPIO peripheral in output modeThere’s more…with the SparkFun Artemis Nano!Turning an LED on and off with a push-buttonGetting readyThe operating principles of the push-buttonHow to do it…Connecting the push-button to the GPIO pinProgramming the GPIO peripheral in input modeThere’s more…with the SparkFun Artemis Nano!Using interrupts to read the push-button stateGetting readyWorking with interrupts using the Mbed OS APIHow to do it...There’s more…SummaryLearn more on Discord
Building a Weather Station with TensorFlow Lite for Microcontrollers
Technical requirementsImporting weather data from WorldWeatherOnlineGetting readyHow to do it…There’s more…Preparing the datasetGetting readyBalancing the datasetFeature scaling with Z-scoreHow to do it…There’s more…Training the model with TensorFlowGetting readyHow to do it…There’s more…Evaluating the model’s effectivenessGetting readyEvaluating the performance with the confusion matrixEvaluating recall, precision, and F-scoreHow to do it…There’s more…Quantizing the model with the TensorFlow Lite converterGetting readyModel quantizationHow to do it…There’s more…Reading temperature and humidity data with the Arduino NanoGetting readyHow to do it…There’s more…Reading temperature and humidity with the DHT22 sensor and the Raspberry Pi PicoGetting readyHow to do it…There’s more…with the SparkFun Artemis Nano!Preparing the input features for the model inferenceGetting readyHow to do it…There’s more…with the SparkFun Artemis NanoOn-device inference with TensorFlow Lite for MicrocontrollersGetting readyHow to do it…There’s more…with the SparkFun Artemis Nano!SummaryLearn more on Discord
Using Edge Impulse and the Arduino Nano to Control LEDs with Voice Commands
Technical requirementsAcquiring audio data with a smartphoneGetting readyCollecting audio samples for KWSHow to do it…There’s moreAcquiring audio data with the Arduino NanoGetting readyCollecting data using a fully supported platform in Edge ImpulseHow to do it…There’s moreExtracting MFE features from audio samplesGetting readyAnalyzing audio in the frequency domainExtracting the Mel-spectrogramHow to do it…There’s moreDesigning and training a CNNGetting readyHow to do it…There’s moreTuning model performance with the EON TunerGetting readyHow to do it…There’s moreLive classifications with a smartphoneGetting readyHow to do it…There’s moreKeyword spotting on the Arduino NanoGetting readyLearning how a real-time KWS application worksHow to do it…There’s moreSummaryLearn more on Discord
Recognizing Music Genres with TensorFlow and the Raspberry Pi Pico – Part 1
Technical requirementsConnecting the microphone to the Raspberry Pi PicoGetting readyConnecting the microphone to the ADC pinHow to do it…There’s more…Recording audio samples with the Raspberry Pi PicoGetting readyRecording audio with ADC and timer interruptsProgramming the ADC with the Raspberry Pi Pico SDKHow to do it…There’s more…with the SparkFun Artemis Nano!Generating audio files from samples transmitted over the serialGetting readyHow to do it…There’s more…Building the dataset for classifying music genresGetting readyUsing the GTZAN dataset for music genre classificationAugmenting the dataset with audio samples taken with the Raspberry Pi PicoChoosing the suitable model input lengthHow to do it…There’s more…with the SparkFun Artemis Nano!Extracting MFCCs from audio samples with TensorFlowGetting readyApplying the Hann windowLeveraging RFFTCalculating the FFT magnitudeThe Mel scale conversionComputing the DCT coefficientsEvaluating the SRAM usage to run MFCCsHow to do it…There’s more…SummaryReferencesLearn more on Discord
Recognizing Music Genres with TensorFlow and the Raspberry Pi Pico – Part 2
Technical requirementsComputing the FFT magnitude with fixed-point arithmetic using the CMSIS-DSP libraryGetting readyEvaluating the hardware capabilities of the Raspberry Pi PicoUsing the CMSIS-DSP Python libraryRepresenting numbers in 16-bit fixed-point formatHow to do it…There’s more…Implementing the MFCCs feature extraction with the CMSIS-DSP libraryGetting readyExtracting the DCT coefficientsHow to do it…There’s more…Designing and training an LSTM RNN modelGetting readyTime series analysis with RNNsDesigning a many-to-one RNN for music genre classificationHow to do it…There’s more…Evaluating the accuracy of the quantized model on the test datasetGetting readyHow to do it…There’s more…Deploying the MFCCs feature extraction algorithm on the Raspberry Pi PicoGetting readyHow to do it…There’s more…with the SparkFun Artemis Nano!Recognizing music genres with the Raspberry Pi PicoGetting readyHow to do it…There’s more…with the SparkFun Artemis Nano!SummaryLearn more on Discord
Detecting Objects with Edge Impulse Using FOMO on the Raspberry Pi Pico
Technical requirementsAcquiring images with the webcamGetting readyBuilding a dataset for FOMOHow to do it…There’s more…Designing the Impulse’s pre-processing blockGetting readyChoosing the correct input image resolutionHow to do it…There’s more…with the SparkFun Artemis Nano!Transfer learning with FOMOGetting readyBehind the design of FOMOLocating objects from heat mapsHow to do it…There’s more…Evaluating the model’s accuracyGetting readyHow to do it…There’s more…Using OpenCV and pySerial to send images over the serial interfaceGetting readySending bytes over the serial with pySerialHow to do it…There’s more…Reading data from the serial port with Arduino-compatible platformsGetting readyHow to do it…There’s more…with the SparkFun Artemis Nano!Deploying FOMO on the Raspberry Pi PicoGetting readyInitializing the signal_t data structureReading the output results from ei_impulse_result_tTransmitting the centroid coordinates to the Python script How to do it…There’s more…with the SparkFun Artemis Nano!SummaryLearn more on Discord
Classifying Desk Objects with TensorFlow and the Arduino Nano
Technical requirementsTaking pictures with the OV7670 camera moduleGetting readyHow to do it…There’s more…Grabbing camera frames from the serial port with PythonGetting readyIntroducing the RGB565 color formatTransmitting images over the serialHow to do it…There’s more...Acquiring QQVGA images with the YCbCr422 color formatGetting readyConverting YCbCr422 to RGB888How to do it…There’s more…Building the dataset to classify desk objectsGetting readyHow to do it…There’s more…Transfer learning with KerasGetting readyBehind the MobileNet network design choicesHow to do it…There’s more…Quantizing and testing the trained model with TensorFlow LiteGetting readyHow to do it…There’s more…Fusing the pre-processing operators for efficient deploymentGetting readyImage resizing with bilinear interpolationHow to do it…There’s more…SummaryLearn more on Discord
Building a Gesture-Based Interface for YouTube Playback with Edge Impulse and the Raspberry Pi Pico
Technical requirementsCommunicating with the MPU-6050 IMU through I2CGetting readyIntroducing the features of the MPU-6050 IMUBasics of the I2C communication protocolProgramming the I2C peripheral with Mbed OSAccessing the I2C peripheral in Raspberry Pi PicoHow to do it…There’s more…with the SparkFun Artemis Nano!Acquiring accelerometer dataGetting readyThe basic principles of the accelerometerHow to do it…There’s more…with the SparkFun Artemis Nano!Building the dataset with the Edge Impulse data forwarder toolGetting readyHow to do it…There’s more…Designing and training the ML modelGetting readyUsing spectral analysis to recognize hand gesturesHow to do it…There’s moreLive classifications with the Edge Impulse data forwarder toolGetting readyHow to do it…There’s more…with the SparkFun Artemis NanoDeveloping a continuous gesture recognition application with Edge Impulse and Arm Mbed OSGetting readyManaging concurrent tasks with Arm Mbed OSFiltering out redundant and spurious predictionsHow to do it…There’s more…with the SparkFun Artemis NanoBuilding a gesture-based interface with PyAutoGUIGetting readyHow to do it…There’s moreSummaryLearn more on Discord

Deploying a CIFAR-10 Model for Memory-Constrained Devices with the Zephyr OS on QEMU
Technical requirementsGetting started with the Zephyr OSGetting readyHow to do it…There’s more…Designing and training a CIFAR-10 model for memory-constrained devicesGetting readyReplacing convolution with depthwise separable convolutionKeeping the model memory requirement under controlHow to do it…There’s more…Evaluating the accuracy of the quantized modelGetting readyHow to do it…There’s more…Converting a NumPy image into a C-byte arrayGetting readyHow to do it…There’s more…Preparing the Zephyr Project structureGetting readyHow to do it…There’s more…Deploying the TensorFlow Lite for Microcontrollers program on QEMUGetting readyHow to do it…There’s more…SummaryReferencesLearn more on Discord
Running ML Models on Arduino and the Arm Ethos-U55 microNPU Using Apache TVM
Technical requirementsGetting familiar with Arduino CLIGetting readyHow to do it…There’s more…with the SparkFun Artemis Nano!Downloading a pre-trained CIFAR-10 model and input test imageGetting readyHow to do it…There’s more…Deploying the model with TVM using the AoT executor on the host machineGetting readyBehind the TVM compilerCode generation with TVMGraph executor versus AoT executorIntroducing microTVM for microcontroller deployment How to do it…There’s more…Deploying the model on the Arduino NanoGetting readyRunning the model inference using the TVM runtime Building an Arduino sketch with the code generated by TVMHow to do it…There’s more…Deploying the model on the Raspberry Pi PicoGetting readyHow to do it…There’s more…with the SparkFun Artemis Nano!Installing the Fixed Virtual Platform (FVP) for the Arm Corstone-300 Getting readyHow to do it…There’s more…Code generation with TVMC for Arm Ethos-U55Getting readyHow to do it…There’s more…Installing the software dependencies to build an application for the Arm Ethos-U microNPUGetting readyThe GNU Arm Embedded ToolchainThe Ethos-U driverThe Ethos-U platformHow to do it…There’s more…Running the CIFAR-10 model inference on the Arm Ethos-U55 microNPUGetting readyAn overview of memory capabilities on Corstone-300 FVP How to do it…There’s more…SummaryLearn more on Discord
Enabling Compelling tinyML Solutions with On-Device Learning and scikit-learn on the Arduino Nano and Raspberry Pi Pico
Technical requirementsHow can we train a model on microcontrollers?Getting readySolving the XOR and NAND problem using a shallow neural networkTraining a neural network with backpropagationHow to do it…There’s more…with the SparkFun Artemis Nano!How can we deploy scikit-learn models on microcontrollers?Getting readyHow to do it… There’s more…with the SparkFun Artemis NanoHow can we power microcontrollers with batteries?Getting readyIncreasing the output voltage by connecting batteries in seriesIncreasing the charge capacity by connecting batteries in parallelConnecting batteries to the microcontroller boardHow to do it…There’s more…with the SparkFun Artemis NanoSummaryReferencesLearn more on Discord
Conclusion
Other Books You May Enjoy
Index

Content preview from TinyML Cookbook - Second Edition

4 Using Edge Impulse and the Arduino Nano to Control LEDs with Voice Commands

Have you ever wondered how smart assistants can enable a hands-free experience with your devices? The answer lies in keyword spotting (KWS) technology, which recognizes the popular wake word phrases of OK Google, Alexa, Hey Siri, or Cortana. By identifying these phrases, the smart assistant wakes up and listens to your commands. Since KWS uses real-time speech recognition models, it must be on-device, always on, and running on a low-power system to be effective.

In this chapter, we will demonstrate the usage of KWS through Edge Impulse by building an application to voice control a light-emitting diode (LED) that emits a colored light (red, green, or blue) a certain ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781837637362

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

TinyML Cookbook - Second Edition

by Gian Marco Iodice

4

Using Edge Impulse and the Arduino Nano to Control LEDs with Voice Commands

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.