AI and Machine Learning for On-Device Development

AI and Machine Learning for On-Device Development

by Laurence Moroney
Released August 2021
Publisher(s): O'Reilly Media, Inc.
ISBN: 9781098101749

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Buy on Amazon Buy on ebooks.com
Start your free trial

Book description

AI is nothing without somewhere to run it. Now that mobile devices have become the primary computing device for most people, it's essential that mobile developers add AI to their toolbox. This insightful book is your guide to creating and running models on popular mobile platforms such as iOS and Android.

Laurence Moroney, lead AI advocate at Google, offers an introduction to machine learning techniques and tools, then walks you through writing Android and iOS apps powered by common ML models like computer vision and text recognition, using tools such as ML Kit, TensorFlow Lite, and Core ML. If you're a mobile developer, this book will help you take advantage of the ML revolution today.

  • Explore the options for implementing ML and AI on mobile devices
  • Create ML models for iOS and Android
  • Write ML Kit and TensorFlow Lite apps for iOS and Android, and Core ML/Create ML apps for iOS
  • Choose the best techniques and tools for your use case, such as cloud-based versus on-device inference and high-level versus low-level APIs
  • Learn privacy and ethics best practices for ML on devices

Table of contents

  1. Preface
    1. Who Should Read This Book?
    2. Why I Wrote This Book
    3. Navigating This Book
    4. Technology You Need to Understand
    5. Conventions Used in This Book
    6. Using Code Examples
    7. O’Reilly Online Learning
    8. How to Contact Us
    9. Acknowledgements
  2. 1. Introduction to AI and Machine Learning
    1. What Is Artificial Intelligence?
    2. What Is Machine Learning?
      1. Moving from Traditional Programming to Machine Learning
      2. How Can a Machine Learn?
      3. Comparing Machine Learning with Traditional Programming
    3. Building and Using Models on Mobile
    4. Summary
  3. 2. Introduction to Computer Vision
    1. Using Neurons for Vision
      1. Your First Classifier: Recognizing Clothing Items
      2. The Data: Fashion MNIST
      3. A Model Architecture to Parse Fashion MNIST
      4. Coding the Fashion MNIST Model
    2. Transfer Learning for Computer Vision
    3. Summary
  4. 3. Introduction to ML Kit
    1. Building a Face Detection App on Android
      1. Step 1: Create the App with Android Studio
      2. Step 2: Add and Configure ML Kit
      3. Step 3: Define the User Interface
      4. Step 4: Add the Images as Assets
      5. Step 5: Load the UI with a Default Picture
      6. Step 6: Call the Face Detector
      7. Step 7: Add the Bounding Rectangles
    2. Building a Face Detector App for iOS
      1. Step 1: Create the Project in Xcode
      2. Step 2: Using CocoaPods and Podfiles
      3. Step 3: Create the User Interface
      4. Step 4: Add the Application Logic
    3. Summary
  5. 4. Computer Vision Apps with ML Kit on Android
    1. Image Labeling and Classification
      1. Step 1: Create the App and Configure ML Kit
      2. Step 2: Create the User Interface
      3. Step 3: Add the Images as Assets
      4. Step 4: Load an Image to the ImageView
      5. Step 5: Write the Button Handler Code
      6. Next Steps
    2. Object Detection
      1. Step 1: Create the App and Import ML Kit
      2. Step 2: Create the Activity Layout XML
      3. Step 3: Load an Image into the ImageView
      4. Step 4: Set Up the Object Detector Options
      5. Step 5: Handling the Button Interaction
      6. Step 6: Draw the Bounding Boxes
      7. Step 7: Label the Objects
    3. Detecting and Tracking Objects in Video
      1. Exploring the Layout
      2. The GraphicOverlay Class
      3. Capturing the Camera
      4. The ObjectAnalyzer Class
      5. The ObjectGraphic Class
      6. Putting It All Together
    4. Summary
  6. 5. Text Processing Apps with ML Kit on Android
    1. Entity Extraction
      1. Start Creating the App
      2. Create the Layout for the Activity
      3. Write the Entity Extraction Code
      4. Putting It All Together
    2. Handwriting and Other Recognition
      1. Start the App
      2. Creating a Drawing Surface
      3. Parsing the Ink with ML Kit
    3. Smart Reply to Conversations
      1. Start the App
      2. Mock a Conversation
      3. Generating a Smart Reply
    4. Summary
  7. 6. Computer Vision Apps with ML Kit on iOS
    1. Image Labeling and Classification
      1. Step 1: Create the App in Xcode
      2. Step 2: Create the Podfile
      3. Step 3: Set Up the Storyboard
      4. Step 4: Edit the View Controller Code to Use ML Kit
    2. Object Detection in iOS with ML Kit
      1. Step 1: Get Started
      2. Step 2: Create Your UI on the Storyboard
      3. Step 3: Create a Subview for Annotation
      4. Step 4: Perform the Object Detection
      5. Step 5: Handle the Callback
      6. Combining Object Detection with Image Classification
      7. Object Detection and Tracking in Video
    3. Summary
  8. 7. Text Processing Apps with ML Kit on iOS
    1. Entity Extraction
      1. Step 1: Create the App and Add the ML Kit Pods
      2. Step 2: Create the Storyboard with Actions and Outlets
      3. Step 3: Allow Your View Controller to be Used for Text Entry
      4. Step 4: Initialize the Model
      5. Step 5: Extract Entities from Text
    2. Handwriting Recognition
      1. Step 1: Create the App and Add the ML Kit Pods
      2. Step 2: Create the Storyboard, Actions, and Outlets
      3. Step 3: Strokes, Points, and Ink
      4. Step 4: Capture User Input
      5. Step 5: Initialize the Model
      6. Step 6: Do the Ink Recognition
    3. Smart Reply to Conversations
      1. Step 1: Create an App and Integrate ML Kit
      2. Step 2: Create Storyboard, Outlets, and Actions
      3. Step 3: Create a Simulated Conversation
      4. Step 4: Get Smart Reply
    4. Summary
  9. 8. Going Deeper: Understanding TensorFlow Lite
    1. What Is TensorFlow Lite?
    2. Getting Started with TensorFlow Lite
      1. Save the Model
      2. Convert the Model
      3. Testing the Model with a Standalone Interpreter
    3. Create an Android App to Host TFLite
      1. Import the TFLite File
      2. Write Kotlin Code to Interface with the Model
      3. Going Beyond the Basics
    4. Create an iOS App to Host TFLite
      1. Step 1: Create a Basic iOS App
      2. Step 2: Add TensorFlow Lite to Your Project
      3. Step 3: Create the User Interface
      4. Step 4: Add and Initialize the Model Inference Class
      5. Step 5: Perform the Inference
      6. Step 6: Add the Model to Your App
      7. Step 7: Add the UI Logic
      8. Moving Beyond “Hello World”: Processing Images
    5. Exploring Model Optimization
      1. Quantization
      2. Using Representative Data
    6. Summary
  10. 9. Creating Custom Models
    1. Creating a Model with TensorFlow Lite Model Maker
    2. Creating a Model with Cloud AutoML
      1. Using AutoML Vision Edge
    3. Creating a Model with TensorFlow and Transfer Learning
    4. Creating Language Models
      1. Create a Language Model with Model Maker
    5. Summary
  11. 10. Using Custom Models in Android
    1. Bridging Models to Android
    2. Building an Image Classification App from a Model Maker Output
    3. Using a Model Maker Output with ML Kit
    4. Using Language Models
    5. Creating an Android App for Language Classification
    6. Summary
  12. 11. Using Custom Models in iOS
    1. Bridging Models to iOS
    2. A Custom Model Image Classifier
      1. Step 1: Create the App and Add the TensorFlow Lite Pod
      2. Step 2: Create the UI and Image Assets
      3. Step 3: Load and Navigate Through the Image Assets
      4. Step 4: Load the Model
      5. Step 5: Convert an Image to an Input Tensor
      6. Step 6: Get Inference for the Tensor
    3. Use a Custom Model in ML Kit
    4. Building an App for Natural Language Processing in Swift
      1. Step 1: Load the Vocab
      2. Step 2: Convert the Sentence to a Sequence
      3. Step 3: Extend Array to Handle Unsafe Data
      4. Step 4: Copy the Array to a Data Buffer
      5. Step 5: Run Inference on the Data and Process the Results
    5. Summary
  13. 12. Productizing Your App Using Firebase
    1. Why Use Firebase Custom Model Hosting?
    2. Create Multiple Model Versions
    3. Using Firebase Model Hosting
      1. Step 1: Create a Firebase Project
      2. Step 2: Use Custom Model Hosting
      3. Step 3: Create a Basic Android App
      4. Step 4: Add Firebase to the App
      5. Step 5: Get the Model from Firebase Model Hosting
      6. Step 6: Use Remote Configuration
      7. Step 7: Read Remote Configuration in Your App
      8. Next Steps
    4. Summary
  14. 13. Create ML and Core ML for Simple iOS Apps
    1. A Core ML Image Classifier Built Using Create ML
      1. Making a Core ML App That Uses a Create ML Model
      2. Add the MLModel File
      3. Run the Inference
    2. Using Create ML to Build a Text Classifier
    3. Use the Model in an App
    4. Summary
  15. 14. Accessing Cloud-Based Models from Mobile Apps
    1. Installing TensorFlow Serving
      1. Installing Using Docker
      2. Installing Directly on Linux
    2. Building and Serving a Model
    3. Accessing a Server Model from Android
    4. Accessing a Server Model from iOS
    5. Summary
  16. 15. Ethics, Fairness, and Privacy for Mobile Apps
    1. Ethics, Fairness, and Privacy with Responsible AI
      1. Responsibly Defining Your Problem
      2. Avoiding Bias in Your Data
      3. Building and Training Your Model
      4. Evaluating Your Model
    2. Google’s AI Principles
    3. Summary
  17. Index

Product information

  • Title: AI and Machine Learning for On-Device Development
  • Author(s): Laurence Moroney
  • Release date: August 2021
  • Publisher(s): O'Reilly Media, Inc.
  • ISBN: 9781098101749