book

AI and Machine Learning for On-Device Development

Name: AI and Machine Learning for On-Device Development
Author: Laurence Moroney
ISBN: 9781098101749

by Laurence Moroney

August 2021

Intermediate to advanced

326 pages

8h 31m

English

O'Reilly Media, Inc.

Audiobook available

Read now

Unlock full access

Preface
Who Should Read This Book?Why I Wrote This BookNavigating This BookTechnology You Need to UnderstandConventions Used in This BookUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgements
1. Introduction to AI and Machine Learning
What Is Artificial Intelligence?What Is Machine Learning?Moving from Traditional Programming to Machine LearningHow Can a Machine Learn?Comparing Machine Learning with Traditional ProgrammingBuilding and Using Models on MobileSummary
2. Introduction to Computer Vision
Using Neurons for VisionYour First Classifier: Recognizing Clothing ItemsThe Data: Fashion MNISTA Model Architecture to Parse Fashion MNISTCoding the Fashion MNIST ModelTransfer Learning for Computer VisionSummary
3. Introduction to ML Kit
Building a Face Detection App on AndroidStep 1: Create the App with Android StudioStep 2: Add and Configure ML KitStep 3: Define the User InterfaceStep 4: Add the Images as AssetsStep 5: Load the UI with a Default PictureStep 6: Call the Face DetectorStep 7: Add the Bounding RectanglesBuilding a Face Detector App for iOSStep 1: Create the Project in XcodeStep 2: Using CocoaPods and PodfilesStep 3: Create the User InterfaceStep 4: Add the Application LogicSummary
4. Computer Vision Apps with ML Kit on Android
Image Labeling and ClassificationStep 1: Create the App and Configure ML KitStep 2: Create the User InterfaceStep 3: Add the Images as AssetsStep 4: Load an Image to the ImageViewStep 5: Write the Button Handler CodeNext StepsObject DetectionStep 1: Create the App and Import ML KitStep 2: Create the Activity Layout XMLStep 3: Load an Image into the ImageViewStep 4: Set Up the Object Detector OptionsStep 5: Handling the Button InteractionStep 6: Draw the Bounding BoxesStep 7: Label the ObjectsDetecting and Tracking Objects in VideoExploring the LayoutThe GraphicOverlay ClassCapturing the CameraThe ObjectAnalyzer ClassThe ObjectGraphic ClassPutting It All TogetherSummary
5. Text Processing Apps with ML Kit on Android
Entity ExtractionStart Creating the AppCreate the Layout for the ActivityWrite the Entity Extraction CodePutting It All TogetherHandwriting and Other RecognitionStart the AppCreating a Drawing SurfaceParsing the Ink with ML KitSmart Reply to ConversationsStart the AppMock a ConversationGenerating a Smart ReplySummary
6. Computer Vision Apps with ML Kit on iOS
Image Labeling and ClassificationStep 1: Create the App in XcodeStep 2: Create the PodfileStep 3: Set Up the StoryboardStep 4: Edit the View Controller Code to Use ML KitObject Detection in iOS with ML KitStep 1: Get StartedStep 2: Create Your UI on the StoryboardStep 3: Create a Subview for AnnotationStep 4: Perform the Object DetectionStep 5: Handle the CallbackCombining Object Detection with Image ClassificationObject Detection and Tracking in VideoSummary
7. Text Processing Apps with ML Kit on iOS
Entity ExtractionStep 1: Create the App and Add the ML Kit PodsStep 2: Create the Storyboard with Actions and OutletsStep 3: Allow Your View Controller to be Used for Text EntryStep 4: Initialize the ModelStep 5: Extract Entities from TextHandwriting RecognitionStep 1: Create the App and Add the ML Kit PodsStep 2: Create the Storyboard, Actions, and OutletsStep 3: Strokes, Points, and InkStep 4: Capture User InputStep 5: Initialize the ModelStep 6: Do the Ink RecognitionSmart Reply to ConversationsStep 1: Create an App and Integrate ML KitStep 2: Create Storyboard, Outlets, and ActionsStep 3: Create a Simulated ConversationStep 4: Get Smart ReplySummary
8. Going Deeper: Understanding TensorFlow Lite
What Is TensorFlow Lite?Getting Started with TensorFlow LiteSave the ModelConvert the ModelTesting the Model with a Standalone InterpreterCreate an Android App to Host TFLiteImport the TFLite FileWrite Kotlin Code to Interface with the ModelGoing Beyond the BasicsCreate an iOS App to Host TFLiteStep 1: Create a Basic iOS AppStep 2: Add TensorFlow Lite to Your ProjectStep 3: Create the User InterfaceStep 4: Add and Initialize the Model Inference ClassStep 5: Perform the InferenceStep 6: Add the Model to Your AppStep 7: Add the UI LogicMoving Beyond “Hello World”: Processing ImagesExploring Model OptimizationQuantizationUsing Representative DataSummary
9. Creating Custom Models
Creating a Model with TensorFlow Lite Model MakerCreating a Model with Cloud AutoMLUsing AutoML Vision EdgeCreating a Model with TensorFlow and Transfer LearningCreating Language ModelsCreate a Language Model with Model MakerSummary

10. Using Custom Models in Android
Bridging Models to AndroidBuilding an Image Classification App from a Model Maker OutputUsing a Model Maker Output with ML KitUsing Language ModelsCreating an Android App for Language ClassificationSummary
11. Using Custom Models in iOS
Bridging Models to iOSA Custom Model Image ClassifierStep 1: Create the App and Add the TensorFlow Lite PodStep 2: Create the UI and Image AssetsStep 3: Load and Navigate Through the Image AssetsStep 4: Load the ModelStep 5: Convert an Image to an Input TensorStep 6: Get Inference for the TensorUse a Custom Model in ML KitBuilding an App for Natural Language Processing in SwiftStep 1: Load the VocabStep 2: Convert the Sentence to a SequenceStep 3: Extend Array to Handle Unsafe DataStep 4: Copy the Array to a Data BufferStep 5: Run Inference on the Data and Process the ResultsSummary
12. Productizing Your App Using Firebase
Why Use Firebase Custom Model Hosting?Create Multiple Model VersionsUsing Firebase Model HostingStep 1: Create a Firebase ProjectStep 2: Use Custom Model HostingStep 3: Create a Basic Android AppStep 4: Add Firebase to the AppStep 5: Get the Model from Firebase Model HostingStep 6: Use Remote ConfigurationStep 7: Read Remote Configuration in Your AppNext StepsSummary
13. Create ML and Core ML for Simple iOS Apps
A Core ML Image Classifier Built Using Create MLMaking a Core ML App That Uses a Create ML ModelAdd the MLModel FileRun the InferenceUsing Create ML to Build a Text ClassifierUse the Model in an AppSummary
14. Accessing Cloud-Based Models from Mobile Apps
Installing TensorFlow ServingInstalling Using DockerInstalling Directly on LinuxBuilding and Serving a ModelAccessing a Server Model from AndroidAccessing a Server Model from iOSSummary
15. Ethics, Fairness, and Privacy for Mobile Apps
Ethics, Fairness, and Privacy with Responsible AIResponsibly Defining Your ProblemAvoiding Bias in Your DataBuilding and Training Your ModelEvaluating Your ModelGoogle’s AI PrinciplesSummary
Index

Content preview from AI and Machine Learning for On-Device Development

Chapter 3. Introduction to ML Kit

In the first two chapters of this book, you took a look at the foundations of machine learning and deep learning to build some basic models. For the rest of the book, you’re going to switch gears and explore how to implement models on mobile devices. A powerful toolkit that allows you to implement preexisting models (aka turnkey scenarios) as well as custom models is Google’s ML Kit. In this chapter you’ll explore how to use ML Kit to run Android or iOS models entirely on your device. The model will stay on-device, giving your users speed and privacy advantages.

Note

I would strongly recommend going through this chapter in detail, particularly if you aren’t super familiar with how to implement add-on libraries for both Android and iOS. I’ll go through it in a lot of detail here, and subsequent chapters will refer back to this one.

There are three main scenarios where ML Kit can be used:

Turnkey solutions where a model already exists in ML Kit to implement what you need
Rapid prototyping using a generic model for a specific task; for example, if you want to build a vision app, don’t have a model for your needs yet, but want to determine if it’s possible with your device
Building apps with custom models like those we explored in Chapter 2

In this chapter I’ll explore some turnkey solutions, so you can understand how to get up and running quickly with ML models in your apps. In the subsequent chapters, we’ll explore how to use ML Kit initially ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Applied Machine Learning and AI for Engineers

Publisher Resources

ISBN: 9781098101732Errata Page

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

AI and Machine Learning for On-Device Development

by Laurence Moroney

Chapter 3. Introduction to ML Kit

Note

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.