book

AI and Machine Learning for On-Device Development

Name: AI and Machine Learning for On-Device Development
Author: Laurence Moroney
ISBN: 9781098101749

by Laurence Moroney

August 2021

Intermediate to advanced

326 pages

8h 31m

English

O'Reilly Media, Inc.

Audiobook available

Read now

Unlock full access

Preface
Who Should Read This Book?Why I Wrote This BookNavigating This BookTechnology You Need to UnderstandConventions Used in This BookUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgements
1. Introduction to AI and Machine Learning
What Is Artificial Intelligence?What Is Machine Learning?Moving from Traditional Programming to Machine LearningHow Can a Machine Learn?Comparing Machine Learning with Traditional ProgrammingBuilding and Using Models on MobileSummary
2. Introduction to Computer Vision
Using Neurons for VisionYour First Classifier: Recognizing Clothing ItemsThe Data: Fashion MNISTA Model Architecture to Parse Fashion MNISTCoding the Fashion MNIST ModelTransfer Learning for Computer VisionSummary
3. Introduction to ML Kit
Building a Face Detection App on AndroidStep 1: Create the App with Android StudioStep 2: Add and Configure ML KitStep 3: Define the User InterfaceStep 4: Add the Images as AssetsStep 5: Load the UI with a Default PictureStep 6: Call the Face DetectorStep 7: Add the Bounding RectanglesBuilding a Face Detector App for iOSStep 1: Create the Project in XcodeStep 2: Using CocoaPods and PodfilesStep 3: Create the User InterfaceStep 4: Add the Application LogicSummary
4. Computer Vision Apps with ML Kit on Android
Image Labeling and ClassificationStep 1: Create the App and Configure ML KitStep 2: Create the User InterfaceStep 3: Add the Images as AssetsStep 4: Load an Image to the ImageViewStep 5: Write the Button Handler CodeNext StepsObject DetectionStep 1: Create the App and Import ML KitStep 2: Create the Activity Layout XMLStep 3: Load an Image into the ImageViewStep 4: Set Up the Object Detector OptionsStep 5: Handling the Button InteractionStep 6: Draw the Bounding BoxesStep 7: Label the ObjectsDetecting and Tracking Objects in VideoExploring the LayoutThe GraphicOverlay ClassCapturing the CameraThe ObjectAnalyzer ClassThe ObjectGraphic ClassPutting It All TogetherSummary
5. Text Processing Apps with ML Kit on Android
Entity ExtractionStart Creating the AppCreate the Layout for the ActivityWrite the Entity Extraction CodePutting It All TogetherHandwriting and Other RecognitionStart the AppCreating a Drawing SurfaceParsing the Ink with ML KitSmart Reply to ConversationsStart the AppMock a ConversationGenerating a Smart ReplySummary
6. Computer Vision Apps with ML Kit on iOS
Image Labeling and ClassificationStep 1: Create the App in XcodeStep 2: Create the PodfileStep 3: Set Up the StoryboardStep 4: Edit the View Controller Code to Use ML KitObject Detection in iOS with ML KitStep 1: Get StartedStep 2: Create Your UI on the StoryboardStep 3: Create a Subview for AnnotationStep 4: Perform the Object DetectionStep 5: Handle the CallbackCombining Object Detection with Image ClassificationObject Detection and Tracking in VideoSummary
7. Text Processing Apps with ML Kit on iOS
Entity ExtractionStep 1: Create the App and Add the ML Kit PodsStep 2: Create the Storyboard with Actions and OutletsStep 3: Allow Your View Controller to be Used for Text EntryStep 4: Initialize the ModelStep 5: Extract Entities from TextHandwriting RecognitionStep 1: Create the App and Add the ML Kit PodsStep 2: Create the Storyboard, Actions, and OutletsStep 3: Strokes, Points, and InkStep 4: Capture User InputStep 5: Initialize the ModelStep 6: Do the Ink RecognitionSmart Reply to ConversationsStep 1: Create an App and Integrate ML KitStep 2: Create Storyboard, Outlets, and ActionsStep 3: Create a Simulated ConversationStep 4: Get Smart ReplySummary
8. Going Deeper: Understanding TensorFlow Lite
What Is TensorFlow Lite?Getting Started with TensorFlow LiteSave the ModelConvert the ModelTesting the Model with a Standalone InterpreterCreate an Android App to Host TFLiteImport the TFLite FileWrite Kotlin Code to Interface with the ModelGoing Beyond the BasicsCreate an iOS App to Host TFLiteStep 1: Create a Basic iOS AppStep 2: Add TensorFlow Lite to Your ProjectStep 3: Create the User InterfaceStep 4: Add and Initialize the Model Inference ClassStep 5: Perform the InferenceStep 6: Add the Model to Your AppStep 7: Add the UI LogicMoving Beyond “Hello World”: Processing ImagesExploring Model OptimizationQuantizationUsing Representative DataSummary
9. Creating Custom Models
Creating a Model with TensorFlow Lite Model MakerCreating a Model with Cloud AutoMLUsing AutoML Vision EdgeCreating a Model with TensorFlow and Transfer LearningCreating Language ModelsCreate a Language Model with Model MakerSummary

10. Using Custom Models in Android
Bridging Models to AndroidBuilding an Image Classification App from a Model Maker OutputUsing a Model Maker Output with ML KitUsing Language ModelsCreating an Android App for Language ClassificationSummary
11. Using Custom Models in iOS
Bridging Models to iOSA Custom Model Image ClassifierStep 1: Create the App and Add the TensorFlow Lite PodStep 2: Create the UI and Image AssetsStep 3: Load and Navigate Through the Image AssetsStep 4: Load the ModelStep 5: Convert an Image to an Input TensorStep 6: Get Inference for the TensorUse a Custom Model in ML KitBuilding an App for Natural Language Processing in SwiftStep 1: Load the VocabStep 2: Convert the Sentence to a SequenceStep 3: Extend Array to Handle Unsafe DataStep 4: Copy the Array to a Data BufferStep 5: Run Inference on the Data and Process the ResultsSummary
12. Productizing Your App Using Firebase
Why Use Firebase Custom Model Hosting?Create Multiple Model VersionsUsing Firebase Model HostingStep 1: Create a Firebase ProjectStep 2: Use Custom Model HostingStep 3: Create a Basic Android AppStep 4: Add Firebase to the AppStep 5: Get the Model from Firebase Model HostingStep 6: Use Remote ConfigurationStep 7: Read Remote Configuration in Your AppNext StepsSummary
13. Create ML and Core ML for Simple iOS Apps
A Core ML Image Classifier Built Using Create MLMaking a Core ML App That Uses a Create ML ModelAdd the MLModel FileRun the InferenceUsing Create ML to Build a Text ClassifierUse the Model in an AppSummary
14. Accessing Cloud-Based Models from Mobile Apps
Installing TensorFlow ServingInstalling Using DockerInstalling Directly on LinuxBuilding and Serving a ModelAccessing a Server Model from AndroidAccessing a Server Model from iOSSummary
15. Ethics, Fairness, and Privacy for Mobile Apps
Ethics, Fairness, and Privacy with Responsible AIResponsibly Defining Your ProblemAvoiding Bias in Your DataBuilding and Training Your ModelEvaluating Your ModelGoogle’s AI PrinciplesSummary
Index

Content preview from AI and Machine Learning for On-Device Development

Chapter 9. Creating Custom Models

In earlier chapters you saw how to use turnkey models for image labeling, object detection, entity extraction, and more. What you didn’t see was how you might be able to use models that you had created yourself, and indeed how you might be able to create them by yourself. In this chapter, we’ll look at three scenarios for creating models, and then in Chapters 10 and 11, we’ll look at incorporating those models into Android or iOS apps.

Creating models from scratch can be difficult and very time consuming. It’s also the realm of pure TensorFlow development and is covered in lots of other books such as my book AI and Machine Learning for Coders (O’Reilly). If you aren’t creating from scratch, and in particular if you’re focused on mobile apps, there are some tools to assist you, and we’ll look at three of them in this chapter:

TensorFlow Lite Model Maker is the preferred choice if you are building an app that fits a scenario that Model Maker supports. It’s not a generic tool for building any type of model, but is designed to support common use cases such as image classification, object detection, and more. It involves little to no neural-network-specific coding, and as such is a great place to start if you don’t want to learn that stuff yet!
Creating models using Cloud AutoML, and in particular the tools in Cloud AutoML designed to minimize the amount of code you have to write and maintain. Similar to TensorFlow Model Maker, the scenarios here ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Applied Machine Learning and AI for Engineers

Publisher Resources

ISBN: 9781098101732Errata Page

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

AI and Machine Learning for On-Device Development

by Laurence Moroney

Chapter 9. Creating Custom Models

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.