AI and Machine Learning for On-Device Development

Chapter 4. Computer Vision Apps with ML Kit on Android

Chapter 3 gave you an introduction to ML Kit and how it can be used for face detection in a mobile app. But ML Kit is far more than that—it gives you the ability to rapidly prototype common vision scenarios, host custom models, or implement other turnkey solution scenarios such as barcode detection also. In this chapter, we will explore some of the models that are available in ML Kit to provide computer vision scenarios, including image labeling and classification, and object detection in both still and moving images. We’ll do this on Android, using Kotlin as the programming language. Chapter 6 will cover the equivalent content using Swift for iOS development.

Image Labeling and Classification

The concept of image classification is a well-known one in machine learning circles, and the staple of computer vision. In its simplest sense, image classification happens when you show an image to a computer and it tells you what that image contains. For example, you show it a picture of a cat, like that in Figure 4-1, and it will label it as a cat.

Image labeling in ML Kit takes this a little further and gives you a list of things seen in the image with levels of probability, so instead of Figure 4-1 just showing a cat, it might say that it sees a cat, flowers, grass, daisies, and more.

Let’s explore how to create a very simple Android app that can label this image! We’ll use Android Studio and Kotlin. If you don’t have them already, ...