Practical AI on iOS
Building with Core ML and Vision
Expert Jon Manning offers a hands-on overview of the new machine learning features built into iOS. Over the course of this 90-minute course, you’ll learn how to apply the Vision and Core ML frameworks to solve practical problems in object detection, face recognition, and more.
Because these frameworks run on-device rather than relying on a cloud processing service, they work quickly—with no network access required. Additionally, user input never has to leave the phone, making these frameworks ideal for situations where the user wants to maintain their privacy. All of these factors and more make the AI features of iOS 11 incredibly appealing. Join in to learn how to build tools that take full advantage of them.
What you'll learn-and how you can apply it
By the end of this live online course, you’ll understand:
- What the Core ML and Vision frameworks are
- How to expand upon your new skills
And you’ll be able to:
- Detect faces and facial features
- Load trained models for use in machine learning
- Detect and classify objects in photos
This training course is for you because...
- You’re a programmer who’s excited about the possibilities of machine learning, and you want to explore the capabilities of iOS 11’s machine learning features.
- You’re interested in machine learning in general and want to see it in action.
- You want to learn how to apply your existing machine learning knowledge to iOS.
- A working knowledge of the Swift programming language
Materials or downloads needed in advance:
- A machine with Xcode 9 installed
- The Inception v3 model downloaded
- A device running iOS 11
Apple’s Machine Learning overview (website) https://developer.apple.com/machine-learning/
About your instructor
Jon Manning is the cofounder of independent game development studio Secret Lab. He's currently working on top-down puzzler Button Squid and the critically acclaimed adventure game Night in the Woods, which includes his interactive dialogue system Yarn Spinner. Jon has written a whole bunch of books for O'Reilly Media about iOS development and game development. He holds a doctorate about jerks on the internet. Jon can be found as @desplesda on Twitter.
The timeframes are only estimates and may vary according to how the class is progressing
Overview of Core ML and Vision (20 minutes)
- Lecture and hands-on exercises: The feature sets available in the new Core ML and Vision frameworks that ship with iOS 11
Vision with face detection (25 minutes)
- Lecture and hands-on exercises: Creating an app that detects faces in provided photos
Break (10 minutes)
Working with AVKit (25 minutes)
- Lecture and hands-on exercises: Getting access to real-time camera input, using a lower-level framework that lets us work with the audio-visual systems built into the device directly
Detecting facial features (20 minutes)
- Lecture and hands-on exercises: Using Vision to describe where certain parts of faces are
Classifying objects in photos (20 minutes)
- Lecture and hands-on exercises: Using Core ML and a pretrained neural network to detect and classify the objects present in the camera’s view