Mobile Deep Learning with TensorFlow Lite, ML Kit and Flutter

Snapchat

The filters used by Snapchat are design overlays added on top of pictures and videos, with the capability of following face movements. These filters are made possible by computer vision. The first step of the algorithm being used by the application is to detect the faces present in an image. It outputs boxes bounding the detected faces. It then marks the facial landmarks—such as eyes, nose, and lips—for each of the detected faces. The output here is generally a two-dimensional point containing x-coordinates and y-coordinates. After the faces and facial features have been detected properly, it uses image processing to correctly place or apply filters on the whole face. The algorithm goes one step further to analyze the key facial ...