After developing the viral Not Hotdog app (that we looked at in Chapter 12), Jian-Yang was originally supposed to build a classifier to recognize all food in existence. In fact, the app was originally supposed to be called SeeFood—an app that can “see” food and know it right away (Figure 13-1). In other words, the “Shazam for Food.” However, the app was too successful for its own good and was acquired by Periscope. The original vision of his investor, Erlich Bachman, remains unfulfilled. In this chapter, our mission is to fulfill this dream.
Where would such a feature be useful? For health nuts, it could look at a dish and provide the nutritional information, including the number of calories. Or, it could scan a few ingredients, and recommend a recipe based on them. Or, it could even look at a product in the market, and check whether it contains any blacklisted ingredients such as specific allergens.
This is an interesting problem to solve for several reasons because it represents several challenges:
There are more than a hundred cuisines around the world, each with hundreds if not thousands of dishes.
It should be right most of the time.
It should run near instantly. ...