Building Intelligent Apps with Cognitive APIs

Chapter 1. What You’ll Learn in This Report

Infusing AI into an application can be difficult and time-consuming. Until recently, you needed both a deep understanding of machine learning and months of development to acquire data, train models, and deploy them at scale. Even then, success was not guaranteed. The path was filled with blockers, gotchas, and pitfalls causing teams to fail to realize value from their AI investments.

Microsoft Azure Cognitive Services remove these challenges. They allow you to infuse your apps, websites, and bots with intelligence using just a few simple lines of code and without a large data science team. With these services you can quickly create applications that hear, speak, understand, and even begin to reason. These capabilities will unlock new experiences and applications for your business. Imagine an app that allows users to take a photograph of a menu and then automatically translate it into another language and retrieve reviews, pictures, and relevant recipes. Or a chatbot that can talk with your users in a customized voice that matches your brand. Imagine infusing this chatbot with even more intelligence, such as being able to recognize pictures of your products, identify any defects, and streamline the return process. These are just a few of the AI-powered features companies are building using the Cognitive Services.

Even companies with deep expertise in AI turn to these services rather than creating their own. When Uber looked at ways to verify a driver’s identity—even if they’d recently cut their hair or changed their glasses—they chose to use the Microsoft Azure Cognitive Services Face API rather than build their own solution, despite their deep knowledge and use of machine learning across the company.

There are several intelligent APIs to choose from: Apple, Amazon, Google, and Microsoft all have offerings. In this report, however, we’ll show you how to work with the Microsoft Azure Cognitive Services to quickly add intelligence to your applications. We will look at a wide range of scenarios, from kiosks that can greet visitors and recommend products to ways to gather real-time insights about crowd behavior or monitor driver safety on the road. Along the way, we will provide C# code snippets showing you how to call a few of the APIs.¹ We assume you have a basic understanding of the Azure platform. If you are completely new to Azure, you can find training materials on the website. We are constantly updating and improving our services, so be sure to check the documentation to learn more about the latest features and functionality for each service.

Here’s a breakdown of what we cover in this report:

Chapter 2, The Microsoft AI Platform

Cognitive Services is just one part of Microsoft’s AI platform, which also includes frameworks, tools, infrastructure, and services for developing AI applications and bringing them to familiar systems like SQL Server and Power BI.

Chapter 3, Understanding Azure Cognitive Services

Today, there are more than 20 Cognitive Services within Microsoft Azure, each with multiple features and options. These services allow you to quickly and simply bring the latest breakthroughs from research into your apps. They’re divided into five categories: Vision, Speech, Language, Decision, and Web Search. We’ll show you what you can achieve with each one and how to build them into your apps.

Chapter 4, Vision

Want to analyze an image or a video? The various Vision services provide a powerful tool for extracting data from images. Recognizing and describing faces, objects, and text are just some of the many features they offer. You get the power of a fully trained deep learning image recognition model and can even customize it to recognize your specific objects.

Chapter 5, Speech

The Speech services cover speech-to-text, text-to-speech, and real-time translation across several languages. You can customize speech models for specific acoustic environments, like a factory floor, or train the service to recognize and pronounce your business’s unique jargon.

Chapter 6, Language

The Language services enable you to analyze, understand, and translate text. You can turn your FAQ into an interactive chatbot with the QnA Maker, extract sentiment and key phrases using Text Analytics, or understand the meaning of a user’s comment using the Language Understanding service.

Chapter 7, Decision

With the Decision services, you can build apps that surface recommendations for informed and efficient decision making. You can use the Personalizer service to provide relevant, engaging, and unique experiences to every user, improving app satisfaction, usability, and engagement, and you can quickly identify problems in time series data using the Anomaly Detector service.

Chapter 8, Web Search

Whether you want to search for an image or use an image to do a search, you can use the Bing Search APIs to bring the power of Microsoft’s Bing search engine to your app.

Chapter 9, Paving the Road Ahead

In this chapter we will provide some examples of customers leveraging the Cognitive Services to add intelligence to their offerings and transform the way they do business.

Chapter 10, Where to Go Next

Finally, we provide some pointers to resources you can turn to if you’d like to broaden your understanding.

¹ The authors would like to thank Winona Azure for creating most of the code snippets for this report.