Search Engine in Startups: What to Optimize

Presented by Liang Wu, Machine Learning Data Scientist at Airbnb

Choosing the correct optimization metric is key to success of a search engine. Unlike in traditional web searches, where clicks are clearly the main objective to optimize, many emerging vertical search engines like E-Commerce search may require a different optimization metric, such as conversions, revenue, and quality. Selection of a good metric may depend on the query type (transactional vs. navigational vs. informational) and also on the goal of a business (profitability vs. growth). For example, a typical product search engine may focus on maximizing the number of transactions and total revenue, while navigational search may aim at minimizing the total number of clicks. In this talk, we will investigate factors needed to be considered when we are in search for a good metric, and we will also walk through an example of designing an optimization metric for a particular business, including how it is selected, mathematically defined, and optimized with a machine learning framework.