Designing Bots

Chapter 19. Analytics and Continuous Improvement

If you torture the data long enough, it will confess anything.

—GORDON TULLOCK

IF YOU’VE EVER MANAGED a website or a mobile app, you probably got addicted to checking your analytics—waking up every morning and seeing that you got more users and more engagement is a delightful experience. It can also be an eye-opening experience to see how users engage with your service, which buttons they click, and which parts they avoid. Great products and great design stems from understanding your users, and data from analytics services about how people really use your bot should be treated like gold.

How Do Bot Analytics Work?

Bots connect to a messaging platform (like Kik, Slack, Facebook Messenger, and others) through an API that lets you send and receive messages, receive events such as being invited to a direct or team conversation, receive presence change events, and so on. When users use your bot, you will be getting a great deal of data about the interaction, and the key challenge is to turn this raw data into insights you can use to understand your users and improve your bot. There are a few common ways you can turn that data into actionable insights:

Raw logs

Many bot builders spend lots of time looking at the raw data, seeing what users are saying to their bot and what the bot is saying back. This approach is time-consuming, but provides an unfiltered view of the conversations your bot is having. This might be easiest way to start, although ...