Chapter 4. Data to Improve Product Design
Data! Data! Data! I can’t make bricks without clay!
Sir Arthur Conan Doyle
When we think of using data to improve designs, we tend to think of A/B tests. A/B testing is no longer just testing the color of a button or the name of a category; it is an integral part of an iterative process. Moreover, A/B testing isn’t the only way to use data to make improvements. Some companies are combining A/B tests with other methods, whether it is usability testing or even customer service conversations, to fine-tune the design.
A/B Testing and Usability Testing
Airbnb runs constant experiments that go beyond just a change of a label or a button color. The booking flow is complex. Add in other variables, including users who are not authenticated, people switching from device to device, and a time lag for confirmation, and suddenly gathering data about design decisions takes on a new complexity. Despite the complications, Airbnb regularly uses an in-house A/B testing platform to make improvements such as tweaking the mobile checkout flow or the price filter on the search page (see Figure 4-1). Usability testing is used before, after, or in tandem with A/B testing, depending on the goal.
Figure 4-1. Airbnb’s pricing filter A/B test (Source: Experiments at Airbnb)
Etsy is another company that relies on A/B testing to make incremental improvements (see ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access