Introduction

Product research doesn’t have to be difficult. It doesn’t have to take a long time and cost a lot of money. It doesn’t have to be done by scientists or expert researchers. It can be quick, cheap, and simple, and the whole team can do it. You just have to remember a few rules and develop a research mindset.

We started this journey with a question: why, in an industry that has been building digital products for decades, do teams still make products that fail? Article after article came up with credible answers: myopic vision, lack of market fit, no differentiation, poor focus, inability to “cross the chasm,” too many negative reviews, and more. But at the root of all of these is one problem: failing to understand the needs of the customer.

When teams want to understand their customers, they turn to market and user research. Market research is collecting and analyzing information about a market that is categorized by the products or services sold within it. It encompasses the characteristics, spending habits, locations, and needs of a business’s target market and the industry as a whole. User research identifies the users’ goals, needs, and motivations using human-centric methods.

While both market research and user research create great insights, what many teams fail to do is build on these insights in a timely manner. By treating research as a special, untouchable project that only a handful of people conduct on an infrequent basis, they miss out on leveraging their findings in the actual making of the product. The outcome is frustration, poor understanding of the market, and a product that isn’t designed for its user.

Sometimes valuable qualitative data can get overlooked because there aren’t concrete “hard numbers” in qualitative research. Numbers matter, although it’s not all numbers.1 It is unfortunate that many executives don’t know how to deal with qualitative research. Instead, they rely on what makes the most sense to them: the numbers. It’s important to realize that in product research, numbers do matter, just as much as the stories, anecdotes, and observations that lead to insights.

Product research is an approach that draws from user research, market research, and product analytics to help any product team arrive at insights in a timely, continuous manner (see Figure P-1).

Product research
Figure P-1. Product research

If you don’t have good data, your conclusions and insights can lead you astray. We’ll show you how to look at all three types of input to connect the dots for smart insights that can lead you to build great products.

Some consider product research to be a waste of time and resources, and if you’re one of these people, we’re so glad you’re here. This is often because the research is conducted poorly with inconsistent results that lack real insights. Product research draws on the strengths of both market and user research and focuses on understanding how your product works for the users it serves. It uses product analytics to inform research questions and relies on behavioral evidence to understand the user. Product research acknowledges the existence of a market and always considers market dynamics when interpreting results and suggesting actions.

Product research isn’t just about conducting surveys, speaking with users, and running analytics. It’s a change in mindset—a new way of thinking that takes our own preconceptions into account. We all have strong predilections, egos, and agendas that get in the way of forming valid and solid insights directly from our users. The product research approach tackles these head on.

Excuses for Not Doing Research

Let’s examine some of the reasons companies give for not doing research. They’re common excuses, and you might have used some of them yourself. We certainly have!

It takes too long.
There are many methods of product research. Some of them, such as multicountry ethnographic studies, will take a long time. But there are many other methods that can be carried out in weeks, days, even hours. You don’t have to spend months to find valuable insights. This book will show you how to begin discovering what you need to know quickly.
We don’t have the budget.
The majority of your research needs can be addressed with budget-friendly methods, iteratively, without compromising quality. In many cases, not starting with a good understanding of customer needs leads to expensive delays, low financial performance, and costly ground-up redesigns. Ask yourself this: do you have the budget to redesign your product? If the answer is no, avoiding product research might be the most expensive mistake you ever make.
We decided what the user needs.
Sometimes teams take days or weeks to discuss what they want to build and decide that is what the user needs. Unfortunately, they do this without including any direct input from users about their needs, current behavior, and motivations. Having a big meeting to discuss your ideas as a team is great, but that does not stand in for research because no users were involved.
We’re not researchers.
No one is born a researcher. Once upon a time, we weren’t researchers either. Research, like any other discipline, is a learnable skill. With the right mindset and some foundational methods, anyone can work with users, make sense of data, and arrive at insights. We hope the set of rules in this book will help you learn to conduct valuable product research.
The product is completely new.
Your team is excited about their new, from-scratch project. How can they do user research when nothing like this exists? There are always ways to gather feedback from your potential users. In fact, you’re taking a big risk if your new product first meets its users at the time of production. Getting early feedback and making changes will create a much better product.
It’s just a small change.
If you’re only making a small change to your product, do you need to conduct research? Many small changes accumulate over time to create big changes, and while it’s great that your organization can ship small changes, you’re still serving a new experience to your users. Small changes can and should be validated through research.
We need the features first.
Agile development, Lean methods, and DevOps allow us to create working software more easily than ever before. In this era of speed and time-to-market, it feels very natural to deliver first and think later. However, that system you’ve shipped so quickly may affect your users just as quickly as you release your features. The product research approach acknowledges the modern IT delivery reality, and the methods in this book can be used with many development approaches, including Agile software delivery. We will talk about Agile and product research in Chapter 9.
It’s not the right time.
It’s always the right time. Research falls into many categories, and each category allows you to answer different questions at different times (we will go into detail about these in Chapter 4). Based on the question you’re trying to answer, there are many ways to employ product research methods without changing your schedule.
We don’t have many users to test with.
You don’t need many users. This may go against what you’ve heard, especially if you have experience with quantitative methods. “What about statistical significance?” we hear you cry. Qualitative research is considered valid when it can capture the essence and richness of what is being observed, even if it doesn’t have statistical significance. There are many methods where you only need 5 to 10 users to discover how your ideas resonate with your audience.
We have enough data.
Data is the starting point of all product research (see Chapter 3). However, while Google Analytics, Omniture, Mixpanel, Appsee, and similar telemetry systems are great for understanding what users are doing, they don’t tell you why users are behaving as they are. It’s only when you combine actual user behavior from a telemetry system with qualitative research that you get solid insights.
We’ll learn during the pilot.
Pilots and betas are good opportunities to get user feedback. However, the cost of making changes at the point of launch is high, and you risk upsetting the willing early adopters you’ve fought to find. Product research across the product development life cycle will give you the same feedback as a pilot, at a much lower cost and at a time when you can still fix your product.

Those who don’t conduct research learn the hard way. Color was a startup that aimed to be the most popular, most fun social network of 2011. The team had good funding and believed strongly in their vision, so they started building. However, they overlooked the ease of use, abundance of content, and simplicity of other social networks at that time, which led to very slow growth and ultimately to a complete shutdown. As founder Bill Nguyen summarized it, “I thought we were going to build a better Facebook. But within 30 minutes I realized, Oh my God, it’s broken.”2 Their assumptions about how great the service would be didn’t hold, and it was too late to fix things. Hindsight is 20/20, but getting the users’ reactions earlier on might have shown warning signs and given the team time to adjust.

Product research skills can be learned. With the right training and mindset, everyone can do it; when planned well, it costs very little. Instead of taking months to yield results, product research takes only a few days, meaning it’s easy to build into existing practices. And when product research is easy, it becomes a habit, creating better products and happier, more engaged teams to build them.

When Do You Do Product Research?

All the time! Product research can and should be used at every stage of developing your product. This is because you need to learn different things at different times along your product journey. We’re going to take some liberties in simplifying the steps of product development as much as we can here and describe the process in three distinct stages.

  • Stage 1 is exploring the value of products or features in the market. This is the phase where you are discovering deeper needs in a broader context. In many cases, you don’t even have a plan to build something: you’re just trying to find out whether it’s a good idea. At this stage, you are trying to understand the problem space. Have you understood the problem correctly? Are you considering the right solutions? Are you planning to build the right solutions for the problem you understand?

  • Stage 2 is the development of the product or feature. Here, research helps you stay on course and allows you to assess the right approach. Your results might invite you to explore alternatives. Now that you’re immersed in the problem, why are you struggling with certain aspects? Do the assumptions you made at the beginning still hold true? Are you building the solution the right way?

  • Stage 3 comes after you have released your product or feature or when you’re working on refining existing features. Research at this stage helps you observe the change in your users’ behavior. Now you can check your assumptions directly with the users and see how their needs are changing because of your product or service.

If you are new to product research, you may be concerned at this point. We can hear some of you busy product people say, “Oh my God, I barely have time to grab coffee! How am I supposed to create time to do research?” We will talk about how this is possible in this book, with many examples from companies that range in size and research budgets.

Building on Different Research Disciplines

Product research draws from different research disciplines, namely user research, market research, and product analytics. While there is some overlap between these disciplines, each discipline has a different focus. Each discipline has subdisciplines that are specialized for particular types of research. Here is an overview of the disciplines that product research builds upon.

User Research

User research studies what a user does with and surrounding the context of a product’s use. It is about working with real humans to understand their motivations, behavior, and needs. It aims to understand how that person employs your product and what happens before, during, and after that experience.

In practice, most user research can be broken down into three categories: generative, descriptive, and evaluative.4

Generative user research

Aims to get a deep, rich understanding of user needs and desires: users’ behaviors, attitudes, and perceptions. It explores problems and possibilities in the early stages of product development. Because generative user research is learning about nuanced practices directly from users, it uses methods like ethnography and contextual interviews, where the researcher spends significant time with participants.

Descriptive user research

Aims to uncover how something works and describe a phenomenon in detail. It helps teams understand how specific parts of the problem arise. It uses methods such as interviews, contextual interviews, and diary studies.

Evaluative user research

Aims to find out how something compares to a known set of criteria. It is also used to confirm that a specific solution can solve the problem you are working on. Usability studies and A/B testing are common evaluative research methods, especially if you already have a digital product or good prototype in hand.

Market Research

Market research involves gathering data about what people want and analyzing that data to help make decisions—for example, about strategies, processes, operations, and growth. Market research strongly shapes what a company does and where it focuses its efforts.

Market research is usually split into four areas: exploratory, descriptive, causal, and predictive.

Exploratory market research

Used when the research problem has a lot of unknowns. It identifies avenues for new and existing product growth. Market exploration usually makes use of secondary data from inside and outside the company, as well as observational studies, expert opinions, and user feedback.

Descriptive market research

Concerned with finding out how things occur, how often, and how they’re connected. Interviews and surveys are popular descriptive market research methods.

Causal market research

Establishes the cause-and-effect relationship between a set of variables. It relies on statistical methods and large data sets, therefore it requires rigor.

Predictive market research

Helps you predict certain market variables. It forecasts what users will want and when they want it, and findings can affect future sales, growth projections, or the development of a product.

Product Analytics

Product analytics is about discovering how your audience uses your product from the data trails they leave. Product analytics can be used to find answers to questions regarding the behavior of a large number of users. It can also be used to formulate and refine questions, as you will see in Chapter 3.

Product analytics can be classified into four subtypes: descriptive, diagnostic, predictive, and prescriptive. For most product analytics, descriptive and diagnostic are what you’ll use. We’ll discuss more details in Chapter 7.

Descriptive analytics

Describes what you know from your data, whether that’s the number of downloads recorded or the percentage of users who leave the site within a minute. It paints a numerical picture of what happened.

Diagnostic analytics

Helps you discover why something is happening. It uses techniques like data discovery, drilldowns, data mining, and correlations.

Predictive analytics

Asks what might happen in the future, based on what has happened in the past. You take the data you have and employ statistical techniques, usually involving machine learning, to predict how users might behave.

Prescriptive analytics

Asks what your next steps should be, based on what you know and what you think users will do in the future. Prescriptive analytics is thus firmly rooted in predictive analytics but is more advanced.

As you can see, there are so many variations and possible approaches that a researcher can take. But which one is the right one? In Chapter 4, we will talk about how you can pick the right research type and research method based on what you want to learn.

A Set of Rules for Product Research

This book is the result of decades of experience in the world of product research. During that time we’ve seen certain patterns that make product research effective and enjoyable. We thought of summarizing these patterns as recipes for any team to use. But we realized that if we covered all permutations of products, services, business models, markets, target audiences, team member skillsets, and more, the number of recipes would be enormous. Research methods also vary greatly from one team to another. Good insights come from using qualitative and quantitative methods together; a text on every combination of these methods would be an encyclopedia, not a book. Even if we had some recipes, we do not know about the exact circumstances of your product, so the recipe might apply somewhat but not exactly.

Instead of creating recipes based on these patterns, we decided to distill them into rules to guide you on your journey. We believe they have more applicability and give you more freedom in adapting them into your own workflow. To cement these rules in real life, we added examples of how different teams apply these rules and what they’ve achieved.

Rule 1: Prepare to be wrong.
We all want our ideas to result in successful, enjoyable, and popular products. But many of our ideas are bad, and some of them are just plain terrible. Product research will show that your brilliant idea wasn’t actually good. You will have to be OK with being wrong more often than you want to be and seek to learn from your users with an open mind.
Rule 2: Everyone is biased, including you.
We are human, and we have many biases. These biases manifest themselves when we speak, when we think, and when we share thoughts—and they lead us to wildly inaccurate conclusions. We can’t get rid of them completely, but we can acknowledge and account for them.
Rule 3: Good insights start with a question.
If you hear yourself saying, “Oh, let’s do a survey!” or “We need personas!” this rule is for you. Good research starts with a question, not with a method or a deliverable. Good research questions come from the data you have—from what you already know. A single crisp, unbiased research question is key to getting good insights.
Rule 4: Plans make research work.
If you are new to research, you will be surprised when you realize that the time you spend with your participants is very short compared with the time you spend preparing for research. Choosing a method, selecting participants, and preparing field guides, protocols, and communication plans all take a lot of time, but you recover this investment during your sessions and analysis.
Rule 5: Interviews are a foundational skill.
In most of our research, we talk to people. Interviewing is the base of all research methods. Even small improvements in your interviewing technique will go a long way toward creating a stronger bond with your participants so that you can learn from them in a richer, more personal way.
Rule 6: Sometimes a conversation is not enough.
While you can learn a lot from a simple conversation, there are many times in product research where you need to look to other techniques to discover insights. Analytics and quantitative data analysis are, of course, one way to fill that gap. Additionally, qualitative techniques that involve working directly with your participants or asking them to complete tasks can reveal insights you never would have obtained through interviews alone.
Rule 7: The team that analyzes together thrives together.
One of the first steps in getting buy-in for your research results is to conduct analysis that includes many of your stakeholders. While this may not be possible all the time, involving your stakeholders in parts of your analysis will help pave the way for them to accept your findings—and, more importantly, act on them.
Rule 8: Insights are best shared.
If an insight is in a report and no one reads it, does it ever get incorporated into a product? There are many ways teams share their product research insights. We have a few tips on how to share them so they stick.
Rule 9: Good research habits make great products.
Research is not a one-and-done endeavor. Teams that bake product research into their normal way of working not only uncover insights regularly but also swiftly shape their product in response to those insights, resulting in ever-better products.

So there you have it: nine rules to help product teams of any size, with any budget, working on any type of project. Like all rules, they’re meant to be broken. In fact, we’ve broken every one of them—sometimes unwittingly, sometimes purposefully. But understanding this guide for product research will allow you to create products that people want to use—and then you can find the exceptions to the rules.

Let’s get started.

Acknowledgments

Adaora Spectra Asala

Adrian Howard

Alex Purwanto

Alper Gökalp

Andrea Saez

Aylin Tokuç

Becky White

Beril Karabulut

Berk Çebi

Bruce McCarthy

Cat Smalls

Chris Skeels

Dan Berlin

Dan Rothstein

Daniel Elizalde

Dilek Dalda

Doğa Aytuna

Emre Ertan

Erde Hushgerry-Aur

Erman Emincik

Esin Işık

Evren Akar

Fernando Oliveira

Gabriela Bufrem

Gökhan Besen

Gregg Bernstein

Hope Gurion

Janna Bastow

Jofish Kaye

Kate Towsey

Kayla Geer

Lan Guo

Levent Atan

Lily Smith

Loui Vongphrachanh

Martin Eriksson

Matt LeMay

Melissa Perri

Michael Zarro

Mona Patel

Murat Erdoğan

Mustafa Dalcı

Nilay Ocak

Orkun Buran

Özge Atçı

Özlem Mis

Pablo Gil Torres

Pelin Kenez

Pınar Yumruktepe

Randy Silver

Rıfat Ordulu

Rob Manzano

Roger Maranan

Sercan Er

Shirin Shahin

Sophie Bradshaw (Editor)

Steve Portigal

Şüheyda Oğan

Takahiro Kuramoto

Theresa Torres

Thomas Carpenter

Tim Herbig

Yakup Bayrak

Yasemin Efe Yalçın

Aras
A few years ago, Ercan Altuğ and Adnan Ertemel planted the idea of a book, and C. Todd lit the fuse. My contributions to this book would have been at best a set of Medium articles without their encouragement and mentorship.

I am truly grateful to everyone who taught me invaluable lessons that led to the practices in this book. I want to thank my amazing managers Jenny Dunlop, Chris Liu, Darrell LeBlanc, Fatih Bektaşoğlu, Eray Kaya, and Hüsnü Erel; my inspiring professors David Davenport, Paul Dourish, Gregory Abowd, Nancy Nersessian, Wendy Newstetter, and Keith Edwards; unbelievable businessmen the İşbecer brothers; and the folks at Kloia and Expertera for their support throughout the project.

And thank you Zip, my dear over-thinker, over-lover, over-joy, for your endless support. And thank you, Derin, for all the pürç you brought to our lives. You wouldn’t believe the amazing things your mother did for you, even if someone writes a book about it one day.

C. Todd

It’s one thing to coauthor a book; it’s another to do it again with the same person. MC (Michael Connors), you are a rock star. Thank you for joining me on another adventure. Aras, it was great to see that our ideas aligned and ultimately became this book. I’m a better person for completing this journey with you two.

To my friends and family, thanks for putting up with yet another “I have to finish a chapter this weekend.” To the greater product management community: You inspire us to write books like this. Keep pushing our discipline forward. We make products that make the world a better place. Thank you!

Michael
It was a pleasure working with everyone who contributed to this book. Many thanks to C. Todd and Aras for including me. Every conversation and working session we had throughout the project was interesting and fun—two of the most optimistic and thoughtful people I’ve ever met! Thanks also to the O’Reilly team for their guidance and input, real pros. And to all of the designers, developers, and other project collaborators that I’ve worked with during my career, as well as the greater design community. They all played a part in forming and shaping the concepts presented here.

You know the exact product, feature, or service your customers need. Or do you?

1 Adi Ignatius, “The Tyranny of Numbers,” Harvard Business Review (September–October 2019), https://hbr.org/2019/09/the-tyranny-of-numbers.

2 Danielle Sacks, “Bill Nguyen: The Boy in the Bubble,” Fast Company (October 19, 2011), https://www.fastcompany.com/1784823/bill-nguyen-the-boy-in-the-bubble.

3 To learn more, see the Agile Alliance Experience Report, “Using Design Methods to Establish Healthy DevOps Practices,” https://www.agilealliance.org/resources/experience-reports/using-design-methods-to-establish-healthy-devops-practices.

4 A good overview of these different approaches can be found in Just Enough Research by Erika Hall (A Book Apart).

Get Product Research Rules now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial