—Paul Graham, Founder and Partner of Y Combinator
Spend time understanding all aspects of the customer value proposition. Ask yourself: why should your customer buy your product? How does your product fit into the rest of his world? What influences his opinion of the product’s value? What is your product displacing—all products displace something—and why your customer should risk making that switch?
—Gary Swart, CEO of oDesk
When I first started doing customer development interviews, I spent hours coming up with a long list of perfect questions that were specific to my audience and product. I wrote down far more questions than I thought I could cover in a 30-minute interview in order to be prepared for any direction the conversation might take.
It didn’t take long to realize that the pages of notes from a single interview came from the first few basic questions.
After all, your biggest risk comes from one of two common errors: that you failed to solve a problem that your customer has, or that you failed to make the solution attractive enough for your customer to choose it.
This chapter starts with the basic questions that I use in almost every interview. These work across a variety of customer and industry types.
We’ll also cover:
Why customers don’t know what they want
What you should be listening and prompting for
Getting subjective answers from objective questions
The second half of this chapter delves into social psychology, but it’s not purely theoretical: understanding how people think will help you ask effective questions. We’ll talk about how to overcome people’s natural limitations and biases (including your own).
I don’t vary my interview script much at all from project to project or company to company. I might add one completely custom question or adapt my tone to fit my audience, but otherwise I follow the same basic script.
If you’re taking notes on a laptop, you can save a template document with about eight blank lines after each question and take notes directly into that. Don’t worry about a more structured form for taking notes; just get down as much detail as possible.
You may be wondering how just five questions can take up a 20-minute interview. These are not the only questions I ask, but they are the only scripted questions I ask. (See the Appendix to learn more about different types of questions and what you can expect to learn from them.)
As the customer talks, I’ll respond with open-ended questions that are triggered by what she just said. Depending on the conversation, these might be things like:
Can you tell me more about how that process goes?
Who is involved in making that decision?
Last time you did ______, how long did it take?
Where did you most recently go to buy ______?
May I ask, why did you come to that conclusion?
Think of these as conversational prompts more than questions. You can use them to keep the customer talking or to redirect him to a slightly different topic. If you’re too busy writing notes to ask these prompt questions, that’s a good problem to have. The customer is talking!
“It’s not the customer’s job to know what they want.” —Steve Jobs
How is it that customers so often don’t seem to know what they want? There’s an obvious tension between customer wants and needs. You may be skeptical that customers can reveal things that product builders (who spend their days immersed in a technology, an industry, a business model) don’t already know.
“Our customers just don’t have good ideas.” I’ve heard that on more than one occasion, and I’d be lying if I told you that I don’t agree.
Most of us have had the maddening experience of building something based on customer feedback, only to find that the customer still isn’t happy (or didn’t buy the product after all). After this happens a few times, many companies conclude that customers are difficult, flaky, or don’t know what they’re talking about. Voice-of-the-customer feedback becomes something to dutifully collect and then mostly ignore.
Given the fact that this book has the word “customer” in the title, I think you can guess how vehemently I feel about that. Don’t ignore what your customers say.
But it’s worth taking some time to dig into exactly why customers are unreliable at telling us what they want and what they’ll buy. And I’m going to start using the word “we” in talking about this—because you and I are subject to these same limitations.
No one knows better than your customer what it’s like to live in his world. But that doesn’t mean that he will be able to automatically put his experiences into words.
Once we get used to limitations, we stop questioning or even noticing them. It doesn’t occur to us to mention constraints if we think of them as just the way things are. Sociologists call these taken-for-granted assumptions.
Our brains are biased toward things that have happened recently. We don’t tend to mention past failures when we’re proposing new solutions. If we’ve abandoned a process or tool, we may not immediately remember what parts of that solution did work.
They know the methods they feel comfortable with and where they have good intentions but terrible follow-through. We often don’t volunteer that we are bad at certain skills or processes.
Proficiency does not require understanding how a tool or process works. “Any sufficiently advanced technology is indistinguishable from magic,” writes science fiction author Arthur C. Clarke, and that isn’t far from how most people view the products they use every day. The fact that I can ask my iPhone a question and get a reasonable answer certainly feels like magic to me. As the people who build products, we’re immersed in our worlds. We’re familiar with what technology and process and automation can do. Our customers may be 10 years behind us.
You need to guide the conversation and set expectations, but also defer to the customer’s experience. To overcome this tension, you’ll be asking questions that have no clear right or wrong answer. You’ll state your questions as objectively as possible and prompt the customer for personal and subjective responses.
It’s your job to uncover those needs. It’s your job to get the details so that you can deduce why or how the previous solutions didn’t work. It’s your job to make him feel comfortable enough to be honest about those things with you.
To get the most out of the five basic customer development questions, you’ll want to know what to listen for. These are the objective and subjective factors that separate prospective customers from buying customers:
How your customers are behaving today (which predicts how they’ll behave tomorrow)
The constraints that affect the choices and actions that your customers take
What frustrates or motivates your customers
How your customers make decisions, spend money, and determine value
For each of these factors, we’ll walk through how they impact decision making and how you can frame your follow-up questions to uncover the most useful details.
Discovering what your customer is doing today is the heart of understanding the problem that you’re setting out to solve.
What your customers are doing today tells you:
Their current behavior is your competition. It doesn’t matter how effective or ineffective their current behavior seems—it’s what they are accustomed to and it works (at least to some degree).
You can learn about how customers are behaving today by prompting with, “Tell me about how you do ______” or “Walk me through how you use ______.”
How you fill in those blanks is important, too. It’s critical to define the problem broadly so that you don’t prematurely constrain what your potential customers say. If you think you’re solving a specific problem, try to move up one level of abstraction and ask the customer about the problem one step up from that.
Abstracting up by one level is what allows you to look beyond incremental, easy-to-copy improvements and see opportunities for disruptive change.
For example, don’t ask about how someone arranges grocery delivery online; ask about how she feeds her family. Don’t ask about how someone uploads and shares files; ask about the last time she worked on a document and needed a coworker’s input.
If TiVo had interviewed customers about how they program their VCRs, they might have gotten feedback that drove them to simplify the programming controls and missed the boat on creating the digital video recording industry. In fact, that’s exactly what the first attempts at improving the VCR looked like.
Compare that to asking customers about the time they missed the last 10 minutes of the final episode of Twin Peaks or the game-winning play in the Super Bowl—it’s easy to imagine how quickly (and emphatically) customers would’ve told you about the problems that inspired pausing live TV, recording by show name instead of time slot, and fast-forwarding through commercials.
You may be tempted to ask immediately about critical events such as a purchase, a registration, or completion of a key task. Asking an abstract question like “tell me about how...” feels pretty far removed from understanding the critical events that drive success.
But these customer decisions are made in the complex matrix of the environment they’re in, the resources they have available, and their capabilities and past experiences. When you understand the factors that go into customers making decisions, it becomes far easier to figure out how to prioritize specifics about making, marketing, and selling a product.
By asking procedural questions, you prompt the customer to tell her story, step by step. This is how you can uncover how she makes sense of her world. She’s likely to skip details, or accidentally omit underlying assumptions. When something doesn’t make sense, prompt her for an explanation.
Customer: “On Sunday night, we look at the calendar and plan the upcoming week...”
Interviewer: “Excuse me, when you say ‘we’, you mean...?”
Customer: “Oh, yes. My husband, myself, and my oldest daughter. She’s in high school, so she keeps track of many of her activities on her own.”
Interviewer: “Thanks. So the three of you look at the calendar and...”
The specific actions that customers take are important, but equally important are the adjacent factors of how, why, when, and with whom. Those are the underlying root causes that make or break products. As your customer talks, be ready to respond with open-ended questions.
Ask someone, “In the future, would you do X?” and you’ll get an inaccurate answer. Some people will be too polite to say no. Others will give optimistic or socially acceptable answers. (Ask any smoker how often he’s told people he plans to quit.)
The choices we make immediately are different from the ones we make in the future; specifically, our future selves are more virtuous. In a study done at the University of Illinois at Urbana-Champaign, people who were asked to choose movies that they’d like to watch in the future chose highbrow or culturally salient films like Schindler’s List or The Piano. But when asked to choose a movie to watch tonight, they opted for more lowbrow choices like Mrs. Doubtfire or Speed. It’s not hard to find echoes of these behaviors in ourselves. We pay for gym memberships but don’t go to the gym; we choose a burger tonight and rationalize that we’ll eat a salad for lunch tomorrow; we promise that we’ll clear up tech debt in our next software release.
At Yammer, a small but vocal set of customers have frequently requested that we enhance our topics functionality (which is similar to hashtags on Twitter or Facebook). But when we look at our analytics data, we can confirm that only a tiny percentage of users have ever added a topic or clicked on a topic in another user’s post (Figure 4-1).
We’ve made multiple attempts to increase topic usage through visual design and functionality changes, but customer behavior remains unchanged. Despite what users promise about their future behavior (“If topics worked better, we’d be able to curate our Yammer network! If we could use topics more, people would use them to build an internal knowledge base!”), the actual usage we see today tells us that most people are not motivated to use this feature.
To get the most accurate answers from your customers, frame your questions to ask about specific events or decisions and focus on the present or recent past. Table 4-1 provides some examples.
How likely would you be to use ______?
Tell me about the last time you used something like ______.
How often does _____ situation occur?
In the past month, how many times has _____ situation occurred?
How much would it cost your company if _____ happened?
Last time ______ happened, how much did it cost your company?
How would your family react if you decided to _____?
Last time you made a significant decision, how did your family react?
When I was conducting customer development interviews at KISSmetrics, I talked with many startup founders who were highly aspirational about how they’d use data to search for patterns and run split tests to optimize their businesses. But many of them admitted that they were not currently collecting any data beyond a standard Google Analytics installation.
“Tell me about how you measure data for your business today...” forced them to describe their current environment and capabilities honestly. That in turn allowed us to focus on the few features that would bring the most value to that set of customers.
When we see customers with problems that they haven’t been able to solve, we tend to think that the reason is either lack of access (“they just don’t have this product”) or lack of motivation (“they just don’t get it”). In other words, if the customer could get his hands on a product that fixed this problem, he’d be all set. Or, if the customer cared enough to challenge the status quo and look for a solution, he’d have figured one out by now.
In reality, customers are influenced by a number of constraints. Not all of them are objective, but that doesn’t reduce their impact. You need to understand the type of constraints limiting your customer because they determine which follow-up questions will be valuable, what type of solutions will be attractive, and what type of solutions will ultimately be effective.
Common constraints that I have noticed include:
Problem is not perceived as a problem
Lack of awareness of what’s possible technologically
Limited resources (environment, time, budget)
Cultural or social expectations that limit behaviors
Let’s talk through each of these so that you understand why they pose problems and how you can guide users to see past them.
People naturally focus on the tasks and processes they are most familiar with. When we have a task to complete, our focus is on completing it rather than optimizing how we complete it. We don’t challenge our assumptions. This is a concept called functional fixedness, described by social psychologist Karl Duncker as “a mental block against using an object in a new way that is required to solve a problem.”
You may be familiar with the classic experiment on functional fixedness. Participants were given a box of tacks, a candle, and a book of matches and challenged to attach the candle to a wall so that no wax dripped to the floor below. The solution requires rethinking the uses for each of those ordinary items. Dump the tacks out of the box and use a tack to attach the box to the wall, forming a “shelf”; then use a bit of candle wax as adhesive to stick the candle to the shelf.
We’re not good at this kind of rethinking. In everyday life, the ability to see an object or process as having only one purpose makes our brains more efficient. But that efficiency makes us terrible at coming up with novel solutions to complicated problems.
In the candle and box experiment, the participants knew that they had a specific problem to solve. They were told that a valid solution existed—it was just a matter of figuring out what it was. In real life, we don’t always realize that there is a problem. Even when we’re experiencing a frustration or lack, we don’t realize how bad it is unless we have something to compare it to. In fact, as author Daniel Pink points out, even offering explicit incentives to “think outside the box” doesn’t work.
Take online shoe shopping. Many people won’t buy shoes without trying them on because it’s hard to get the right fit—even if you’re buying the same brand and size that you’ve worn for years. Shoefitr is a Pittsburgh-based company that uses 3D imaging to measure the internal dimensions of shoes. The customer can look at the measurements of a shoe he’s browsing and compare them to the shoes that are already on his feet.
When user researcher Grace O’Malley talked to customers, they described different strategies for getting the right shoe size. O’Malley explains, “Some people would buy multiple pairs in different sizes and return the ones that didn’t fit. A lot of people would just make a shoe work, by putting insoles in or in other scrappy ways. They just dealt with it. Some refused to buy shoes online. But no one asked for a solution that would help you find the shoes that would fit your feet in advance of buying them.”
For example, when I worked at Yodlee, an online banking solution provider, I spent a lot of time thinking about the problems around online bill payment. One thing I noticed was that people use a lot of mental shorthand: we say “paying bills online” to refer to a collection of activities that may include circling due dates on a physical calendar, calculating balances, confirming which checks have cleared, and resetting a forgotten password as well as the quick step of confirming payment accounts and clicking Submit on the banking website.
Tony McCaffrey, CTO at Innovation Accelerator, uses what he calls the “generic parts technique.” “Break each object into its parts and ask two questions,” says McCaffrey. “Can it be broken down further? Does your description imply a use? If so, describe it more generically.”
In my experience, when customers talked about paying bills online, they were satisfied with how long they spent on the activity. When they broke that familiar activity into its component generic parts, it made them stop and question the amount of time and effort they expended on the task.
When your customers describe their behaviors, you want them to realize that every action and assumption that is second nature is actually a decision that they are making—and that better options may exist.
Another way is to challenge the unrealized problem is to create some implied comparisons between what the customer’s life is like now and what it would be like in a future where his problem is solved. This doesn’t mean that you should talk about the specifics of your product! Instead, you’ll want to dig into how the problem is causing pain to the customer. Is it wasting an hour of his time that he’d rather spend with his family? Is it costing an amount of money equivalent to what he spends on gas each week? Is it creating social friction that’s hurting a relationship?
Problems without conceivable solutions feel more like facts than problems. It may not occur to people to complain about something if they believe that there is no way it can be remedied. The solutions we suggest are rooted in what we perceive as possible. This is why customers tend to suggest incremental improvements rather than fundamental changes.
As the people who build products, we’re immersed in our worlds. We’re familiar with what technology and process and automation can do. Our customers are not.
Think about a solution like Siri. Just a couple of years ago, most people’s experience with voice recognition was limited to those nightmarish automated interactive voice response systems where you find yourself yelling “1, ENGLISH, NO, 3, TALK TO OPERATOR, OPERATOR, OPERATOR!” at your phone and then hanging up in disgust. How could a customer ask for anything resembling Siri if that was her only experience with natural-language processing and voice recognition?
Even when a customer can conceive of a solution that doesn’t exist today, she’s unlikely to share it for fear of sounding foolish. We don’t want to admit that we can’t tell the difference between something difficult and something impossible. Who wants the embarrassment of inadvertently asking for something ridiculous?
It’s not easy to get customers to think outside the realm of what they know is possible, but it can be done.
There’s a question that I’ll reference throughout this book, and it’s this: “Forget about what’s possible. If you could wave a magic wand and solve anything, what would you do?”
The magic wand question takes a huge burden off your customer’s shoulders. It tells him that he doesn’t have to worry about what is possible. There are no wrong answers. When you frame the interview in this way, you tend to hear, “Well, this is ridiculous but...” followed by legitimate problems and creative ideas. The ideas that customers propose are usually not possible or practical, but they can lead you to ones that are.
“I wish I could just sit on people’s shoulders—like the devil on their shoulder—and ask ‘Why?’ right at the moment when they abandon my website.” That response to the magic wand question helped KISSmetrics design KISSinsights, an on-site survey tool. It perfectly captured the frustration that our customers were feeling.
Sometimes customers are constrained by the environment they are working or living in. People living in dorms or apartments or working in cubicles may not be able to take advantage of a solution that requires them to make changes to their physical space. Parents of young children may not be able to benefit from a solution that requires two hands or rapt concentration. (Product designer Anne Halsall writes, “For the first three weeks of our son’s life, I didn’t touch a computer I couldn’t use with one hand...an iPad mini immediately became my primary computer.”)
Many environmental limitations are not permanent. Consider noise, poor weather, crowded physical space, unreliable Internet access, or the distance between points A and B. They simply add a small amount of additional friction, which is often enough to inhibit customer behaviors.
Of course, most customers face some sort of resource constraints. It’s critical to understand which resources are scarce. A busy working parent is less likely to spend time in order to save money; a college student often has the opposite inclination. At KISSmetrics, many of our early customers had limited engineering resources. This meant that they were willing to trade a shorter installation and setup process for a longer one that could be accomplished mostly by nonengineers.
At Yammer, we often encounter users with a set of internal rules for how they use our product: “I read all the time, but I only respond to a question if it’s asked by one of my peers. I’m not comfortable responding to managers.”
There are no limitations in the software. It’s even unlikely that their particular company has articulated guidelines on who talks with whom. That doesn’t matter. A hierarchical work culture can influence behavior even if the rules are entirely unspoken.
Does your customer feel that she will be judged negatively if she tries your solution and it is not successful? If so, how can you reduce her sense of risk in decision making? Does your customer feel that she needs permission before changing behaviors? If so, you’ll want to focus heavily on the other stakeholders who she perceives as granting that permission. (See How Your Customers Make Decisions, Spend Money, and Determine Value later in this chapter.)
Another cultural obstacle is that a product may conflict with how the customer wishes to view herself.
A couple years ago, I was working with a startup team that wanted to solve the problem of people losing expensive items. They asked people in their personal networks if they knew anyone who was always losing things and quickly found dozens of professionals who routinely lost iPhones, laptops, ski equipment, and other expensive items. The interviewees admitted that they wasted hours each week and spent hundreds or even thousands of dollars replacing lost items.
But when the team started investigating how much these prospective customers were willing to spend on a solution, they ran into resistance. One customer finally admitted, “I’ve just gotten used to budgeting extra money to replace the stuff I’ll inevitably lose. I’m OK with that.”
Even if you objectively know that you lost your laptop last month and your keys twice this week, you may not think of yourself as a careless person. Buying a product to keep you from losing things is akin to admitting that you are.
This is a great example of where Steve Blank’s earlyvangelist diagram in Chapter 3 should be keeping you honest. These customers may have seemed ideal, but they were not actively seeking a solution. Time to pivot and seek out a different target audience.
Social and cultural limitations aren’t always obvious to your customer, either. Sometimes customers are fully aware that they don’t want to, or would be uncomfortable, using your product. Often they are not. Try asking the customer to envision herself fixing the problem, and watch for body language or changes in her tone of voice. Does she seem relieved or excited at the prospect of fixing the problem? Or does she seem hesitant or conflicted?
People are not strictly rational. Our decisions are not based solely on logical, economic criteria. We choose more expensive options because they look cooler; we reject potential solutions because something about them makes us feel uncomfortable.
As your potential customers describe what they’re doing today, listen carefully for hints of what frustrates them and what motivates them.
No matter how effective your product or service is, it still requires effort and investment from the customer. To maintain that effort and investment, your customer needs positive feedback. What motivates one customer may turn off another. You’ll need to uncover what incentivizes your customer as well as what types of frustration he considers a deal-breaker.
One of the biggest demotivators that I’ve run across, in multiple contexts, is uncertainty. Being unsure of how a service will work, or how to best interact with a product, creates such a high level of discomfort that customers may choose to disengage. This specific issue is worth exploring because it creates a big disconnect for us.
As people who envision and build new products, we tend to be unusually comfortable with uncertainty. Our customers are not.
“How does it work? How should I use it? What will happen if I...?” are the first questions that people ask every time I introduce them to a new feature.
You’ll also want to figure out what makes your customer feel successful. Is he motivated by mastering a task, by seeing visible progress, or by performing a task better than his peers?
The incentives you provide should align with what your customer values. For customers who are competitive, earning a spot atop a leaderboard is motivating. It does little to nothing for customers who value hearing “thank you” from someone they’ve helped.
The person who uses the product isn’t always the one who buys it. Toys and clothing are used by kids and purchased by parents. Medicines and medical devices are used by patients, prescribed by doctors, and (typically) paid for by medical insurance. But the relationship between the primary user and the other invisible stakeholders—people or organizations that make decisions or have influence—is not always that clear.
You’ll want to ask which people or groups have this problem directly and which experience it secondhand. If the customer doesn’t mention anyone else, you can try asking about which other people are present when the problem occurs, or which other people they discuss the problem with.
Kids and spouse (desire to minimize conflict)
Social circle (desire to avoid judgment)
Whoever holds the checkbook/credit card/purchase requisition forms
People who provide scarce and needed skills, such as an engineering team or legal counsel
People who ensure compliance, such as IT security, finance, or legal counsel
People who use the product with the customer (for products that require collaboration or a network effect)
This chapter may have felt a bit dense and theoretical, but we’re about to head back into tactical territory. In the next chapter, we’ll walk through the interview to give you a sense of how it flows and what to be prepared for. After you read Chapter 5, you’ll be ready to get out of the building.
 A company called Gemstar created a product called VCR+ that gave each television program a unique 4-digit code. Instead of having to manually input a channel and program a timeslot—which required having your VCR set to the correct time, its own private nightmare—customers could look up the desired program in their TV Guide or local newspaper television listings and key in that code (http://en.wikipedia.org/wiki/Video_recorder_scheduling_code). It was an improvement—assuming you hadn’t accidentally thrown out your TV listings or misread the 4-digit code or simply forgotten to enter the code before you left for work in the morning.
 Daniel Read, George Loewenstein, and Shobana Kalyanaraman. “Mixing Virtue and Vice: Combining the Immediacy Effect and the Diversification Heuristic.” Journal of Behavioral Decision Making. (12) pp. 257–273, 1999.
 Tech debt is a term used in engineering to refer to shortcuts or suboptimal decisions made in coding, often in order to meet a deadline. As with monetary debt, the implication is that there will be consequences if the debt is not paid off in the future.
 According to online running shoe retailer Running Warehouse, 65% of all returned shoes were due to improper fit. In the first two years of using Shoefitr, fit-related returns fell by 23%, which resulted in a 2.5% increase in profit margins (http://www.runblogrun.com/2012/05/shoefitr-use-of-online-fitting-application-increases-rate-of-returns-decreases-release-from-shoe-fit.html).
 This fear is wonderfully captured in the goofy movie Austin Powers: The Spy Who Shagged Me, when Dr. Evil has traveled back in time to 1969 and is threatening the President:
Dr. Evil: “Mr. President, after I destroy Washington, D.C...I will destroy another major city every hour on the hour. That is, unless of course you pay me... [dramatic pause] one hundred billion dollars.”
His request is so ridiculous that his audience, instead of cowering in fear, bursts out laughing at him: “Dr. Evil, this is 1969! That amount of money doesn’t even exist! That’s like saying, I want a kajillion bajillion dollars!”
 This story is so good that I have to quote some more from it: “What I realized was that before I became a parent, I never really thought about the one-armed people of the world. And by that I simply mean the people who, because of their work or the circumstances of their lives, don’t have much of an opportunity to use any device that requires two hands. Once you start looking for these people you see them everywhere... They don’t use devices unless they are small and light enough to be operated one-handed. Parents of small children certainly fall into this category. So do people with active jobs who are often standing or traveling” (http://contextsensitive.quora.com/Seeking-the-one-armed-man).