UX for Lean Startups

Introduction

I’m a little tired of the phrase “Get out of the building.”

Don’t get me wrong. It’s a wildly important concept. If you’re not familiar with Steve Blank and Eric Ries and the whole Lean Startup methodology, you should be. It’s good stuff.

Steve and Eric talk a lot about getting out of the building, and there’s an excellent reason for that. Getting out of the building is a great way to make products that people want to buy. Sadly, sometimes it’s easier to understand the importance of something than it is to put it into practice.

Of course, the goal of getting out of the building is to get you in better touch with your users. If you have a question about how your product should work or what feature you should build next, the answer is not in that airless conference room you’re trapped in. It’s not on the whiteboard. It’s not the result of an endless brainstorming session with other people at your company. Even your CEO does not have the answer.

Your answer is somewhere in the world outside the building, and your customers can help you find it. You just need to know how to learn from them. This is one of the most important concepts of Lean Startup. You need to validate things with your customers early and often in order to keep learning. If you missed that part of Eric and Steve’s work, you should take another look. It’s kind of a big deal.

If you’ve been paying attention to the rest of the talk about Lean Startup, then you’ve heard that user experience is also incredibly important. You have to know how to design simple things, A/B test them, and then iterate. Oh, and don’t forget continuous deployment, Agile development, and Minimum Viable Products.

Of course, if you’ve ever tried to do any of those things, you’ve probably figured out that they can all be ridiculously hard to do.

Take “getting out of the building” as an example. Did you realize that there’s a whole industry built around this? There are huge numbers of people who have studied this in school and who do this professionally. These people are often called things like “user researchers,” and they know all sorts of things about what kind of research to do and when and how to get the right sort of information from people. They figure out what’s wrong with your product and explain what you need to do to fix it. And they charge you a lot of money for it.

The same goes for user experience design. There are people who have been designing products for years. They’ve studied great designs and terrible ones. They’ve spent their careers trying to design things simple enough for people to understand and delightful enough for people to want to use.

Unfortunately, there is a terrible shortage of these types of people, so it’s very likely that you’re trying to build a product without one. Or maybe you were lucky enough to find a designer, but she’s mostly good at visual design or she’s never worked in anything other than a waterfall environment. Or you might even be a designer or user researcher, but you’ve never worked in the terribly chaotic and insane world that is a new startup.

As one of the above-mentioned types of people, you’re now being told that you need to get out of the building; and to design a responsive, elegant, simple interface; and to do it all agilely; and could you also raise a few million dollars and hire a bunch of rockstar engineers and...your old job with a steady paycheck and weekends off is starting to look pretty good right about now, isn’t it?

So why am I telling you all of this? You know how hard it is. You don’t need me going on and on about it.

Here’s the deal. I’m going to assume that you’ve already learned how hard all this can be. I’m going to make it easier for you.

Instead of pontificating about how important design is or giving you marginally relevant examples of how other companies did things, I’m just going to give you some tools to help you get out of the building, design a simple product, and validate all your assumptions. I won’t make you an expert, but I’ll teach you some useful tricks so that you can build a better product. Maybe, with a little practice, you’ll be able to build something that people actually want to use.

I’ll even throw in a few tips about how to measure your successes and failures so that you don’t keep doing the same stupid things over and over and over. Think how much time that will save.

So welcome to the world outside the building. Let me show you where to go next.

What Is Lean UX, Anyway?

Lean UX is more than a buzzword. In some ways, it’s a fundamental change in the way we design products. In another way, it’s a bunch of stuff that a lot of us have been doing for a long time.

In fact, a lot of people who have been doing user-centered design or Agile design are confused by what all the fuss is about Lean UX. There’s a reason for that. A lot of Lean UX is going to seem very familiar to people who are used to Agile design and user-centered design.

But Lean UX also introduces a few new things that aren’t found in other design practices. It takes many of the best parts of the design practices that have come before it, and it adds its own twist. Let’s look at the things that define Lean User Experience Design.

Lean UX Is About Validating Hypotheses

Much like the rest of Lean Startup, Lean UX centers around validating hypotheses. This is quite a departure from traditional design, which often revolves around fulfilling a designer or product owner’s “vision.”

Instead of thinking of a product as a series of features to be built, Lean UX looks at a product as a set of hypotheses to be validated. In other words, we don’t assume that we know what the user wants. We do customer interviews and research in order to develop a hypothesis about what a customer might want, and then we test that hypothesis in various ways to see if we were right. And we keep doing that every time we make a change to the product.

Let’s look at an example before and after applying Lean UX practices.

Before Lean UX, Company X might have said, “We need to record user comments for each product and display them on the product page.” The product manager would have spent time writing a spec for comments on product pages. A designer might have come in and designed a great comment experience. Engineers would then have written code that matched the spec and the design. The whole process might have taken two or three months.

Figure 1. The Lean loop

At the end of the process, the product pages would allow users to leave comments. But so what? Would this make users spend more money on products? Would they stick around for longer? Would users even bother to leave comments? What is the actual benefit to the company for spending those three months on building a comment system rather than on something else?

The problem with this approach is that it starts with a feature rather than a hypothesis. It assumes that comments on product pages are necessary rather than identifying a user problem, coming up with an idea that might solve that user problem, and designing a test to see if the idea is true.

Here’s a leaner way to approach the problem.

Let’s say that, instead of deciding that comments are necessary, Company X says, “We need to find a way to improve our revenue. Based on our preliminary research, we believe that allowing users to leave comments on product pages will cause customers to become more engaged and buy more items. How can we figure out if that’s true?”

Now this has been restated in a way that allows the company to validate the hypothesis and easily change the feature if it turns out to be the wrong way to improve the metric.

Once Company X has a hypothesis—that adding comments will increase revenue—the Lean User Experience Designer could figure out the best way to validate or invalidate that hypothesis. That might be something as simple as gathering some comments from users about products and adding those comments to the page manually to see if having comments on the page affects whether users buy more products.

Could the result still be adding a feature that enables user comments on product pages? Sure! The difference is that you will know whether this is the right feature to build long before you spend a lot of time and money building it. You’ll also be able to get some great customer feedback on exactly how the feature should work, so you don’t have to go back and fix it later.

Lean UX isn’t about adding features to a product. It’s about figuring out which metrics drive a business, understanding what customer problems we can solve to improve those metrics, generating ideas for fixing those customer problems, and then validating whether or not we were correct.

If that all seems rather daunting, don’t worry. I’ll share some very specific methods for doing this later in the book.

But It’s Not Just About Validating Hypotheses...

I was speaking with one startup, and they explained that they had done a small test of a particular new feature and measured their conversion funnel. About 3% of people had converted to using the feature. Unfortunately, they had no way to tell whether 3% conversion was a fantastic outcome or a complete failure because nobody had ever built this sort of feature before.

This is a trap I see a lot of Lean startups fall into: the inability to interpret whether or not a hypothesis has been validated.

Of course, when possible, Lean User Experience Design encourages designers to A/B test their changes, and A/B tests are typically quite easy to interpret—either a change makes a statistically significant difference or it doesn’t. I’ll have lots more on this later in the book. However, there are other types of tests to run, and they aren’t always as easy to interpret as a straight A/B test.

Sometimes, when you’re trying to validate hypotheses in new industries, it can be tough to know what sort of results constitute a success. With Lean UX, you must learn how to design the right kinds of tests in order to know when your hypotheses have been validated. And sometimes you just have to rely on plain old qualitative research and good judgment. We’ll cover that in later chapters.

Lean UX Is User Centered

I can’t tell you how often I’ve talked to designers who are trained in User-Centered Design (UCD) and they’ve told me, “Oh, you’re prototyping and talking to users? I do that! I must be doing Lean UX!”

Figure 2. All of these things are used in both UCD and Lean UX

And that’s partially true. Lean UX borrows heavily from UCD, largely because UCD is awesome. For example, Lean Startup didn’t come up with the idea of learning from users. Hell, some of us have been doing that for decades, and we learned from people who had been doing it even longer.

This is great, because it means that getting product feedback is not a new science. Lean teams don’t have to figure out how to learn from users on their own. There are lots of people practicing UCD and doing user research who can help guide them. In fact, if you want to learn to be a fabulous Lean User Experience Designer, you could do a lot worse than starting by getting a good grounding in User-Centered Design.

But It’s Not Just About User -Centered Design...

Whenever my UCD friends say they’re doing Lean UX because they make interactive prototypes or run usability studies, I have to correct them.

Lean UX does borrow very heavily from UCD, but it also adds many of its own flourishes. For example, UCD doesn’t really have a strong opinion about things like frequent iterations, validating hypotheses, or Agile teams. Lean does. UCD also doesn’t have any concept of measuring design outcomes in a scientific manner, while that is central to the Lean methodology.

Don’t think of Lean UX as something that is replacing UCD or that is better or worse than UCD. It’s a set of tools that includes much of what is so wonderful about User-Centered Design—mainly its relentless focus on the user.

I’m not going to teach you to be a fantastic User-Centered Designer in this book, but hopefully I’ll show you enough tricks to get your product off the ground and teach you how to incorporate great User-Centered Design into your process.

Lean UX Is Agile

After there was User-Centered Design, there was Agile Design. And Lean UX steals a lot of great stuff from Agile, too.

For example, Agile Design includes cross-functional teams where designers and developers work directly together on a project rather than treating the design department as an external agency. The cross-functional team is critical to Lean UX. By including developers, designers, and product owners in the majority of decisions, it makes the entire design process faster and easier to change when things aren’t going well.

Agile gets rid of a lot of documentation and specifications in order to be more...well, agile. Lean also eschews a lot of traditional documentation. Instead of always creating a PRD and a pixel-perfect mockup of every screen, both Lean and Agile concentrate on providing the type of documentation that is most useful for communicating design to the team. Why write a 300-page Word document when a flow diagram or a sketch will do the trick?

Perhaps most importantly, both Agile Design and Lean UX get rid of the waterfall methodology of the past where research and design could take months or even years before getting delivered to engineering as a “final spec,” which was often wrong.

Like Agile, Lean focuses on working quickly and in short cycles in order to reduce the amount of time until the team gets feedback on the product.

But It’s Not Just About Being Agile...

While Lean UX takes a huge number of best practices from Agile Design, that’s not the whole story.

Lean replaces traditional user stories that can be declared “done” by a product owner with User Hypotheses that can be validated or invalidated. In other words, a feature isn’t finished when it’s shipped to a user. A feature is often simply a method of validating whether allowing a new customer behavior is good or bad for the business.

When working in Agile Design environments, I would often be asked to check a feature in a staging environment in order to determine if it was finished and could be accepted. In Lean UX, that question can’t be answered until the feature has been used by customers and the results on key metrics have been measured. And even then, the feature isn’t really “finished.” It’s just ready for its next iteration.

It may seem like a small distinction, but the concept of measuring design output is something that has the potential to dramatically change the product design industry for the better. Unsurprisingly, I’ll cover measuring design in much more detail in later chapters.

Lean UX Is Data Driven

Speaking of data, Lean UX is solidly data driven.

One of the most important ideas behind Lean UX is that we don’t assume that a new design or feature is automatically better than what came before it. We test everything. When we ship a new feature, we test to see if it made a significant impact on important user behaviors. When we change text or the timing of an email, we test to see which version performs better. When we make a change to a user flow or a navigational structure, we make sure we haven’t inadvertently made things worse for users.

Even more importantly, we use this deploy-and-test process as a feedback loop for the designers. If we made a change that we fully expected to improve things, but it actually hurt key metrics, we use that as a basis to investigate what went wrong. Which assumptions were made that were incorrect? Which hypotheses have been invalidated?

Figure 3. You can’t learn if you don’t measure

By testing every iteration of a feature’s design, we help the designer learn more about real user behavior. We also have specific metrics that we can use to show the return on investment for design projects.

This is something that is unique to Lean UX, in my experience. Of course, designers and user researchers have been running qualitative usability tests on their products for many, many years, but this sort of quantitative measurement of the actual effect of specific design changes on shipped products was not widespread until recently, especially not at startups.

Some members of the design community have also resisted it quite vigorously because they feel that “design can’t be measured” or that somehow quantifying the impact of design detracts from the purity of the process. This, in my opinion, is utter bullshit.

The way something is designed can either improve a product’s usability and desirability or destroy it. It is in our best interest, as designers and product owners, to measure the real impact that our work has on the bottom line of our products and companies. But don’t worry. We can fight about this later in the book.

But It’s Not Just About Data...

One fairly common argument against Lean UX is that somehow all this data is removing the vision from the craft. This may be the root of why many designers are so resistant to the idea of data-driven design. They’ve heard about Google testing 41 shades of blue, and they think that all design will be reduced to this sort of “just test everything” approach.

This couldn’t be further from the truth. While it’s true that things like picking the best color for links is something that is easily done with a multivariate test, it’s also true that real design changes, not just tiny optimizations, are made by designers and people with vision and passion for a product.

Just because we are testing how our changes affect the bottom line of the product doesn’t mean that we’re not making the critical design decisions. Instead of thinking of Lean design as being driven by data, you can think of it as being informed by data. Data can tell us things like what people are doing and where people are falling out of funnels, but only good research and design will tell us why that’s happening and how to fix it.

Data-driven design isn’t about simply trying everything you can think of and picking the things that convert best. It’s about combining good design practices with good testing practices in order to create a more successful product. If you’re not convinced yet, I’ll go into the right and wrong way to test designs later in the book.

Lean UX Is Fast and Cheap (Sometimes)

Lean often gets misunderstood as a synonym for “as cheap as possible.” This simply isn’t true. Lean Startup, like Lean UX, has nothing to do with whether a company is bootstrapped or Fortune 500. A Lean Startup does not mean a cheap startup. It never has.

However, I have found that Lean UX is frequently faster and cheaper than traditional UX, almost entirely because Lean UX strives to eliminate waste.

Imagine for a moment how much it costs to do a full round of traditional user research followed by a month or two of design and usability testing before writing a line of code. I don’t actually have to imagine that. I worked at an agency that did that sort of work, and the answer is easily in the tens of thousands of dollars, and sometimes significantly more. Remember, that’s before you’ve validated your initial product idea at all.

Now, imagine that you do all that work, but after you launch your product nobody buys it, because nobody particularly likes your idea.

For example, perhaps you’ve heard of a company called Webvan. They spent somewhere in the neighborhood of $400 million building an automated warehouse system just to figure out that people weren’t yet ready to shop for groceries online.

Instead of building before you validate, imagine that you start with a few lightweight experiments to test your product hypothesis, realize that it’s absolutely wrong and that nobody will pay for it, and keep iterating and pivoting until you find something that people are interested in buying.

For example, instead of building an entire infrastructure capable of delivering groceries to millions of people, Webvan could have experimented with a static website, a limited range of products, and a concierge service. Or they could have started out by partnering with stores that already sold groceries. In fact, if they’d started out that way, they might have been able to save enough money to survive until people were ready to get all their groceries delivered.

Either way, they would have avoided spending a lot of time and money on building something nobody wanted in favor of spending a little time and money finding something people might pay them for.

I’m no economist, but I’d say that the second version of that story is a whole lot cheaper than the first one. And that’s the point, really. Lean UX saves you time and money not because you’re refusing to invest in good design. Lean UX saves you money by keeping you from spending lots of time and money on stuff nobody wants.

Lean UX still requires a solid investment in the tools and people necessary to do good design. It just keeps them from designing the wrong things.

But It’s Not Just About Being Cheap and Fast...

Lean UX is not bad UX, and building a good user experience is not cheap. You can’t refuse to spend any money on research and design and then say you’re being Lean. You’re not. You’re being stupid.

In this book, I’m going to provide you with a lot of tips that will help you move faster and get better results with less waste. But I’m not going to tell you how to design a fabulous product for free, because that’s just not possible.

Design and research still take time and money to do correctly. But by spending your time and money wisely, at least you’ll wind up with a great product that people want to use. The rest of this book will teach you how to do your research and design with less waste and how to learn the right things at the right time.

Lean UX Is Iterative (Always)

A huge part of Lean UX involves something called the Minimum Viable Product (MVP), which I’m going to talk about at length later on in the book. It gets its own chapter! The short version of the MVP definition is that you’re going to build the smallest possible thing you can in order to conclusively validate or invalidate a hypothesis.

There is a very, very important concept that many people seem to miss, however. Once you’ve created an MVP, you need to keep working on it. You see, if everything is a hypothesis that you need to validate, then once you’ve validated that hypothesis, you need to act on it.

Lean UX is all about gathering actionable metrics. Of course, the key to being successful with actionable metrics is that you must do something with them. They’re not something that just make you feel good about yourself. They are important tools for deciding what to do next.

What this creates is an incredibly iterative process, where you’re constantly building small things, learning from them, and then continuing to build—or occasionally destroy—based on what you’ve learned.

To keep the cycle going, you must keep iterating and building. I have seen too many bad Lean Startups with products that are littered with abandoned or half-built features. Their intentions were good. They were trying a lot of different things but then just leaving them to rot.

Trying new things constantly and then abandoning them without further study or work is not iterating. That’s flailing. And, more importantly, it’s what leads to wildly overcomplicated products with a weird mix of abandoned features used by a small percentage of users.

One startup I worked with had no fewer than three different ways to search its site at one time. Instead of iterating on the first version, it kept creating new versions of search with slightly different features, leading to one of the most confusing experiences I’ve ever seen for users who just wanted to search for a product. I’ll talk about ways to avoid this sort of problem throughout the book.

With Lean UX, you need to constantly be improving your product, not just by adding new features, but also by improving the experience of the features you have already built and killing underperforming features.

Nothing is ever really finished. It’s just ready for its next iteration.

But It’s Not Just About Iterating...

I’m kidding. It is just about iterating.