UX for Business

Probability

Don’t Be Lucky; Be Smart

One of the most shocking things any designer ever said to me was that I was really lucky to get so many good A/B test results. I didn’t know what to say. Instead, I started asking questions and soon understood that he thought A/B tests were a way to test your guesses. He thought I was guessing because he was guessing. He had no idea whether a new design would be better or worse and was relying on the A/B test to measure his luck.

Good UX is not a matter of lucky guesses. Guessing is not design. Design gets results intentionally. I hope, after reading this far, you are already starting to agree.

If you understand value in the business model and you have diagnosed all the big problems, there is one general principle that will ensure you get the most out of your designs: luck. Or the technical term for luck: probability. Your designs won’t be better because you are luckier, they will be better because you understand how luck works in UX.

All Things Being Equal, What Will a User Click?

Instead of relying on luck, let’s break down the behavior of one click to understand how it works.

Imagine a menu down the left side of a website, a very common design pattern. Maybe it is a list of eight different categories of fireworks. Which one will a user click?

If you don’t understand probability, you might look at the categories and guess why one kind of fireworks will be more popular, or something like that. But I can tell you that statistically, the first item in the menu is probably getting the most clicks.

What?! But we don’t even know what website we’re talking about or which things are in the menu! How can we possibly know that?

Either I am a wizard or I just know how luck works.

Unfortunately, I am not a wizard.

Probability Intuition: You Can’t Do the Second Thing Before You Do the First Thing

Let’s deconstruct that one menu click and understand what happens in that moment.

Think about a user who also doesn’t know which menu item to pick (i.e., every first-time user ever). They see a list of options, but do they see all of the options at the same time? Nope. They read the first one first, then the next one, then the next one.

Right?

And if they see a menu option that sounds like it might be the one they want, do they keep reading? No! In real life, they just click the first option that seems reasonable. So which item is seen by the most users? The first one!

When a million people do that, you get the most clicks on the first menu item, second most on the second, third most on the third, and so on. Because the only people who see the last item in the menu are the people who have not clicked all the other menu items on the way down.

That’s probability. The chance of something happening. The chance of less people clicking more than more people is pretty low.

Effort and Time Both Work This Way

Imagine that we ask 1,000 users to complete a form. Will more people answer 5 questions or 10 questions? Similar to the menu example before, everybody who answers 10 questions has also answered 5 questions because you can’t get to the 10th question without going through the first 5.

This is the reason usability exists. Because the less effort you need to do something, the more people will be willing to do that thing. It is a fact that more people will do less effort than more effort, because everybody who is willing to do more effort is also willing to do less effort.

One more example, to really make sure this idea is clear: time. In Google Analytics, one useful graph (a histogram) shows how long visitors stayed on your site. There is a bar in the graph for 0–10 seconds, 11–30 seconds, 31–60 seconds, and so on.

That graph always has roughly the same shape. Why? Probability, of course.

The bar representing the people who stayed for 0–10 seconds is usually the biggest. Even if a typical user spends 10 minutes on the site, they have to go through 10 seconds on the site first, before continuing for another 9 minutes and 50 seconds, on average. It is easier to spend less time than more time.

Incentives (Motivations) Also Work This Way!

Psychological probability!

If I ask 100 people to mow my lawn, a few suckers nice people might do it. If I also offer them a thank-you dance, more of them will do it (probably). Who doesn’t want to see me do a thank-you dance?!

If we offer people good emotions or to relieve negative emotions, it motivates them. Or we could say it increases the probability that they will act. When we offer negative emotions, we don’t get a lot of volunteers. In other words, it decreases the probability.

Incentives are simply emotional reasons to act (or not). Prizes! Appreciation! Fame! Exclusive access!

In UX we sometimes have the great power/responsibility to design incentives. No matter how many people would post on social media for rational reasons, more people will post to get likes and followers and feel good about it.

Likes and followers are features, designed by a UX person; likes and followers have changed the world.

“Yeah, but Who Would Do That?”

You don’t need a deep mathematical understanding of probability to be a good designer. You just need a feel for it. We will see more examples throughout the book (which will make it clearer), but I want to introduce one more effect of probability first: the chance of a real user experiencing something.

The more users you have, the more often anything can happen. The opposite is also true. A 1% chance of a user deleting a file by accident might sound small, and if you only have 10 users, you might not see that problem very often. However, if you have 100 million users, an unlucky user is probably deleting their file right now!

Frequent usage also increases probability. If users work with files every day, it is almost a guarantee they will accidentally delete a file a couple times per year. What a pain in the ass! Once-a-year users still might do it eventually, but the chance is smaller overall.

A common debate in UX is: How many users do you need to test? It’s only a debate because a lot of people don’t understand probability. They think about user testing backward. When you do a survey, for example, you need a lot of answers because you want the results to represent a broad population of people. But user testing isn’t trying to represent the population of users; it’s trying to find the most common problems! Five testers are very likely to uncover any problem that affects at least 30% of users, because there is a high chance it will affect one of your five testers. The more you see that problem among five testers, the bigger the problem is! In fact, if all five users have the same experience, good or bad, more than 70% of all users will have a similar experience. The less likely the problem or the more accurately you want to measure something, the more users you need.

UX Is a Numbers Game

One way to think about UX is that we are designing and measuring behavior. UX, fundamentally, really isn’t about the pixels and devices; it is about the way people behave in a designed environment.

Unfortunately, with so many users, so far away from us, we can’t really see what everybody is doing. UX is mostly invisible in that way. Which is why we need to measure everything! Data makes UX visible.

For that reason, I personally don’t think anyone can be a very effective UX designer without comparing the visual design patterns with the data patterns.

Data + Understanding Probability = Insights

With data, UX is suddenly visible, in the form of numbers. When we can see the behavior that our designs cause (think: symptoms), we can use that to improve our designs.

You might think, “I do UI design, so my UX is visible!” Not so fast, pixel pusher. Whether you designed it or not, visual UI designs only give you half of the information you need for UX work, and it’s not the reliable half. Without data and user feedback, you’re still guessing!

But data isn’t the whole enchilada either. If we only look at the data, we still don’t know what that data means. To close this information gap, we should always compare the visual UI designs with the data and user feedback to see if they make sense together.

For example, if your menu items are ordered 1, 2, 3, 4, 5 on the screen, but the data says the popularity of those items is 1, 3, 2, 5, 4, that’s interesting! Are 3 and 5 extra popular, or are 2 and 4 not as popular as they should be? Since you understand probability, you recognize that the design and the data are “out of sync” with each other.

Like everything in the VDP framework, we need the user side and the business side to make sense together. Then we can ask why.

More Users, More Predictable

Your intuition might say that user data gets more complicated with more users, but actually no!

If your menu items are 1, 3, 2, 5, 4 in the data, but you only have 10 users, you can’t draw any conclusions yet. One guy might click 3 and 5 every day and throw everything off because you’re only measuring 9 other users.

Less users = less confidence in your data. Emotionally and statistically.

But with 10 million users, that guy can click 3 and 5 all day, every day, and it won’t matter. The behavior of 9,999,999 other users will be so much bigger that guy will be irrelevant overall. Either the data will stabilize at 1, 2, 3, 4, 5 like you expect, or you can feel confident that 1, 3, 2, 5, 4 is the true order of popularity for your menu items.

More users also means more niche use cases, but we’ll cover that later.

Probability Is How We Optimize

All of these probability-based principles add up to one big type of UX work: optimization. Conversion Rate Optimization (CRO) is a job title in UX for people who specialize in this type of design analysis. But even on products and services as different as B2B software, social networks, and games, the same principles of probability apply. We use them to ensure that we create the most value, as often as possible.

Probability isn’t about making something work; it’s about making it work better:

• Redesign menus, sequences of steps or clicks, and anything else that has an order to get more attention on the valuable stuff.

• Reduce effort and increase positive incentives to get more people to try or finish creating value.

• Measure the important stuff so we can see behavior and compare it to the data patterns we expect to see.

Suddenly, you might feel a little luckier!

Design Is Redesign: Your Second Try Is Often Luckier Than the First

When you approach design with your probability hat on, you will notice that the second time you design something it often works better. The reason is another consequence of this whole VDP thing: you can take advantage of new information.

After designing something, launching it, measuring it, and watching people use it, you now understand much more about your design than you did while you were making it. You have more information. Your diagnosis will be better, and you will realize where you can improve the probability of creating value. So, you will!

Design is a process, but not just a one-time process. We iterate! That means we do it again and again as we learn more. If you think your second version is good, wait until you see the genius ideas that appear for your sixth version!

There Is a Lifetime of Nuance to This

There are many other factors that combine with probability to determine what people will click. For example, a user must be ready to make a decision before they can make a decision. Sounds obvious, but if you have an ecommerce store, maybe the buy button shouldn’t be on the first page. Users might not be ready to buy yet! Instead, it should be on the page where they first get the information they need to decide to buy.

Another version of UX probability is: How many users does a problem affect? Sure, maybe it is just a little problem, but if 100 million users will have that little problem every day, the numbers say it is a big problem! The lack of an Edit button on X comes to mind. It’s not one typo. It’s billions of typos. But there is a business reason (and maybe technical reasons) not to fix it, so they don’t.

And yet another version of UX probability is, What is the default? If option A is selected by default, then you can bet your dog it will be chosen by more people than option B, which must be actively selected.

Probability is everywhere. We will look at more scenarios throughout the book.

You will need to research all the nuances of your users to understand which problems or opportunities should be seen by the most users, the most times, and have the biggest effect. But you can always trust that approach.

If probabilistic design doesn’t work, you’re probably ignoring the real probability that is affecting the user.

Probabilistic Design, in Summary

Using probability in design isn’t always a purely statistical thing, like an A/B test or a survey sample size. Much more often you will need a sense of what is more likely to happen and what is less likely to happen.

Go through your design and consider how you have structured your menus, ordered your content, laid out your buttons, and how much effort and time it takes the user to see, find, use, or complete everything that is required. How many times will it happen, and how many users will it affect?

Make sure the most valuable actions and information are at the top, or first in the sequence, or the fewest clicks from where the user lands by default. Make sure the most painful mistakes are hard to do accidentally.

And think beyond the visual interface. Make sure good and bad feelings are aligned with the user value and business value you’re trying to create. Efficiency problems minimize effort, and entertainment problems maximize good feelings.