Lean Analytics

Chapter 28. What to Do When You Don’t Have a Baseline

We’ve tried to describe some useful baselines. But if you’ve read through the past seven chapters, you’ll know that these numbers are rudimentary at best: you want churn below 2.5%; you want users to spend 17 minutes on your site if you’re in media or UGC; fewer than 2.5% of people will interact with content; 65% of your users will stop using your mobile app within 90 days. For many metrics, there’s simply no “normal.”

The reality is you’ll quickly adjust the line in the sand to your particular market or product. That’s fine. Just remember that you shouldn’t move the line to your ability; rather, you need to move your ability to the line.

Nearly any optimization effort has diminishing returns. Making a website load in 1 second instead of 10 is fairly easy; making it load in 100 milliseconds instead of 1 second is much harder. Ten milliseconds is nearly impossible. Eventually, it’s not worth the effort, and that’s true of many attempts to improve something.

That shouldn’t be discouraging. It’s actually useful, because it means that as you approach a local maximum, you can plot your results over time and see an asymptote. In other words, the rate at which your efforts are producing diminishing results can suggest a baseline, and tell you it’s time to move to a different metric that matters.

Consider the 30-day optimization effort for a site that’s trying to convince visitors to enroll, shown in Figure 28-1. At first, out of over 1,200 ...