Escaping the Build Trap

Chapter 16. Understanding the Direction and Setting Success Metrics

At Marquetly, I was working with the VPs of product, Karen and Joe, and we were figuring out how to quantify their product initiative in order to take it to Jen.

“Now let’s go back over the data that we already have,” I suggested. “What are our current retention and acquisition rates?”

Joe pulled up the data that the team had gathered during its exploration of product initiatives. “We currently retain users at a rate of 40% after six months. That isn’t great.”

“No, that’s definitely not great because we’re also planning to spend more acquiring users in the coming months. That means we’ll be burning through some money if we can’t retain them,” said Karen.

“Okay, so we know those numbers. Now let’s look at our problem. We found that users wanted more course variety. What percentage of them, roughly, do we think want that?” I asked them.

“Well,” Karen said, “we have two points of data we can pull from to approximate that. We left Qualaroo running on our site for about a month, and we’re finding that, of the people who answered, about 55% of them said they were looking for more course variety. We now have statistical significance around that number, too. So that means we are potentially losing out on 82,500 people every month signing up. Not all of those people are going to convert, but the upside to solving this problem is high.”

“And that’s only part of the potential, too,” I said. “Let’s look at those numbers for ...