Lean Mobile App Development

Keep the differences between variations subtle

The difference between A and B in our example is very subtle, and that is for a reason. If the difference between the two variations is too big, you will not know what it is that you are testing:

The preceding is a brilliant example of what not to do. If your onboarding split test shows that variant A leads to a 61% conversion and variant B leads to a 66% conversion rate, then what does the outcome prove? Not only is the difference in conversion percentages not very convincing, it also is not clear what has led to a slightly better conversion. Was it the background color that did the trick? Or ...