3. On Method
A/B and Multivariate Testing
We’ve talked about an approach to experimentation. In this chapter, we’ll look at two methods you can use to actually run your experiments online: A/B testing and multivariate testing (MVT).
What are A/B and MVT?
A/B and MVT are methods for running controlled experiments online. Programmer Jesse Farmer describes it as a cage match: send in your champion against a challenger (or several challengers) and out comes a winner.
These methods let you compare alternative versions of the same page or element running simultaneously on your website by randomly assigning visitors to different groups, then comparing the data to see which version produces the best outcome for your goal, for example, increased conversion. ...