Practical A/B Testing

Opting for an Equivalence A/B Test

The name says it all. An equivalence A/B test aims to conclude that two versions are the same or have no meaningful differences.

Let’s take a second to highlight the differences between equivalence and non-inferiority A/B tests. For a non-inferiority test, the goal is to prove that version A is as good as version B. However, suppose version B is worse to some degree than version A. In that case, the inferiority test will still result in a positive outcome. In contrast, an equivalence test’s objective is to prove that version A is the same as version B and that version B is the same as version A.

The beauty of the equivalence A/B test is that it pushes you to identify what change in your success metric you ...