Statistics for Data Science and Analytics

8More than Two Samples or Categories

In previous chapters, we looked at the popular A/B test and how to test for statistical significance of the results. In this chapter, we will see how to compare multiple samples and whether they differ from one another. After completing this chapter, you should be able to:

• Construct an RC table to compare categorical data across multiple categories.
• Perform a chi-square test to assess whether categories differ in a statistically significant way.
• Construct a table to compare numeric data across multiple samples or treatments.
• Conduct ANOVA to test whether multiple samples with numeric data differ in a statistically significant way.
• Explain how the problem of testing for statistical significance becomes more complex with multiple comparisons.
• Explain how multi-arm bandits deal with A/B and multiple testing from an optimization perspective.

8.1 Count Data—RC Tables

The field of behavioral economics has called into question many truisms of classical economics, such as the idea that demand rises as prices go down. The business consultant McKinsey and Company recounts the story of a jewelry store owner trying to sell a line of jewelry that was proving difficult to move.

Several strategies accomplished nothing, and the owner asked the staff to ...