Experiment!: Planning, Implementing and Interpreting

8.13 Summary

• A statistical hypothesis test answers a question that contains a statistical parameter and a word expressing a difference. “Is the sample mean different from the target value?’ is an example of such a question.
• The null hypothesis assumes that any apparent difference is due to sampling error – natural, random variation in the data. It is coupled to a reference distribution that describes all possible outcomes of the experiment if it is completely governed by chance.
• To ensure that this reference distribution is valid, the order of the experimental treatments must be randomized. Otherwise, drift in the conditions could produce apparent effects that would be wrongly attributed to the experimental treatments.
• The alternative hypothesis is the opposite of the null hypothesis and generally represents an effect that we want the experiment to reveal. If the significance test discredits the null hypothesis, the alternative hypothesis is supported and we say that the result is statistically significant.
• The confidence level is the risk of making a Type I error, or the probability of rejecting the null hypothesis when it is true. The power is the risk of making a Type II error: accepting the null hypothesis when it is false.
• The one-sample t-test is used for comparing a sample mean to a target value. The two-sample t-test is used for comparing the means of two independent samples to each other. If the samples are dependent, the paired t-test is used, which is a special case ...