Illuminating Statistical Analysis Using Scenarios and Simulations

26Sample Size, Variance, and Uncertainty

Statistical Scenarios—Assorted Sample Sizes and Variances

Let's review:

With our investigation, what happens as we increase or decrease our sample sizes?

What happens if the sample variances happen to be higher or lower?

Let's see how various changes to the “base case” of the previous chapter impact the original -value of 0.04. We'll change one aspect at a time, leaving all the others as they were. The formulaic calculations and the simulation histogram are given for each case. For all the simulations, we'll assume the null hypothesis of equal population means for females and males is true, so the difference in the population means is zero. We'll make both the males' and females' population means equal 3.0. We want to see how frequently we should expect to get two sample means that are 0.5 apart or more simply by chance when the two population means are in fact equal. The uncertainty, as usual, is driven by the variances and sample sizes. We'll use an alpha-level of 0.05 (95% confidence level) for declaring statistical significance. Let's start by reviewing the base case.

(A) Base case.

The sample variances are 3.1 and 2.9, and the sample sizes are 100 and 100.

1. for standard error
2. standard errors fit into the actual difference ...