In Exercise 1d we drew samples of 25 graduates from the DEUCE program, but in Exercise 2 we will draw samples of n = 100 and n = 4. Watch what happens and think about how you can explain how statistical power is influenced by sample size.
To simulate drawing a larger sample, enter the following values into the WISE Power Applet:
Press enter/return after placing the new values in the appropriate boxes.
For this exercise, do ten simulations of drawing a sample of 100 cases and record the results below. You don’t need to record means; just select the button for “Reject H0” or “Fail to Reject” for each of the ten simulations.
First, do ten simulations of drawing a sample of 100 cases and record the results below.
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | |
| Reject H0 | ||||||||||
| Fail to Reject |
2a. Power (as shown in the applet) =
Next, do ten simulations of drawing a sample of 4 cases and record the results below.
(Set sample size n = 4, press Enter)
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | |
| Reject H0 | ||||||||||
| Fail to Reject |
2b. Power (as shown in the applet) =
2c. How many times out of 10 did you Reject H0 for each of the two scenarios?
n = 4:
n = 100:
If nothing else is changed, power is greater with a larger sample size because:
(select True or False for each before you check your answers):
| True | False | |
|---|---|---|
| 2d. The effect size is larger. | ||
| 2e. The alpha error is smaller. | ||
| 2f. The alpha error is larger. | ||
| 2g. The population variance is smaller. | ||
| 2h. The variance of the alternate sampling distribution is smaller. | ||
| 2i. More of the alternate sampling distribution exceeds the critical value. |
Check your answers:
2d. The effect size is larger.
False. The alpha error rate that is set by the researcher does not change with sample size, although the p-value of the test may change.
2e. The alpha error is smaller.
False. The effect size is the difference between the null mean and the alternate mean divided by the standard deviation. These values are not affected by the sample size.
2f. The alpha error is larger.
False. The alpha error rate that is set by the researcher does not change with sample size, although the p-value of the test may change.
2g. The population variance is smaller.
False. The population variance is not affected by the sample size. The variance of the sampling distributions of the means is smaller when the sample is larger, but the population variance is a constant value.
2h. The variance of the alternate sampling distribution is smaller.
True. With larger n, the variance is smaller for both the null and alternate sampling distributions, leading to less overlap and greater power.
2i. More of the alternate sampling distribution exceeds the critical value.
True. The mean of the alternate sampling distribution is greater than the critical value, and with a larger sample the variance of the sampling distribution is smaller, so more of the sampling distribution exceeds the critical value.
We have seen that the power of a statistical test increases as the sample increases. Let’s review the reasons behind this increase in power. You may have noticed that the sampling distribution of the mean was much wider when the sample size was reduced from 100 to 4. This occurred because sample means are much more variable when the sample size is small. Regardless of sample size, the average sample mean, taken across infinite samples, would be equal to the population mean. However, the precision of sample means is less when the sample size is small. This can be observed as the greater variability in sampling distributions for the sample size of 4. When the sampling distributions are more variable, it is more likely that a sample taken from the alternative distribution will be mistaken for a sample taken from the null distribution. This mistake leads to failure to reject the null hypothesis, which corresponds to lower power. The figures below show sampling distributions for samples of 100 compared to samples of 4. Although the difference between the means is the same, there is much less overlap of the sampling distributions for the larger samples.
 |
 |
2j. Explain to a classmate why statistical power increases as the sample size increases.
