.We can see from the formula that two variables impact the effect size: the difference between the means of the alternative and null populations and the standard deviation of the two populations. Below are two images that may be useful for examining the impact of standard deviation and mean differences on power.
Same Standard Deviation, Different Means
 |
 |
When the difference between the means of the null and the alternative distributions is increased, the sampling distributions do not change shape, but they are further apart on the x-axis. Note that when the difference is increased, the sampling distributions overlap to a lesser extent, and it is less likely that a random sample from the alternative distribution will be mistaken for a sample taken from the null distribution. More of the sampling distribution for the alternate population (the pink distribution) is greater than the critical value (red dashed line), so it is more likely that a random sample mean will lead to rejection of the null hypothesis.
Same Mean Difference, Different Standard Deviation
When the standard deviation is smaller, the sampling distributions are narrower, but the means remain the same distance apart on the x-axis. The sampling distributions overlap to a lesser extent because they are narrower. Compare the proportion of the alternate (red) distribution that is above or below the critical value in the two scenarios.
The following questions are designed to test your understanding of the factors that affect power.
Below are key statistics for each of two new training programs, SLAM and DUNK.
Do you expect that a test of statistical significance would have greater power for the SLAM program or the DUNK program? Why? Respond to the following true/false statements. See if you can answer all statements correctly before you check your answers.
| T | F | Question | |
|---|---|---|---|
| 1j. The test of the SLAM program will have greater power because SLAM has a larger mean than DUNK. | |||
| 1k. The test of the SLAM program will have greater power because SLAM has a larger standard deviation than DUNK. | |||
| 1l. The test of the DUNK program will have greater power, because a program with smaller standard deviation has greater power. | |||
| 1m. The test of the SLAM program will have greater power because SLAM has a greater effect size. | |||
| 1n. The test of the DUNK program will have greater power than the test for the SLAM program if sample size and alpha are the same, because DUNK has a greater effect size. | |||
| 1o. Power is the same for the tests of the two programs because the samples have the same size. | |||
| 1p. Holding all other factors constant, a larger difference between the null and alternate population means will always yield greater power. | |||
| 1q.Holding all other factors constant, power is greater when the variance of the null and |
Check your answers:
1j. The test of the SLAM program will have greater power because SLAM has a larger mean than DUNK.
False. Although a larger mean contributes to a larger effect size, the effect size also depends on the standard deviation.
1k. The test of the SLAM program will have greater power because SLAM has a larger standard deviation than DUNK.
False. If everything else is the same, power is less when the standard deviation is larger.
1L. The test of the DUNK program will have greater power, because a program with smaller standard deviation has greater power.
Not necessarily true. If everything else is the same, power is greater when the standard deviation is smaller. However, power also depends on the difference between the null and alternative means.
1m. The test of the SLAM program will have greater power because SLAM has a greater effect size.
False. Check your computations for the effect size for each program. The program with the larger effect has greater power if sample size and alpha are the same
1n. The test of the DUNK program will have greater power than the test for the SLAM program if sample size and alpha are the same, because DUNK has a greater effect size
True. The effect size for DUNK is 1.0 while the effect size for SLAM is only .8, and the program with the larger effect size has greater power.
1o. Power is the same for the tests of the two programs because the samples have the same size
False. The programs differ in other ways that affect power.
1p. Holding all other factors constant, a larger difference between the null and alternate population means will always yield greater power.
True. If the standard deviations, sample sizes, and alpha levels are the same, then the difference between the population means is the only variable left that affects power. In this case, a larger difference between means indicates less overlap between the two sampling distributions, and so there is greater power to detect the difference..
1q. Holding all other factors constant, power is greater when the variance of the null and alternate populations is greater.
False. If all other factors are the same, power is less when variability is greater. This is because the sampling distributions are more spread out and so they overlap more, making it more difficult to detect a difference.
