As the researcher, we decide on the value of alpha, typically at .05 or .01. Alpha is the error rate we are willing to accept for the error of rejecting the null hypothesis if it were true. We require stronger evidence to reject the null hypothesis if we set alpha at .01 than if we use alpha of .05.
The figures show how power and alpha error are related. When we reduce alpha error (represented by the dark blue area of the null distribution to the right of the red dashed line), we also reduce power (the pink area in the alternate distribution to the right of the dashed line).
For this example, use one-tailed alpha α = .01 (z = 2.326). In this case, we will reject the null hypothesis only if a sample mean is so large that it would occur less than 1% of the time given the null hypothesis is true. You do not need to draw additional samples for this problem; you can use the data recorded for samples drawn in Exercise 1 (μ0 = 500, σ = 100, n = 25, α = .05 , z = 1.645).
Data from Exercise 1
ACE Program (µ1 = 580)
| Trial | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
|---|---|---|---|---|---|---|---|---|---|---|
| Mean | 579 | 574 | 594 | 600 | 541 | 585 | 578 | |||
| Z-Score | 3.96 | 3.72 | 4.69 | 4.99 | 2.04 | 4.23 | 3.92 |
DEUCE Program (µ1 = 520)
| Trial | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
|---|---|---|---|---|---|---|---|---|---|---|
| Mean | 509 | 511 | 513 | 502 | 492 | 513 | 533 | |||
| Z-Score | 0.44 | 0.54 | 0.65 | 0.11 | -0.41 | 0.65 | 1.63 |
3a. Using alpha of .01 instead of .05, how many times could you reject the null hypothesis for your results in Exercise 1? (How many times is Z > 2.326?)
| α = .05 (from #1) | α = .01 | |
| Reject for ACE Program (μ1 = 580) |
||
| Reject for DEUCE Program (μ1 = 520) |
3b. What is the power for each of these tests? You can use the applet below to calculate power for the tests using alpha α = .01. (Set n = 25 and μ0 = 500 for all tests ; use μ1 = 580 for ACE and μ1 = 520 for DEUCE). Remember to press ‘Enter’ after each change to the applet.
| α = .05 (from #1) | α = .01 | |
| Power for ACE Program (μ1 = 580) |
.991 | |
| Power for DEUCE Program (μ1 = 520) |
Check your power calculations
Answer: With alpha = .01, power for the ACE program is .953, and for the DEUCE program power is .092.
If nothing else changes, why is power less when alpha is .01 than when alpha is .05?
Answer: When alpha is .01 instead of .05, the criterion is moved toward the tail of the null distribution, requiring a larger sample mean to reject the null hypothesis. Less of the sampling distribution for the alternate population falls above the criterion, making it less likely that we will observe a sample mean large enough to reject the null hypothesis.
