Suppose we wish to determine the statistical power of a test of the null hypothesis that the population mean is 100 if in fact the population mean is 105 with a population standard deviation of 10, and we use a sample size of 25 cases and set alpha error to .05 for a one-tailed test.
Statistical power is equal to (1 – beta error), so to find statistical power we can solve for Zβ.
We can rearrange the terms in Formula 1 to solve for Zβ :
Using the BEAN acronym, we wish to solve for B because power is (1 – beta error). We need to specify the other three terms: E, A, and N.
Effect size d = (μ1 – μ0) / σ = (105 – 100) / 10 = 0.50.
Alpha error rate is set at .05, which corresponds to a Z score of 1.645 for a one-tailed test.
N is set at 25.
Thus, Zβ = 1.645 – 5 (0.50) = 1.645 – 2.500 = -0.855
Zβ corresponds to the Z score for the critical value on the pink (alternate hypothesis) sampling distribution. We will reject the null hypothesis if we observe a score greater than this critical value. We consult a Z table or the WISE p–z converter applet to find that the probability of observing a Z score greater than ‑0.855 is .804. This is the value given by the WISE power applet.
Thus, in this scenario our statistical power is about 80%. If we collect data and conduct a test of statistical significance, there is an 80% chance that the test will attain statistical significance, and a 20% chance that the test will fail to detect statistical significance.
Sample Size Calculation
We can rearrange the terms in Formula 1 to solve for n.
Suppose we wish to have a sample large enough to have power of 90% to detect a difference between means of 4.0 where the standard deviation is 10.0, using one-tailed alpha error rate of .01. Recalling BEAN, we need to specify B, E, and A to solve for N.
- B: Zβ corresponds to the Z score on the pink distribution where 90% of the distribution falls above that score. We can use the WISE p–z converter or a Z table to find Zβ = -1.282.
- E: The effect size d = (μ1 – μ0) / σ = (4.0 / 10.0) = 0.40.
- A: The alpha error rate of .01 corresponds to Zα = 2.326.
Applying the formula for n:
Thus, we need a sample size of about 82 to attain the desired level of power in this scenario.
It is important to note carefully that the sign on Zβ is often negative. Subtracting a negative value is equivalent to adding a positive value.