WISE Power Tutorial – Discussion Questions for Statistical Power
Your friend Bumble plans to conduct a study to determine whether people who are exposed to an advertisement for a power drink have a more favorable view of that product compared to people not exposed to the advertisement. He has some questions and asks for your help.
1. Bumble says, “I know that statistical power is supposed to be larger if I increase the sample size, but I don’t understand why. Can you explain why sample size is related to power?”
2. Bumble asks you, “Would a sample of 50 people be large enough?” How do you respond? What more do you need to know and why?
3. Bumble says, “I really would like to design my study to be able to show that my advertisement is effective. How many cases do I need to make SURE I can reject the null hypothesis?” How do you respond?
4. Bumble says, “I don’t want to make an alpha error where I conclude that my advertisement is effective when it really isn’t effective. If I set my alpha error smaller (.001 instead of .05), will that give me much more power?” Explain to Bumble how alpha error is related to power.
5. Bumble says, “I think the advertisement will be more effective if I show it to people three times instead of just once.” If Bumble is right, how would power be affected?
6. Bumble conducted a study of his advertisement for a power drink, and he found that those who saw the advertisement had a significantly more favorable view of the product than those who did not see the advertisement, p = .001. Bumble concluded that he had strong evidence that the advertisement produced a very large and important effect. Evaluate Bumble’s conclusion and interpret his findings.
(Hint: Consider the distinctions between statistical significance, practical significance, and large effect size. How is sample size related to statistical power? Is a statistically significant effect necessarily an important effect? Can one obtain a statistically significant effect with a small effect?)
