We are interested in whether program B graduates score better than 555 on the GRE Quantitative section. We start with the assumption that Program B graduates score the same as people who did not participate in a training program. We can use a z value to test this assumption.
We assume that program B's mean is 555. To reject this assumption, we need to demonstrate that our sample results would be surprising if our null hypothesis is true. Surprising is, of course, a vague term.
What qualifies as "surprising?"
Often, results are considered surprising if a sample mean this far or farther from the hypothesized mean would occur less than 5% of the time by chance (or less than 1% of the time if we wish to be more conservative). These values are called alpha levels. Alpha levels refer to a criterion of oddity, telling us how surprising a result has to be in order to lead us to reject our null hypothesis (initial assumption).
In the previous example, you judged a result as surprising if it would have occurred less than 5% of the time given that the null hypothesis is true. When a result is termed "surprising," we reject the null hypothesis. If we reject the null hypothesis we are claiming that it is unlikely that the mean score for the population of all graduates from the training program is 555.
Another Question
Now, suppose that we sampled 10 students from Program B, and that these 10 students scored an average of 645 on the GRE quantitative section. This yields a z score of 2.05. Given the null hypothesis above and an alpha level of .05, what do you conclude? (Note: you should refer to your z-table for this problem).
a) Population mean for Program B may be 555 (null hypothesis cannot be rejected). This is called failing to reject the null hypothesis.
b) Population mean for Program B is not likely to be 555 (null hypothesis is not likely). This is called rejecting the null hypothesis.