Criterion is a measure of the willingness of a respondent to say ‘Signal Present’ in an ambiguous situation. The choice of a criterion may depend on perceived consequences of outcomes. For example, if the consequences are costly for saying ‘Signal Present’ when the signal actually is absent, then a respondent may generally be less willing to say ‘Signal Present.’ On the other hand, if the consequences are more costly for failing to detect a signal when it is present, then a respondent may be more willing to say ‘Signal Present.’
Researchers may be interested in comparing response bias (i.e., criterion) for groups of individuals who differ in various ways. For example, are older people more reluctant to say ‘Signal Present’ when the situation is ambiguous?
Setting a criterion is equivalent to setting alpha error in a hypothesis testing situation. Alpha error is the chance a researcher is willing to take that a test will be found ‘statistically significant’ when there is no real effect. We define the Criterion as the z score on the Signal Absent distribution. A larger value of the Criterion implies that the respondent requires stronger evidence before saying that the signal is present.
Exercise 5 (Criterion):
In the SDT applet, Move the “d’ =” box in the Normal Distributions panel to obtain a value of d’ near 1.8. Now move the Criterion box to the far right.
a. Slowly move the Criterion box in the Normal Distributions panel from the far right to the far left, and back again. Explain what happens to the hit and false alarm rates as you shift criterion. How do the rates of change in hits and false alarms differ? What happens to d’ as you shift criterion?
Check Your Answers
When criterion is at the far right, the respondent will never say ‘Signal Present.’ Consequently, the hit and the false alarm rates are both zero. When the criterion is at the far left, the respondent will always say ‘Signal Present,’ and the hit and false alarm rates are both 1.00. As the criterion moves from the far right to the left, the respondent becomes more willing to say ‘Signal Present.’ When d’ is positive, as it is here, the hit rate increases more rapidly than the false alarm rate until the criterion reaches the point where the two normal curves intersect. From there on, the false alarm rate grows more quickly. d’ is not affected by changes in the criterion.
b. Use the applet to model the performance of Anita, who had hit and false alarm rates of .90 and .30, respectively. What is her d’ and her Criterion (shown in the boxes)?
Check Your Answers
Anita has d’ = 1.81 and Criterion = .52.
c. Use the applet to model the performance of Bob, who had hit and false alarm rates of .68 and .09, respectively. What are his d’ and his Criterion?
Check Your Answers
Bob has d’ = 1.81 and Criterion = 1.34.
d. Compare the performance of Anita and Bob in simple language, avoiding technical SDT terminology.
Check Your Answers
Anita and Bob have equal sensitivity to the signal, but compared to Bob, Anita will say ‘Signal Present’ at a lower level of stimulation.
e. How is the Criterion value related to the Signal Absent distribution?
Check Your Answers
When the signal is absent, the stimulation on any given trial may be more or less like a signal. The criterion represents the minimum level of stimulation needed for the respondent to say ‘Signal Present.’ The computed Criterion is the z-score of this stimulation value on the Signal Absent distribution. Thus, criterion reflects the willingness of the respondent to make a false alarm (i.e., say ‘Signal Present’ when it is absent).
f. Set d’ = 1.8, and move the Criterion to about 3. What are the hit and false alarm rates?
Check Your Answers
With d’ = 1.8 and Criterion = 3, the hit rate is about .115 and the false alarm rate is about .001.
g. Continue with d’ = 1.8 and move the Criterion to about 0. What are the hit and false alarm rates?
Check Your Answers
With d’ = 1.8 and Criterion = 0, the hit rate is about .96 and the false alarm rate is about .49.
h. If a false alarm is very costly relative to the benefit of a hit, which Criterion is better, 3 or 0, and why?
Check Your Answers
If false alarms are very costly, the higher criterion is better. The ratio of hits to false alarms is greater than 100:1, while with the second criterion the ratio is only about 2:1.
Questions, comments, difficulties? See our technical support page or contact us: wise@cgu.edu.
