The receiver-operating characteristic (ROC) is a graphic representation of the relationship between the underlying Signal Absent and Signal Present distributions. This fundamental signal detection graphic is essentially a curve fitting a scatterplot that shows the relationship between false alarm rates on the x-axis, and hit rates on the y-axis.
Exercise 6: ROC
To see how ROCs are formed, you need to look at both the Normal Distributions and the ROC panels (the leftmost and center panels in the SDT applet). On the ROC panel, there is a yellow dot on the ROC curve which is connected to the Criterion box on the Normal Distributions panel. This point marks an observer’s hit rate and false alarm rate based upon this individual’s current criterion.
a. Move d’ from 0 to 3, and note how the ROC changes. Describe how the ROC reflects sensitivity.
Check Your Answers
All ROCs connect the lower left corner where the hit and false alarm rates of both zero, with the upper right corner where hit and false alarm rates are both 1.00. When d’ = 0, the ROC is a straight line from the lower left to the upper right. This is because the signal and noise distributions overlap perfectly, and so the hit and false alarm rates are equal to each other at every criterion. If we reduce criterion in an effort to increase hits, we find that false alarms increase exactly as much as the hits. When d’ = 3, the hit rates rise much more quickly than false alarm rates as the criterion moves toward the left from the extreme right. Thus, the ROC curve rises very quickly toward the top left corner. As the criterion becomes more liberal (smaller, to the left), the false alarm rates begin to increase. When the criterion crosses the intersection of the two normal curves, the false alarm rate rises more quickly than the hit rate. With greater sensitivity, the ROC is higher and to the left. Sensitivity can also be measured by the area between the ROC and the diagonal line. This area is larger when d’ is larger, and it is zero when d’ is zero.
b. Compare the ROCs generated by Anita and Bob. Anita had a false alarm rate = .3 and hit rate = .9 while these rates for Bob were .09 and .68, respectively.
Check Your Answers
Anita and Bob each generate one point on the same ROC. They have the same ROC, and thus the same d’ value.
c. Chris has the same hit rate as Anita (.90) and the same false alarm rate as Bob (.09). Find d’ and criterion for Chris. Compare the performance of Chris to that of Anita and Bob (recall that Anita and Bob both had a d’ = 1.81 and criterion of .52 and 1.34, respectively).
Check Your Answers
For Chris, d’ = 2.62 and criterion = 1.34. Thus, Chris has the same criterion as Bob. However, Chris is more sensitive than Bob, as reflected by the larger d’ and the more convex ROC.
Questions, comments, difficulties? See our technical support page or contact us: wise@cgu.edu.
