Observer Evaluation of Computer-Aided Detection: Second Reader versus Concurrent Reader Scenario

摘要

We are comparing the performance of computer-aided detection (CAD) used as a second reader to concurrent-use CAD. We have designed a multi-reader multi-case (MRMC) observer study using fixed-size mammographic background images with fixed intensity Gaussian signals added in two experiments. A CAD system was developed to automatically detect these signals. The two experiments utilized signals of different contrast levels to assess the impact of CAD when the standalone CAD sensitivity was superior (low contrast) or equivalent (high contrast) to the average reader in the study. Seven readers participated in the study and were asked to review 100 images, identify signal locations, and rate each on a 100-point scale. A rating of 50 was used as a cutpoint and provided a binary classification of each candidate. Readers read the case set using CAD in both the second-reader and concurrent-reader scenarios. Results from the different signal intensities and reading paradigms were analyzed using the area under the Free-response Receiver Operating Characteristics curves. Sensitivity and the average number of FPs/image were also determined. The results showed that CAD, either used as a second reader or as a concurrent reader, can increase reader sensitivity but with an increase in FPs. The study demonstrated that readers may benefit from concurrent CAD when CAD standalone performance outperforms average reader sensitivity. However, this trend was not observed when CAD performance was equivalent to the sensitivity of the average reader.