A constrained formulation for the receiver operating characteristic (ROC) curve based on probability summation.

Area of Science:

• Radiology
• Medical Imaging Analysis
• Statistical Modeling

Background:

• Receiver Operating Characteristic (ROC) curves are standard for evaluating diagnostic accuracy.
• Conventional ROC formulations can produce unrealistic curve shapes, especially with limited false-positive data.
• Probability summation is a key mechanism influencing observer performance.

Purpose of the Study:

• To propose and validate a principled, constrained formulation of the ROC curve.
• To compare the constrained ROC formulation against the conventional one using real observer data.
• To ensure realistic curve fitting and reliable accuracy estimation.

Main Methods:

• Fitted both conventional and constrained binormal ROC formulations to 150 datasets from observer studies.
• Data involved chest radiograph interpretation by 20 readers for 5 abnormalities.
• Used maximum-likelihood procedures with normally distributed latent variables to estimate ROC curves and Area Under the Curve (Az).

Main Results:

• Both ROC formulations performed similarly on symmetric data.
• Conventional ROC curves showed unrealistic upward "hooks" with asymmetric data (few false positives).
• Constrained ROC curves avoided hooks, estimated larger Az values with smaller standard errors, and guaranteed realistic fits.

Conclusions:

• The constrained ROC formulation offers a realistic and reliable alternative to conventional methods.
• It accurately describes observer ratings and guarantees realistic performance curves.
• Estimated parameters are interpretable and useful for predicting localization accuracy.