Risk-adjusted training and evaluation for breast cancer detection

Area of Science:

• Medical imaging analysis
• Machine learning in healthcare
• Radiomics and quantitative imaging

Background:

• Current breast cancer detection models use Free-response Receiver Operating Characteristic (FROC) for performance evaluation.
• FROC does not account for the varying clinical impact of missed or detected lesions.
• A need exists to incorporate clinical prognosis and risk imbalance into machine learning model evaluation for medical object detection.

Purpose of the Study:

• To propose a novel risk-adjusted FROC (raFROC) metric for evaluating breast cancer detection models.
• To better reflect the clinical significance of lesions in model performance assessment.
• To improve the clinical utility of machine learning in medical imaging.

Main Methods:

• Developed risk-adjusted FROC (raFROC) by adapting the standard FROC methodology.
• Implemented a risk-adjusted adaptation of focal loss (raFocal) for model training.
• Evaluated the proposed methods on two independent breast MRI datasets comprising 1535 lesions in 1735 subjects.

Main Results:

• The proposed raFROC metric demonstrated clinical potential and advantages over traditional evaluation methods.
• Utilizing raFocal improved raFROC results and patient-level performance of the nnDetection model.
• Performance improvements were achieved without compromising standard FROC evaluation.

Conclusions:

• raFROC offers a more clinically relevant evaluation framework for breast cancer detection and medical object detection.
• Incorporating risk adjustment in both evaluation metrics and loss functions enhances model performance and clinical applicability.
• The proposed methods represent a significant step towards more accurate and clinically meaningful AI in medical imaging.