Risk-adjusted training and evaluation for breast cancer detection

Breast cancer detection, and broadly medical object detection, revolves around discovering and rating lesions. One of the most common ways of measuring performance is FROC (Free-response Receiver Operating Characteristic), which calculates sensitivity at predefined thresholds of false positives per case. However, depending on the clinical context, not all lesions might be of equivocal impact on the long-term outcome of a patient. Some lesions missed e.g. in screening might be detected in the subsequent screening round without impacting the clinical prognosis, whilst missing others might significantly detoriate prognosis and treatment pathways. It is therefore desirable to develop and include consideration of clinical prognosis/risk imbalance in the way machine learning models are developed and evaluated. In this work, we propose risk-adjusted FROC (raFROC), an adaptation of FROC that constitutes a first step on reflecting the underlying clinical need more accurately. Experiments on two independent breast magnetic resonance imaging (MRI) datasets with a total of 1535 lesions in 1735 subjects showcase the clinical potential of the proposed metric and its advantages over traditional evaluation methods. Additionally, by utilizing a risk-adjusted adaptation of focal loss (raFocal) we are able to improve the raFROC results and patient-level performance of nnDetection, at no expense of the regular FROC.