Reproducibility--an important factor determining the quality of computer-aided detection (CAD) systems

Area of Science:

• Radiology
• Medical Imaging
• Artificial Intelligence in Medicine

Background:

• Mammography is a key tool for breast cancer screening.
• Computer-aided detection (CAD) systems aim to improve mammography interpretation.
• Reproducibility of CAD system markings is crucial for clinical adoption.

Purpose of the Study:

• To evaluate the reproducibility of markings generated by a commercial computer-aided detection (CAD) system on mammography films.
• To assess the sensitivity of the CAD system in detecting malignant lesions, including microcalcifications and masses.

Main Methods:

• One hundred unilateral mammography examinations with histopathologically proven malignant lesions were retrospectively scanned three times using a CAD system.
• Reproducibility and sensitivity (strict and broader sense) were determined for microcalcifications and masses.
• The study included cases with suspicious microcalcification clusters, masses, or both.

Main Results:

• Only 18% of mammography examinations showed identical CAD markings across all three scans, indicating low reproducibility.
• The CAD system achieved strict sensitivity of 23.4% for microcalcifications and 51.1% for masses.
• False negative rates were 12.8% for microcalcifications and 9.6% for masses.

Conclusions:

• The reproducibility of the evaluated CAD system is currently insufficient for routine clinical use.
• Improvements in CAD system characteristics are needed to enhance its value as a 'second reader' in mammography interpretation.
• Further development is required to ensure reliable and consistent CAD performance in clinical practice.