A quantitative analysis of the improvement provided by comprehensive annotation on CT lesion detection using deep...

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Radiology

Background:

• Hospital data often has incomplete radiologist annotations due to time constraints.
• Evaluating deep learning models on partially labeled data can lead to inaccurate performance estimations.

Purpose of the Study:

• To quantitatively assess the impact of annotated lesion percentage on universal lesion detection (ULD) algorithm performance.
• To investigate how varying levels of annotation in CT images affect ULD model accuracy.

Main Methods:

• Trained a multi-view feature pyramid network with position-aware attention (MVP-Net) for ULD.
• Utilized three versions of the DeepLesion dataset: OriginalDL (partial labels), EnrichedDL (full labels), and UnionDL (combined).
• Trained separate MVP-Net models on each dataset to evaluate annotation impact.

Main Results:

• Model performance significantly dropped when tested on fully labeled data after training on partially labeled data.
• Increasing annotated lesions in training data improved sensitivity, with diminishing returns following a power law.
• EnrichedCNN and UnionCNN showed improved sensitivity (66.0%, 67.8%) compared to OriginalCNN (56.1%) on the EnrichedDL test set.

Conclusions:

• Expanded the DeepLesion dataset by annotating 21,775 additional lesions.
• Fully labeled CT images prevent overestimation of ULD algorithm performance and increase sensitivity.
• Findings have significant implications for future CT lesion detection research and development.