The Three-Class Annotation Method Improves the AI Detection of Early-Stage Osteosarcoma on Plain Radiographs: A Novel...

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Oncology

Background:

• Developing high-performance artificial intelligence (AI) models for rare diseases is hindered by limited data availability.
• Osteosarcoma detection on plain radiographs presents a significant challenge for AI due to data scarcity.
• Conventional single-class annotation methods may not fully capture the nuances required for accurate rare disease identification.

Purpose of the Study:

• To evaluate a novel three-class annotation method for AI training data preparation.
• To compare the performance of a three-class (3C) annotation model against a conventional single-class (1C) annotation model in detecting osteosarcoma.
• To determine if enhanced annotation strategies can improve AI model performance in rare disease detection with limited data.

Main Methods:

• Two annotation methods were developed for a dataset of 468 osteosarcoma and 378 normal radiographs.
• A conventional single-class (1C) annotation and a novel three-class (3C) annotation (labeling intramedullary, cortical, extramedullary tumor components) were applied.
• Identical U-Net-based AI architectures were used, with performance evaluated on an independent validation set.

Main Results:

• Both models achieved high diagnostic accuracy (AUC: 0.99 for 3C vs. 0.98 for 1C).
• The 3C model demonstrated superior operational characteristics, maintaining balanced sensitivity (93.28%) and specificity (92.21%) at a cutoff of 0.2, unlike the 1C model's compromised specificity (83.58%).
• The 3C model maintained diagnostic accuracy at substantially lower thresholds, showing identical false-negative rates at the 25th percentile despite significantly different cutoff values.

Conclusions:

• Anatomically informed three-class annotation enhances AI model performance for rare disease detection without requiring additional training data.
• The 3C annotation method improves AI model stability at lower thresholds, optimizing training efficacy in data-limited scenarios.
• Thoughtful annotation strategies are crucial for maximizing AI performance in rare disease diagnostics.