Performance Comparison Between Two Versions of a Commercial Artificial Intelligence System for Chest Radiograph...

Area of Science:

• Radiology and Medical Imaging
• Artificial Intelligence in Healthcare
• Computer-Aided Diagnosis

Background:

• Deep learning-based artificial intelligence (AI) systems are increasingly used for analyzing medical images.
• Gleamer ChestView is an AI system designed for chest X-ray analysis.
• Evaluating performance updates in AI diagnostic tools is crucial for clinical adoption.

Purpose of the Study:

• To compare the diagnostic performance of two versions (1.5.0 and 1.5.4) of the Gleamer ChestView AI system.
• To assess the AI system's effectiveness across multiple thoracic findings on chest radiographs.
• To validate improvements in AI diagnostic accuracy and reliability.

Main Methods:

• A retrospective multicenter study involving 187 chest radiographs from six centers.
• Comparison of Gleamer ChestView versions 1.5.0 and 1.5.4 on the same dataset.
• Ground truth established by chest CT (49 cases) and consensus of three radiologists (138 cases).
• Evaluation of performance metrics including sensitivity, specificity, precision, and F1 score.

Main Results:

• Overall performance improved from version 1.5.0 to 1.5.4, with higher accuracy (87.7% vs 92.5%) and F1 score (0.816 vs 0.881).
• Version 1.5.4 demonstrated increased precision (75.0% vs 85.2%) and specificity (86.9% vs 93.1%).
• Nodule detection precision increased significantly (47.8% to 73.3%) with maintained sensitivity.

Conclusions:

• Gleamer ChestView version 1.5.4 shows superior lesion-specific performance compared to version 1.5.0.
• The updated AI system offers fewer false positives and higher diagnostic confidence.
• Systematic version-to-version validation supports the implementation of updated AI diagnostic systems.