How threshold customisation affects the performance of a multiclass X-ray AI model for primary care triage: a...

Area of Science:

• Artificial Intelligence in Medical Imaging
• Radiology Workflow Optimization
• Deep Learning for Chest X-ray Analysis

Background:

• Commercially available deep learning (DL) models for multiclass chest X-ray (CXR) analysis require adaptation for specific clinical settings.
• Primary care settings necessitate AI tools that prioritize safety and minimize false negatives.

Purpose of the Study:

• To describe the threshold optimization process for a commercial CXR DL model.
• To evaluate the model's diagnostic performance at various thresholds.
• To estimate the potential operational impact of AI-enabled triage in primary care.

Main Methods:

• Retrospective diagnostic performance evaluation of a multiclass CXR DL model.
• Threshold-based analysis using data from primary care clinics and a tertiary hospital in Singapore.
• Inclusion of 816 adult frontal CXRs, with exclusion of pediatric studies and non-standard views.

Main Results:

• At a threshold of 0.10, sensitivity increased to 93.2% and negative predictive value (NPV) to 91.3%, with specificity at 71.7%.
• Lowering the threshold from 0.15 to 0.10 improved sensitivity and NPV, acceptable for a safety-focused triage workflow.
• The trade-off prioritized minimizing false negatives, crucial for primary care safety.

Conclusions:

• Threshold optimization is critical for tailoring AI models to clinical workflows.
• Adjusting the operating threshold allows prioritization of sensitivity and NPV for safe AI-assisted triage.
• Collaborative efforts between radiology and clinical teams are essential for effective AI implementation in primary care.