Utilizing a Digital Swarm Intelligence Platform to Improve Consensus Among Radiologists and Exploring Its...

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Radiology

Background:

• Radiologists are crucial for diagnostic decisions and AI algorithm development.
• Low inter-reader reliability (IRR) in interpreting complex cases is a significant challenge.
• Team-based decisions can be hindered by interpersonal biases, limiting participation.

Purpose of the Study:

• To investigate a novel digital swarm platform to enhance consensus and mitigate bias in radiological image interpretation.
• To compare the IRR of swarm-based consensus votes against individual, majority, and AI algorithm predictions.

Main Methods:

• Two cohorts (radiologists and residents) used a real-time, blinded digital swarm platform to grade meniscal lesions on knee MR exams.
• Consensus votes were benchmarked against clinical (arthroscopy) and radiological standards.
• IRR was calculated using Cohen's kappa and compared across different decision-making methods (individual, majority, swarm) and an AI algorithm.

Main Results:

• The attending radiologist cohort showed a 23% improvement in IRR with swarm votes (k=0.34) over majority votes (k=0.11).
• Resident cohorts demonstrated significant IRR improvements: 23% for a 3-resident swarm (k=0.25) and 30% for a 5-resident swarm (k=0.37) compared to majority votes.
• Swarm consensus votes outperformed individual and majority decisions in both cohorts and surpassed predictions from a state-of-the-art AI algorithm.

Conclusions:

• A digital swarm platform effectively enhances IRR and consensus in radiological interpretation, overcoming limitations of individual and majority voting.
• This AI-inspired collaborative approach shows promise in improving diagnostic accuracy and reliability in radiology.
• The swarm consensus method demonstrated superior performance compared to both human majority decisions and current AI algorithms for meniscal lesion detection.