Collaborative Integration of AI and Human Expertise to Improve Detection of Chest Radiograph Abnormalities

Purpose To develop a collaborative artificial intelligence (AI) system that integrates eye gaze data and radiology reports to improve diagnostic accuracy in chest radiograph interpretation by identifying and correcting perceptual errors. Materials and Methods This retrospective study used public datasets REFLACX (Reports and Eye-Tracking Data for Localization of Abnormalities in Chest X-rays) and EGD-CXR (Eye Gaze Data for Chest X-rays) to develop a collaborative AI solution, named Collaborative Radiology Expert (CoRaX). It uses a large multimodal model to analyze image embeddings, eye gaze data, and radiology reports, aiming to rectify perceptual errors in chest radiology. The proposed system was evaluated using two simulated error datasets featuring random and uncertain alterations of five abnormalities. Evaluation focused on the system's referral-making process, the quality of referrals, and its performance within collaborative diagnostic settings. Results In the random masking-based error dataset, 28.0% (93 of 332) of abnormalities were altered. The system successfully corrected 21.3% (71 of 332) of these errors, with 6.6% (22 of 332) remaining unresolved. The accuracy of the system in identifying the correct regions of interest for missed abnormalities was 63.0% (95% CI: 59.0, 68.0), and 85.7% (240 of 280) of interactions with radiologists were deemed satisfactory, meaning that the system provided diagnostic aid to radiologists. In the uncertainty-masking-based error dataset, 43.9% (146 of 332) of abnormalities were altered. The system corrected 34.6% (115 of 332) of these errors, with 9.3% (31 of 332) unresolved. The accuracy of predicted regions of missed abnormalities for this dataset was 58.0% (95% CI: 55.0, 62.0), and 78.4% (233 of 297) of interactions were satisfactory. Conclusion The CoRaX system can collaborate efficiently with radiologists and address perceptual errors across various abnormalities in chest radiographs. Keywords: Perception, Convolutional Neural Network (CNN), Deep Learning Algorithms, Radiology-Pathology Integration, Unsupervised Learning, CoRaX, Perceptual Error, Referral, Deferral Supplemental material is available for this article. © RSNA, 2025 See also commentary by Levi and Laghi in this issue.