Leveraging ChatGPT for Report Error Audit: An Accuracy-Driven and Cost-Efficient Solution for Ophthalmic Imaging...

Area of Science:

• Ophthalmology
• Artificial Intelligence
• Medical Imaging Analysis

Background:

• Accurate ophthalmic imaging reports (FFA, B-scan ultrasound) are critical for clinical decisions.
• Current reporting workflows involving residents, technicians, and ophthalmologists are time-consuming and error-prone.
• Evaluating AI's role in auditing these reports is crucial for workflow optimization.

Purpose of the Study:

• To assess ChatGPT-4o's effectiveness in auditing errors in fundus fluorescein angiography (FFA) and ocular B-scan ultrasound reports.
• To evaluate the potential of ChatGPT-4o to reduce time and costs in ophthalmic reporting.
• To compare AI's error detection accuracy against human reviewers.

Main Methods:

• GPT-4o analyzed 100 FFA and 80 ocular B-scan reports for errors like incorrect eye laterality and anatomical descriptions.
• GPT-4o's accuracy was benchmarked against retinal specialists, general ophthalmologists, and ophthalmic technicians.
• Cost-effectiveness analysis and a real-world validation with 20 erroneous reports were conducted.

Main Results:

• GPT-4o achieved a 79.0% error detection rate, comparable to general ophthalmologists (78.0%).
• AI integration reduced report review time by 86% (0.27h vs. 2.17-3.19h) and costs from $0.21 to $0.03 per report.
• In real-world tests, GPT-4o detected 18/20 errors with no false positives.

Conclusions:

• ChatGPT-4o effectively audits ophthalmic imaging reports, identifying common errors.
• Implementing AI can alleviate ophthalmologist workload, streamline reporting, and cut costs.
• This technology has the potential to improve clinical workflow and patient outcomes.