Optimizing GPT-4 Turbo Diagnostic Accuracy in Neuroradiology through Prompt Engineering and Confidence Thresholds

Area of Science:

• Artificial Intelligence in Medicine
• Medical Imaging Analysis
• Neuroradiology

Background:

• Large language models (LLMs) face diagnostic challenges in medical imaging, with current misdiagnosis rates between 30-50%.
• Integrating LLMs like GPT-4 Turbo into diagnostic imaging requires methods to enhance accuracy.

Purpose of the Study:

• To evaluate the impact of prompt engineering and confidence thresholds on LLM diagnostic accuracy in neuroradiology.
• To reduce misdiagnosis rates in AI-assisted medical image interpretation.

Main Methods:

• Analysis of 751 neuroradiology cases from the American Journal of Neuroradiology.
• Utilized GPT-4 Turbo with customized prompts to enhance diagnostic precision.
• Implemented a 90% confidence threshold and reformatting of responses to list five diagnostic candidates.

Main Results:

• Baseline GPT-4 Turbo diagnostic accuracy was 55.1%.
• Prompt engineering and confidence threshold increased diagnostic precision to 72.9%.
• The candidate list provided the correct diagnosis in 85.9% of cases, reducing misdiagnosis to 14.1%.

Conclusions:

• Strategic prompt engineering and high confidence thresholds significantly improve LLM diagnostic precision in neuroradiology.
• These methods substantially reduce misdiagnosis rates, enhancing LLM utility.
• Further research is needed to optimize these approaches for clinical implementation, balancing accuracy and utility.