Enhancing Patient Communication With Chat-GPT in Radiology: Evaluating the Efficacy and Readability of Answers to...

Area of Science:

• Artificial Intelligence in Healthcare
• Medical Informatics
• Radiology Patient Education

Background:

• Artificial intelligence (AI) tools like ChatGPT offer potential for patient education.
• Evaluating AI's utility in answering common patient questions about medical imaging is crucial.

Purpose of the Study:

• To assess the accuracy, relevance, and readability of ChatGPT responses to patient imaging questions.
• To examine the impact of a simple prompt on the quality of AI-generated answers.

Main Methods:

• Developed 22 common imaging-related patient questions across key categories.
• Posed questions to ChatGPT with and without a prompt for accuracy and clarity.
• Evaluated 264 responses for accuracy, consistency, relevance, and readability (Flesch Kincaid Grade Level) by radiologists and patient advocates.

Main Results:

• ChatGPT responses were 83% accurate unprompted and 87% prompted.
• Prompting significantly improved response consistency (72% to 86%) and full relevance (67% to 80%).
• Readability remained high (average Flesch Kincaid Grade Level of 13.0-13.6), failing to meet recommended eighth-grade level.

Conclusions:

• ChatGPT shows potential for accurate, consistent, and relevant patient imaging information.
• Prompts enhance response quality but do not improve readability; high complexity necessitates human oversight.
• Further research and cautious implementation are needed for AI in patient-facing health materials.