Advancing patient education in PRRT through large language models: challenges and potential.

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Oncology Patient Education

Background:

• Growing use of AI chatbots for patient education.
• Need to assess AI chatbot performance in delivering complex medical information.
• Focus on accuracy, readability, and conciseness for patient understanding.

Purpose of the Study:

• Evaluate ChatGPT 4o and DeepSeek V3 performance for Peptide Receptor Radionuclide Therapy (PRRT) patient inquiries.
• Compare AI chatbot accuracy, readability, and conciseness.
• Identify potential medical inaccuracies in AI-generated patient education content.

Main Methods:

• Submitted 12 common PRRT questions to ChatGPT 4o and DeepSeek V3.
• Nine professionals assessed responses via blinded survey (accuracy, conciseness, readability).
• Statistical analysis using Mann-Whitney U and Chi-square tests.

Main Results:

• No significant difference in accuracy (ChatGPT 4o: 4.43, DeepSeek V3: 4.56) or readability (ChatGPT 4o: 4.38, DeepSeek V3: 4.25).
• ChatGPT 4o responses were significantly more concise (4.55 vs 4.24; P=0.0013).
• 7-8% of responses contained medical inaccuracies, with no significant difference between models.

Conclusions:

• Both AI chatbots demonstrate strong performance for PRRT patient education.
• ChatGPT 4o offers superior conciseness in its responses.
• Medical inaccuracies necessitate physician oversight for AI-driven patient education.