ChatGPT 3.5 Better Improves Comprehensibility of English, than Spanish, Generated Responses to Osteosarcoma Questions

Area of Science:

• Artificial Intelligence in Healthcare
• Natural Language Processing for Patient Education
• Health Communication

Background:

• Health literacy and language barriers impede patient understanding of medical information, impacting outcomes.
• Most patient materials exceed the average patient's reading level.
• Limited bilingual physicians create communication challenges for diverse patient populations.

Purpose of the Study:

• To evaluate ChatGPT 3.5's accuracy and appropriateness for patient questions in English and Spanish.
• To assess the comprehensibility of ChatGPT 3.5 responses and their modifiability.

Main Methods:

• Twenty osteosarcoma patient questions were input into ChatGPT 3.5.
• Responses were assessed for clinical appropriateness and accuracy by expert reviewers.
• Comprehensibility was measured using Flesch Reading Ease Score and Flesch-Kincaid Grade Level, with modifications tested.

Main Results:

• All ChatGPT 3.5 responses were suitable for patient information platforms.
• No significant difference in initial comprehensibility was found between English and Spanish responses.
• Modified responses showed a statistically significant difference in comprehensibility, favoring English.

Conclusions:

• ChatGPT 3.5 generated factually accurate and clinically appropriate responses in both languages.
• The AI model demonstrated the ability to simplify its responses upon request.
• ChatGPT 3.5 was more effective in enhancing the comprehensibility of English responses compared to Spanish.