Identifying Unmonitored Tone Drift in Patient Education Materials Transcribed to a Lower Reading Level by a Large...

Area of Science:

• Neurosurgery
• Artificial Intelligence in Healthcare
• Medical Communication

Background:

• Neurosurgical patient education materials (PEMs) often exceed recommended readability levels, hindering patient comprehension and informed consent.
• Simplifying these materials can inadvertently alter their tone and introduce bias.
• The impact of large language models (LLMs) like ChatGPT on the sentiment and emotional tone of simplified neurosurgical PEMs is not well understood.

Purpose of the Study:

• To evaluate the changes in sentiment and emotional tone of neurosurgical patient education materials after simplification to a lower reading level using ChatGPT.

Main Methods:

• Analyzed 336 neurosurgical PEMs for readability, sentiment, and emotion.
• Simplified materials to a seventh-grade reading level using GPT-4.0.
• Assessed readability using six standard indices and sentiment/emotion via VADER and NRC Emotion Lexicon, with paired t-tests for significance.

Main Results:

• Significant improvements in readability were observed across all neurosurgical topics (P < .001).
• Sentiment shifted towards increased positivity, with decreased disgust and modest increases in sadness, surprise, and joy.
• Fear and anger showed no significant change, but overall, simplification yielded large readability gains with small, measurable alterations in emotional tone.

Conclusions:

• AI-assisted simplification of neurosurgical PEMs enhances readability but introduces subtle, potentially persuasive shifts in sentiment and emotional tone.
• These affective shifts warrant careful monitoring when deploying LLMs for patient-facing materials.
• While current PEMs present communication barriers, providers must be cautious about AI-driven modifications.