Potential misinformation in large language model descriptions of upper extremity diseases

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Patient Education

Background:

• The proliferation of online health information necessitates accurate and reliable sources.
• Large language models (LLMs) are increasingly used to access health information, raising concerns about potential misinformation.
• Upper extremity conditions represent a common area of patient inquiry and self-management.

Purpose of the Study:

• To evaluate the accuracy and impact of large language model (LLM) responses concerning upper extremity conditions.
• To assess whether LLMs can effectively correct misinformation when prompted.
• To investigate the influence of LLM-generated content on patient health agency and decision-making.

Main Methods:

• A large language model was prompted with queries about common upper extremity conditions.
• The LLM was subsequently prompted to identify and correct misinformation within its own generated responses.
• Analysis focused on the persistence of misinformation, changes in user thinking patterns, and reported impact on self-management.

Main Results:

• Despite prompts to correct misinformation, the LLM exhibited persistent inaccuracies in its discussions of upper extremity conditions.
• LLM responses tended to reinforce unhelpful health-related thinking patterns.
• There was a notable reduction in the sense of independent health management (agency) among users interacting with the LLM.

Conclusions:

• Current large language models demonstrate limitations in providing accurate and unbiased information on upper extremity conditions.
• The tendency for LLMs to perpetuate misinformation and reduce patient agency poses a risk to effective health management.
• Further development and rigorous validation are required before LLMs can be reliably used for patient education on medical conditions.