Exploring the Capacity of Large Language Models to Assess the Chronic Pain Experience: Algorithm Development and...

Area of Science:

• Artificial Intelligence in Healthcare
• Natural Language Processing for Clinical Applications
• Pain Medicine and Management

Background:

• Chronic pain affects over 20% of the global population, causing significant individual and economic burdens.
• Personalized pain assessment through written narratives (WNs) offers insights beyond standardized questionnaires.
• Evaluating WNs is time-consuming for clinicians, highlighting the need for efficient assessment tools.

Purpose of the Study:

• To evaluate the potential of large language models (LLMs) in assisting clinicians with pain assessment from written narratives.
• This is the first study to explore LLM application for assessing patient-reported pain in written narratives.

Main Methods:

• GPT-4 was prompted to score and explain pain severity and disability from 43 fibromyalgia patient narratives.
• GPT-4 scores were quantitatively compared against expert scores using correlation and agreement metrics.
• Experts qualitatively analyzed GPT-4's score explanations for accuracy and clinical applicability.

Main Results:

• GPT-4 demonstrated comparable performance to experts, with high agreement (weighted percentage > 0.95) and moderate correlations with standardized measures.
• Low error rates (RMSE for severity and disability) were observed, indicating reliable scoring.
• Experts found GPT-4's ratings and explanations adequate, though a slight tendency to overestimate pain was noted.

Conclusions:

• LLMs show significant potential for facilitating the assessment of written pain narratives in fibromyalgia patients.
• Automated LLM-based assessment offers a novel approach to enhance understanding and evaluation of patient pain experiences.
• Integrating LLMs can streamline pain assessment, leading to improved patient care and tailored interventions for chronic pain.