Utilizing Large Language Models to Enhance Patient-Reported Outcome Measures: Application to the EQ-5D-5L and...

Area of Science:

• Health Informatics
• Psychometrics
• Natural Language Processing

Background:

• Patient-reported outcome measures (PROMs) are crucial for assessing health status.
• Developing and adapting PROMs, such as the EQ-5D-5L, requires robust methods.
• Large language models (LLMs) offer potential for analyzing patient-generated text data.

Purpose of the Study:

• To evaluate the efficacy of LLMs in identifying potential bolt-on dimensions for the EQ-5D-5L.
• To assess LLM-generated item wordings against established criteria.
• To explore LLMs as a tool for developing and adapting PROMs.

Main Methods:

• GPT-4o analyzed free-text data from 1,977 celiac disease patients.
• Prompts guided LLM to identify potential EQ-5D-5L bolt-on dimensions and draft item wordings.
• Evaluations included comparison with qualitative analysis and topic modeling, Kappa agreement, and item wording suitability assessment.

Main Results:

• The LLM identified 12 potential bolt-on dimensions, with 9 overlapping with qualitative analysis.
• Text-entry level agreement between LLM and qualitative methods was generally moderate to almost perfect (median Kappa=0.68).
• LLM-generated item wordings for the top 4 dimensions scored highly (4.0-4.4/5) on suitability.

Conclusions:

• LLMs show significant potential for supporting the development and modification of PROMs using patient text data.
• Further research is needed to assess transferability across different diseases and data sources.
• Incorporating patient and stakeholder input throughout the process is recommended.