Utilizing Large Language Models to Enhance Patient-Reported Outcome Measures: Application to the EQ-5D-5L and...

Large language models (LLMs) may be useful tools for the development/adaptation of patient-reported outcome measures. As a methodological proof-of-concept, we evaluated the use of LLMs to support the identification of potential EuroQol 5-Dimension 5-Level (EQ-5D-5L) bolt-on dimensions, using patient-reported free-text data.

We used GPT-4o to analyze text data from 1977 members of the Dutch Celiac Association, who completed the EQ-5D-5L and narratively described the impact of celiac disease on their lives. Prompts were designed to identify potential EQ-5D-5L bolt-on dimensions and produce preliminary item wordings for selected dimensions. Evaluations comprised comparisons of dimensions identified using 2 alternative approaches (qualitative analysis and topic modeling) conducted on a subset of 85 text entries, text-entry level agreement (Kappa) between LLM and qualitatively identified dimensions, and suitability of LLM-generated item wordings assessed against existing criteria.

The LLM identified 12 potential bolt-on dimensions to the EQ-5D-5L, of which 9 were also identified using qualitative analysis, and 5 using topic modeling. Text-entry-level agreement between the LLM and qualitative approaches was "moderate," "substantial," or "almost perfect," with 2 exceptions of slight/fair agreement (median Kappa = 0.68, IQR = 0.56-0.76). Sensitivity analyses using 4 other LLMs produced similar agreement results. The LLM-generated item wordings for the 4 most common dimensions scored 4.0 to 4.4 out of 5 when assessed against existing criteria.

This study demonstrates the potential of LLMs to support the development/modification of patient-reported outcome measures based on patient-reported text data. Further research should assess the approach's transferability across disease areas and data sources, while better incorporating patient/stakeholder input throughout.