Large Language Models for Thematic Summarization in Qualitative Health Care Research: Comparative Analysis of Model...

Area of Science:

• Computational Linguistics
• Qualitative Health Research
• Health Informatics

Background:

• Growing importance of analyzing expert textual online data in healthcare.
• Large language models (LLMs) offer potential for computational linguistics and qualitative research.
• Scaling thematic analysis of large datasets requires robust interpretation methods.

Purpose of the Study:

• Investigate LLM capabilities in analyzing expert textual data.
• Explore LLMs for scaling the interpretation phase of topic modeling.
• Compare LLM-derived interpretations with human interpretations of qualitative data.

Main Methods:

• Collected and preprocessed data from an online nurse forum.
• Applied Latent Dirichlet Allocation (LDA) for topic modeling.
• Compared human categorization and interpretation with LLM-generated analysis.

Main Results:

• Substantial agreement (80%) between LLM and human interpretation of themes.
• LLMs provide detailed subtheme explanations, aligning with and expanding human themes.
• LLMs identified coherence and complementarity missed by human evaluation.

Conclusions:

• LLMs can automate interpretation in qualitative research.
• Challenges remain in evaluating LLM-generated themes.
• LLMs show promise for enhancing the scalability and depth of qualitative data analysis.