Large Language Models for Large-Scale, Rigorous Qualitative Analysis in Applied Health Services Research.

Area of Science:

• Health Services Research
• Qualitative Methodology
• Artificial Intelligence in Healthcare

Background:

• Large language models (LLMs) offer potential for enhancing qualitative analysis efficiency in health services research.
• Methodological guidance and evidence for LLM integration in real-world qualitative research are limited.
• Federally Qualified Health Centers (FQHCs) provide essential primary care, necessitating efficient research methods.

Purpose of the Study:

• To develop a model- and task-agnostic framework for integrating human and LLM methods in qualitative analysis.
• To demonstrate the application of this framework in a multi-site study of diabetes care at FQHCs.
• To assess the impact of LLM assistance on qualitative data synthesis and deductive coding.

Main Methods:

• Developed a flexible framework for human-LLM qualitative analysis applicable to various research aims.
• Implemented LLM-assisted qualitative synthesis of researcher summaries to generate comparative feedback reports.
• Utilized LLM-assisted deductive coding of 167 interview transcripts for intervention refinement.

Main Results:

• LLM assistance facilitated timely feedback delivery to healthcare practitioners.
• Enabled the incorporation of large-scale qualitative data, informing theory and practice changes.
• Demonstrated enhanced efficiency and preserved rigor in qualitative analysis through human-LLM collaboration.

Conclusions:

• LLMs can be effectively integrated into applied health services research to improve efficiency.
• The developed framework supports diverse analytic goals in qualitative research.
• This study offers guidance for innovative LLM applications in qualitative health research.