Developing and evaluating human-led and large language model-assisted hybrid deductive-inductive workflows for...

Area of Science:

• Health Informatics
• Qualitative Research Methods
• Artificial Intelligence in Healthcare

Background:

• Qualitative analysis of large datasets is time-consuming.
• Integrating artificial intelligence (AI) tools like large language models (LLMs) offers potential for efficiency.
• Developing and validating hybrid human-AI workflows is essential for reliable research.

Purpose of the Study:

• To develop and compare a human-led versus an LLM-assisted hybrid workflow for qualitative thematic analysis.
• To assess the feasibility and effectiveness of LLM-assisted qualitative analysis in health research.

Main Methods:

• A hybrid deductive-inductive approach was applied to 122 transcripts from a heart failure study.
• Human-led analysis used Dedoose software; LLM-assisted analysis used ChatGPT Edu with an eleven-prompt protocol.
• Outputs were compared, themes integrated, and quotation fidelity manually verified.

Main Results:

• Human and LLM analyses yielded complementary themes, with moderate to substantial overlap.
• Robustness testing showed consistent core concepts but variability in theme labels.
• Quotation fidelity revealed a mix of verbatim, paraphrased, and hallucinated excerpts, highlighting the need for verification.

Conclusions:

• LLM-assisted qualitative analysis is feasible for large-scale health research within HIPAA-compliant environments.
• Human oversight is essential for contextual interpretation, quotation verification, and ensuring theme-quotation support.
• LLMs function best as analytic partners, necessitating transparent workflows with human validation for responsible AI integration.