Utilizing AI-Powered Thematic Analysis: Methodology, Implementation, and Lessons Learned

Area of Science:

• Medical research
• Qualitative analysis
• Artificial intelligence applications

Background:

• Large language models (LLMs) show potential for automating qualitative research tasks like data analysis and coding.
• Existing research on LLM performance in qualitative studies lacks clear implementation guidelines.
• Artificial intelligence (AI) is poised to revolutionize healthcare, education, and research methodologies.

Purpose of the Study:

• To provide practical, step-by-step methods and prompts for implementing LLMs in qualitative analysis.
• To develop and evaluate a customized generative pre-trained transformer (Custom-GPT) based on thematic analysis framework.
• To assess the consistency and alignment of LLM-generated themes with manual coding.

Main Methods:

• Developed a Custom-GPT model aligned with Braun and Clarke's six-step thematic analysis framework.
• Evaluated the Custom-GPT model's performance across three distinct datasets.
• Utilized Google's NotebookLM for triangulation and comparison with manual coding and thematic interpretation.

Main Results:

• The Custom-GPT model produced consistent thematic structures closely matching manual coding across datasets.
• Identified challenges including response variability, lack of AI decision-making transparency, and the need for repeated prompting.
• Human intervention was necessary between steps to refine outputs and maintain methodological integrity.

Conclusions:

• LLMs present significant opportunities for improving qualitative thematic analysis efficiency.
• Human oversight remains essential to address LLM limitations and ensure the reliability of AI-assisted qualitative research.
• Further research is needed on ethical frameworks, domain-specific LLMs, and advanced prompt engineering for responsible AI integration.