The AI-Reflexivity Checklist (ARC): A Pre-Analysis Pause for LLM-Assisted Coding.

Area of Science:

• Qualitative Health Research
• Artificial Intelligence in Medicine
• Health Informatics

Background:

• Artificial intelligence (AI) is increasingly used in qualitative health research, but current reflexivity guidelines inadequately address algorithmic influence.
• Existing methods for assessing AI in qualitative analysis are limited, especially concerning culturally nuanced or emotionally complex data.

Purpose of the Study:

• To introduce the AI-Reflexivity Checklist (ARC), a novel pre-analysis tool for qualitative health research.
• To establish appropriate human-in-the-loop (HITL) postures for large language model (LLM)-assisted qualitative coding.
• To integrate algorithmic influence assessment into early-stage qualitative analysis.

Main Methods:

• Development of the AI-Reflexivity Checklist (ARC) based on literature from science and technology studies, AI-assisted qualitative analysis, and workflow models.
• Operationalization of five decision domains (descriptive scope, contextual variation, experiential depth, ethical exposure, output reversibility) into sequential prompts.
• Defining HITL postures (delegate, assist/augment, human-led) based on task characteristics and risk assessment.

Main Results:

• ARC provides a structured, evidence-informed checkpoint before AI implementation in qualitative coding.
• The checklist guides the selection of HITL posture, permitting automation for simple tasks and mandating human oversight for complex or sensitive ones.
• ARC addresses limitations in conventional reflexivity by including algorithmic actors and mitigating risks associated with indiscriminate automation.

Conclusions:

• The AI-Reflexivity Checklist (ARC) enhances reflexivity in AI-assisted qualitative health research by incorporating algorithmic considerations.
• ARC promotes a balanced approach to AI adoption, ensuring human oversight is tailored to the complexity and ethical implications of the analysis.
• This tool offers a practical method for researchers to document and review human-AI interaction, improving the rigor and trustworthiness of qualitative health data analysis.