AI-assisted qualitative analysis of formative feedback to support student-centered learning in physiology

Area of Science:

• Educational Technology
• Artificial Intelligence in Education
• Higher Education Pedagogy

Background:

• Large courses present challenges for instructors in timely analysis of student feedback.
• Student-centered learning (SCL) requires effective synthesis of student input for instructional adaptation.
• Formative feedback is crucial for identifying learning gaps and informing teaching strategies.

Purpose of the Study:

• To evaluate the efficacy of large language models (LLMs) in analyzing formative feedback for student-centered learning (SCL).
• To compare the speed and thematic reliability of LLM analysis against human analysis of student feedback.
• To explore the potential of LLMs in supporting instructors with large volumes of student input.

Main Methods:

• Utilized Claude Sonnet 4.0 and ChatGPT 4.1 to analyze 63 anonymous student responses to a 'Muddiest Point' prompt.
• Conducted 20 analysis runs to assess consistency and reliability of LLM performance.
• Compared LLM analysis time and thematic identification with a human reviewer's assessment.

Main Results:

• Both LLMs consistently identified
• Ventilation and Lung Mechanics
• " as the primary area of student confusion.
• LLMs completed the analysis significantly faster (average 19.6s/31.0s) than a human reviewer (32 minutes).
• Thematic reliability was observed across LLM runs, aligning with human analysis.

Conclusions:

• LLMs offer a highly efficient method for processing formative feedback, enabling instructors to adapt teaching in real-time.
• This technology supports student-centered learning and educational equity by incorporating all student voices.
• While promising for formative assessment, LLM variability necessitates further refinement for high-stakes summative evaluations.