Related Experiment Video
Updated: Apr 7, 2026

Using Learning Outcome Measures to assess Doctoral Nursing Education
Published on: June 21, 2010
Can Large Language Models Replicate Systematic Review Outcome Classifications in Medical Education? A Pilot Study
Giuliano Romano1, Emilio Romano2, Michelle Rau1
1Oakland University William Beaumont School of Medicine, Rochester, MI United States of America.
Abstract:
Systematic reviews in medical education often classify outcomes using the Kirkpatrick framework, but manual coding is time-consuming and subjective. We conducted a proof-of-concept study testing ChatGPT (GPT-5, August 2025 release) on 32 full-text articles from a published systematic review of sepsis education. Agreement with human-coded outcomes was modest: 50% percent agreement, unweighted κ = 0.170 (95% CI 0.000-0.458), weighted κ = 0.351 (95% CI 0.074-0.629). Most disagreements were between adjacent levels.
Supplementary Information:
The online version contains supplementary material available at 10.1007/s40670-026-02639-1.
More Related Videos
13:44Project-Based Learning Guidelines for Health Sciences Students: An Analysis with Data Mining and Qualitative Techniques
Published on: December 9, 2022
10:39Qualitative and Quantitative Validation of Tools with Rating Scales Aimed at Assessing the Quality of University Service-Learning
Published on: August 29, 2025
Related Concept Videos
Nursing Process for Patient and Caregiver Teaching III: Evaluation and Documentation
Nurses can use several methods to evaluate patient outcomes. For example, oral questions can assess cognitive learning,...
Types of Records II: Educational and Administrative Records
Nursing Evaluation
Guidelines for Writing Outcome
Patient outcomes reflect the patient's response to the goal rather than what the nurse aims to achieve. Terminology should be observable and measurable to avoid the reader's interpretation. The desired outcome should be realistic and achievable in the designated care timeframe. Expected outcomes should align with adjunctive therapies. The outcome should enhance care...
Levels of Organization
Molecules Are Composed of Atoms, and Biomolecules Are Assembled from Molecules:
The most basic levels include atoms, molecules, and biomolecules. Atoms, the smallest unit of ordinary matter, are composed of a nucleus and electrons. Molecules...
Heart Failure IV: Classification and Diagnostic Evaluation