Medical student reliance on artificial intelligence in nephrology education

Area of Science:

• Medical Education
• Artificial Intelligence
• Nephrology

Background:

• Large language models (LLMs) like ChatGPT are increasingly integrated into medical education.
• Concerns exist regarding student engagement with AI-generated recommendations, particularly in complex subjects like renal physiology.
• Foundational understanding in renal physiology is critical for clinical application.

Purpose of the Study:

• To investigate how first-year medical students interact with AI-generated answers in pediatric nephrology and urology.
• To determine if students change their answers to align with AI recommendations, and under what conditions.
• To assess the impact of AI on learning foundational science versus clinical reasoning.

Main Methods:

• Fifty-seven first-year medical students answered paired foundational science and clinical questions.
• Students reviewed ChatGPT-generated answers (correct or incorrect) and re-answered questions.
• Change-to-match behavior was analyzed using logistic regression, accounting for item difficulty and student clustering.

Main Results:

• Students changed answers to match ChatGPT in 22.3% of cases.
• Answer changes were more frequent for foundational science questions (24.1%) than clinical questions (20.4%).
• Foundational question type significantly increased the likelihood of answer changes (OR 1.57).

Conclusions:

• Medical students selectively rely on AI recommendations, particularly when uncertain and with foundational content.
• AI reliance is context-dependent, emphasizing the need for tailored AI literacy training.
• Findings underscore the importance of domain-specific AI literacy in nephrology education.