More Than Scores: AI-Assisted Instruction in Long-term Knowledge Retention and Critical Thinking Skills for Diagnostic Education

To compare the effectiveness of a generative artificial intelligence (GenAI)-assisted case-based learning model versus traditional teaching in a clinical diagnostics course.

A quasi-experimental study was conducted with 84 fourth-year medical students cluster-assigned to an AI group (n = 42, using DeepSeek platform) or a control group (n = 42, using textbooks and clinical guidelines). Both groups received equal contact time (120 min/session) and feedback frequency (two iterative rounds). Assessments included theoretical knowledge tests at baseline, immediately post-course, and at 6-month follow-up, as well as critical thinking (CTDI-CV), self-directed learning (SRSSDL-CV), and teaching satisfaction. Linear mixed models with class as random intercept and baseline scores as fixed covariates were used, with Benjamini-Hochberg FDR correction for multiple comparisons (q-value < 0.05 considered statistically significant).

No significant baseline differences were observed. Post-course theoretical scores showed comparable improvements across all knowledge domains (all q > 0.05). At 6 months, the AI group demonstrated significantly attenuated knowledge decay in symptomatology (adjusted difference = 2.1, 95% CI [0.5, 3.7], q = 0.015) and laboratory/instrumental domains (adjusted difference = 2.4, 95% CI [0.2, 4.6], q = 0.042), while physical examination showed no significant difference (q = 0.335). The AI group also achieved higher total critical thinking scores (adjusted difference = 4.72, 95% CI [1.35, 8.09], q = 0.012), with significant gains in open-mindedness, systematicity, inquisitiveness, and self-confidence (all q < 0.05), and higher self-directed learning scores (adjusted difference = 5.18, 95% CI [1.76, 8.60], q = 0.003). Teaching satisfaction in course discussions, teamwork, and environment was significantly higher in the AI group (all P < 0.05).

GenAI‑assisted learning was associated with improvements in critical thinking, self‑directed learning, and satisfaction, and showed promise for supporting knowledge retention. These preliminary single‑center findings suggest AI may supplement traditional instruction, though multi‑center validation is needed.

The online version contains supplementary material available at https://doi.org/10.1007/s40670-026-02830-4.