Independent judgment before viewing generative AI recommendations reduces susceptibility to misleading advice: a...

Generative AI can support preservice teachers' pedagogical decisions, but fluent recommendations may also redirect attention away from diagnostic classroom evidence. We compared an independent-judgment-before-recommendation sequence with a recommendation-before-judgment sequence under expert-keyed and misleading generative AI recommendations.

A three-arm randomized scenario-based experiment enrolled 240 preservice teachers from one Chinese university. Group A recorded an independent judgment before viewing an AI recommendation, Group B viewed the recommendation before its first recorded judgment, and Group C completed the tasks without AI as a contextual benchmark and did not enter the primary sequence × recommendation-quality hypothesis test. Groups A and B completed 12 fixed-order items, including six expert-keyed and six misleading recommendations. Expert-keyed joint accuracy-agreement with the expert-designated plan and rationale-was the primary outcome. After data-quality exclusions, 218 participants remained in the main analysis. Participant-clustered logistic generalized estimating equations tested sequence, recommendation quality, and their interaction.

Under misleading recommendations, Group A had higher expert-keyed joint accuracy than Group B, OR = 1.912, 95% CI [1.413, 2.589], p < 0.001. The estimate remained significant among all 160 randomized A/B participants with available outcomes, OR = 1.814, 95% CI [1.345, 2.448], p < 0.001, and was similar after version adjustment. The difference appeared in both final plan accuracy and final rationale accuracy. Group A was also less likely to reproduce the misleading recommendation completely, OR = 0.403, 95% CI [0.296, 0.549], p < 0.001. Under expert-keyed recommendations, the between-group contrast was not statistically significant, OR = 0.821, 95% CI [0.598, 1.128], p = 0.225.

Recording an independent judgment before viewing AI was associated with lower susceptibility to misleading recommendations in the selected controlled scenarios. The result supports further evaluation of independent-first workflows in teacher education, but it does not isolate a single cognitive mechanism or establish transfer to authentic classroom decision-making. These findings extend research on cognitive-forcing designs to contextualized pedagogical judgments that require both instructional action selection and evidence-based justification.