Exploring the effect of GenAI on learning outcomes in higher education: a three-level meta-analysis

As generative artificial intelligence (GenAI) becomes increasingly integrated into higher education, greater clarity is needed regarding its impact on student learning.

This study conducted a three-level meta-analysis of 36 empirical studies, synthesizing 132 effect sizes from 7,229 participants. Learning outcomes were classified using the Learning Outcomes Thematic Group (LOTG) framework, and seven study-level moderators were examined.

The results indicate a significant medium overall effect of GenAI on learning outcomes (g = 0.499). Stronger effects were found for understanding, cognitive and creative outcomes (g = 0.669) and higher-order learning (g = 0.504), with moderate effects for dispositions (g = 0.452) and attainments (g = 0.363). Evidence was insufficient for the Using and Membership/inclusion/self-worth outcome categories. Teaching method was the only significant moderator, with collaborative learning (g = 1.026) and blended learning (g = 0.633) yielding the strongest effects, while other moderators showed no significant influence.

These findings suggest that GenAI is most effective when embedded in interactive and collaborative pedagogies. The study introduces a GenAI-Learning Alignment Perspective and outlines implications for instructional design, assessment practices, and teacher professional development.