AI in action: Changes to student perceptions when using generative artificial intelligence for the creation of a...

Area of Science:

• Educational Technology
• Artificial Intelligence in Education
• Assessment Design

Background:

• The increasing prevalence of artificial intelligence (AI) necessitates novel assessment strategies to ensure authentic student learning.
• A pilot study introduced a collaborative group multimedia assessment allowing AI use to explore student experiences and perceptions.
• Research aimed to understand how AI integration impacts student views on its role in learning and assessment.

Purpose of the Study:

• To investigate student experiences with AI in a collaborative multimedia assessment.
• To determine if AI use alters student perceptions of AI's utility in educational contexts.
• To explore the gap between perceived and actual AI applications in student assessments.

Main Methods:

• An exploratory qualitative case study design was employed with 40 undergraduate students in a capstone Pharmacology unit.
• Thematic analysis, guided by an AI role-based conceptual framework, was applied to student logbooks.
• Student perceptions of AI use were analyzed pre- and post-assessment activities.

Main Results:

• Students initially viewed AI roles as Arbiter (49%), Oracle (41%), and Quant (10%), akin to a personal tutor.
• Actual AI use was limited, primarily as an Oracle (86%) for idea generation and Quant (14%) for data analysis.
• No instances of AI being used for generating written text for the final assessment were recorded.

Conclusions:

• A discrepancy exists between students' perceived and actual AI utilization, highlighting uncertainty in AI-integrated assessments.
• Clear guidelines are essential for educators and students to navigate AI-supported learning and maintain assessment integrity.
• Further research is needed to develop robust frameworks for assessing AI-assisted academic work.