Opportunities and Challenges of Generative AI in Postgraduate Health Professions Education Assessments From Educator...

The application of artificial intelligence (AI) is increasingly valuable as a tool and assistant in many areas of clinical and academic medicine. Generative AI (GenAI) creates new content used by large language models, which can generate language that strongly resembles or even improves on that of humans. Learners and educators in many areas of education are using GenAI for essays and assessments, raising issues regarding learning and assessment. GenAI is also raising new concerns in health professions education (HPE), an area of health professions training that sometimes has different aims and assessment methods compared to its clinical counterparts. HPE needs to assess levels of knowledge and understanding of pedagogy, and the use of GenAI presents challenges to its current assessments, which are predominantly written.

The study aimed to investigate educators' and learners' perspectives on the opportunities and challenges presented by GenAI in postgraduate HPE assessments. It particularly focused on perspectives of how GenAI may influence the future of assessment and essay-based assessments in HPE.

Informed by a constructivist paradigm, a qualitative approach was adopted, undertaking 8 semistructured interviews conducted via Microsoft Teams. Purposive sampling ensured a mixture of educators and learners in current HPE courses from a range of health care professions. Data were thematically analyzed.

There was no difference between educator and learner perspectives. Four themes were identified: AI is here, students are at a disservice if we do not embrace it; AI as an opportunity to rethink HPE assessments; AI is a "gray area"; and AI is fallible.

The findings present AI as an external catalyst, highlighting the current internal desire for assessment change within HPE. It offers opportunities for creative, authentic assessments that reflect real-life academic and clinical practice, aiming to develop competent future HPE educators and keep courses relevant. These findings contribute to the debate around the future potential and development of AI in HPE assessments.