Artificial Intelligence Enhanced Content Review for Curriculum Evaluation in Nursing Education: A Pilot Study

Curricular evaluation is core to nursing education and can benefit from artificial intelligence-based automation. Large language models offer new opportunities for scalable, structured document analysis.

We developed a rubric to identify elements related to health equity, including whether the content addresses vulnerable groups, social factors in health, and clinical applications.

A chatbot using retrieval-augmented generation (RAG) was configured with structured rubric-based instructions. The reliability of chatbot ratings of 9 lecture slide sets from one nurse-midwifery course was compared to ratings from 3 faculty reviewers.

The chatbot achieved reliability in repeated ratings of 84% to 91% across assessed topics. Chatbot alignment with faculty raters across topics was more variable and ranged from 56% to 89% agreement.

An RAG-based large language model system demonstrates promising reliability for automated curriculum screening. This tool can be used to assist, but not replace, human insights and summative judgments.