Related Experiment Video
Updated: Oct 2, 2026
Exploring Deep Space - Uncovering the Anatomy of Periventricular Structures to Reveal the Lateral Ventricles of the Human Brain
Published on: October 22, 2017
Comparison of artificial intelligence and lecturers' grading in the evaluation of handwritten open-ended anatomy
Mert Nahir1, Abdulkerim Kasap2, Canan Bayraktar Nahir3
1Department of Anatomy, Faculty of Medicine, Tokat Gaziosmanpaşa University, Tokat, Turkey. mert.nahir@gop.edu.tr.
Purpose:
This study evaluated the performance of artificial intelligence (AI) chatbots in grading handwritten open-ended anatomy examinations and examined whether providing a standardized answer key improved agreement with lecturers' grading.
Methods:
Examination papers from 81 s-year dental students were included. The examination comprised 10 open-ended questions, each worth 10 points. Two anatomists graded all papers using a standardized answer key, and the mean of their scores was used as lecturers' grading. Four AI chatbots-ChatGPT, Gemini, Copilot, and DeepSeek-graded each paper under two conditions: without an answer key and with an answer key and detailed grading criteria. Agreement with lecturers' grading was assessed using the intraclass correlation coefficient (ICC).
Results:
Differences were observed among the nine grading methods (p < 0.001). No significant differences were found between lecturers' grading and ChatGPT scores obtained either without or with the answer key. ChatGPT showed the highest agreement with lecturers' grading after the answer key was provided (ICC = 0.923, 95% CI [0.883, 0.950]), followed by ChatGPT without the answer key (ICC = 0.904, 95% CI [0.855, 0.937]). Gemini and Copilot showed good agreement without the answer key, but agreement decreased when the answer key was provided. DeepSeek demonstrated the lowest agreement under both conditions.
Conclusion:
AI chatbot grading performance varied across models. ChatGPT produced scores most closely aligned with lecturers' grading, whereas providing a standardized answer key did not consistently improve performance. AI-assisted grading may support assessment in anatomy education, but human oversight and structured grading criteria remain essential, particularly in high-stakes examinations.
