Automated grading of anatomical objective structured practical examinations using decision trees: An artificial...

Area of Science:

• Medical Education
• Artificial Intelligence
• Anatomy and Physiology

Background:

• Objective Structured Practical Examinations (OSPEs) are effective for assessing anatomical knowledge but are resource-intensive.
• Traditional OSPEs often use short-answer or fill-in-the-blank questions, requiring significant human grading resources.
• Online anatomy and physiology courses may reduce students' opportunities for OSPE practice.

Purpose of the Study:

• To evaluate the accuracy of Decision Trees (DTs) in grading OSPE questions.
• To establish a foundation for an intelligent, online OSPE tutoring system.

Main Methods:

• Utilized OSPE data from McMaster University's anatomy and physiology course (Winter 2020).
• Trained a Decision Tree (DT) for each of the 54 OSPE questions using 90% of the dataset via 10-fold validation.
• DTs were trained on unique words from correct student answers and used to grade the remaining 10% of the data.

Main Results:

• Decision Trees (DTs) achieved an average accuracy of 94.49% across all 54 OSPE questions.
• DT grading accuracy was comparable to that of human staff and faculty.

Conclusions:

• Machine learning algorithms, specifically DTs, are highly effective for OSPE grading.
• DTs show significant potential for developing intelligent, online OSPE tutoring systems.