Developing and validating a knowledge-based AI assessment system for learning clinical core medical knowledge in...

Area of Science:

• Medical Education Technology
• Artificial Intelligence in Healthcare
• Otolaryngology Training

Background:

• Clinical core medical knowledge (CCMK) learning is vital for medical trainees.
• Developing adaptive assessment systems for CCMK is challenging, particularly with AI.
• Extracting expert CCMK requires innovative approaches beyond traditional methods.

Purpose of the Study:

• To develop a multi-expert knowledge-aggregated adaptive assessment scheme (MEKAS).
• To utilize knowledge-based AI for facilitating CCMK in otolaryngology (CCMK-OTO) learning.
• To validate MEKAS effectiveness via a one-month training program at a tertiary hospital.

Main Methods:

• MEKAS employed repertory grid technique and case-based reasoning to aggregate expert knowledge.
• A longitudinal study compared an experimental group (EG) using MEKAS with a control group (CG).
• Training effectiveness was assessed via pre/post-test CCMK-OTO scores and technology acceptance questionnaires.

Main Results:

• EG trainees (undergraduate and residents) showed significant CCMK-OTO score improvements (P < 0.001 and P = 0.042).
• CG undergraduate trainees did not show significant improvement (P = 0.228).
• EG participants reported high satisfaction with MEKAS (average scores 3.8-4.1/5.0).

Conclusions:

• MEKAS effectively facilitates CCMK-OTO learning and knowledge aggregation.
• The system demonstrates potential for broader application in other medical subjects.
• Further large-scale validation is recommended to assess scalability and generalizability.