Employing UMLS for generating hints in a tutoring system for medical problem-based learning

Area of Science:

• Medical Education
• Artificial Intelligence in Education
• Cognitive Science

Background:

• Problem-based learning (PBL) is effective for clinical reasoning but strains medical faculty.
• Traditional intelligent tutoring systems (ITSs) offer limited feedback on non-standard solutions.
• There is a need for ITSs that support creative exploration in PBL.

Purpose of the Study:

• To develop an alternative ITS architecture for medical PBL.
• To leverage domain ontologies for generating effective feedback.
• To provide feedback that acknowledges partially correct student reasoning.

Main Methods:

• Utilized a domain ontology (UMLS) within the METEOR tutoring system.
• Implemented a hint generation strategy based on concept hierarchy and co-occurrence.
• Evaluated system-generated hints using expert agreement on a Likert scale.

Main Results:

• System-generated hints achieved an average expert agreement score of 4.44.
• Hints providing partial correctness feedback were significantly preferred.
• Human expert hints received an average score of 4.2.

Conclusions:

• The proposed ITS strategy effectively supports medical PBL by providing nuanced feedback.
• Leveraging domain ontologies enhances the feedback capabilities of ITSs.
• This approach can increase the scalability of facilitated PBL training.