Using theory-informed data science methods to trace the quality of dental student reflections over time

Area of Science:

• Medical Education
• Data Science in Education
• Health Professions Education

Background:

• Reflection quality is crucial for developing expertise in health professions.
• Assessing reflection depth and elements in student writing is challenging.
• Data science offers novel approaches to analyze educational data at scale.

Purpose of the Study:

• To apply data science methods to analyze the quality of reflections from dental students over four years.
• To evaluate the progression of reflection depth (No, Shallow, Deep) and specific reflective elements.
• To develop machine learning models for automatic detection of reflection qualities.

Main Methods:

• Analysis of 1,500 reflections from 369 dental students.
• Evaluation of reflection depth and six key elements (Description, Analysis, Feeling, Perspective, Evaluation, Outcome).
• Development of machine learning models using linguistic features to detect reflection qualities.

Main Results:

• A dramatic increase in shallow reflections (20% to 66%) was observed within the first year.
• Deep reflections showed only a gradual rise (2% to 26%) over four years.
• Machine learning models demonstrated reliable detection for Description and Evaluation, with moderate performance for Analysis, Feelings, and Perspectives. The Outcome model suffered from overfitting.

Conclusions:

• While students progress to more shallow reflection, deep reflection development remains limited.
• The presence of specific reflective elements like Feelings and Analysis needs targeted educational intervention.
• Data science and machine learning can effectively analyze reflection quality, informing personalized learning support for skill development.