Related Experiment Videos
Evaluating an instrument for the peer review of inpatient teaching
Thomas J Beckman1, Mark C Lee, Charles H Rohren
1Division of General Internal Medicine, Mayo Clinic and Mayo Clinic Foundation, Rochester, MN 55905, USA. Beckman.Thomas@mayo.edu
Abstract:
The purpose of this study was to assess an instrument for the peer review of inpatient teaching at Mayo. The Mayo Teaching Evaluation Form (MTEF) is an instrument, based on the Stanford seven-category educational framework, which was developed for the peer review of inpatient teaching. The MTEF has 28 Likert-scaled items derived from the Stanford Faculty Development Program form (SFDP-26), the Mayo electronic evaluation form and three additional items. In this study three physician-evaluators used the MTEF to evaluate 10 attending physicians on the Mayo general internal medicine hospital services. Cronbach's alphas were used to assess the internal consistency of the MTEF, and Kendall's coefficient of concordance was used to summarize the inter-rater reliability. Results of this study reveal that the MTEF is internally consistent, based on average ratings across all evaluators (Cronbach's alpha=0.894). Stanford categories with the highest alphas are Self-Directed Learning, Learning Climate, Communication of Goals, and Evaluation. Categories with lower alphas are Feedback, Understanding and Retention, and Control of Teaching Session. Additionally, the majority of items on the MTEF show significant agreement across all evaluators, and teacher enthusiasm was among the most reliable items. In conclusion, the MTEF is overall internally consistent for the peer review of inpatient teaching at Mayo. Hence, the MTEF may be a useful element in the peer evaluation of teaching at our institution.
Insights
The Mayo Teaching Evaluation Form (MTEF) demonstrates strong internal consistency and inter-rater reliability for assessing inpatient teaching. This peer review instrument may be valuable for evaluating teaching quality in medical education.
Area of Science:
- Medical Education
- Healthcare Quality Improvement
- Academic Medicine
Background:
- Effective peer review of inpatient teaching is crucial for medical education.
- Existing evaluation tools may require refinement for specific institutional contexts.
- The Stanford seven-category educational framework provides a basis for structured teaching assessment.
Purpose of the Study:
- To assess the reliability and validity of the Mayo Teaching Evaluation Form (MTEF).
- To evaluate the MTEF's utility as an instrument for peer review of inpatient teaching.
- To determine the internal consistency and inter-rater reliability of the MTEF.
Main Methods:
- Development of the MTEF based on the Stanford framework and existing forms.
- Application of the MTEF by three physician-evaluators to 10 attending physicians.
- Statistical analysis using Cronbach's alpha for internal consistency and Kendall's coefficient of concordance for inter-rater reliability.
Main Results:
- The MTEF exhibited strong internal consistency (Cronbach's alpha=0.894).
- High reliability was observed in categories such as Self-Directed Learning, Learning Climate, Communication of Goals, and Evaluation.
- Significant agreement among evaluators was noted for most items, with teacher enthusiasm being highly reliable.
Conclusions:
- The MTEF is a reliable and internally consistent instrument for the peer evaluation of inpatient teaching.
- The MTEF shows potential as a valuable tool for enhancing teaching quality assessment in medical institutions.
- Further validation may support the MTEF's broader application in peer review processes.