Frame-of-Reference Training: Establishing Reliable Assessment of Teaching Effectiveness

Area of Science:

• Medical Education
• Health Professions Education
• Faculty Development

Background:

• Accurate assessment of health professionals' teaching performance is crucial.
• Faculty often lack training in providing reliable peer assessments.
• Workplace-based assessments require standardized rating methods.

Purpose of the Study:

• To determine the effectiveness of Frame-of-Reference (FOR) training for assessing medical faculty teaching performance.
• To evaluate if FOR training improves the accuracy and reliability of peer ratings.
• To assess the feasibility of FOR training in a medical school setting.

Main Methods:

• 21 Harvard Medical School faculty participated in pilot and implementation studies.
• Participants underwent FOR training, focusing on a criterion-based peer assessment instrument.
• Trained faculty rated pre- and post-training recorded lectures, with scores compared to expert-derived scores.

Main Results:

• A significant increase in rating agreement between participants and experts was observed post-training.
• Participants demonstrated improved ability to accurately categorize and score teaching performances after FOR training.
• FOR training proved to be a feasible method for enhancing faculty assessment skills.

Conclusions:

• Frame-of-Reference training is an effective method for teaching faculty to accurately and reliably assess medical lectures.
• Trained peer observers can provide valuable feedback to medical school instructors and continuing education presenters.
• FOR training offers a time-efficient approach for performance assessment and feedback beyond traditional methods.