Related Experiment Video
Updated: Jun 5, 2026

Using Learning Outcome Measures to assess Doctoral Nursing Education
Published on: June 21, 2010
Relying on others' reliability: challenges in clinical teaching assessment
Elaine M Zibrowski1, Kathryn Myers, Geoff Norman
1Schulich School of Medicine & Dentistry, University of Western Ontario, London, Ontario, Canada. elaine.zibrowski@schulich.uwo.ca
Background:
The quality of the data generated from internally created faculty teaching instruments often draws skepticism. Strategies aimed at improving the reliability and validity of faculty teaching assessments tend to revolve around literature searches for a replacement instrument(s).
Purpose:
The purpose was to test this "search-and-apply" method and discuss our experiences with it within the context of observational assessment practice.
Method:
In a naturalistic pilot test, two previously validated faculty assessment instruments were paired with a global question. The reliability of both metrics was estimated.
Results:
Generalizability analyses indicated that for both pilot tested faculty teaching instruments, the global question was a more reliable measure of perceived clinical teaching effectiveness than a multiple-item inventory. Item analysis with Cronbach's coefficient alpha suggested redundant instrument content. Rater error accounted for the greatest proportion of the variance and straight-line responses occurred in approximately 28% of residents' appraisals.
Conclusions:
The results of the present study draw attention to one of the common fallacies surrounding instrument-based assessment in medical education; the solution to improving one's assessment practice primarily involves identifying a previously published instrument from the literature. Academic centers need to invest in ongoing quality control efforts including the pilot testing of any proposed instruments.
Related Concept Videos
Nursing Process for Patient and Caregiver Teaching I: Assessment and Diagnosis
It is critical to determine the patient's learning needs during the assessment. Determination of learning needs compounds data from the...
Obedience
Nursing Process for Patient and Caregiver Teaching III: Evaluation and Documentation
Nurses can use several methods to evaluate patient outcomes. For example, oral questions can assess cognitive learning, patient...
Accountability and Responsibility of a Nurse I
Accountability and Responsibility of a Nurse II
For example, a nurse demonstrating respect and compassion might listen attentively to a patient's concerns, provide comfort...
Surveys