Related Experiment Video
Updated: Oct 12, 2025

The Collective Trust Game: An Online Group Adaptation of the Trust Game Based on the HoneyComb Paradigm
Published on: October 20, 2022
The critical role of direct observation in entrustment decisions
Matthew Sibbald1, Muqtasid Mansoor2, Michael Tsang2
1McMaster Faculty of Health Sciences Education Research, Innovation and Program (MERIT), McMaster University, Ontario, Canada.
Background:
Entrustment decisions may be retrospective (based on past experiences with a trainee) or real-time (based on direct observation). We investigated judgments of entrustment based on assessor prior knowledge of candidates and based on systematic direct observation, conducted in an objective structured clinical exam (OSCE).
Methods:
Sixteen faculty examiners provided 287 retrospective and real-time entrustment ratings of 16 cardiology trainees during OSCE stations in 2019 and 2020. Reliability and validity of these ratings were assessed by comparing correlations across stations as a measure of reliability, differences across postgraduate years as an index of construct validity, correlation to standardized in-training exam (ITE) as a measure of criterion validity, and reclassification of entrustment as a measure of consequential validity.
Results:
Both retrospective and real-time assessments were highly reliable (all intra-class correlations >0.86). Both increased with a year of postgraduate training. Real-time entrustment ratings were significantly correlated with standardized ITE scores; retrospective ratings were not. Real-time ratings explained 37% (2019) and 46% (2020) of variance in examination scores vs. 21% (2019) and 7% (2020) for retrospective ratings. Direct observation resulted in a different level of entrustment compared with retrospective ratings in 44% of cases (p = <0.001).
Conclusions:
Ratings based on direct observation made unique contributions to entrustment decisions.
Related Concept Videos
Naturalistic Observations
Actor-Observer Effect
Obedience
Decision Making: Traditional Method
First, a specific claim about the population parameter is decided based on the research question and is stated in a simple form. Further, an opposing statement to this claim is also stated. These statements can act as null and alternative hypotheses, out of which a null hypothesis would be a...
Reason and Intuition
Data Collection by Observations
An astronomer viewing the motion and brightness of stars in the sky and recording the data is an example of observational data collection. A botanist recording...

