Related Experiment Video
Updated: Jul 13, 2026
09:09
Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody
Published on: September 27, 2024
Undesired variance due to examiner stringency/leniency effect in communication skill scores assessed in OSCEs
Peter H Harasym1, Wayne Woloschuk, Leslie Cunning
1Department of Community Health Sciences, University of Calgary, Calgary, AB, Canada.
Summary
Examiner variability significantly impacts medical student communication scores in Objective Structured Clinical Examinations (OSCEs). Item Response Theory analysis revealed examiner leniency/stringency accounted for substantial score variance, affecting pass-fail outcomes.
Area of Science:
- Medical Education
- Health Professions Education
- Communication Skills Assessment
Background:
- Physician-patient communication is a crucial, learnable clinical skill.
- Medical schools emphasize teaching and evaluating communication skills.
- Objective Structured Clinical Examinations (OSCEs) are commonly used for assessment.
Purpose of the Study:
- To investigate sources of error variance in OSCE-based communication skill assessments.
- To determine the impact of different examiners on student communication scores.
- To quantify the influence of examiner stringency/leniency on OSCE evaluations.
Main Methods:
- Utilized Item Response Theory (IRT) and a Multifaceted Rasch model.
- Analyzed communication performance data from 6 OSCE stations for medical clerks.
- Employed the Calgary-Cambridge guide for performance rating.
Main Results:
- Examiner variance (0.86) significantly exceeded examinee variance (0.20).
- Examiner stringency/leniency demonstrated high reliability (0.91).
- Approximately 11% of student outcomes changed when using IRT-adjusted 'true' scores.
Conclusions:
- Examiner variability is a major factor influencing OSCE communication scores.
- Standardized examiner training is recommended to mitigate bias.
- Using IRT-derived 'true' scores may improve the fairness of pass-fail decisions.
Related Concept Videos
01:21Barriers to Effective Communication II
The barriers to effective communication also include cultural barriers, semantic barriers, gender barriers, and time constraints.
Cultural barriers:
Differences in values, beliefs, religion, knowledge, and tradition can significantly impact communication. Awareness of nonverbal cues is critical, especially when conversing with a patient from a different culture. What appears appropriate in one culture may be inappropriate in another.
Semantic barriers:
As a result of their tendency to use...
Cultural barriers:
Differences in values, beliefs, religion, knowledge, and tradition can significantly impact communication. Awareness of nonverbal cues is critical, especially when conversing with a patient from a different culture. What appears appropriate in one culture may be inappropriate in another.
Semantic barriers:
As a result of their tendency to use...
01:30Barriers to Effective Communication I
A communication barrier is any distortion or interruption during a conversation, resulting in miscommunication of the message. A good communicator should know these barriers and continuously check for the listener's understanding by obtaining feedback.
Communication barriers include the following:
Physiological barriers: They are limitations caused by a person's health condition or disability, such as hearing loss, poor eyesight, illness, or unconsciousness. An example to overcome this barrier...
Communication barriers include the following:
Physiological barriers: They are limitations caused by a person's health condition or disability, such as hearing loss, poor eyesight, illness, or unconsciousness. An example to overcome this barrier...
01:29Theory of Attribution II: Kelley's Covariation Theory
Attribution theory plays a crucial role in social psychology, helping to explain how individuals interpret the causes of behavior. One prominent model within this field is Harold Kelley's covariation theory, which provides a systematic approach to determining whether internal traits or external circumstances drive a person's actions. The model posits that individuals rely on three key types of information—consensus, consistency, and distinctiveness—to make these judgments.Consensus: Comparing...
01:29Non-Verbal Cues
Non-verbal communication extends beyond gestures and facial expressions to include vocal elements known as paralanguage. Paralanguage consists of non-verbal vocal cues such as pitch, loudness, speech rate, pauses, and non-verbal vocalizations like laughter, sighs, and moans. These elements not only accompany speech but also provide critical emotional and contextual information.The Role of Paralanguage in CommunicationParalanguage adds depth to spoken language by conveying emotions and...
01:17Correspondence Bias
Correspondence bias, also referred to as the fundamental attribution error, describes the tendency to attribute another person’s behavior to internal characteristics rather than situational influences. This cognitive bias leads individuals to overlook external factors that may be influencing actions, thereby fostering potentially inaccurate assessments of others’ intentions and dispositions.Empirical Evidence for Correspondence BiasResearch has consistently demonstrated the prevalence of...
01:15Variance
The deviations show how spread out the data are about the mean. A positive deviation occurs when the data value exceeds the mean, whereas a negative deviation occurs when the data value is less than the mean. If the deviations are added, the sum is always zero. So one cannot simply add the deviations to get the data spread. By squaring the deviations, the numbers are made positive; thus, their sum will also be positive.The standard deviation measures the spread in the same units as the data.