Related Experiment Video
Updated: Sep 18, 2025
10:17
Improving Student Outcomes with an Adaptable Molecular Cloning Course-Based Undergraduate Research Experience
Published on: November 15, 2024
1.2K
Clerkship Assessment Reliability Based on Assessor Type and Assessor-Student Contact Frequency: A Generalizability
Summary
Clinical performance assessments (CPAs) are more reliable when completed by residents with daily student contact. Understanding assessor characteristics can improve the trustworthiness of medical education assessment systems.
Area of Science:
- Medical Education
- Assessment Reliability
- Clinical Performance Assessments
Background:
- Workplace-based assessments (WBAs), including clinical performance assessments (CPAs), are crucial components of medical clerkship assessment systems.
- Variability in CPA reliability across clerkships suggests a need to investigate contributing factors.
- Assessor characteristics, such as type and frequency of student contact, may influence assessment reliability.
Purpose of the Study:
- To explore the reliability of clinical performance assessments (CPAs) in medical clerkships.
- To examine the impact of assessor characteristics, specifically assessor type (resident vs. attending) and assessor-student contact frequency (daily vs. not daily), on CPA reliability.
Main Methods:
- Analysis of 4,062 CPAs from third-year medical students across five clerkships.
- Generalizability analysis incorporating clerkship, assessor type, and contact frequency.
- Decision studies to determine the number of assessments needed for a reliability of 0.7 (D0.7).
Main Results:
- Assessments by residents were more reliable (φ=0.62) than by attendings (φ=0.48).
- Assessments by those with daily contact were more reliable (φ=0.63) than those with less frequent contact (φ=0.31).
- Residents with daily contact demonstrated the highest reliability (φ=0.38).
Conclusions:
- Assessor type and frequency of student contact significantly influence CPA reliability.
- These factors can explain inter-clerkship variability in assessment reliability.
- Optimizing assessor characteristics in assessment system design can enhance reliability and stakeholder trust.
More Related Videos
09:00Author Spotlight: Validation of SICOLE-R for Assessing Cognitive and Reading Skills in Spanish-Speaking Children and Its Role in Personalized Education
Published on: August 16, 2024
921
08:40Isokinetic Robotic Device to Improve Test-Retest and Inter-Rater Reliability for Stretch Reflex Measurements in Stroke Patients with Spasticity
Published on: June 12, 2019
7.6K
Related Concept Videos
02:16Surveys
15.4K
Often, psychologists develop surveys as a means of gathering data. Surveys are lists of questions to be answered by research participants, and can be delivered as paper-and-pencil questionnaires, administered electronically, or conducted verbally. Generally, the survey itself can be completed in a short time, and the ease of administering a survey makes it easy to collect data from a large number of people.
15.4K
01:29Reliability and Validity
13.2K
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
13.2K
02:13The Representativeness Heuristic
16.2K
The representative heuristic describes a biased way of thinking, in which you unintentionally stereotype someone or something. For example, you may assume that your professors spend their free time reading books and engaging in intellectual conversation, because the idea of them spending their time playing volleyball or visiting an amusement park does not fit in with your stereotypes of professors.
16.2K
01:22Case Studies
12.1K
There are many research methods available to psychologists in their efforts to understand, describe, and explain behavior and the cognitive and biological processes that underlie it.
12.1K
02:30Naturalistic Observations
16.0K
If you want to understand how behavior occurs, one of the best ways to gain information is to simply observe the behavior in its natural context. However, people might change their behavior in unexpected ways if they know they are being observed. How do researchers obtain accurate information when people tend to hide their natural behavior? As an example, imagine that your professor asks everyone in your class to raise their hand if they always wash their hands after using the restroom. Chances...
16.0K
02:20Correlations
34.0K
Correlation means that there is a relationship between two or more variables (such as ice cream consumption and crime), but this relationship does not necessarily imply cause and effect. When two variables are correlated, it simply means that as one variable changes, so does the other. We can measure correlation by calculating a statistic known as a correlation coefficient. A correlation coefficient is a number from -1 to +1 that indicates the strength and direction of the relationship between...
34.0K