Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Systematic Error: Methodological and Sampling Errors01:15

Systematic Error: Methodological and Sampling Errors

8.7K
In the case of systematic errors, the sources can be identified, and the errors can be subsequently minimized by addressing these sources. According to the source, systematic errors can be divided into sampling, instrumental, methodological, and personal errors.
Sampling errors originate from improper sampling methods or the wrong sample population. These errors can be minimized by refining the sampling strategy. Defective instruments or faulty calibrations are the sources of instrumental...
8.7K
Reliability and Validity01:29

Reliability and Validity

12.9K
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
12.9K
Types of Errors: Detection and Minimization01:12

Types of Errors: Detection and Minimization

8.8K
Error is the deviation of the obtained result from the true, expected value or the estimated central value. Errors are expressed in absolute or relative terms.
Absolute error in a measurement is the numerical difference from the true or central value. Relative error is the ratio between absolute error and the true or central value, expressed as a percentage.
Errors can be classified by source, magnitude, and sign. There are three types of errors: systematic, random, and gross.
Systematic or...
8.8K
Hindsight Biases01:12

Hindsight Biases

3.5K
Hindsight bias leads you to believe that the event you just experienced was predictable, even though it really wasn’t. In other words, you knew all along that things would turn out the way they did. Can you relate this to the phrase "Hindsight is 20/20" now? 
3.5K
Theory of Attribution II: Kelley's Covariation Theory01:29

Theory of Attribution II: Kelley's Covariation Theory

1.1K
Attribution theory plays a crucial role in social psychology, helping to explain how individuals interpret the causes of behavior. One prominent model within this field is Harold Kelley's covariation theory, which provides a systematic approach to determining whether internal traits or external circumstances drive a person's actions. The model posits that individuals rely on three key types of information—consensus, consistency, and distinctiveness—to make these judgments.Consensus:...
1.1K
Errors In Hypothesis Tests01:14

Errors In Hypothesis Tests

4.4K
When performing a hypothesis test, there are four possible outcomes depending on the actual truth (or falseness) of the null hypothesis and the decision to reject or not.
4.4K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

The certification status of generalist physicians and the mortality of their patients after acute myocardial infarction.

Academic medicine : journal of the Association of American Medical Collegesยท2001
Same author

The validity of long cases.

Medical educationยท2001
Same author

Setting performance standards for medical practice: a theoretical framework.

Medical educationยท2001
Same author

Clinical teaching: maintaining an educational role for doctors in the new health care environment.

Medical educationยท2000
Same author

A review of the evaluation of clinical teaching: new perspectives and challenges.

Medical educationยท2000
Same author

What should we do about unreliable scores?

Medical educationยท2000

Related Experiment Video

Updated: May 6, 2026

Problem-Solving Before Instruction PS-I: A Protocol for Assessment and Intervention in Students with Different Abilities
10:26

Problem-Solving Before Instruction PS-I: A Protocol for Assessment and Intervention in Students with Different Abilities

Published on: September 11, 2021

3.2K

The answer key as a source of error in performance assessment.

J Norcini1, R Reshetar, R Lipner

  • 1American Board of Internal Medicine, 3624 Market Street, 19104, Philadelphia, PA, USA, jnorcini@ABIM.org.

Advances in Health Sciences Education : Theory and Practice
|November 2, 2013
PubMed
Summary

Answer key construction variability has minimal impact on measurement error in performance-based assessments. This study found that while some error exists, it does not hinder the use of video-based tests for evaluating cardiologists.

More Related Videos

Operant Procedures for Assessing Behavioral Flexibility in Rats
08:30

Operant Procedures for Assessing Behavioral Flexibility in Rats

Published on: February 15, 2015

20.9K
Universal Screening for Prevention of Reading, Writing, and Math Disabilities in Spanish
14:43

Universal Screening for Prevention of Reading, Writing, and Math Disabilities in Spanish

Published on: July 18, 2020

7.7K

Related Experiment Videos

Last Updated: May 6, 2026

Problem-Solving Before Instruction PS-I: A Protocol for Assessment and Intervention in Students with Different Abilities
10:26

Problem-Solving Before Instruction PS-I: A Protocol for Assessment and Intervention in Students with Different Abilities

Published on: September 11, 2021

3.2K
Operant Procedures for Assessing Behavioral Flexibility in Rats
08:30

Operant Procedures for Assessing Behavioral Flexibility in Rats

Published on: February 15, 2015

20.9K
Universal Screening for Prevention of Reading, Writing, and Math Disabilities in Spanish
14:43

Universal Screening for Prevention of Reading, Writing, and Math Disabilities in Spanish

Published on: July 18, 2020

7.7K

Area of Science:

  • Medical Education
  • Psychometrics

Background:

  • Performance-based assessments are crucial for evaluating clinical skills.
  • Variability in answer key construction can introduce measurement error.

Purpose of the Study:

  • To assess the impact of answer key construction variability on measurement error.
  • To compare methods for reducing this error in video-based assessments.

Main Methods:

  • A video-based format assessing arteriogram interpretation was used.
  • Generalizability theory and aggregate scoring were applied to estimate error variance.

Main Results:

  • Scorer variability was less than case variability.
  • Several scorers significantly reduced measurement error.

Conclusions:

  • Answer key construction error is relatively small.
  • This error should not impede the use of performance-based item formats.