Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Correcting performance-rating errors in oral examinations.

M R Raymond1, L C Webb, W M Houston

  • 1American College Testing, Iowa City, IA.

Evaluation & the Health Professions
|February 8, 1991
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

A Case of Acute Rheumatism Treated with Full Doses of Quinine.

The North-Western medical and surgical journal·2023
Same author

Formalin against Flies.

The Indian medical gazette·2017
Same author

Generation of a high-density rat EST map.

Genome research·2001
Same author

Ocular Munchausen's syndrome, a costly disorder.

Canadian journal of ophthalmology. Journal canadien d'ophtalmologie·1999
Same author

How well does the psychiatry residency in-training examination predict performance on the American Board of Psychiatry and Neurology. Part I. Examination?

The American journal of psychiatry·1996
Same author

The use of cognitive taxonomies in licensure and certification test development: reasonable or customary?

Evaluation & the health professions·1995
Same journal

The "Twilight Zone" Is a Danger Zone: Why the Occupational-Clinical Divide in Burnout Assessment Is a False Dichotomy.

Evaluation & the health professions·2026
Same journal

Evaluating Equity in AI-Supported Functional Assessment: Agreement Between Clinician Judgment and Digital Metrics in Stroke Rehabilitation.

Evaluation & the health professions·2026
Same journal

Psychometric Properties of the Arabic Version of the PROMIS Sleep Disturbance 8b Short Form Among Nurses.

Evaluation & the health professions·2026
Same journal

Commentary: Systemic Inequities in Japan's Technical Intern Training Program (TITP): Health, Labor, and Legal Vulnerabilities of Foreign Trainees.

Evaluation & the health professions·2026
Same journal

Application of Patient-Reported Outcome Measurements in Traditional Chinese Medicine Clinical Trials for Musculoskeletal Disorders in China: A Registry-Based Analysis.

Evaluation & the health professions·2026
Same journal

Divergent Socioeconomic Pathways to Biologically Uncontrolled Diabetes by Gender: A Bayesian Analysis of NHANES 2021-2023.

Evaluation & the health professions·2026
See all related articles

Oral examinations can be unreliable due to evaluator leniency or stringency. This study introduces a regression method to correct these rating errors, improving competence assessment accuracy and potentially altering pass/fail decisions.

Area of Science:

  • Medical Education
  • Psychometrics
  • Performance Assessment

Background:

  • Oral examinations are common for assessing competence but suffer from inconsistent reliability.
  • Evaluator leniency and stringency introduce significant error into performance ratings.
  • Existing methods for correcting rating bias are complex or limited.

Purpose of the Study:

  • To present and evaluate a simple regression-based method for identifying and correcting leniency and stringency errors in oral examinations.
  • To provide corrected performance estimates for examinees and bias indices for raters.
  • To assess the impact of bias correction on pass/fail decisions in a medical specialty examination.

Main Methods:

  • A regression model, adapted from Wilson (1988) and Cason & Cason (1984, 1985), was used to analyze rating data.

Related Experiment Videos

  • The method estimates individual performance adjusted for rater leniency/stringency.
  • Rater bias indices and other statistical properties of the rating data were generated.

    • Main Results:

    • The regression method successfully identified significant levels of leniency and stringency in oral examination ratings.
    • Correcting for these biases altered pass/fail outcomes for approximately 6% of examinees.
    • The method provided valuable statistics for evaluating the quality of rating data.

    Conclusions:

    • The proposed regression method offers a practical approach to enhance the reliability of oral examinations.
    • Adjusting for rater bias can significantly impact examinee outcomes, underscoring the importance of bias correction.
    • Further research is needed to refine the procedure and explore its broader applications.