Jove
Visualize
Contact Us

Related Experiment Videos

Whose criterion standard is it anyway?

Gregory Ethan Stone1

  • 1Foundations of Education, The University of Toledo, Mailstop #923, OH 43566 USA. gregory.stone@utoledo.edu

Journal of Applied Measurement
|April 25, 2006
PubMed
Summary

Judge characteristics significantly influence criterion-referenced standards in professional testing. These standards are sample-dependent, highlighting the need for further research into judge decision-making for reliable testing programs.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same journal

Development of a Short Form of the CPAI-A (Form B) with Rasch Analyses.

Journal of applied measurement·2021
Same journal

Evaluating the Impact of Multidimensionality on Type I and Type II Error Rates using the Q-Index Item Fit Statistic for the Rasch Model.

Journal of applied measurement·2021
Same journal

Diabetes Distress in Emerging Adults: Refining the Problem Areas in Diabetes-Emerging Adult Version using Rasch Analysis.

Journal of applied measurement·2021
Same journal

A Psychometric Replication of Fan (1998) Item Response Theory and Classical Test Theory: An Empirical Comparison of their Item/Person Statistics.

Journal of applied measurement·2021
Same journal

The Development of the Mental Toughness Situational Judgment Test: A Novel Approach to Assessing Mental Toughness.

Journal of applied measurement·2021
Same journal

Using the Rasch Model to Measure Comprehension of Fraction Addition.

Journal of applied measurement·2021
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Area of Science:

  • Professional Testing
  • Psychometrics
  • Health Professions Education

Background:

  • Criterion-referenced standards are crucial for professional testing programs.
  • Evidence suggests judge characteristics (e.g., age, gender) influence decision-making.
  • Understanding these influences is key to standard setting validity.

Purpose of the Study:

  • To explore the impact of judge characteristics on criterion-referenced standards.
  • To investigate differences in judgment-making within health-care related professional boards.
  • To assess the sample dependency of established examination cutoff points.

Main Methods:

  • Utilized two health-care professional boards with judge panels (26 and 30 members).
  • Employed a simplified Objective Standard Setting exercise to define examination cutoff points.
  • Applied multi-faceted Rasch analyses to detect and explore judge differences.

Main Results:

  • Significant differences were observed among panel judges based on examined characteristics.
  • These differences were not always consistent across all characteristics.
  • Results indicate criterion-referenced standards are linked to judges' normative experiences and are sample-dependent.

Conclusions:

  • Criterion-referenced standards are inherently influenced by the normative experiences of the judges.
  • The sample dependency of these standards raises concerns about their universal applicability and predictive validity.
  • Further research is needed to ensure the meaningful independence and validity of criterion-referenced standards in professional testing.

Related Experiment Videos