Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Stereotypes, Prejudice, and Discrimination02:55

Stereotypes, Prejudice, and Discrimination

89.6K
Humans are very diverse and although we share many similarities, we also have many differences. The social groups we belong to help form our identities (Tajfel, 1974). These differences may be difficult for some people to reconcile, which may lead to prejudice toward people who are different. Prejudice is a negative attitude and feeling toward an individual based solely on one’s membership in a particular social group (Allport, 1954; Brown, 2010). Prejudice is common against people who...
89.6K
Response Surface Methodology01:16

Response Surface Methodology

64
Response Surface Methodology (RSM) is a collection of statistical and mathematical techniques used to develop, improve, and optimize processes. It is particularly valuable when many input variables or factors potentially influence a response variable.
The process of RSM involves several key steps:
64
Rational Emotive Behavior Therapy01:24

Rational Emotive Behavior Therapy

27
Cognitive-behavioral therapies (CBTs) are grounded in the belief that our thoughts profoundly influence our emotions and actions. Advocates of CBT emphasize three core assumptions: first, that cognitions are identifiable and measurable; second, that they are central to psychological functioning; and third, that irrational or maladaptive beliefs can be replaced with rational and adaptive ones. This transformative approach to therapy has paved the way for specific models such as Albert...
27
Triarchic Theory of Intelligence01:24

Triarchic Theory of Intelligence

5.4K
Robert Sternberg's triarchic theory of intelligence posits that intelligence is composed of three distinct but interrelated components: analytical, creative, and practical intelligence.
5.4K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Exploring the Influence of Response Styles on Continuous Scale Assessments: Insights From a Novel Modeling Approach.

Educational and psychological measurement·2024
Same author

Modeling Rating Order Effects Under Item Response Theory Models for Rater-Mediated Assessments.

Applied psychological measurement·2023
Same author

Diagnostic Classification Model for Forced-Choice Items and Noncognitive Tests.

Educational and psychological measurement·2023
Same author

A Mixture IRTree Model for Performance Decline and Nonignorable Missing Data.

Educational and psychological measurement·2020
Same author

Utilizing response times in cognitive diagnostic computerized adaptive testing under the higher-order deterministic input, noisy 'and' gate model.

The British journal of mathematical and statistical psychology·2019

Related Experiment Video

Updated: May 11, 2025

Evaluating Usability Aspects of a Mixed Reality Solution for Immersive Analytics in Industry 4.0 Scenarios
06:02

Evaluating Usability Aspects of a Mixed Reality Solution for Immersive Analytics in Industry 4.0 Scenarios

Published on: October 6, 2020

2.1K

Understanding Rater Cognition in Performance Assessment: A Mixed IRTree Approach.

Hung-Yu Huang1

  • 1National Cheng Kung University, Taiwan.

Applied Psychological Measurement
|April 17, 2025
PubMed
Summary
This summary is machine-generated.

This study introduces a mixed item response theory (IRT) model to differentiate between professional and novice raters in assessments. Accounting for rater types improves judgment validity and reduces estimation bias in performance evaluations.

Keywords:
IRTreeitem response theoryrater cognitionrater-mediated assessments

More Related Videos

Three Laboratory Procedures for Assessing Different Manifestations of Impulsivity in Rats
09:12

Three Laboratory Procedures for Assessing Different Manifestations of Impulsivity in Rats

Published on: March 17, 2019

9.4K
Operant Procedures for Assessing Behavioral Flexibility in Rats
08:30

Operant Procedures for Assessing Behavioral Flexibility in Rats

Published on: February 15, 2015

20.7K

Related Experiment Videos

Last Updated: May 11, 2025

Evaluating Usability Aspects of a Mixed Reality Solution for Immersive Analytics in Industry 4.0 Scenarios
06:02

Evaluating Usability Aspects of a Mixed Reality Solution for Immersive Analytics in Industry 4.0 Scenarios

Published on: October 6, 2020

2.1K
Three Laboratory Procedures for Assessing Different Manifestations of Impulsivity in Rats
09:12

Three Laboratory Procedures for Assessing Different Manifestations of Impulsivity in Rats

Published on: March 17, 2019

9.4K
Operant Procedures for Assessing Behavioral Flexibility in Rats
08:30

Operant Procedures for Assessing Behavioral Flexibility in Rats

Published on: February 15, 2015

20.7K

Area of Science:

  • Educational Measurement
  • Psychometrics
  • Cognitive Psychology

Background:

  • Human raters in assessments face challenges in accurately judging ratee competencies.
  • Rater cognition research indicates personal preferences and impersonal judgments impact evaluation validity.
  • Existing models may not adequately account for the heterogeneity of rater expertise.

Purpose of the Study:

  • To introduce a novel mixed IRTree-based model for rater judgments (MIM-R).
  • To differentiate between professional and novice raters within assessment contexts.
  • To improve the validity of rater judgments and reduce estimation bias.

Main Methods:

  • Sequential application of ideal-point and dominance item response theory (IRT) models.
  • Development of a mixed IRTree-based model (MIM-R) to capture rater cognition.
  • Simulation studies to evaluate model parameter recovery and bias.

Main Results:

  • The MIM-R model demonstrated satisfactory recovery of its parameters in simulations.
  • Neglecting the mixed nature of raters leads to biased estimations, especially with more novice raters.
  • The model successfully identifies professional and novice raters based on their judgmental processes.

Conclusions:

  • Considering the mixed nature of raters is crucial for accurate assessment validity.
  • The MIM-R model offers a robust approach to analyzing rater cognition and expertise.
  • Empirical application in creativity assessment highlights the practical implications of the MIM-R model.