Related Experiment Video

Updated: Jan 18, 2026

Operant Procedures for Assessing Behavioral Flexibility in Rats

Operant Procedures for Assessing Behavioral Flexibility in Rats

Published on: February 15, 2015

Do you see what I see? Mitigating rater effects on direct behavior ratings-multi-item scales (DBR-MIS) through

Abigail E Pruitt¹, Kara M Styck¹, Annika L Hodges¹

¹Department of Psychology, Northern Illinois University.

School Psychology (Washington, D.C.)

|June 5, 2025

Abstract:

The purpose of our study was to compare the effectiveness of rater training and statistical adjustment at mitigating rater effects and improving the accuracy of direct behavior ratings-multi-item scales (DBR-MIS) scores targeting academic engagement and disruptive behavior. Results from a many-facet Rasch measurement analysis with a sample of video clips of 15 middle school students rated by 10 graduate students indicated that raters significantly differed in their tendencies toward severity/leniency and that rater training was only successful at reducing between-rater differences on disruptive behavior. Unfortunately, neither rater training nor statistical adjustment improved DBR-MIS score accuracy when compared to direct observation, though improved accuracy was noted for ratings on a single DBR-MIS disruptive behavior item (i.e., "noisy"). School personnel may wish to consider rater training when using DBR-MIS to assess disruptive behavior, and future research should explore the use of statistical modeling to develop customized rater training that incorporates the idiosyncrasies of individual raters. (PsycInfo Database Record (c) 2026 APA, all rights reserved).

More Related Videos

Testing for Metacognitive Responding Using an Odor-based Delayed Match-to-Sample Test in Rats

Testing for Metacognitive Responding Using an Odor-based Delayed Match-to-Sample Test in Rats

Published on: June 18, 2018

Pavlovian Conditioned Approach Training in Rats

Pavlovian Conditioned Approach Training in Rats

Published on: February 4, 2016

Related Concept Videos

Halo Effect

Halo Effect

The halo effect is a cognitive bias in which an individual's overall impression influences judgments about their specific traits. This psychological phenomenon leads people to associate positive characteristics with those they perceive as generally good and negative characteristics with those they view as bad. This effect is particularly influential in social perception, professional evaluations, and decision-making processes.The Psychological Basis of the Halo EffectThe halo effect is rooted...

Response Surface Methodology

Response Surface Methodology

Response Surface Methodology (RSM) is a collection of statistical and mathematical techniques used to develop, improve, and optimize processes. It is particularly valuable when many input variables or factors potentially influence a response variable.
The process of RSM involves several key steps:

Articles linked to this work by shared authors, journal, and citation graph.

Utility of the social skills improvement system-rating scales for capturing dynamic social constructs: Evidence using the measurement model of derivatives.

Child development·2024

Evaluating the factor structure and measurement invariance of the Academic Competence Evaluation Scales-Short Form.

School psychology (Washington, D.C.)·2023

Using many-facet rasch measurement and generalizability theory to explore rater effects for direct behavior rating-multi-item scales.

School psychology (Washington, D.C.)·2022

Dimensionality of the State-Trait Inventory of Cognitive and Somatic Anxiety.

Assessment·2020

Examining Rater Effects on the Classroom Assessment Scoring System.

Child development·2020

The SSIS SEL Brief Scales-Student Form: Initial development and validation.

School psychology (Washington, D.C.)·2020

See all related articles