Related Experiment Video
Updated: Aug 5, 2026
07:34
Perceptual and Category Processing of the Uncanny Valley Hypothesis' Dimension of Human Likeness: Some Methodological Issues
Published on: June 3, 2013
Using Explanatory Item Response Theory to Study Rating-Scale Design Effects on Response Style Discrimination
Munevver Ilgun Dibek1, Daniel Bolt2
1TED University, Ankara, Türkiye.
Educational and Psychological Measurement
|July 31, 2026
Summary
This study introduces an explanatory Item Response Theory (IRT) method to analyze how rating scale design affects response style sensitivity. Findings show category number influences discrimination, but overall effects are weakened by significant residual variability.
Area of Science:
- Psychometrics
- Statistical Modeling
- Survey Methodology
Background:
- Rating scale design features can influence how respondents answer survey items.
- Understanding item sensitivity to response styles is crucial for accurate data interpretation.
- Previous research has explored rating scale variations, but a unified framework for analyzing their impact on response style sensitivity is lacking.
Purpose of the Study:
- To develop and apply an explanatory Item Response Theory (IRT) methodology to investigate the impact of rating scale design features on item sensitivity to response style.
- To model response style as a latent trait within a multidimensional nominal response model (MNRM).
- To examine how manipulated rating scale features (number of categories, labeling, polarity) predict item discrimination on the response style dimension.
Main Methods:
- Utilized item response data from a two-part experimental study manipulating rating scale features.
- Applied a multidimensional nominal response model (MNRM) within an explanatory IRT framework.
- Treated item discrimination parameters on the response style dimension as outcomes predicted by design features.
Main Results:
- The number of rating scale categories was generally associated with stronger response-style discrimination.
- Rating scale wave, labeling, and polarity conditions showed less influence on response style discrimination.
- Substantial residual variability in response style discrimination was observed at both within- and between-item levels, indicating design features have relatively weak effects.
Conclusions:
- The developed MNRM within an explanatory IRT framework is feasible for exploring item-level sensitivities to response style.
- The framework allows evaluation of design effects against residual heterogeneity in response style discrimination.
- Significant residual heterogeneity explains inconsistencies in findings regarding the effects of design characteristics across studies.
Related Concept Videos
01:16Response Surface Methodology
Response Surface Methodology (RSM) is a collection of statistical and mathematical techniques used to develop, improve, and optimize processes. It is particularly valuable when many input variables or factors potentially influence a response variable.
The process of RSM involves several key steps:
The process of RSM involves several key steps:
02:01Group Design
The most basic experimental design involves two groups: the experimental group and the control group. The two groups are designed to be the same except for one difference— experimental manipulation. The experimental group gets the experimental manipulation—that is, the treatment or variable being tested—and the control group does not. Since experimental manipulation is the only difference between the experimental and control groups, we can be sure that any differences between the two are due to...
02:01Factorial Design
Factorial Analysis is an experimental design that applies Analysis of Variance (ANOVA) statistical procedures to examine a change in a dependent variable due to more than one independent variable, also known as factors. Changes in worker productivity can be reasoned, for example, to be influenced by salary and other conditions, such as skill level. One way to test this hypothesis is by categorizing salary into three levels (low, moderate, and high) and skills sets into two levels (entry level...
01:29Theory of Attribution II: Kelley's Covariation Theory
Attribution theory plays a crucial role in social psychology, helping to explain how individuals interpret the causes of behavior. One prominent model within this field is Harold Kelley's covariation theory, which provides a systematic approach to determining whether internal traits or external circumstances drive a person's actions. The model posits that individuals rely on three key types of information—consensus, consistency, and distinctiveness—to make these judgments.Consensus: Comparing...
01:15Theory of Attribution I: Correspondent Inference Theory
Correspondent inference theory, proposed by Jones and Davis in 1965, seeks to explain how individuals infer stable personality traits from observed behaviors. It suggests that people attribute actions to underlying dispositions rather than external circumstances, particularly when the behavior appears intentional and socially significant.Voluntary Behavior and Dispositional AttributionAccording to this theory, individuals are more likely to attribute behavior to personal traits when it appears...
01:26Causes of Similarity-Dissimilarity Effect
The similarity-dissimilarity effect, a fundamental concept in social psychology, explains how interpersonal similarities and differences influence attraction and social interactions. This effect is supported by three key psychological perspectives: balance theory, social comparison theory, and consensual validation.Balance Theory and Cognitive ConsistencyBalance theory, developed by Fritz Heider, posits that individuals seek cognitive consistency in their relationships. When two people share...