Related Experiment Video
Updated: May 16, 2026
10:26
Problem-Solving Before Instruction (PS-I): A Protocol for Assessment and Intervention in Students with Different Abilities
Published on: September 11, 2021
How IRT can solve problems of ipsative data in forced-choice questionnaires
Anna Brown1, Alberto Maydeu-Olivares
1School of Psychology, Keynes College, University of Kent, Canterbury, United Kingdom. A.A.Brown@kent.ac.uk
Psychological Methods
|November 15, 2012
Summary
Multidimensional forced-choice (MFC) questionnaires can reduce response biases. Item response theory (IRT) modeling using Thurstone
Area of Science:
- Psychometrics
- Psychological Measurement
- Quantitative Psychology
Background:
- Multidimensional forced-choice (MFC) questionnaires present items measuring different attributes in blocks, requiring participants to rank order items.
- Comparative formats in MFC questionnaires can mitigate response biases common in single-stimulus (rating/Likert) scales.
- Traditional scoring of MFC instruments yields ipsative data, distorting individual profiles, construct validity, criterion-related validity, and reliability.
Purpose of the Study:
- To address the limitations of traditional scoring methods for MFC questionnaires.
- To advocate for the use of item response theory (IRT) models suitable for comparative data.
- To demonstrate the effectiveness of Thurstonian IRT modeling for scoring MFC instruments.
Main Methods:
- Applying item response theory (IRT) models, specifically Thurstone's law of comparative judgment, to score MFC data.
- Utilizing Thurstonian IRT modeling (Brown & Maydeu-Olivares, 2011) to analyze existing MFC questionnaires.
- Comparing the outcomes of IRT-based scoring with traditional ipsative scoring methods.
Main Results:
- Thurstonian IRT modeling adequately scores MFC questionnaires, even those with complex features.
- IRT-estimated scores derived from MFC instruments are free from the distortions associated with ipsative data.
- The proposed IRT approach resolves issues in validity and reliability caused by ipsative scoring.
Conclusions:
- Inadequate scoring is the root cause of ipsative data problems in MFC questionnaires.
- Item response theory (IRT) models, particularly Thurstonian IRT, offer a robust solution for scoring comparative data.
- IRT-based scoring enhances the validity and reliability of MFC questionnaire data, overcoming ipsative limitations.
Related Concept Videos
02:16Surveys
Often, psychologists develop surveys as a means of gathering data. Surveys are lists of questions to be answered by research participants, and can be delivered as paper-and-pencil questionnaires, administered electronically, or conducted verbally. Generally, the survey itself can be completed in a short time, and the ease of administering a survey makes it easy to collect data from a large number of people.
01:29Measures of Intelligence
Psychologists measure intelligence by using standardized tests that produce a score known as the intelligence quotient or IQ. To understand IQ tests, it's important to recognize the key principles behind their construction: validity, reliability, and standardization.
Validity refers to how well a test measures what it claims to measure. An intelligence test should accurately assess intelligence rather than another characteristic, like anxiety. Criterion validity is one way to evaluate this; it...
Validity refers to how well a test measures what it claims to measure. An intelligence test should accurately assess intelligence rather than another characteristic, like anxiety. Criterion validity is one way to evaluate this; it...
01:17Cochran's Q Test
Cochran's Q Test is a nonparametric statistical test used to determine if there are potential differences in the outcomes of three or more related groups on a binary (yes/no) or dichotomous outcome. It is essentially an extension of the McNemar Test, which is limited to two related samples - Cochran's Q test can handle three or more related samples, making it more versatile in scenarios where subjects are measured under multiple conditions. The test statistic follows a Chi-Square distribution,...
01:21Methods of Documentation III: PIE
Problem-intervention-evaluation (PIE) is a systematic approach to documentation used in healthcare settings for clinical decision-making and patient care planning. It is a structured approach to organizing patient data based on problems, interventions, and evaluations. Here's a breakdown of its key features and considerations:
02:01Group Design
The most basic experimental design involves two groups: the experimental group and the control group. The two groups are designed to be the same except for one difference— experimental manipulation. The experimental group gets the experimental manipulation—that is, the treatment or variable being tested—and the control group does not. Since experimental manipulation is the only difference between the experimental and control groups, we can be sure that any differences between the two are due to...
01:09Ion-Exchange Chromatography
Ion-exchange chromatography, or IEC, is a technique for separating ions based on their affinity for the stationary phase. The stationary phase is a cross-linked polymer resin with covalently attached ionic functional groups. The functional groups can be either positively charged (cation exchangers) or negatively charged (anion exchangers). A cation exchanger consists of a polymeric anion and active cations, while an anion exchanger is a polymeric cation with active anions. The choice of...