Related Experiment Video
Updated: Oct 2, 2025
08:27
Applying an eMASS Customization Program as a Research Tool to Evaluate Consumer Benefits
Published on: September 27, 2019
7.0K
Reliability coefficients for multiple group item response theory models.
Björn Andersson1, Hao Luo2, Kseniia Marcq1
1University of Oslo, Norway.
Summary
This study introduces new reliability coefficients for psychological and educational assessments, accounting for group differences. The methods provide accurate estimates and confidence intervals, crucial for precise measurement across diverse populations.
Area of Science:
- Psychometrics
- Educational Measurement
- Item Response Theory (IRT)
Background:
- Score reliability is essential for measurement precision in psychological and educational assessments.
- Reliability is population-dependent and can vary significantly across different demographic groups.
- Existing methods often fail to account for group-specific latent distributions and differential item functioning in reliability estimation.
Purpose of the Study:
- To introduce novel group-specific and overall reliability coefficients for sum scores and ability estimates.
- To develop methods based on a multiple group item response theory model to address population dependence.
- To provide accurate confidence intervals for these reliability estimates.
Main Methods:
- Development of group-specific and overall reliability coefficients within a multiple group item response theory framework.
- Derivation of confidence intervals using asymptotic theory.
- Evaluation of the proposed estimators and confidence intervals through a comprehensive simulation study.
Main Results:
- The developed reliability estimators demonstrate largely unbiased performance.
- Confidence intervals derived using asymptotic theory are accurate, particularly with moderately large sample sizes.
- The approach was successfully exemplified using the Montreal Cognitive Assessment (MoCA) across education level groups.
Conclusions:
- The proposed item response theory-based approach effectively estimates reliability coefficients that account for group differences.
- The methods provide accurate and reliable measures of score precision for diverse populations.
- Recommendations are provided for the practical application of these advanced reliability estimation techniques in applied research.
Related Concept Videos
01:29Reliability and Validity
13.3K
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
13.3K
01:25Multiple Regression
3.2K
Multiple regression assesses a linear relationship between one response or dependent variable and two or more independent variables. It has many practical applications.
Farmers can use multiple regression to determine the crop yield based on more than one factor, such as water availability, fertilizer, soil properties, etc. Here, the crop yield is the response or dependent variable as it depends on the other independent variables. The analysis requires the construction of a scatter plot...
Farmers can use multiple regression to determine the crop yield based on more than one factor, such as water availability, fertilizer, soil properties, etc. Here, the crop yield is the response or dependent variable as it depends on the other independent variables. The analysis requires the construction of a scatter plot...
3.2K
01:20Kendall's Coefficient of Concordance
583
Kendall's Coefficient of Concordance (W), also known as Kendall's W, is a non-parametric statistical measure used to assess the agreement or concordance between multiple raters or judges when they rank a set of items. It is often used when you have ordinal data (ranks) and you want to see if there is consistency or consensus among the raters. It is widely applied in research areas such as psychology, medicine, and social sciences, where multiple judges are asked to rank or rate subjects...
583
01:21Friedman Two-way Analysis of Variance by Ranks
322
Friedman's Two-Way Analysis of Variance by Ranks is a nonparametric test designed to identify differences across multiple test attempts when traditional assumptions of normality and equal variances do not apply. Unlike conventional ANOVA, which requires normally distributed data with equal variances, Friedman's test is ideal for ordinal or non-normally distributed data, making it particularly useful for analyzing dependent samples, such as matched subjects over time or repeated measures...
322
01:11Calculating and Interpreting the Linear Correlation Coefficient
6.6K
The correlation coefficient, r, developed by Karl Pearson in the early 1900s, is numerical and provides a measure of strength and direction of the linear association between the independent variable, x, and the dependent variable, y. Hence, it is also known as the Pearson product-moment correlation coefficient. It can be calculated using the following equation:
6.6K
01:20Comparing the Survival Analysis of Two or More Groups
327
Survival analysis is a cornerstone of medical research, used to evaluate the time until an event of interest occurs, such as death, disease recurrence, or recovery. Unlike standard statistical methods, survival analysis is particularly adept at handling censored data—instances where the event has not occurred for some participants by the end of the study or remains unobserved. To address these unique challenges, specialized techniques like the Kaplan-Meier estimator, log-rank test, and...
327