A Confirmatory Analysis of Item Reliability Trends (CAIRT): Differentiating True Score and Error Variance in the
Johannes Hartig1, Britta Hölzel2, Helfried Moosbrugger3
1a German Institute for International Educational Research , Frankfurt am Main , Germany.
Abstract:
Numerous studies have shown increasing item reliabilities as an effect of the item position in personality scales. Traditionally, these context effects are analyzed based on item-total correlations. This approach neglects that trends in item reliabilities can be caused either by an increase in true score variance or by a decrease in error variance. This article presents the Confirmatory Analysis of Item Reliability Trends (CAIRT) that allows estimating both trends separately within a structural equation modeling framework. Results of a simulation study prove the CAIRT method to provide reliable and independent parameter estimates; the power exceeds the analysis of item-total correlations. We present an empirical application to self- and peer ratings collected in an Internet-based experiment. Results show that reliability trends are caused by increasing true score variance in self-ratings and by decreasing error variance in peer ratings.
Related Concept Videos
Reliability and Validity
Theory of Attribution II: Kelley's Covariation Theory
Comparing Experimental Results: Student's t-Test
One-Way ANOVA: Equal Sample Sizes
Different sample means can result in different values for the variance estimate: variance between samples. This is because the variance between samples is calculated as the product of the sample size and the variance between the...
Statistical Analysis: Overview
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
Friedman Two-way Analysis of Variance by Ranks


