Reliability and true-score measures of binary items as a function of their Rasch difficulty parameter.

Area of Science:

• Psychometrics
• Educational Measurement
• Item Response Theory

Background:

• Accurate estimation of true-score measures and reliability is crucial for effective test development and score interpretation.
• Existing methods may require extensive data collection for reliability and true-score estimation.
• The dichotomous Rasch model provides a framework for item calibration and ability estimation.

Purpose of the Study:

• To derive formulas for expected true-score measures and reliability of binary items based on Rasch difficulty.
• To demonstrate the utility of these formulas in test development, score analysis, and simulation studies.
• To enable estimation of true-score measures and reliability without additional data collection post-calibration.

Main Methods:

• Utilizing the dichotomous Rasch model for item calibration.
• Developing theoretical formulas for expected true-score measures (domain score, true score variance, error variance) and reliability.
• Assuming normal or logistic trait (ability) distributions.
• Applying formulas to norm-referenced and criterion-referenced interpretations.

Main Results:

• Formulas were derived for expected true-score measures and reliability as a function of Rasch item difficulty.
• The proposed formulas allow for estimation of these psychometric properties without further data collection after Rasch calibration.
• Demonstrated application in developing tests with desired psychometric properties and comparing item subsets.

Conclusions:

• The derived formulas offer theoretical and practical value for psychometricians and test developers.
• These formulas streamline the process of estimating key measurement properties for binary items.
• Facilitates informed decisions in test construction and analysis using calibrated Rasch item banks.