Related Experiment Video
Updated: Jul 16, 2026
09:36
Measurement of Spatial Stability in Precision Grip
Published on: June 4, 2020
Revisiting reliability and measurement precision: Towards a unified perspective
Jimmy de la Torre1, Klaas Sijtsma2, Rodrigo Schames Kreitchmann3
1Faculty of Education, The University of Hong Kong, Hong Kong, China.
Summary
This study unifies reliability and measurement precision concepts from Classical Test Theory (CTT), Item Response Theory (IRT), and Cognitive Diagnosis Models (CDMs). It proposes a common framework using R-squared for consistent reliability estimation across psychometric applications.
Area of Science:
- Psychometrics
- Educational Measurement
- Statistical Modeling
Background:
- Classical Test Theory (CTT), Item Response Theory (IRT), and Cognitive Diagnosis Models (CDMs) offer distinct approaches to reliability and measurement precision.
- Existing frameworks present conceptual differences, hindering consistent application and interpretation.
- A unified perspective is needed to bridge these psychometric models.
Purpose of the Study:
- To revisit and clarify reliability and measurement precision concepts across CTT, IRT, and CDMs.
- To propose a unified psychometric framework integrating these diverse models.
- To introduce novel reliability indices applicable across different measurement paradigms.
Main Methods:
- Conceptual analysis of reliability and precision in CTT, IRT, and CDMs.
- Mathematical integration linking continuous score estimators (CTT/IRT) to mastery probabilities (CDMs).
- Development of reliability indices based on the coefficient of determination (R-squared).
Main Results:
- Demonstrated a conceptual link between continuous score estimators and discrete mastery probabilities.
- Introduced R-squared based reliability indices as a common measure of association.
- Established applicability of the proposed indices to both continuous and discrete classifications.
Conclusions:
- A unified framework clarifies reliability estimation across CTT, IRT, and CDMs.
- Consistent reliability reporting is facilitated, promoting psychometric practice coherence.
- Transparent interpretation of test results is enhanced through a unified reliability perspective.
Related Concept Videos
03:37Uncertainty in Measurement: Accuracy and Precision
Scientists typically make repeated measurements of a quantity to ensure the quality of their findings and to evaluate both the precision and the accuracy of their results. Measurements are said to be precise if they yield very similar results when repeated in the same manner. A measurement is considered accurate if it yields a result that is very close to the true or the accepted value. Precise values agree with each other; accurate values agree with a true value.
01:52Accuracy and Precision
Scientists typically make repeated measurements of a quantity to ensure the quality of their findings and to evaluate both the precision and the accuracy of their results. Measurements are said to be precise if they yield very similar results when repeated in the same manner. A measurement is considered accurate if it yields a result that is very close to the true or the accepted value. Precise values agree with each other; accurate values agree with a true value. Highly accurate measurements...
01:52Accuracy and Precision
Scientists typically make repeated measurements of a quantity to ensure the quality of their findings and to evaluate both the precision and the accuracy of their results. Measurements are said to be precise if they yield very similar results when repeated in the same manner. A measurement is considered accurate if it yields a result that is very close to the true or the accepted value. Precise values agree with each other; accurate values agree with a true value. Highly accurate measurements...
01:11Statistical Analysis: Overview
When we take repeated measurements on the same or replicated samples, we will observe inconsistencies in the magnitude. These inconsistencies are called errors. To categorize and characterize these results and their errors, the researcher can use statistical analysis to determine the quality of the measurements and/or suitability of the methods.
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
01:20Random and Systematic Errors
Scientists always try their best to record measurements with the utmost accuracy and precision. However, sometimes errors do occur. These errors can be random or systematic. Random errors are observed due to the inconsistency or fluctuation in the measurement process, or variations in the quantity itself that is being measured. Such errors fluctuate from being greater than or less than the true value in repeated measurements. Consider a scientist measuring the length of an earthworm using a...
01:20Random and Systematic Errors
Scientists always try their best to record measurements with the utmost accuracy and precision. However, sometimes errors do occur. These errors can be random or systematic. Random errors are observed due to the inconsistency or fluctuation in the measurement process, or variations in the quantity itself that is being measured. Such errors fluctuate from being greater than or less than the true value in repeated measurements. Consider a scientist measuring the length of an earthworm using a...