A consistency measure for psychometric measurements
Maximilian Karl Scharf1, Anna Warzybok1, Sabine Hochmuth2
1Medical Physics and Cluster of Excellence "Hearing4all," Carl von Ossietzky Universität Oldenburg, Oldenburg, Germany.
The Journal of the Acoustical Society of America
|October 3, 2025
Summary
This study introduces a new consistency measure for psychometric testing to identify unreliable results. This method helps ensure data accuracy in adaptive tracking procedures.
Area of Science:
- Psychology
- Psychophysics
- Data Analysis
Background:
- Adaptive tracking in psychophysics can yield erroneous results due to subject inattention or external factors.
- Existing methods may struggle to automatically detect inconsistent psychometric measurement outcomes.
Purpose of the Study:
- To develop and validate a novel consistency measure for psychometric testing.
- To enable post hoc, automated detection of unreliable data in adaptive tracking procedures.
Main Methods:
- A multi-state psychometric model was developed to rate measurement outcomes.
- The model calculates log likelihood differences between single and interleaved psychometric functions.
- A binary classifier evaluated various consistency measures using simulated and empirical data.
Main Results:
- The proposed consistency measure effectively classified inconsistent tracks.
- The new measure outperformed stimulus level spectrum for predicting consistency.
- A threshold of 10 for the German matrix sentence test demonstrated 60% sensitivity and 80% specificity.
Conclusions:
- The developed consistency measure provides a reliable method for automated data quality assessment in psychophysics.
- This approach enhances the trustworthiness of results from adaptive tracking procedures.
- The measure is applicable to psychometric functions modeled by sigmoid curves.
Related Concept Videos
01:29Measures of Intelligence
8.3K
Psychologists measure intelligence by using standardized tests that produce a score known as the intelligence quotient or IQ. To understand IQ tests, it's important to recognize the key principles behind their construction: validity, reliability, and standardization.
Validity refers to how well a test measures what it claims to measure. An intelligence test should accurately assess intelligence rather than another characteristic, like anxiety. Criterion validity is one way to evaluate this;...
Validity refers to how well a test measures what it claims to measure. An intelligence test should accurately assess intelligence rather than another characteristic, like anxiety. Criterion validity is one way to evaluate this;...
8.3K
03:37Uncertainty in Measurement: Accuracy and Precision
99.7K
Scientists typically make repeated measurements of a quantity to ensure the quality of their findings and to evaluate both the precision and the accuracy of their results. Measurements are said to be precise if they yield very similar results when repeated in the same manner. A measurement is considered accurate if it yields a result that is very close to the true or the accepted value. Precise values agree with each other; accurate values agree with a true value.
99.7K
01:29Reliability and Validity
13.7K
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
13.7K
01:11Statistical Analysis: Overview
14.5K
When we take repeated measurements on the same or replicated samples, we will observe inconsistencies in the magnitude. These inconsistencies are called errors. To categorize and characterize these results and their errors, the researcher can use statistical analysis to determine the quality of the measurements and/or suitability of the methods.
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
14.5K
01:20Random and Systematic Errors
14.3K
Scientists always try their best to record measurements with the utmost accuracy and precision. However, sometimes errors do occur. These errors can be random or systematic. Random errors are observed due to the inconsistency or fluctuation in the measurement process, or variations in the quantity itself that is being measured. Such errors fluctuate from being greater than or less than the true value in repeated measurements. Consider a scientist measuring the length of an earthworm using a...
14.3K
01:22Self-Report Tests of Personality
770
Self-report inventories are objective personality assessments that use multiple-choice items or numbered scales, typically ranging from 1 (strongly disagree) to 5 (strongly agree). They are often called Likert scales after Rensis Likert. These inventories are widely used due to their ease of administration and cost-effectiveness. One of the most prominent examples is the Minnesota Multiphasic Personality Inventory (MMPI), initially developed in the 1940s to assess abnormal personality traits.
770