Multivariate models provide an effective psychometric solution to the variability in classification accuracy of
Laura Cutler1, Matthew Greenacre2, Christopher A Abeare1
1Department of Psychology, Neuropsychology Track, University of Windsor, Windsor, Ontario, Canada.
The Clinical Neuropsychologist
|August 10, 2022
Summary
Multivariate models using the D-KEFS Stroop test demonstrated superior accuracy in identifying valid performance compared to single-score cutoffs. This approach balances sensitivity and specificity, improving classification in clinical and disability evaluations.
Area of Science:
- Neuropsychology
- Psychometrics
Background:
- The D-KEFS Stroop is a potential performance validity test (PVT), but prior research yielded inconsistent findings.
- Previous studies on D-KEFS Stroop validity cutoffs produced diverging conclusions.
Purpose of the Study:
- To evaluate the classification accuracy of existing D-KEFS Stroop validity cutoffs.
- To assess the efficacy of multivariate models and a novel D-KEFS Stroop Index for performance validity testing.
Main Methods:
- Two independent samples (mild TBI patients, disability benefit applicants) were used.
- Classification accuracy was computed against four criterion PVTs.
- Age-corrected scaled scores (ACSSs) and multivariate models were analyzed.
Main Results:
- Individual subtest cutoffs (ACSS ≤6) often lacked adequate specificity.
- More conservative cutoffs improved specificity but reduced sensitivity.
- Multivariate models (≥3 failures at ACSS ≤6 or ≥2 failures at ACSS ≤5) and the D-KEFS Stroop Index showed high classification accuracy (74.6-93.3%).
Conclusions:
- Multivariate approaches offer robust performance validity assessment, mitigating sample- and instrument-specific variability.
- This method balances sensitivity and specificity effectively.
- The D-KEFS Stroop Index shows promise as a reliable PVT.
More Related Videos
Related Concept Videos
01:29Reliability and Validity
13.1K
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
13.1K
01:19Expected Frequencies in Goodness-of-Fit Tests
2.6K
A goodness-of-fit test is conducted to determine whether the observed frequency values are statistically similar to the frequencies expected for the dataset. Suppose the expected frequencies for a dataset are equal such as when predicting the frequency of any number appearing when casting a die. In that case, the expected frequency is the ratio of the total number of observations (n) to the number of categories (k).
2.6K
01:29Measures of Intelligence
7.8K
Psychologists measure intelligence by using standardized tests that produce a score known as the intelligence quotient or IQ. To understand IQ tests, it's important to recognize the key principles behind their construction: validity, reliability, and standardization.
Validity refers to how well a test measures what it claims to measure. An intelligence test should accurately assess intelligence rather than another characteristic, like anxiety. Criterion validity is one way to evaluate this;...
Validity refers to how well a test measures what it claims to measure. An intelligence test should accurately assess intelligence rather than another characteristic, like anxiety. Criterion validity is one way to evaluate this;...
7.8K
01:11Variability: Analysis
183
Measures of variability are statistical metrics that reveal the dispersion pattern within a dataset. They are pivotal in biostatistics, providing insights into the heterogeneity within health and biological data. Variability signifies the degree to which data points diverge from one another, helping researchers understand the potential range of values and associated uncertainty within the data.
The range is a simple measure of variability, indicating the difference between the highest and...
The range is a simple measure of variability, indicating the difference between the highest and...
183
01:21Friedman Two-way Analysis of Variance by Ranks
285
Friedman's Two-Way Analysis of Variance by Ranks is a nonparametric test designed to identify differences across multiple test attempts when traditional assumptions of normality and equal variances do not apply. Unlike conventional ANOVA, which requires normally distributed data with equal variances, Friedman's test is ideal for ordinal or non-normally distributed data, making it particularly useful for analyzing dependent samples, such as matched subjects over time or repeated measures...
285
01:20Spearman's Rank Correlation Test
983
Spearman's rank correlation test, also known as Spearman's rho, is a nonparametric method for assessing the strength and direction of association between two variables. This test is particularly valuable when the data distribution is unknown or when the assumption of normality does not hold. Named after the English psychologist and statistician Dr. Charles Edward Spearman, it serves as the nonparametric counterpart to Pearson's correlation coefficient.
Spearman's test calculates...
Spearman's test calculates...
983