A Multi-Trait Multi-Method Examination of Psychometric Instrument Performance in Autism Spectrum Disorder

Area of Science:

• Neurodevelopmental Disorders
• Psychometrics
• Autism Spectrum Disorder Research

Background:

• Rater-based measures (e.g., parent reports) in Autism Spectrum Disorder (ASD) research may exhibit significant across-trait covariance.
• This covariance can reduce measurement precision for specific traits and potentially explain discrepancies between parent reports and performance-based assessments.

Purpose of the Study:

• To investigate method-associated bias in Autism Spectrum Disorder (ASD) samples.
• To quantify the contribution of measurement methodology to score variance in both rater-based and performance-based assessments.

Main Methods:

• Employed a multi-trait, multi-method research design.
• Examined performance and rater-instrument pairs for attention, inhibition, working memory, motor coordination, and core ASD features in an ASD sample (n=83).

Main Results:

• Rater-based scores demonstrated substantial methodology bias, accounting for 57% of score variance.
• Performance-based scores exhibited considerably weaker methodology bias, explaining 22% of score variance.

Conclusions:

• The significant influence of methodology on rater-based scores validates concerns about their use in ASD research.
• Findings underscore the need to consider method bias when interpreting rater-based data in ASD trait assessment.