A Multi-Trait Multi-Method Examination of Psychometric Instrument Performance in Autism Spectrum Disorder

Anecdotal evidence has suggested that rater-based measures (e.g., parent report) may have strong across-trait/within-individual covariance that detracts from trait-specific measurement precision; rater measurement-related bias may help explain poor correlation within Autism Spectrum Disorder (ASD) samples between rater-based and performance-based measures of the same trait. We used a multi-trait, multi-method approach to examine method-associated bias within an ASD sample (n = 83). We examined performance/rater-instrument pairs for attention, inhibition, working memory, motor coordination, and core ASD features. Rater-based scores showed an overall greater methodology bias (57% of variance in score explained by method), while performance-based scores showed a weaker methodology bias (22%). The degree of inter-individual variance explained by method alone substantiates an anecdotal concern associated with the use of rater measures in ASD.