Autism Knowledge Assessments: A Closer Examination of Validity by Autism Experts.

Area of Science:

• Psychology
• Neurodevelopmental Disorders
• Educational Assessment

Background:

• Autism knowledge assessments are crucial for education and research.
• Prior psychometric validation is important but may not capture all validity aspects.
• Expert review is essential for ensuring the accuracy and clarity of assessment content.

Purpose of the Study:

• To conduct a substantive validity review of four autism knowledge assessments.
• To evaluate the accuracy and ambiguity of assessment questions using expert feedback.
• To identify common issues in autism knowledge assessment questions.

Main Methods:

• Recruited 69 autism experts from editorial boards of peer-reviewed autism journals.
• Experts reviewed four autism knowledge assessments for accuracy and ambiguity.
• Analyzed expert feedback to identify themes in problematic questions.

Main Results:

• 34% of questions were flagged as potentially problematic for accuracy.
• 17% of questions were flagged as potentially problematic for ambiguity.
• Identified three themes: oversimplification of research, insufficient recognition of heterogeneity, and lack of clarity.

Conclusions:

• Substantive validity of autism knowledge assessments requires rigorous expert evaluation.
• Potentially problematic questions should be modified or removed to enhance assessment validity.
• Future assessments should incorporate feedback from diverse experts and respondents to ensure accuracy and clarity.