A Preliminary Evaluation of Interrater Reliability and Concurrent Validity of Open-Ended Indirect Assessment

Indirect assessments are a commonly used component of functional behavior assessment by behavior analysts in practice who work with individuals with severe behavior disorders. Although used frequently, closed-ended indirect assessments have repeatedly been shown to have low to moderate interrater reliability and poor concurrent validity with functional analysis. Recently, the use of open-ended interviews has become more commonly adopted in applied clinical practice, despite no studies evaluating the psychometric properties of such assessments. In the present study, we evaluated the interrater reliability and concurrent validity of an open-ended functional assessment interview. We compared the results of two open-ended indirect assessments conducted with a common caregiver and subsequently conducted functional analyses in an attempt to validate hypotheses generated from the interviews. Interrater agreement for the open-ended interviews was higher than previous research on closed-ended interviews (75%); however, concurrent validity with functional analysis was relatively poor (50%). We discuss these findings in the context of assessment and treatment for severe behavior disorders, as well as best practice methods during functional behavior assessment.