Relative precision, efficiency and construct validity of different starting and stopping rules for a computerized...

Area of Science:

• Psychiatry and Behavioral Sciences
• Clinical Psychology
• Health Services Research

Background:

• Substance abuse treatment programs face increasing demands for efficient problem assessment.
• Clinical decision-making requires accurate and timely measurement of substance use disorders.
• Existing assessment tools may be lengthy, impacting treatment efficiency.

Purpose of the Study:

• To evaluate the efficiency, precision, and construct validity of computerized adaptive testing (CAT) for the Global Appraisal of Individual Needs' (GAIN) Substance Problem Scale (SPS).
• To determine optimal starting and stopping rules for shortening the GAIN-SPS using CAT.
• To facilitate more efficient diagnosis of substance use problems in treatment-seeking populations.

Main Methods:

• A simulation study using data from 1,048 adolescents and adults in substance abuse treatment.
• Computerized adaptive testing (CAT) was employed to administer a shortened version of the GAIN-SPS.
• CAT performance was assessed using metrics including standard errors, item count, measurement bias, classification accuracy (Cohen's kappa), and correlations with full-scale measures and associated instruments.

Main Results:

• The CAT approach demonstrated efficiency in administering the GAIN-SPS, requiring fewer items on average.
• CAT yielded precise person measures with minimal bias and high correlation with full-scale measures.
• Classification accuracy of CAT-derived diagnoses was comparable to clinical classifications, supporting construct validity.

Conclusions:

• Computerized adaptive testing (CAT) is an efficient and valid method for administering the GAIN Substance Problem Scale (SPS).
• CAT facilitates accurate and precise assessment of substance use problems, supporting clinical decision-making.
• This approach can improve the efficiency of substance abuse treatment programs.