Comparing the predictive validity of DUI risk screening instruments: development of validation standards

Area of Science:

• Forensic Psychology
• Substance Abuse Research
• Statistical Modeling

Background:

• Driving under the influence (DUI) screening instruments are crucial for assessing recidivism risk.
• Previous validations of these instruments may be compromised by varying base rates and selection ratios.
• Lack of standardized evaluation methods hinders reliable comparisons of instrument efficacy.

Purpose of the Study:

• To compare the predictive efficacy of established DUI screening instruments.
• To demonstrate how base rate variations and selection ratios influence efficacy assessments.
• To develop robust evaluation standards for risk prediction instruments.

Main Methods:

• Analyzed data from 4815 DUI offenders.
• Examined the impact of base rates and selection ratios on traditional predictive efficacy measures.
• Compared the predictive performance of 10 previously validated DUI instruments.
• Introduced a novel measure of predictive efficacy less sensitive to base rate and selection ratio variations.

Main Results:

• Three instruments consistently performed well at specific thresholds.
• Different evaluation standards yielded significantly divergent conclusions on instrument efficacy.
• Established new standards for validating risk prediction instruments based on the analyses.

Conclusions:

• Failure to apply uniform evaluation standards can result in erroneous conclusions about the relative predictive efficacy of risk assessment tools.
• The developed standards aim to ensure valid and equivalent comparisons of risk prediction instruments.
• Standardized evaluation is essential for accurate risk assessment in DUI populations.