Comparing countdown- and IRT-based approaches to computerized adaptive personality testing.

Area of Science:

• Psychological assessment
• Psychometrics
• Computational psychology

Background:

• Computerized adaptive testing (CAT) is increasingly used for personality assessment due to its efficiency.
• Limited research directly compares item response theory-CAT (IRT-CAT) and countdown method-CAT (CM-CAT).

Purpose of the Study:

• To compare the efficiency and validity of IRT-CAT and CM-CAT algorithms for personality assessment.
• To evaluate item savings, classification accuracy, and validity across different CAT methods.

Main Methods:

• Real-data simulations using responses from the Schedule for Nonadaptive and Adaptive Personality (SNAP) (N = 8,690).
• Evaluation of three CAT algorithms: IRT-CAT, IRT-CAT with a 5-item minimum, and CM-CAT.
• Assessment metrics included item savings, classification accuracy, and convergent/discriminant validity.

Main Results:

• All CAT methods required 18%-86% fewer items than traditional testing.
• All CATs showed lower classification accuracy and validity compared to traditional methods.
• IRT-CAT with a 5-item minimum demonstrated the best balance of item savings and psychometric properties.

Conclusions:

• CAT significantly reduces the number of items needed for personality assessment.
• IRT-CAT, particularly with a minimum item requirement, offers an optimal trade-off between efficiency and assessment quality.
• This study provides a framework for comparing CAT methods in personality assessment.