Overview and current management of computerized adaptive testing in licensing/certification examinations

Area of Science:

• Psychometrics
• Educational Measurement
• Computerized Adaptive Testing

Background:

• Computerized adaptive testing (CAT) has been utilized in high-stakes professional examinations since 1994.
• CAT implementation aims to enhance the precision of ability estimations in medical and health licensing examinations.
• Item response theory (IRT) underpins most CAT systems, assuming stable examinee and item characteristics.

Purpose of the Study:

• To outline the implementation process and components of Computerized Adaptive Testing (CAT).
• To discuss the management aspects and challenges associated with CAT systems.
• To evaluate the suitability of CAT for national licensing examinations.

Main Methods:

• The study details a 5-step implementation process for CAT, including feasibility assessment and item bank establishment.
• Key components of the CAT algorithm are identified: item bank, starting item, item selection rule, scoring procedure, and termination criterion.
• CAT management strategies such as content balancing, item analysis, and bank updates are described.

Main Results:

• CAT has been successfully implemented in major U.S. examinations like the NCLEX-RN and NREMT certification.
• The 5-step implementation framework and 5 core algorithm components provide a structured approach to CAT deployment.
• Challenges include the costs associated with CAT platform construction and required computer technology.

Conclusions:

• Computerized adaptive testing (CAT) presents a promising approach for national licensing examinations to achieve more accurate examinee ability estimations.
• Measurement theory provides a robust foundation for implementing CAT in high-stakes testing environments.
• CAT's structured methodology and IRT basis support its adoption for standardized assessments.