Constructing tests for skill assessment with competence-based test development

Area of Science:

• Psychometrics
• Educational Measurement
• Cognitive Science

Background:

• Competence-based test development aims to create maximally informative tests about an individual's skill set.
• Existing methods like the competency deletion procedure can be complex and computationally intensive.
• Test construction often requires balancing informativeness with practical constraints and developer input.

Purpose of the Study:

• To introduce a streamlined competency deletion procedure for efficient test construction.
• To integrate test developer preferences regarding competencies into the test development process.
• To evaluate the performance of tests developed using the streamlined procedure in accurately assessing competence states.

Main Methods:

• A streamlined version of the competency deletion procedure was developed, focusing on essential information for test construction.
• A method was devised to incorporate test developer preferences directly into the selection of competencies.
• The performance of the resulting tests was evaluated based on their ability to accurately infer competence states from item responses.

Main Results:

• The streamlined procedure effectively reduces computational complexity while retaining test informativeness.
• Incorporating developer preferences did not significantly compromise the tests' ability to uncover competence states.
• The developed tests demonstrated strong performance in accurately identifying individuals' underlying skill sets.

Conclusions:

• The streamlined competency deletion procedure offers an efficient and flexible approach to competence-based test development.
• This method allows for the creation of informative tests that align with both psychometric principles and practical developer needs.
• The findings support the use of this enhanced procedure for developing accurate and efficient assessments of individual competencies.