A competency addition procedure for constructing minimal, yet maximally informative tests for skill assessment

Competence-based test development is a novel method for constructing tests that are as informative as possible about the competence state (the set of skills an individual possesses) underlying item responses. If desired, the tests can also be minimal, meaning that no item can be removed without reducing their informativeness. Consider a set of competencies, each encompassing the skills required to solve a particular item. An individual masters an item if their competence state includes all the skills in the item's associated competency. A reduct is defined by a competency set that is as informative about individuals' competence states as a larger set, and from which no competency can be removed without reducing its informativeness. Test development can be based on the reduct, since including only one item for each competency in the reduct yields a test that is as informative as a test that contains additional items and is minimal with respect to this property. This work introduces the competency addition procedure, an iterative method that successively collects competencies until the resulting set forms a reduct. The competencies to be added can be selected either randomly or based on test developer preferences, thereby favoring competencies with more desirable characteristics. The procedure is illustrated in three real-life applications for the assessment of arithmetic skills, involving the construction of a test from scratch, the improvement of an existing test, and the shortening of an existing test.