A new adaptive testing algorithm for shortening health literacy assessments

Area of Science:

• Health Informatics
• Measurement Theory
• Health Literacy Assessment

Background:

• Low health literacy negatively impacts health outcomes and online resource utilization.
• Brief assessment tools are crucial for practical adoption in healthcare settings.
• Traditional test development via item-response theory necessitates extensive pretesting.

Purpose of the Study:

• To develop a novel classification method for creating brief health literacy assessment instruments.
• To enable the development of assessments without large participant pretesting.
• To create instruments suitable for computerized adaptive testing (CAT).

Main Methods:

• A new algorithm integrating measurement decision theory (MDT) and information theory was developed.
• The algorithm was applied to secondary data from two health assessment tests: health term familiarity and health numeracy.
• Data from 52 participants (60 items) and 165 participants (8 items) were analyzed.

Main Results:

• The method achieved 88.5% correct classification in the familiarity dataset, reducing test length by approximately 50%.
• In the numeracy dataset, a two-class scheme yielded 96.9% correct classification with a 35.7% test length reduction.
• A three-class scheme for numeracy correctly classified 93.8% of subjects, reducing test length by 17.7%.

Conclusions:

• Measurement decision theory (MDT)-based approaches offer a viable alternative to item-response theory.
• These MDT methods are well-suited for developing brief, efficient health literacy assessments.
• The findings support the use of MDT for computerized adaptive testing in the health domain.