Portfolios for determining initial licensure competency

Because attempts to improve initial licensure examinations have not been grounded in measurement theory, partial and inadequate remedies have led to a cycle of refutations, defenses and political polarization.

The author reviewed the psychometric literature, focusing on high-stakes professional decisions. Editorials in the dental literature and position papers of involved organizations often use words from this literature without incorporating its fundamental concepts.

The reliability of one-shot initial licensure examinations is estimated to be approximately r = .40, which is a value well under the standard for such tests in other professions. Validity has not been investigated rigorously, but the one-shot format and proposals to remove live patients certainly would reduce validity. The use of portfolios--a small number of evaluations in several realistic task domains--is a viable means of achieving psychometric standards for initial licensure decisions.

Boards are charged with making valid and reliable licensure decisions, not with conducting examinations. At a minimum, they must define the competencies of beginning practitioners and establish the psychometric criteria for their decisions (neither of which are done currently). Gathering data then can be delegated to whoever is best qualified to meet these standards.