Using Item Response Theory Models in Scaling Severity Scores in Alcohol Research-A Tutorial

Questionnaires are frequently used in the assessment of drug and alcohol dependence.

Typically, sum scores are used for aggregation, which come with several underlying assumptions. If untested psychometrically, the reliability and sufficiency of a sum score as a measure of the characteristic of interest remain unclear. A simple sum score, for example, does not weight the items and therefore each item contributes with the same level of importance to the total. However, weighting more severe symptoms higher would often be desirable. Additionally, sum scores do not reflect the unreliability of the questionnaire items; i.e., the proneness of an item to measurement error. Unreliability reduces the effects of interest, e.g., an analysis of group differences on a score will underestimate true effects. Latent variable models account for measurement error and provide reliable estimates of a person's score on the underlying constructs.

In this paper, we showcase the application of item response theory models from psychometrics for the analysis of categorical questionnaire data. Using non-technical language, we introduce the most common models and discuss model parameters, model fit, and group comparisons. These aspects allow for the evaluation of the assumptions underlying a sum score.

We introduce these concepts with the help of real data from the Danish alcohol and drug consumption survey from 2011, using items on alcohol harms to others. The data and the full reproducible code in STATA and R are provided. Reporting guidelines are given as well as a discussion of pitfalls and limitations.