Methods for measuring utilization of mental health services in two epidemiologic studies

Psychiatric epidemiologic studies often include two or more sets of questions regarding service utilization, but the agreement across these different questions and the factors associated with their endorsement have not been examined. The objectives of this study were to describe the agreement of different sets of mental health service utilization questions that were included in the American Indian Service Utilization Psychiatric Epidemiology Risk and Protective Factors Project (AI-SUPERPFP), and compare the results to similar questions included in the baseline National Comorbidity Survey (NCS).

Responses to service utilization questions by 2878 AI-SUPERPFP and 5877 NCS participants were examined by calculating estimates of service use and agreement (kappa) across the different sets of questions. Logistic regression models were developed to identify factors associated with endorsement of specific sets of questions.

In both studies, estimates of mental health service utilization varied across the different sets of questions. Agreement across the different question sets was marginal to good (kappa = 0.27-0.69). Characteristics of identified service users varied across the question sets.

Neither survey included data to examine the validity of participant responses to service utilization questions. RECOMMENDATIONS FOR FURTHER RESEARCH: Question wording and placement appear to impact estimates of service utilization in psychiatric epidemiologic studies. Given the importance of these estimates for policy-making, further research into the validity of survey responses as well as impacts of question wording and context on rates of service utilization is warranted.