Reliability of self-reported health service use: evidence from the women with co-occurring disorders, and violence...

In behavioral health services research, self-reporting provides comprehensive information on service use, but may have limited reliability because of recall bias and misclassification. This study examines test-retest reliability of self-reported health service use, factors affecting reliability, and the impact of inconsistent reporting on the robustness of cost estimates using the test-retest data from the Women, Co-occurring Disorders, and Violence Study (n = 186). Reliability varies widely across service types: moderate to substantial (k = 0.65-0.94) for any use; slight to substantial (ICC = 0.12-0.93) for quantity of use; and none to moderate (k = -0.06-0.79) for service content, but is not affected by psychiatric symptom severity. Cost estimates do not differ according to the use of test or retest data. Findings suggest that self-reporting provides reliable data on service quantity and is adequate for economic evaluations. However, self-reporting of treatment content in highly specified service categories (e.g., individual counseling during residential treatment) may not be reliable.