The Influence of Respondent Characteristics on the Validity of Self-Reported Survey Responses

To examine concordance between member self-reports and the organization's administrative claims data for two key health factors: number of chronic conditions, and number of prescription drugs.

Medicare Advantage plan claims data and member survey data from 2011 to 2012.

Mailed surveys to 15,000 members, enrolled minimum 6 months, drawn from a random sample of primary care physician practices with at least 200 members.

Descriptive statistics were generated for extent of concordance. Multivariable logistic regressions were used to analyze the association of selected respondent characteristics with likelihood of concordance.

Concordance for number of chronic conditions was 58.4 percent, with 27.3 percent under-reporting, 14.2 percent over-reporting. Concordance for number of prescription drugs was 56.6 percent with 38.9 percent under-reporting, 4.5 percent over-reporting. Number of prescriptions and assistance in survey completion were associated with higher likelihood of concordance for chronic conditions. Assistance in survey completion and number of chronic conditions were associated with higher concordance, and age and number of prescriptions were associated with lower concordance, for prescription drugs.

Self-reported number of chronic conditions and prescription medications are not in high concordance with claims data. Health care researchers and policy makers using patient self-reported data should be aware of these potential biases.