Changing priorities for improvement: the impact of low response rates in patient satisfaction

A disturbing trend in patient satisfaction research has been a willingness to accept low response rates as inevitable. However, it may not be appropriate to generalize data based on low responses to the full population of patients, since to do so may threaten the validity of the findings.

Satisfaction data were collected from 19,556 inpatients discharged in 1994 from 76 hospitals using the 69-item NCG Patient Viewpoint Survey, an instrument that primarily uses a set of five response options, which are transformed to a 0- to 100-point scale. Surveys were sent to random samples of 100 to 1,400 patients, and were followed by postcard reminders. For each hospital sample, results for the "First 30%" were compared with those for "All Respondents," or the total number of respondents, for which the average response rate was 58%.

Results on individual scale scores and the subsequent improvement priorities for individual hospitals had a 50-50 chance of being different when the First 30% responses were compared with the All Respondents responses. For 9 out of 13 survey scales, the scores were significantly different between the First 30% and All Respondents when data were aggregated across all hospitals. For 42% of the 76 hospitals, a different set of scales would be identified as those most in need of improvement.

The capriciousness of within-hospital differences based on the First 30% versus All Respondents brings into question the utility of patient satisfaction data based on low response rates even with a reliable instrument and with controlled, consistent data collection methods across hospitals. Target response rates should be set at 50% or higher. Additional research on the effects of response rates on patient satisfaction data are recommended.