Using telephone interviews to reduce nonresponse bias to mail surveys of health plan members

To assess the nonresponse bias associated with mail-survey returns and the potential for telephone interviews with nonrespondents to reduce that bias.

A mail survey about health care experiences was conducted with samples of 800 members in each of four health plans. Subsequent attempts were made to interview nonrespondents by telephone.

Response rates for the mail surveys averaged 46%; the telephone effort raised the average to 66%. On 17 of 19 measures of health status or need and use of health services, mail respondents were in poorer health and needed more services than interviewed nonrespondents. Thirteen of 36 reports and ratings of health care also differed significantly between the two groups. Based on administrative data, telephone interviews of mail nonrespondents improved the demographic representativeness of the responding samples. Adjusting mail returns to sample population characteristics could not replicate the dual-mode results.

Returns to mail surveys are likely to be related to survey content and hence are potentially biased. Nonresponse to phone surveys is less directly related to survey content. Telephone interviews with mail nonrespondents not only increase response rates but also can produce less biased samples than mail-only protocols.