Survey Protocols, Response Rates, and Representation of Underserved Patients: A Randomized Clinical Trial

Surveys often underrepresent certain patients, such as underserved patients. Methods that improve their response rates (RRs) would help patient surveys better represent their experiences and assess equity and equity-targeted quality improvement efforts.

To estimate the effect of adding an initial web mode to existing Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey protocols and extending the fielding period on RR and representativeness of underserved patient groups.

This randomized clinical trial included 36 001 patients discharged from 46 US hospitals from May through December 2021. Data analysis was performed from May 2022 to September 2023.

Patients were randomized to 1 of 6 survey protocols: 3 standard HCAHPS protocols (mail only, phone only, mail-phone) plus 3 web-enhanced protocols (web-mail, web-phone, web-mail-phone).

RR and number of respondents per 100 survey attempts (yield) were calculated and compared for each of the 6 survey protocols, overall, and by patient age, service line, sex, and race and ethnicity.

A total of 34 335 patients (median age range, 55-59 years; 59.3% female individuals and 40.7% male individuals) were eligible and included in the study. Of the respondents, 6.9% were Asian American or Native Hawaiian or Other Pacific Islander, 0.7% were American Indian or Alaska Native, 11.5% were Black, 17.4% were Hispanic, 61.0% were White, and 2.6% were multiracial. Of the 6 protocols, RRs were highest in web-mail-phone (36.5%), intermediate for the 3 two-mode survey protocols (mail-phone, web-mail, web-phone, 30.3%-31.1%), and lowest for the 2 single-mode protocols (mail only, phone only, 22.1%-24.3%). Web-mail-phone resulted in the highest yield for 3 racial and ethnic groups (Black, Hispanic, and White patients) and second highest for another (multiracial patients). Otherwise, the highest or second highest yield was almost always a 2-mode protocol. Mail only was the lowest-yield protocol for Black, Hispanic, and multiracial patients and phone-only was the lowest-yield protocol for White patients; these 2 protocols tied for lowest-yield for Asian American or Native Hawaiian or Other Pacific Islander patients. Gains from multimode approaches were often 2 to 3 times as large for Asian American or Native Hawaiian or Other Pacific Islander, Black, Hispanic, and multiracial patients as for White patients. Web-mail-phone had the highest RR for 6 of 8 age groups and 4 of 5 combinations of service line and sex.

In this randomized clinical trial, web-first multimode survey protocols significantly improved the RR and representativeness of patient surveys. The best-performing protocol based on RR and representativeness was web-mail-phone. Web-phone performed well for young and diverse patient populations, and web-mail for older and less diverse patient populations. The US Centers for Medicare & Medicaid Services will allow hospitals to use the web-mail, web-phone, and web-mail-phone protocols for HCAHPS administration beginning in 2025.