Who's Not Talking? Nonresponse Bias in Healthcare Employee Well-Being Surveys

Employee well-being surveys are essential tools used by healthcare leaders to assess workforce functioning, such as burnout, team dynamics, and perceptions of support, but surveys frequently have low response rates, which may skew results. Research on nonresponse bias is limited because of the difficulty in sourcing data on outcomes of interest from nonrespondents. This study aimed to examine whether nonrespondents and respondents differed on key outcomes of interest to healthcare leaders to understand whether results of an employee well-being survey were valid. Specifically, we examined differences between respondents and nonrespondents in terms of demographics, turnover over one-year postsurvey, and employee functioning such as productivity and work outside of regular work hours. By using objective data as a proxy for physician functioning, our innovative approach allowed us to study nonresponse bias without relying on a follow-up survey of nonrespondents. The goal was to inform leaders about potential biases that impact survey conclusions and, therefore, better interpret the survey results in decision-making.

The longitudinal study included physicians (N = 348) and advanced practitioners (APs) (i.e., physician assistants, nurse practitioners, and certified nurse midwives; N = 143) from obstetrics and gynecology clinics in the Midwest and Northwest United States, who were invited to complete an employee well-being survey in 2021. Data on demographics, turnover, and other workplace environment indicators-i.e., productivity measured by relative value units (RVUs), work outside of regular work hours, duration of encounters, and appointment cancellations-were collected from electronic health records (EHRs) and human resources information systems (HRIS). Employment status was tracked for 1.25 years post-survey. The study examined demographic differences (i.e., age, gender, race/ethnicity, marital status), assessed the relative risk of turnover at each quarter over 1.25 years, and evaluated differences in productivity and workplace variables between respondents and nonrespondents. For relative risk, we observed turnover differences between retirement age and below retirement age subgroups.

AP nonrespondents had a nearly 10 times higher risk in the full sample and a 12 times risk in the below-retirement age sample of turnover in the quarter after the survey was deployed. Physician nonrespondents below retirement age had a 5 times relative risk of turnover in the two quarters postsurvey. Among APs, nonrespondents were significantly older and more likely to be married; no differences existed for physicians.

Results demonstrate that individuals at higher risk within an organization, as indicated by higher turnover risk and lower productivity, are less likely to fill out employee surveys. This suggests that employee survey results are skewed by nonresponse bias with respect to outcomes of interest, and that relying solely on survey data may lead to incorrect conclusions about workforce functioning, and subsequently, interventions that do not meet the needs of those most at risk within the organization. In addition to the valuable qualitative insights that surveys provide, healthcare leaders should leverage alternative data-collection methods, such as EHRs and HRIS data, to augment survey data and find out how nonrespondents differ from respondents. In this way, they can gain a comprehensive understanding of employee functioning to inform procedural and policy changes to enhance employee well-being and decrease negative outcomes, such as turnover and low productivity.