Surgeon-level reporting presented by funnel plot is understood by doctors but inaccurately interpreted by members of...

Risk-adjusted outcome data for general surgeons practicing in the United Kingdom were published for the first time in 2013 with the aim of increasing transparency, improving standards, and providing the public with information to aid decision making. Most specialties used funnel plots to present their data. We assess the ability of members of the public (MoP), medical students, nonsurgical doctors (NSD), and surgeons to understand risk-adjusted surgical outcome data.

A fictitious outcome dataset was created and presented in the form of a funnel plot to 10 participants from each of the aforementioned group. Standard explanatory text was provided. Each participant was given 5 minutes to review the funnel plot and complete a questionnaire. For each question, there was only 1 correct answer.

Completion rate was 100% (n = 40). No difference existed between NSD and surgeons. A significant difference for identification of the "worst performing surgeon" was noted between surgeons and MoP (p < 0.01) and between NSD and MoP (p < 0.01). Half of medical students and MoP claimed they would use this information to aid decision making compared with 80% of doctors. MoP reported the funnel plot significantly "more difficult" to interpret than surgeons did (p < 0.01) and NSD (p < 0.01).

MoP found these data significantly more "difficult to understand" and were less likely to both spot "outliers" and use this data to inform decisions than doctors. Surgeons should be aware that outcome data may require an alternative method of presentation to be understood by MoP.